From a broad question to a testable quality process

Automated Interface Testing: Safely Validate APIs, Files, and Legacy Data

What should actually be tested?

A useful automated test separates the layers. That prevents one green test run from promising more than it actually checked.

The first two iterations focused on parsing and structure. Iteration 3 adds a small HTTP transport slice. The remaining layers stay separate: not every legacy file needs OAuth, and not every API needs a filename check. A common result model keeps the reports comparable.

Interface types and starting points

Web APIs, files, and legacy exports are the three main areas of the project. We started with a local JSON file, added CSV and fixed-width profiles, then implemented a deliberately limited HTTP adapter and a bounded SFTP adapter. Directory polling, multi-file correlation, and message queues remain later topics.

1. Web APIs: contract plus behavior

An HTTP response is more than a data object. Method, URL, headers, status code, timing, authentication, and error format are part of the contract. A schema can check that customerId is a string and items is an array; it cannot decide by itself whether an unauthorized GET /orders/123 should return 403 or 404.

That is why OpenAPI is a useful outer HTTP contract. Its data models can use JSON-Schema-compatible rules, while contract tests cover methods, statuses, headers, requests, and responses. The validator is one part of contract testing—not a replacement for it.

2. Files: read safely, then validate

A file delivered through FTP or SFTP has properties outside its content: naming, location, delivery window, compression, encoding, checksum, and a signal that writing is complete. A valid CSV can still be truncated if the sender is uploading it while the receiver starts reading.

A complete file adapter therefore needs a sequence: find the file, verify stability or a marker, transfer it safely, record hash and size, parse it, validate it, move it to quarantine or processing, and report the result.

3. Legacy interfaces: turn reality and rules into a contract

Older interfaces often lack a maintained schema. Samples, DDL scripts, field descriptions, COBOL copybooks, RPG source such as RPGLE, or an existing importer are useful evidence—but they are not automatically the business truth. A column may be VARCHAR(10) in DDL and still allow only a small set of status codes.

The sensible order is to derive a schema draft, review it with the business, approve it deliberately, and store it as a versioned contract. Automatic generation saves work; it does not decide which deviation is an error and which is an accepted exception.

Which schema format fits?

The obvious formats serve different purposes. The question is which part of the contract needs to be described.

Why JSON Schema is our starting point

JSON Schema is practical for a first step because both the contract and the test data can be JSON. It describes types, required fields, arrays, ranges, patterns, and compositions. It can be versioned, checked, and validated in many languages.

Our first executable example describes a delivery with an ID, quantity, email address, and postal address:

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "delivery.schema.json",
  "title": "Delivery",
  "type": "object",
  "additionalProperties": false,
  "required": ["deliveryId", "quantity", "email", "address"],
  "properties": {
    "deliveryId": { "type": "string", "minLength": 1 },
    "quantity": { "type": "integer", "minimum": 1 },
    "email": { "type": "string", "format": "email" },
    "address": { "$ref": "refs/address.schema.json" }
  }
}

The address lives in refs/address.schema.json. The example enables format assertions explicitly, so an obviously invalid email is reported as a contract violation. That checks the format—not whether a mailbox exists or can be reached.

Architecture: a small core with open adapters

The current implementation is one Java 21 Maven module. The core defines project-owned result objects and a validator interface. JSON, CSV, fixed-width, and HTTP adapters encapsulate their parsers and profile rules; the CLI uses Picocli, and separate writers produce text or JSON. Spring Boot is not required for this use case.

The shared result contract stays intentionally small: status, valid, and an ordered list of findings. Source, timestamp, schema ID, hash, and HTTP metadata are possible future additions, not current fields. Secrets, tokens, and complete personal payloads should never be copied into a report by default.

Testing files automatically

Iteration 1 validated an existing local JSON file. Iteration 2 added local CSV and fixed-width files; Iteration 3 adds a local HTTP call described by a profile; Iteration 4 retrieves and validates exactly one stable SFTP file; Iteration 5 selects exactly one matching file from an SFTP directory. Multi-file lifecycles, claim/lock protocols, and operational orchestration remain separate concerns.

A complete file flow is larger than a schema check

1. Check the expected path, filename, delivery window, and extension.
2. Confirm that the file is complete—through a temporary suffix, done marker, stable size, or checksum.
3. Transfer it safely and record technical metadata.
4. Validate encoding and file format before applying business rules.
5. Parse records and validate them against the format profile.
6. Check business rules, quantities, and duplicates.
7. Move failures to quarantine without altering the original and produce a traceable report.

Testing web APIs with contracts and authentication

The current API adapter focuses on one known endpoint and checks a limited but useful set of properties:

• reachability, TLS, and timeout behavior;
• HTTP method, URL, headers, and content type;
• expected status codes for success and defined failure cases;
• request and response schemas;
• authentication, pagination, rate limits, correlation IDs, and repeatability as future layers.

Authentication should be represented by a profile such as oauth2-client-credentials, api-key, mtls, or basic. Concrete values belong in environment variables, a secret store, or CI configuration. Neither schemas nor reports should contain real tokens, private certificates, or passwords.

Iteration 3 already covers a valid request, a request violation, an unexpected status, a missing required header, an incorrect response type, malformed JSON, and a transport failure. Authorization profiles, rate limits, and multi-step business scenarios remain later iterations.

Describing legacy interfaces after the fact

A schema created after the system went live is not an inferior substitute for an official specification. It is often the most realistic way to turn implicit knowledge into an explicit, testable contract.

1. Collect samples and DDL.
2. Draft the schema and mark uncertain assumptions.
3. Ask domain owners about required fields, codes, and ranges.
4. Define valid and invalid fixtures.
5. Approve version 1 and add it to the test runs.
6. Treat new deviations as deliberate schema changes, not as silent “make it green” exceptions.

From validation to quality assurance

A useful result separates two questions: Did the data violate the contract, or could the check not be performed reliably? The CLI expresses that boundary with three outcomes:

Findings name a data path, schema path, keyword, and message. Their ordering is stable. Automation can therefore compare the actual report with an expected example instead of checking only “red or green.”

The idea as a living Project Page

The article is meant to grow with the project. The Project Page records the current position, the logbook preserves the chronology, and a sub-article appears only when an iteration has its own fixtures, code, measurements, or meaningful failed attempt. That keeps the architecture honest: later work does not look as if it had been known on day one.

The first development iterations

The ZEUS Interface Quality repository now contains four completed, executable vertical slices. The next entries are options, not promises of existing features.

Repository, CI, and the shared JSON report remain the foundation. Iteration 6 combines directory selection with delivery-ID correlation, claiming, recovery, success archiving, and quarantine. The SFTP adapter now checks not only the content but also the delivery’s transition through the validation process. OpenAPI derivation, secret management, mixed-format deliveries, and cross-file business rules can now be evaluated separately.

Logbook

Newest entries appear first. Earlier entries remain as historical snapshots; the current status belongs at the beginning of the page.

Open questions

• Which OpenAPI or secret-management extension is the next clearly bounded HTTP step?
• Which context should an operational report add without unnecessarily expanding the contract?
• What file-size, runtime, and resource limits are needed before production use?
• When does YAML need its own contract instead of being parsed into a JSON-compatible model?
• What quarantine and retry model is the minimum standard for file adapters?
• Which license should the first public release use?

What schema validation does not do

A schema cannot infer business meaning, move data from a counterparty, or repair a broken credential. It also does not answer availability, security, performance, compatibility, completeness, or retry questions. Those are separate test layers—and keeping them separate is what makes the approach maintainable.

Glossary

Frequently asked questions