One contract for delimiters, positions, and real file failures

Automatically Validate CSV and Legacy Files: Profiles for Interfaces

The broader approach is described on the English Project Page. Iteration 1 covers JSON Schema; this article follows the next vertical slice from a file profile to a reproducible result.

One result contract for multiple file formats

The validator uses a different adapter for each input format, but the result remains the same project object. A CI script does not need to know whether a finding came from JSON, CSV, or a fixed-width file.

A missing CSV value is a problem in delivered data. A broken profile or undecodable input prevents the check itself. Those cases need different treatment in operations.

Describe CSV with a small profile

CSV is not a complete contract. Delimiter, encoding, header, and column rules turn a sequence of fields into a meaningful, testable delivery. Iteration 2 uses a deliberately small JSON profile:

{
  "format": "csv",
  "version": 1,
  "delimiter": ";",
  "encoding": "UTF-8",
  "header": ["deliveryId", "quantity", "email"],
  "columns": [
    {
      "name": "deliveryId",
      "type": "string",
      "required": true,
      "minLength": 1
    },
    {
      "name": "quantity",
      "type": "integer",
      "required": true,
      "minimum": 1
    },
    {
      "name": "email",
      "type": "email",
      "required": true
    }
  ]
}

The header is fixed in both content and order. A file with swapped columns is not “almost correct” when the contract is positional. Column rules stay with the column, so a finding can identify both its data and profile path.

Validate CSV files automatically

The valid fixture contains the same small delivery idea as the JSON test, now with semicolon delimiters. Quoted fields are handled by the CSV parser; profile validation begins after the records can be read.

deliveryId;quantity;email
DEL-1001;3;ops@example.org
DEL-1002;12;"warehouse@example.org"

Choose the adapter explicitly with --input-format csv, or leave the default AUTO in place. The CLI reads format from the profile and selects CSV or fixed-width.

$jar = "target/zeus-interface-quality-0.1.0-SNAPSHOT.jar"
$profile = "src/test/resources/fixtures/csv/delivery.csv-profile.json"
$input = "src/test/resources/fixtures/csv/valid-delivery.csv"

java -jar $jar validate --input-format csv --schema $profile --input $input --report json
$LASTEXITCODE

The invalid fixture contains three instead of an integer:

{
  "status" : "INVALID",
  "valid" : false,
  "findings" : [ {
    "instancePath" : "/rows/0/quantity",
    "schemaPath" : "/columns/quantity/type",
    "keyword" : "type",
    "message" : "Zeile 2, Spalte quantity: 'three' ist keine ganze Zahl"
  } ]
}

/rows/0/quantity means the first data record and its quantity column. The human-readable message includes the physical line number after accounting for the header.

Validate fixed-width and legacy files

Fixed-width files have no delimiter and often no header. A field’s meaning comes from its position; one extra character can shift every field after it. The profile therefore fixes record length and field boundaries.

{
  "format": "fixed-width",
  "version": 1,
  "encoding": "UTF-8",
  "recordLength": 31,
  "sourceLanguage": "COBOL",
  "columns": [
    { "name": "recordType", "start": 1, "length": 1, "type": "string", "required": true, "pattern": "^D$" },
    { "name": "deliveryId", "start": 2, "length": 10, "type": "string", "required": true, "minLength": 1 },
    { "name": "quantity", "start": 12, "length": 5, "type": "integer", "required": true, "minimum": 1 },
    { "name": "email", "start": 17, "length": 15, "type": "email", "required": true }
  ]
}

Positions start at one and are measured after decoding. Field boundaries are trimmed by default so padded values can still be checked. A wrong record length is its own finding; the line is not reinterpreted with shifted fields.

DDEL-1001      3ops@example.org

The invocation is the same except for the adapter and profile:

$jar = "target/zeus-interface-quality-0.1.0-SNAPSHOT.jar"
$profile = "src/test/resources/fixtures/fixedwidth/delivery.cobol-profile.json"
$input = "src/test/resources/fixtures/fixedwidth/valid-delivery.dat"

java -jar $jar validate --input-format fixed-width --schema $profile --input $input --report json
$LASTEXITCODE

COBOL and RPG as provenance, not coupling

A legacy definition may come from a COBOL copybook, RPG/RPGLE, DDS, or a database description. The first validator does not need to parse the language. It needs the approved layout description that came out of that work.

For that reason, a fixed-width profile may contain sourceLanguage: COBOL or sourceLanguage: RPG. The metadata documents provenance; it does not change the parsing algorithm. A future generator could create a draft from a copybook, RPGLE definition, or DDL, but approving that draft remains a domain and engineering decision.

Read findings and exit codes

{
  "status" : "INVALID",
  "valid" : false,
  "findings" : [ {
    "instancePath" : "/records/0/quantity",
    "schemaPath" : "/columns/quantity/type",
    "keyword" : "type",
    "message" : "Zeile 1, Spalte quantity: 'three' ist keine ganze Zahl"
  } ]
}

The fields status, valid, and findings remain stable across adapters. Automation should use status, path, and keyword; the current human-readable messages are not the long-term machine contract.

Why the core stays technology-neutral

The shared core knows result objects and the validator interface. JSON Schema, CSV, and fixed-width profiles live in separate adapters. Apache Commons CSV is an implementation detail of the CSV adapter; it is not part of the core contract.

The automated evidence

The tests cover valid files, type errors, missing required values, wrong CSV headers, wrong column counts, malformed CSV quoting, fixed-width length errors, and invalid profiles. The local evidence does not yet prove SFTP retrieval, delivery windows, or business correctness.

What is deliberately not included

• no FTP or SFTP adapter and no authentication;
• no filename, delivery-window, hash, or quarantine workflow;
• no business rules across records or files;
• no automatic derivation from DDL, COBOL copybooks, DDS, or RPGLE;
• no byte-level fixed-width semantics or special EBCDIC profiling;
• no size or runtime limits beyond the current development scope.