Package scriptella.driver.json
Streaming JSON input/output provider, selected with driver="json".
Supports standard JSON and JSON Lines.
Configuration and values
| Setting | Default | Meaning |
|---|---|---|
| url | Standard input/output | File or URL relative to the ETL file; use separate read/write connections. |
| format | Filename inference | json or jsonl; explicit configuration takes precedence. |
| encoding | UTF-8 | Input decoding and output encoding. |
| readonly | false | Skip scripts without opening output. |
| JSON value | Reader representation | Writer mapping |
|---|---|---|
| Object | Insertion-ordered map | String-keyed map |
| Array | List | List |
| String | String | String, escaped automatically |
| Boolean | Boolean | Boolean |
| Integer | Long, or BigInteger beyond its range | Byte, Short, Integer, Long, BigInteger |
| Fraction/exponent | BigDecimal | BigDecimal and finite Float/Double |
| Null | null | null |
Runnable JSON-to-database, database-to-JSON, JSON-to-JSON and JSON Lines examples
are included in the examples ZIP under json/, with commands and expected
results in json/README.md. In the source tree they live under
samples/json/. The binary and examples ZIPs include Jackson Core;
Maven applications must add the optional provider dependency explicitly.
Connection properties: format=json or format=jsonl, encoding=UTF-8 (default). The URL uses normal Scriptella
relative URL resolution; omit it to read standard input. Decoding rejects malformed
bytes. Standard JSON input accepts one initial UTF-8 BOM; JSON Lines rejects BOMs. Construction opens no input or output.
Each query opens and closes its input; closing standard input leaves System.in open.
For standard JSON, an empty query selects a root array, or emits a root object as one row.
A nonempty query selects exactly one array using a simple path, such as
response.customers, batches[0].customers or
['response.data']["customers"]. Identifiers contain ASCII letters,
underscores and digits (not a leading digit). Bracket keys use JSON string escapes
with single or double quotes; single quotes additionally allow escaped apostrophes.
Indexes are nonnegative Java ints without leading zeros. Outer whitespace is trimmed;
whitespace between segments, JSONPath operators and expression substitution are unsupported.
Missing segments, wrong containers and non-array targets fail.
Each callback exposes row: insertion-ordered maps, lists, strings,
booleans, null, Long integers (BigInteger beyond the Long range), and BigDecimal
fractions/exponents. Scalar and null array items each emit a row. Other names delegate
to the enclosing parameters. Existing expressions support row.customer.name,
row.items[0].price and row['external.code']. Missing fields
retain existing expression behavior. Parsing preserves numeric precision; expressions
and destination JDBC conversions can change it. No date inference is performed.
Input must contain exactly one document. Empty input, comments, trailing commas, trailing root values and duplicate keys in materialized records fail. Repeated selected keys at path ancestor objects also fail, including keys occurring after rows were emitted. Duplicate unrelated keys are ignored. Syntax validation continues through unrelated subtrees and after the selection. Already executed callbacks cannot be undone. A successful query counts as one statement, including an empty selection.
Memory is proportional to the current record, path depth and parser token buffers, not document size. Unrelated subtrees are skipped and field-name symbol caching is disabled. A very large single record can exhaust available memory. Jackson Core 2.22.4 uses per-factory limits of 1000 nesting levels, 1000 characters per number, 20,000,000 per string and 50,000 per field name. Parser constraints remain enabled; document length and token count have no fixed limit. Parsing checks cancellation between tokens and records; cancellation cannot interrupt a blocked I/O read.
Failures close query content and input, including downstream callback failures. Use separate connections for input and output. The reader does not implement transactions. For a minimal Maven runtime, add Jackson Core explicitly alongside scriptella-drivers and the chosen JDBC driver: provider dependencies are optional. The full Ant distribution carries the intact Jackson jar and its license notices.
Writing
Use unquoted ?{expression} for every dynamic JSON value,
including strings. The result retains its JSON type; strings are quoted
and escaped automatically. For numeric customer_id=123,
"id": ?{customer_id} emits "id": 123.
"$customer_id", "${customer_id}" and
"?{customer_id}" are literal JSON strings and are not evaluated.
To construct a dynamic string, put the whole string expression in an unquoted
?{...} placeholder. Reference variables directly inside expressions,
such as customer_id or row.customer.name.
Dollar substitution remains available in supported configuration contexts,
such as url="${output.file}" and
<include href="${template}.json"/>, and in other drivers' text
substitution contexts. It does not interpolate JSON template strings.
These rules apply to both JSON and JSON Lines.
Each script is one JSON object template. Unquoted ?{expression}
placeholders occupy complete value positions and use existing Scriptella expressions:
{"id": ?{customer_id}, "customer": {"name": ?{customer_name}},
"items": ?{row.items}, "source": "import", "flags": [true, ?{active}]}
Keys are static JSON strings. Placeholder-like text inside strings, including
${name}, remains literal. Comments, trailing commas, duplicate decoded
keys and extra root content are rejected. Templates preserve field order. The latest
compiled template is reused only when freshly opened resource content compares equal;
dynamic includes are reopened on every invocation.
Values support null, strings, booleans, Byte/Short/Integer/Long, BigInteger, BigDecimal, finite Float/Double, string-keyed maps and lists. Numbers retain their existing precision; expressions may change it. Cycles, duplicate map keys, non-string keys, non-finite numbers and all other types are rejected. Dates and times (including JDBC dates) require explicit formatting or conversion to epoch numbers. Arrays, beans, streams and arbitrary iterables are not implicitly converted. Shared acyclic values are allowed. Output value nesting is limited to 500 levels.
Successful scripts count as one statement each. readonly=true skips
scripts without opening output or incrementing the count. The first operation fixes
connection direction, including a skipped script. The destination is opened only after
the first whole record is validated and serialized. Zero invocations leave existing
files untouched and absent files absent, even with eager initialization. Omit the URL
to write standard output, which is flushed without closing System.out.
For standard JSON, records are emitted into one JSON array in invocation order. Memory includes the current object, its serialized record, and the most recent compiled template, rather than the complete export. Standard JSON commit writes the closing bracket, flushes and closes the output; these failures propagate before ETL success. Closing without commit performs cleanup only. Output is nontransactional: failures can leave incomplete files, and I/O failures can leave partial records. Mapping failures never append part of a record. A failed writer cannot continue or commit successfully, even if an onerror handler handles the original error. There is no rollback of bytes or cross-connection atomicity.
On parse or mapping failure, previously emitted records remain, but no bytes of the failing mapped record are appended. Opening output may truncate an existing file even if opening or the first write fails. Write failures can leave partial records. After a handled error the writer rejects further scripts and commit; cleanup does not add a closing bracket. Cancellation and rollback likewise leave an unfinished array. Cancellation is checked before evaluation and emission; blocked output I/O cannot be interrupted by these checks. A downstream database failure triggers rollback and cleanup, without undoing output bytes.
Finalization writes the closing bracket, flushes, then closes. Failure at any of these stages fails execution, even if the bytes already form valid JSON. Cleanup is attempted after finalization write/flush failure, with cleanup errors suppressed on the original cause. A close failure also fails commit. The engine logs failures from its final cleanup hook without replacing the execution failure. Connections commit in engine order: another connection may already have committed when a later one fails. Neither syntactically valid output nor buffered records prove a complete successful export; consumers must also check execution success. Output I/O diagnostics include destination, record number and emission/finalization stage; template errors include line, column and value path without printing record contents.
JSON Lines
Explicit format=json or format=jsonl takes precedence.
When omitted, the resolved URL filename selects JSON Lines for .jsonl
or .ndjson, case-insensitively, ignoring query parameters and fragments.
All other extensions, missing extensions and omitted URLs default to standard JSON.
Reading and writing use identical selection. Contents are never inspected to select
the format.
Queries must be empty. Each physical line contains exactly one complete JSON value,
exposed as one row, including scalar, null and array values. Empty files
yield zero rows. Blank lines, BOMs, multiline values, duplicate keys and extra values
on a line fail immediately. LF and CRLF are accepted; a final newline is optional.
Bare CR is rejected. Errors report physical line and record number, with parser
column details in the cause. A malformed line is validated before its callback;
earlier callbacks remain executed. There is no skipping or recovery.
Scripts use the same object templates and typed values as standard JSON. Each invocation emits one compact object followed by LF, without array brackets or commas between records. Embedded string newlines are escaped. Memory includes the current input line and materialized record, or the current serialized output record. Very large individual lines can exhaust memory. Cancellation is checked while collecting lines and parsing values.
Lazy opening, encoding, direction, readonly behavior, statement counts, template caching and failure rules are shared with standard JSON. Commit flushes and closes without adding framing. A handled error, cancellation or rollback can leave complete lines that represent only a partial export; execution must still succeed before consumers treat output as complete. An I/O failure can leave an incomplete last line. No atomic replacement, append/resume or content-based format detection is provided.
-
ClassDescriptionStreaming JSON provider.Streaming input or lazily opened output; construction performs no I/O.JSON configuration, selection, framing or I/O failure.