STXT Transform
1. Introduction
This document is STXT-TRANSFORM-SPEC; the other specifications cite it by that name.
This document defines STXT Transform: the operations that take the content of an STXT document to JSON, bring it back from JSON and complete it with external data. There are three:
| Operation | Input | Output |
|---|---|---|
| Export | An STXT document | The same content in JSON |
| Import | A JSON document | The same content as an STXT document |
| Fill | An STXT document with Slots, and JSON data | A complete STXT document |
A Slot is the place of a document that waits for a piece of data. Fill puts in every Slot the data that belongs to it.
Its purpose is that an STXT document can be the source of the programs that already read JSON, and that the data already existing in JSON can move to STXT, without writing a conversion of its own for every namespace.
The three operations are defined on JSON: export produces it, import reads it and fill takes its data from it. The formats that represent the same data (objects, lists and values), such as YAML or NestedText, go through JSON with their own tools: this specification does not deal with them, and an implementation does not need to read or write them. Node formats, such as XML or KDL, have no single correspondence with JSON and will be dealt with in sections of their own, probably for export only: writing them from the document and its definition is simple, and reading them needs a parser for each format. Export is enough for the intended use: writing in STXT and taking the content as input to programs that read other formats.
Export and import rely on the definition of the namespace of the document, whether a schema or a template. The text of a document does not say which value is a number, which node is a list or how its elements are named: its definition does. That is why these two operations need little more information than the document and its definition, and why this specification requires for them, for now, that the document has a namespace and that the namespace has a definition.
Fill needs neither a definition nor a namespace: what every Slot does is written in the document itself.
This specification changes neither the syntax of STXT-SPEC nor the validation of STXT-SCHEMA-SPEC: it is a layer that comes after parsing and validation, and optional like them.
1.1 Date and status of this specification
This specification carries its own date and its own status in the Last modif and
Status fields of its Metadata, independent of the other STXT specifications' and with
the meaning fixed by STXT-SPEC §1.1. It is in Genesis: under
construction, and any part of it may change without notice.
It depends on STXT-SPEC and STXT-SCHEMA-SPEC.
Today it holds the presentation of the three operations and two examples: export and import with JSON, and a first sketch of the fill of Slots. The rules of JSON and those of Slots are still to be written.
1.2 Relation to the canonical tree
STXT-TREE-SPEC already defines a JSON representation of a document: the canonical tree. Its purpose is a different one. The canonical tree describes the result of parsing, with every node as an object that carries its name, its canonical name, its namespace, its form and its value, and it exists to compare implementations. It needs no definition, and every value is text.
The export of this specification produces the JSON that someone designing that data directly in JSON would write: objects with one key per node, arrays, numbers and booleans. It needs the definition, and it is the form meant to be consumed by a program.
1.3 Relation to templates
A document with Slots is not a @stxt.template. A template
(STXT-TEMPLATE-SPEC) describes the structure of every
document of a namespace and is used to validate them. A document with Slots is
one document, with or without a namespace, with values missing: its Slots are filled with
external data, for example a JSON text, and the result is a complete document, which
is validated like any other.
To avoid the confusion, this specification never calls the document with Slots a "template".
2. Terminology
The keywords "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" must be interpreted according to RFC 2119 and RFC 8174: they carry that meaning only when they appear in uppercase, as here.
Terms such as node, namespace, schema, template, type and cardinality retain their meaning in STXT-SPEC, STXT-SCHEMA-SPEC and STXT-TEMPLATE-SPEC.
Additional definitions:
- Definition: the schema or the template of the namespace of a document, as in STXT-DISCOVERY-SPEC.
- Export: the operation that produces, from an STXT document and its definition, a JSON document.
- Import: the operation that produces, from a JSON document and a definition, an STXT document.
- Slot: the place of an STXT document that fill replaces with a piece of data.
- Document with Slots: an STXT document that contains one or more Slots.
- Fill: the operation that produces a complete STXT document from a document with Slots and some data.
3. An example: STXT and JSON
This section is not normative. It shows the direction of export and import with a simple document.
A document:
# The web server of the office
Server (com.example.servers):
Name: web-01
Active: true
Ports:
Port: 80
Port: 443
Owner:
Person: Ana Ruiz
Email: ana@example.com
Notes >>
Restart after every deployment.
Do not switch off during office hours.And the definition of its namespace, here a template:
Template (@stxt.template): com.example.servers
Structure >>
Server (com.example.servers): GROUP
Name: (1)
Active: (1) BOOLEAN
Ports: (1) GROUP
Port: (+) NATURAL
Owner: (?) GROUP
Person: (1)
Email: (1) EMAIL
Notes: (?) TEXT3.1 Export
The export of the document to JSON:
{
"Name": "web-01",
"Active": true,
"Ports": [80, 443],
"Owner": {
"Person": "Ana Ruiz",
"Email": "ana@example.com"
},
"Notes": "Restart after every deployment.\nDo not switch off during office hours."
}
What the definition decides:
| In the document | In the definition | In JSON |
|---|---|---|
Name: web-01 |
No type | A string |
Active: true |
BOOLEAN |
A boolean |
Port: 80 |
NATURAL |
A number |
Ports, with two Port |
GROUP with a single child, which repeats |
An array. The name Port does not appear |
Owner, with two children |
GROUP with different children |
An object, with one key per child |
Notes >> |
TEXT |
A string, with the lines separated by \n |
- Without a definition,
80andtruewould be text, and nothing would tell a list from a node with a single child. - The comment is not exported: it is not part of the tree of the document.
- The root node does not appear: the JSON is the content of
Server.
By contrast, the Ports node in the canonical tree of
STXT-TREE-SPEC, which is [80, 443] in the export:
{
"name": "Ports",
"canonicalName": "ports",
"namespace": "com.example.servers",
"form": "inline",
"value": "",
"children": [
{
"name": "Port",
"canonicalName": "port",
"namespace": "com.example.servers",
"form": "inline",
"value": "80",
"children": []
},
{
"name": "Port",
"canonicalName": "port",
"namespace": "com.example.servers",
"form": "inline",
"value": "443",
"children": []
}
]
}
3.2 Import
Import is the way back. It receives the JSON above and the root node it must produce,
Server (com.example.servers), and returns this document:
Server (com.example.servers):
Name: web-01
Active: true
Ports:
Port: 80
Port: 443
Owner:
Person: Ana Ruiz
Email: ana@example.com
Notes >>
Restart after every deployment.
Do not switch off during office hours.The definition supplies what the JSON does not carry:
- The name
Portof every element of thePortsarray. - The namespace of every node.
- The form of every node. If the definition only allows one, it is that one. If it allows both, as
TEXTdoes, the one that loses less is chosen: a block if the text has line breaks or starts with blanks, which an inline value would trim; inline otherwise.Notesis written as a>>block because its text has more than one line.
Import accepts a string, a number or a boolean as the value of any node with no children,
whatever its type in the definition, and writes it as text: Port gets the same from 80
as from "80". That way the data of formats with no types, such as NestedText, where
every value is a string, is imported too. What does not fit its type, such as "abc" in
Port, is rejected by the validation of the imported document.
The imported document has the same tree as the original. Only the comment has been lost.
4. An example: fill of Slots
This section is not normative. It is a first sketch of Slots and of how they combine: a complete example and, after it, the rules it follows, grouped by topic.
4.1 The example
A document with Slots:
Order (com.example.orders): <number>
Date: <date>
Currency: EUR
Customer: {customer}
Name: <name>
Email: <email>
# The phone is optional
Phone: <?phone>
Line: [lines]<product>
Quantity: <quantity>
Price: <price>
Note: <?note>
Shipment: [shipments]
Carrier: <carrier>
Reference: ${number}/${carrier}
Parcel: [parcels]
Instructions >>
Order ${number}, dated ${date}.
Deliver in the morning.
Label >>
[shipments]
Order ${number}
Carrier: ${carrier}The data, in JSON:
{
"number": "2026-0142",
"date": "2026-10-12",
"customer": {
"name": "Athena Bookshop",
"email": "orders@athena.example"
},
"lines": [
{ "product": "A5 notebook", "quantity": 40, "price": 3.25, "note": "Blue cover" },
{ "product": "HB pencil", "quantity": 200, "price": 0.45 }
],
"shipments": [
{ "carrier": "Royal Mail", "parcels": ["R-01", "R-02"] },
{ "carrier": "DHL", "parcels": ["D-07"] }
]
}
And the document that fill produces:
Order (com.example.orders): 2026-0142
Date: 2026-10-12
Currency: EUR
Customer:
Name: Athena Bookshop
Email: orders@athena.example
Line: A5 notebook
Quantity: 40
Price: 3.25
Note: Blue cover
Line: HB pencil
Quantity: 200
Price: 0.45
Shipment:
Carrier: Royal Mail
Reference: 2026-0142/Royal Mail
Parcel: R-01
Parcel: R-02
Shipment:
Carrier: DHL
Reference: 2026-0142/DHL
Parcel: D-07
Instructions >>
Order 2026-0142, dated 2026-10-12.
Deliver in the morning.
Label >>
Order 2026-0142
Carrier: Royal Mail
Label >>
Order 2026-0142
Carrier: DHLLine and Shipment are repeated once per element, and every Shipment looks up its
carrier and its parcels in its own element; ${number} finds it in the root of the data.
Phone does not come out because the customer has no phone, and the navigation does not
look for it in the root. Note comes out in the first Line and not in the second.
4.2 The Slots
| Slot | What it does | In the example |
|---|---|---|
[list] |
Repeats the node, with its children, once for every element of the array list |
Line, Shipment, Parcel |
{object} |
Enters the object object: the Slots of the node and of its children are looked up in it |
Customer: {customer} |
<var> |
Puts the data item var as the value of the node; it closes the value |
Date: <date> |
${var} |
Puts the data item var inside a text |
Reference: ${number}/${carrier} |
? |
Makes the Slot optional (§4.7) | Phone: <?phone> |
The first three are the navigation of the node and go at the start of its value; ${var}
goes in the text that follows it.
- The name of a Slot has at least one character, and only ASCII letters, digits,
-and_:<number>,[lines],{customer-data},${id_2}. It is looked up as it is among the keys of the data, case-sensitively:<Date>does not finddate.
4.3 Navigation
- The value of a node with Slots starts with its navigation:
[list]and{object}, as many as needed and in any order, and at most one<var>at the end. They apply from left to right, and each one repeats or enters over what the previous one leaves:Address: {customer}[addresses]enters the customer and repeats the node for each of their addresses. There can be blanks between them and before the text:{customer} [addresses]is the same as{customer}[addresses]. What follows is text (§4.4). <var>puts the data item as the value of the node and closes the value: only blanks can follow it.Line: [lines]<product>repeats the node and puts in each one itsproduct;Line: [lines]<product> (units)andLine: <product>[lines]are errors. The node can have children:Order: <number>hasDate,Customerand the rest.- If the element of an array is an object, fill enters it. If it is a value, it is the value of the node:
Parcel: [parcels]producesParcel: R-01andParcel: R-02. That node cannot have anything but the navigation: no text, no children, no more block lines, no further navigation after it. Ifparcelsbrings values,Parcel: [parcels] (box),Parcel: [parcels]with a childWeight: <weight>,Parcel: [parcels]{size}andParcel: [parcels]<weight>are errors.
4.4 Text
- In the text,
${var}puts a data item, and a value can carry several:Reference: ${number}/${carrier}producesReference: 2026-0142/Royal Mail. Everything that has the form${name}is a Slot, also in a block:echo "${HOME}"expects a data itemHOME.Date: ${date}produces the same asDate: <date>when the data item is in the current element; if it is further up, only${date}finds it (§4.6). - To write
${as text, write$${:echo "$${HOME}"producesecho "${HOME}". It is the only escape. A$$not followed by{is copied as it is. - The rest of the text is copied as it is: a value with no Slots (
Currency: EUR), what does not have the form of a Slot (${},${a b}) and also<x>,[x]and{x}, which in the text do not repeat, enter or put a value.Label: See [note]is copied as it is, and in a block with HTML or XML<p>and<br>are text.
4.5 Blocks
- In a
>>block, the first line plays the role of the start of the value. If it starts with[list],{object}or<var>, it is the navigation of the node: it has to be valid navigation as a whole, with nothing else, it applies as in an inline node and it does not pass to the filled document.Label >>starts with[shipments]and produces one block per shipment. If the first line starts with anything else, the whole block is text, including the[list],{object}and<var>of any of its lines.
| First line | Result |
|---|---|
[shipments], {customer} [addresses], <?notes> |
Navigation |
[shipments] Order, <number> Order, {customer} ${name}, [looks]{like-navigation} x |
Error: there is something after the navigation |
A Markdown link, such as [Home](index); <html> or <p>Hello</p> in an HTML block |
Error: they start with something in the form of a Slot |
<!DOCTYPE html>, <html lang="en">, <?xml version="1.0"?>, See <b>[note]</b> |
Text: they do not start with a Slot of the navigation |
<var>puts the data item as the content of the block and closes the value, as in an inline node: the block cannot have more lines. ANotes >>block whose only line is<?notes>puts the notes, with their line breaks, or removes the node if there are no notes; with the line${notes}it produces the same when there is a data item.- In the text of a block, each
${var}is replaced by its data item and the rest is copied as it is: inside the block,Carrier: ${carrier}is text, not a node.
4.6 The data
- The navigation looks up its data item only in the current element: the root of the data, or the object or array element that the previous navigation entered, of the node itself or of the nodes that contain it. If the data had a
notein the root,Note: <?note>would not take it, and the secondLinewould still have noNote. ${var}looks up its data item first in the current element and, if the key is not there, goes up: the object or element that contains it, and so on up to the root of the data. That is how everyShipmentfinds thenumberofReference: ${number}/${carrier}.- A data item whose value is
nullcounts as absent. In the text, it also ends the lookup:${?note}in an element with"note": nullputs nothing, even if there is anotefurther up. - An empty array produces no node and is not an error of fill: if the node was required, the validation of the filled document will say so. A
nullelement is as if it were not there: it produces no node. An array that mixes values and objects is an error, and so is an array inside another array. - A data item that does not fit its Slot is an error: an object where
<var>or${var}expects a value, or a value where[list]expects an array.
4.7 Optional Slots
- Every Slot is required unless it carries a
?right after its opening delimiter:[?list],{?object},<?var>and${?var}. If a line had noquantity, or ifcustomerwere missing, fill would fail. A?outside the delimiters is text:Note: ?<note>is copied as it is, andLink: https://example.com/search?${query}keeps its?. - With no data item,
[?list],{?object}and<?var>remove the node, with its children.${?var}only removes its own piece, and the rest of the text and the node stay:Reference: ${number}${?revision}producesReference: 2026-0142if there is norevision, andPhone: ${?phone}would give an emptyPhone:. Only the navigation decides whether a node appears and how many times.
4.8 The filled document
- The filled node has the form it has in the document with Slots:
Node: <var>comes out inline andNode >>comes out as a block. A data item with line breaks in an inline node is an error. In a block, it produces several lines. - The filled document has the tree its text would give when parsed, with the values normalized as in any document: an inline value with no blanks at the start or the end (STXT-SPEC §5.1), and a block with no blanks at the end of its lines and no empty lines at the end (§10.2 and §10.3). This applies alike to what the data puts and to the text around it:
A: ${t} ${?missing}with nomissingleavesA: x, without the trailing blank. The blanks at the start of a block line and the empty lines in between are content and are kept. - The comment does not pass to the filled document.
- A document with Slots is not validated. The one that is validated is always the filled document.