jq · lesson 30 of 32 · Event streams

fromstream & truncate_stream

fromstream(events) builds values back from events: it gives one value each time an event closes a top-level value. 1 | truncate_stream(events) cuts the first key or index off every path, so each child of the top level becomes a value of its own.

Example 1Events written by hand (the data isn't used)

Document

{
  "customers": [
    { "code": "ADA", "name": "Ada", "city": "London" },
    { "code": "LIN", "name": "Linus", "city": "Helsinki" },
    { "code": "GRA", "name": "Grace", "city": "New York" }
  ],
  "orders": [
    { "no": 101, "customer": "ADA", "item": "keyboard", "qty": 2 },
    { "no": 102, "customer": "GRA", "item": "monitor", "qty": 1 },
    { "no": 103, "customer": "ADA", "item": "mouse", "qty": 3 },
    { "no": 104, "customer": "LIN", "item": "cable", "qty": 10 }
  ]
}

Query

fromstream([["a"], 1], [["b"], 2], [["b"]])

[["a"], 1] A leaf event: put 1 under the key a.
[["b"], 2] Another leaf: put 2 under b.
[["b"]] A closing event for the top-level object, so fromstream gives the value it has built.

Result

{"a":1,"b":2}

Try it in the playground

Example 2Drop a field from every record

Document

{
  "customers": [
    { "code": "ADA", "name": "Ada", "city": "London" },
    { "code": "LIN", "name": "Linus", "city": "Helsinki" },
    { "code": "GRA", "name": "Grace", "city": "New York" }
  ],
  "orders": [
    { "no": 101, "customer": "ADA", "item": "keyboard", "qty": 2 },
    { "no": 102, "customer": "GRA", "item": "monitor", "qty": 1 },
    { "no": 103, "customer": "ADA", "item": "mouse", "qty": 3 },
    { "no": 104, "customer": "LIN", "item": "cable", "qty": 10 }
  ]
}

Query

.customers | fromstream(tostream | select(length == 1 or all(.[0][]; . != "city")))

.customers | fromstream( Rebuild the customers from the events that pass the filter.
tostream Every event of the array.
select(length == 1 or Keep the closing events, so every value still ends properly, and…
all(.[0][]; . != "city") …every other event whose path doesn't mention city, however deep it is.

Result

[
  {"code":"ADA","name":"Ada"},
  {"code":"LIN","name":"Linus"},
  {"code":"GRA","name":"Grace"}
]

Try it in the playground

Example 3One result per order

Document

{
  "customers": [
    { "code": "ADA", "name": "Ada", "city": "London" },
    { "code": "LIN", "name": "Linus", "city": "Helsinki" },
    { "code": "GRA", "name": "Grace", "city": "New York" }
  ],
  "orders": [
    { "no": 101, "customer": "ADA", "item": "keyboard", "qty": 2 },
    { "no": 102, "customer": "GRA", "item": "monitor", "qty": 1 },
    { "no": 103, "customer": "ADA", "item": "mouse", "qty": 3 },
    { "no": 104, "customer": "LIN", "item": "cable", "qty": 10 }
  ]
}

Query

.orders as $o | fromstream(1 | truncate_stream($o | tostream))

.orders as $o Save the array: the events given to truncate_stream are computed from null, so they can't read .orders themselves.
fromstream( Builds values back…
1 | truncate_stream( …from events whose paths lost their first step, the order's index…
$o | tostream …the events of the whole array. Each order is now a top-level value of its own.

Result

{"no":101,"customer":"ADA","item":"keyboard","qty":2}
{"no":102,"customer":"GRA","item":"monitor","qty":1}
{"no":103,"customer":"ADA","item":"mouse","qty":3}
{"no":104,"customer":"LIN","item":"cable","qty":10}

Try it in the playground

Good to know

fromstream(tostream) gives the input back, whatever it is.
Leave a leaf event out, as the second example does, and its key is simply absent from the rebuilt value.
truncate_stream drops the events whose path is not longer than the cut, so scalars directly in the array vanish: only the objects and arrays inside it come out as values.

fromstream & truncate_stream

Examples

Example 1Events written by hand (the data isn't used)

Document

Query

Result

Example 2Drop a field from every record

Document

Query

Result

Example 3One result per order

Document

Query

Result

Good to know