# macaw_data v0.5.3 - Table of Contents

> Macaw's data layer: the interaction event, the data model, the source contract, derived sources, and the transforms and aggregates they share with designs.

## Pages

- [macaw_data](readme.md)
- [Changelog](changelog.md)

## Modules

- [Macaw.Data.Aggregate](Macaw.Data.Aggregate.md): The aggregate behavior (spec/04 §1.2): a statistic, one column's readings reduced to one
value (visualize#909).
- [Macaw.Data.Bounded](Macaw.Data.Bounded.md): One call in a monitored process of its own, its cost counted rather than timed: the
conformance kit's runner (spec/01 §9), ported from visualize's `Visualize.Bounded`.
- [Macaw.Data.Delivery](Macaw.Data.Delivery.md): One answer of a source: its rows, as a table, and what it actually served (spec/01 §3).
- [Macaw.Data.Derived](Macaw.Data.Derived.md): A derived source: a source computed from other sources by a function, rows in and rows
out (spec/02). Ported from visualize's `Visualize.VirtualSource` (visualize#643, D-194).
- [Macaw.Data.Derived.Context](Macaw.Data.Derived.Context.md): The context a derived source's function of two arguments is called with (spec/02 §2.3):
each input's schema as data and its delivery's served facts. It holds no clock and no
variables: a variable is an input like any other.
- [Macaw.Data.Derived.Engine](Macaw.Data.Derived.Engine.md): Where derived sources run (spec/02 §8): a **value** its owner holds and calls from its
own callbacks, as a `Macaw.Data.Source.Push` is held. One owner, and no second process
between the owner and its sources' rows. On a Macaw page the owner is `macaw_compose`'s
runtime.
- [Macaw.Data.Derived.Functions](Macaw.Data.Derived.Functions.md): A host's derived-source functions (spec/02 §7), written as Elixir in the host's own
module and tested with ExUnit
- [Macaw.Data.Derived.Join](Macaw.Data.Derived.Join.md): Joined fields (spec/02 §1.4): a reader of an input reads each per-identity source's
answer for its own identity. Its rows carry, beside their own fields, every output
field but the identity, named `"<id>.<field>"`, a string, never an atom, `nil` where the
output holds nothing for the row's entity; its schema describes them as the outputs do.
- [Macaw.Data.Derived.Output](Macaw.Data.Derived.Output.md): A derived source's answer read as any source's (spec/02 §8.5): a `Macaw.Data.Source`
over an engine's last good output at one path, made by `Macaw.Data.Derived.Engine.source/3`.
- [Macaw.Data.Derived.Sandbox](Macaw.Data.Derived.Sandbox.md): The sandbox a derived source's function runs in (spec/02 §6, visualize D-194).
- [Macaw.Data.Derived.Sandbox.Allowlist](Macaw.Data.Derived.Sandbox.Allowlist.md): What a stored derived-source function may call (spec/02 §6.2, visualize D-194): the allowlist
visualize#643 decided on 2026-10-15, as **explicit** `{function, arity}` lists — never "a module
minus a few".
- [Macaw.Data.Derived.Sandbox.Cache](Macaw.Data.Derived.Sandbox.Cache.md): Compiled functions by source (spec/02 §6.3, visualize D-194): a **value**, held by whatever owns
the functions (the engine, a builder), not a process or a table, as the schema
cache is (spec/01 §1.12).
- [Macaw.Data.Derived.Sandbox.Compiled](Macaw.Data.Derived.Sandbox.Compiled.md): A stored function checked and compiled once (spec/02 §6.3, visualize D-194): `hash`, the
SHA-256 of its source; `fun`, the closure; `arity`; and `warnings`, the compiler's,
each `{line, message}`, printed nowhere.

- [Macaw.Data.Event](Macaw.Data.Event.md): The one interaction shape every component emits (Macaw spec 02 §8).
- [Macaw.Data.FFT](Macaw.Data.FFT.md): The fast Fourier transform of a real or complex sequence (spec/04 §4.11): a radix-2
Cooley–Tukey decimation in time over lists, `O(N log N)`, pure Elixir. The engine of the
`spectrum` transform, ported from visualize's `Visualize.Data.FFT`.
- [Macaw.Data.Operations](Macaw.Data.Operations.md): `macaw_data`'s own package of operations, the registry, and running them (spec/04).
- [Macaw.Data.Operations.Aggregates](Macaw.Data.Operations.Aggregates.md): The statistics of `macaw_data`'s package (spec/04 §5), each an aggregate
(`Macaw.Data.Aggregate`) ported from visualize's `Visualize.Data`
- [Macaw.Data.Operations.Aggregates.Count](Macaw.Data.Operations.Aggregates.Count.md): `count`: the number of readings (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Delta](Macaw.Data.Operations.Aggregates.Delta.md): `delta`: the last reading minus the first (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Deviation](Macaw.Data.Operations.Aggregates.Deviation.md): `deviation`: the square root of the sample variance (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Distinct](Macaw.Data.Operations.Aggregates.Distinct.md): `distinct`: the number of distinct readings, a category by spelling (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Extent](Macaw.Data.Operations.Aggregates.Extent.md): `extent`: `[min, max]` of the readings (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Max](Macaw.Data.Operations.Aggregates.Max.md): `max`: the greatest reading, numbers or times (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Mean](Macaw.Data.Operations.Aggregates.Mean.md): `mean`: the arithmetic mean, a float (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Median](Macaw.Data.Operations.Aggregates.Median.md): `median`: the middle reading, or the mean of the two middles (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Min](Macaw.Data.Operations.Aggregates.Min.md): `min`: the least reading, numbers or times (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Quantile](Macaw.Data.Operations.Aggregates.Quantile.md): `quantile`: the `p`-quantile by linear interpolation between the sorted readings, at
`i = (n − 1) · p`, the last index clamped (spec/04 §5).

- [Macaw.Data.Operations.Aggregates.Slope](Macaw.Data.Operations.Aggregates.Slope.md): `slope`: the least-squares slope of the points the readings are (spec/04 §5): a number
`v` at place `i` is the point `[i, v]`, a reading `[x, y]` (or `{x, y}`) the point
itself. `nil` for fewer than two points, or when every `x` is equal.

- [Macaw.Data.Operations.Aggregates.Sum](Macaw.Data.Operations.Aggregates.Sum.md): `sum`: the sum of numbers, `0` of none (spec/04 §5).
- [Macaw.Data.Operations.Aggregates.Variance](Macaw.Data.Operations.Aggregates.Variance.md): `variance`: the sample variance, `Σ(v − mean)² / (n − 1)` (spec/04 §5).
- [Macaw.Data.Operations.Option](Macaw.Data.Operations.Option.md): An operation's option declaration and the check of a value against it (spec/04 §1.3).
- [Macaw.Data.Operations.Registry](Macaw.Data.Operations.Registry.md): The transforms and aggregates of the enabled packages, by name (spec/04 §2), built by
`Macaw.Data.Operations.registry/1`.
- [Macaw.Data.Operations.Transforms](Macaw.Data.Operations.Transforms.md): The table operations of `macaw_data`'s package (spec/04 §4), each a transform
(`Macaw.Data.Transform`) ported from visualize's `Chart.Transform.Data`, and what they
share: reading a column by either spelling of its name, a reading as seconds, and
answering rows as the table `outputs/2` names.
- [Macaw.Data.Operations.Transforms.Aggregate](Macaw.Data.Operations.Transforms.Aggregate.md): `aggregate` (spec/04 §4.10): one row per distinct value of `by`, in order of first
appearance (a list of fields groups by every one, a `nil` value a group of its own),
holding `by`'s columns and, under `as`, the named aggregate (`fn`) of the group's
readings of `field`, `nil`s dropped; without `by`, one row of `as` alone. A pair field
`[x, y]` reads each row as the point `[x, y]`, `x` in seconds when it is a time.
- [Macaw.Data.Operations.Transforms.Bin](Macaw.Data.Operations.Transforms.Bin.md): `bin` (spec/04 §4.9): the readings of `field`, `nil`s dropped, in bins over their
extent: `n` equal-width bins for an integer `thresholds`, else at the inner thresholds
given. A reading belongs to the first bin `[x0, x1)` holding it, or to the last when it
is the extent's maximum. One row per bin: `x0`, `x1`, `y0` (`0`), `as` (the bin's count,
or with `fn` the named aggregate of its readings) and `values`.

- [Macaw.Data.Operations.Transforms.Filter](Macaw.Data.Operations.Transforms.Filter.md): `filter` (spec/04 §4.1): the rows, in order, whose reading of `field` passes `test`
against `value`. `present` and `absent` test `nil`; `eq` and `ne` compare with a
category one value whichever spelling it has (`Macaw.Data.Operations.same?/2`); the
orderings compare under term order, and a `nil` reading never passes one.

- [Macaw.Data.Operations.Transforms.Fold](Macaw.Data.Operations.Transforms.Fold.md): `fold` (spec/04 §4.7): wide columns to long rows, one per input row per listed column
the row has, in row then `fields` order; the row's other columns carried, the folded
ones dropped by spelling; the column's name under the first name of `as`, its value
under the second. A listed column the rows lack, and a `nil` reading, yield no row.

- [Macaw.Data.Operations.Transforms.Lttb](Macaw.Data.Operations.Transforms.Lttb.md): `lttb` (spec/04 §4.4): shape-preserving downsampling by Largest-Triangle-Three-Buckets
(`Macaw.Data.Reduce.lttb/3`) to `n` rows, whole rows kept, gaps passing through. `n` is
frame-bound: `"plot"` is the plot's width in whole pixels. A reducer (spec/04 §7).

- [Macaw.Data.Operations.Transforms.M4](Macaw.Data.Operations.Transforms.M4.md): `m4` (spec/04 §4.5): per pixel column the first, last, least and greatest rows
(`Macaw.Data.Reduce.m4/3`), deduplicated in order, so every column's extremes survive.
`width` is frame-bound. A reducer (spec/04 §7).

- [Macaw.Data.Operations.Transforms.Sort](Macaw.Data.Operations.Transforms.Sort.md): `sort` (spec/04 §4.6): the rows stably sorted by the reading of `field` under term
order, ascending or descending.

- [Macaw.Data.Operations.Transforms.Spectrum](Macaw.Data.Operations.Transforms.Spectrum.md): `spectrum` (spec/04 §4.11): a time series to its one-sided amplitude spectrum by FFT
(`Macaw.Data.FFT.spectrum/3`). The interval is `every`, else the median positive gap of
the `time` column, else 1; the samples are the newest power of two rows with a numeric
reading of `field`; each row holds `as`'s names for the frequency and the magnitude, and
a third, when `as` gives one, for the phase.

- [Macaw.Data.Operations.Transforms.Stack](Macaw.Data.Operations.Transforms.Stack.md): `stack` (spec/04 §4.8): d3-shape's stack (`Macaw.Data.Stack`) over the series `fields`,
in the order and with the offset given, a `nil` reading `0`. One row per series in
`fields` order, per datum in data order: the datum's columns plus `key`, `series` (its
stacking position, 0 the bottom), `value`, `y0` and `y1`.

- [Macaw.Data.Operations.Transforms.Sum](Macaw.Data.Operations.Transforms.Sum.md): `sum` (spec/04 §4.10): `aggregate` with `fn: "sum"`, `as` `"sum"` by default. It is
vocabulary a design writes, not a deprecated alias.

- [Macaw.Data.Operations.Transforms.Take](Macaw.Data.Operations.Transforms.Take.md): `take` (spec/04 §4.2): the last `n` rows, or the first with `side: :start`; `n` at or
above the count is every row and zero is none.

- [Macaw.Data.Operations.Transforms.Window](Macaw.Data.Operations.Transforms.Window.md): `window` (spec/04 §4.3): the rows whose reading of `field` lies in
`[now − from, now − to)`, in order. A given `to` is exclusive, an absent `to` closes the
window at `now` inclusively, an absent `from` is open, so adjacent windows partition a
column. `now` is the context's, else the newest reading. With `latest`, each declared
identity's newest row of them (`Macaw.Data.Window.latest_rows/3`).

- [Macaw.Data.Ops](Macaw.Data.Ops.md): What a derived source's stored function calls to reach the registered operations
(spec/04 §6): an aggregate of values, or a transform over rows, by name, with the options
a design's node would carry.
- [Macaw.Data.Parameter](Macaw.Data.Parameter.md): A parameter: the one shape for every typed option, a source's or a component's
(spec/01 §2.2).
- [Macaw.Data.Quantity](Macaw.Data.Quantity.md): A fixed list of quantities, each with its facts: a base unit, a kind, bounds, additivity,
and display families by unit system (spec/03 §8–§13).
- [Macaw.Data.Reduce](Macaw.Data.Reduce.md): Shape-preserving downsampling (spec/01 §2.4): the two reducers finishing uses to bring a
delivery within a budget.
- [Macaw.Data.Schema](Macaw.Data.Schema.md): A source's schema: what each of its fields means (spec/01 §1.2).
- [Macaw.Data.Schema.Cache](Macaw.Data.Schema.Cache.md): Schemas by `{feed, version}`, as a value (spec/01 §1.12, visualize D-136).
- [Macaw.Data.Source](Macaw.Data.Source.md): The source contract (spec/01 §2): anything a component's slot reads. One contract
serves feeds, queries, derived sources, state sources and exports. It is a strict
superset of visualize's `Visualize.Feed`: an existing feed is a source with no
parameters.
- [Macaw.Data.Source.Conformance](Macaw.Data.Source.Conformance.md): The source conformance kit: the contract of spec/01 §2–§4 as checks every source runs
(spec/01 §9).
- [Macaw.Data.Source.Description](Macaw.Data.Source.Description.md): What a source is (spec/01 §2.1): its schema, its parameters, the capabilities it
honors, and whether it answers every viewer the same.
- [Macaw.Data.Source.Finish](Macaw.Data.Source.Finish.md): Finishing (spec/01 §2.4): Macaw does what a source did not.
- [Macaw.Data.Source.Push](Macaw.Data.Source.Push.md): One push subscription, as a value (spec/01 §4).
- [Macaw.Data.Source.Request](Macaw.Data.Source.Request.md): What a reader asks a source for (spec/01 §2.3).
- [Macaw.Data.Stack](Macaw.Data.Stack.md): d3-shape's stack (spec/04 §4.8): a baseline for each datum, so layers of data stack on
one another. The engine of the `stack` transform, ported from visualize's
`Visualize.Shape.Stack`.
- [Macaw.Data.Table](Macaw.Data.Table.md): Columnar typed data: the currency of deliveries, derived sources and components
(spec/01 §1.1).
- [Macaw.Data.Transform](Macaw.Data.Transform.md): The transform behavior (spec/04 §1.1): one step of a data pipeline, a table in and a
table out, declared once so the validator, a drawer's forms and vocabulary, the JSON
forms and the sandbox all read the same declaration (visualize#886).
- [Macaw.Data.Unit](Macaw.Data.Unit.md): Units named by their UCUM codes (spec/03 §1–§7, D-138, D-155).
- [Macaw.Data.Window](Macaw.Data.Window.md): Windows, boxes and regions over a source's rows (spec/01 §2.3, §2.4).