Macaw.Data.Delivery (macaw_data v0.6.0)

Copy Markdown View Source

One answer of a source: its rows, as the source gave them, and what it actually served (spec/01 §3).

A delivery is a handful of terms beside its rows, never per row:

  • rows: the rows as the source gave them (macaw#185): a list of row maps or tuples, a map or keyword list of columns, an Nx.Tensor, a Table.Reader source, or a Macaw.Data.Table. Nothing is packed or rewritten at construction: rows/1 reads them as they are, a row list uncopied, and table/2 packs them into a table for a reader that wants columns, normalizing there and only there. A source that has a table delivers it, and it passes as it is. A gap stays nil in either form: absence is never zero.
  • names: the column names of tuple rows or a tensor, in order, for whoever must read them by name (named_rows/1, table/2, finishing); nil otherwise.
  • types: column types for table/2 to pack with, %{} to infer them.
  • schema: the version of the schema the rows were produced under (Macaw.Data.Schema.version/1), never the schema itself, which Macaw.Data.Schema.Cache holds by version; nil when the source has none.
  • served: what this answer served, which may differ from what was asked:
    • window: a range [from, to], a box over a :point window field %{west: w, south: s, east: e, north: n}, or nil;
    • resolution: the spacing of the rows as served (seconds, or a name such as :minute), or nil;
    • capped: whether the source stopped short of the window it was asked for;
    • reduced_by: who brought the rows within budget, :source or :runtime, or nil when nobody had to;
    • params: the parameters it was answered for, a map;
    • note: a line for a person ("coarsened from raw"), or nil.

  • origin: where it came from, %{source: name, request: digest, as_of: time | nil}, or nil.

The shape of visualize's Visualize.Data.Delivery, whose facts are flat and whose reduced_by is :feed or :visualize, is spec/01 §3's here.

Examples

iex> alias Macaw.Data.Delivery
iex> rows = [%{t: 1, v: 3}, %{t: 2, v: nil}]
iex> d = Delivery.new(rows, served: [window: [0, 10]], schema: "ab12")
iex> Delivery.rows(d) === rows
true
iex> {d.served.window, d.served.capped, d.served.reduced_by}
{[0, 10], false, nil}
iex> d |> Delivery.table!() |> Macaw.Data.Table.column(:v)
[3, nil]

Summary

Types

origin()

Where an answer came from.

served()

What one answer served (spec/01 §3).

t()
window()

A window as served: a range, a box over positions, or none.

Functions

empty?(delivery)

Whether the delivery has no rows.

named_rows(delivery)

The rows as maps, for a reader that reads them by name (finishing, push, a reducer): rows/1, with tuple rows and a tensor's named by names (Macaw.Data.Table.named/2). Tuple rows without names raise ArgumentError: they cannot be read by name, and are never dropped for it.

new(rows, opts \\ [])

A delivery of rows, any source Macaw.Data.Table.rows/1 reads, kept as given, with served: (a keyword list or map of the served facts), schema: (a version), origin:, names: (tuple rows' or a tensor's column names, in order) and types: (for table/2).

rows(delivery)

The rows a delivery carries, as Macaw.Data.Table.rows/1 reads its source: a row list itself, uncopied, its tuple rows unchanged; a table, a column map or a tensor one row at a time. Nothing is rewritten.

table(delivery, opts \\ [])

The rows packed into a Macaw.Data.Table (spec/01 §1.1), for a reader that wants columns: {:ok, table} or {:error, faults}. A table passes as it is. The delivery's types: and names: apply, and opts may give others (types: merged over them). Packing normalizes (an integer beside floats reads back a float, a naive time a UTC DateTime); the delivery's rows are never changed by it.

table!(delivery, opts \\ [])

As table/2, raising ArgumentError naming the faults.

Types

origin()

@type origin() ::
  %{source: term(), request: String.t(), as_of: DateTime.t() | nil} | nil

Where an answer came from.

served()

@type served() :: %{
  window: window(),
  resolution: term(),
  capped: boolean(),
  reduced_by: :source | :runtime | nil,
  params: map(),
  note: String.t() | nil
}

What one answer served (spec/01 §3).

t()

@type t() :: %Macaw.Data.Delivery{
  names: [Macaw.Data.Table.name()] | nil,
  origin: origin(),
  rows: Macaw.Data.Table.source() | Macaw.Data.Table.t(),
  schema: String.t() | nil,
  served: served(),
  types: %{required(Macaw.Data.Table.name()) => Macaw.Data.Table.type()}
}

window()

@type window() ::
  [term()]
  | %{west: number(), south: number(), east: number(), north: number()}
  | nil

A window as served: a range, a box over positions, or none.

Functions

empty?(delivery)

@spec empty?(t()) :: boolean()

Whether the delivery has no rows.

named_rows(delivery)

@spec named_rows(t()) :: [map()]

The rows as maps, for a reader that reads them by name (finishing, push, a reducer): rows/1, with tuple rows and a tensor's named by names (Macaw.Data.Table.named/2). Tuple rows without names raise ArgumentError: they cannot be read by name, and are never dropped for it.

new(rows, opts \\ [])

@spec new(Macaw.Data.Table.source() | Macaw.Data.Table.t(), keyword()) :: t()

A delivery of rows, any source Macaw.Data.Table.rows/1 reads, kept as given, with served: (a keyword list or map of the served facts), schema: (a version), origin:, names: (tuple rows' or a tensor's column names, in order) and types: (for table/2).

Raises ArgumentError naming the fault: a key that is not a delivery's or a served fact, a value a field does not take, a delivery of a delivery, or rows of no shape a source has. The rows themselves are not read: a source's rows are its own until a reader reads them.

rows(delivery)

@spec rows(t()) :: [Macaw.Data.Table.row()]

The rows a delivery carries, as Macaw.Data.Table.rows/1 reads its source: a row list itself, uncopied, its tuple rows unchanged; a table, a column map or a tensor one row at a time. Nothing is rewritten.

table(delivery, opts \\ [])

@spec table(t(), keyword()) :: {:ok, Macaw.Data.Table.t()} | {:error, term()}

The rows packed into a Macaw.Data.Table (spec/01 §1.1), for a reader that wants columns: {:ok, table} or {:error, faults}. A table passes as it is. The delivery's types: and names: apply, and opts may give others (types: merged over them). Packing normalizes (an integer beside floats reads back a float, a naive time a UTC DateTime); the delivery's rows are never changed by it.

table!(delivery, opts \\ [])

@spec table!(t(), keyword()) :: Macaw.Data.Table.t()

As table/2, raising ArgumentError naming the faults.