# `Macaw.Data.Derived`
[🔗](https://github.com/dcoai/macaw_data/blob/v0.5.3/lib/macaw/data/derived.ex#L1)

A derived source: a source computed from other sources by a function, rows in and rows
out (spec/02). Ported from visualize's `Visualize.VirtualSource` (visualize#643, D-194).

It is **data**: `id`, its inputs (`from`), its output's `schema`, and its function as
source text (`elixir`, checked by `Macaw.Data.Derived.Sandbox`) or a host function's
name (`function`, defined with `defsource`, spec/02 §7), so a document holds it and its
forms write it. With `per:` the function runs once per entity of one input and its rows
are joined back by that input's declared identity. Every name is a string.

    iex> {:ok, health} =
    ...>   Macaw.Data.Derived.new(
    ...>     id: :health,
    ...>     from: [hosts: %{last: 20}],
    ...>     per: :identity,
    ...>     schema: %{
    ...>       host: %{type: :category, identity: true},
    ...>       state: %{type: :category, values: [:ok, :warning, :error]}
    ...>     },
    ...>     elixir: "fn rows -> %{state: :ok} end"
    ...>   )
    iex> {health.id, health.from}
    {"health", [{"hosts", %{last: 20}}]}
    iex> Macaw.Data.Derived.conform(health, [%{host: "a", state: :ok}])
    {:ok, [%{"host" => "a", "state" => :ok}]}
    iex> Macaw.Data.Derived.conform(health, [%{host: "a", state: :unknown}])
    {:error, {0, "state", :not_in_values}}
    iex> Macaw.Data.Derived.resolve([["health"], ["ops", "health"]], [:ops], :health)
    {:ok, ["ops", "health"]}

# `error`

```elixir
@type error() :: {list(), term()}
```

A fault: the path from the source to the value, and why (spec/02 §1.1).

# `input`

```elixir
@type input() :: %{
  :name => name(),
  :rows => [map()],
  :schema => Macaw.Data.Schema.t(),
  optional(:delivery) => Macaw.Data.Delivery.t()
}
```

An input as its owner read it (spec/02 §1.3, §2.2): its name, its rows in its window,
its schema, and the delivery it came in.

# `name`

```elixir
@type name() :: String.t() | {:source, String.t()} | [String.t(), ...]
```

What an input names (spec/02 §2.1): a short id, a source, or a path.

# `t`

```elixir
@type t() :: %Macaw.Data.Derived{
  elixir: String.t() | nil,
  from: [{name(), window()}, ...],
  function: String.t() | nil,
  id: String.t(),
  per: nil | :identity | {:identity, name()},
  schema: Macaw.Data.Schema.t()
}
```

# `window`

```elixir
@type window() ::
  nil
  | %{last: pos_integer()}
  | %{
      optional(:window) => list(),
      optional(:box) => map(),
      optional(:region) => map(),
      optional(:latest) => boolean()
    }
```

How much of an input the function needs (spec/02 §2.2).

# `argument`

```elixir
@spec argument(t(), [input()]) :: map()
```

The first argument a function is called with (spec/02 §2.2): the inputs as one map from
key to rows. A stored function reads a key, a row's field and a row's category value as
it reads its own literals, a word's atom, else the string; a host's function reads each
key as its string and each row as its source gave it.

# `conform`

```elixir
@spec conform(t(), term()) ::
  {:ok, [map()]} | {:error, {non_neg_integer() | nil, String.t() | nil, term()}}
```

An answer held to the source's schema (spec/02 §1.2): `{:ok, rows}`, the rows keyed by
their fields' strings, or `{:error, {index, field, reason}}` for the first row that does
not hold, never the value.

# `cycles`

```elixir
@spec cycles(%{required(list()) => [list()]}) :: [[list(), ...]]
```

Every cycle among definitions whose inputs are resolved (spec/02 §3): `reads` maps each
path to the paths it reads. Each strongly connected component that holds a cycle, a
source that reads itself included, is its members' paths, sorted; the cycles are sorted
too. Sources downstream of a cycle are in none.

    iex> Macaw.Data.Derived.cycles(%{["a"] => [["b"]], ["b"] => [["a"]], ["c"] => [["a"]], ["d"] => [["d"]]})
    [[["a"], ["b"]], [["d"]]]

# `dotted`

```elixir
@spec dotted([String.t() | atom()]) :: String.t()
```

A path shown dotted (spec/02 §3).

    iex> Macaw.Data.Derived.dotted(["ops", "health"])
    "ops.health"

# `format_error`

```elixir
@spec format_error(error()) :: String.t()
```

One fault as a line a person reads, the root named `derived source` (spec/02 §1.1).

    iex> Macaw.Data.Derived.format_error({[:per], {:per, :ambiguous}})
    "per: is :identity over several inputs: name the input, {:identity, input}"

# `from_json`

```elixir
@spec from_json(binary(), keyword()) :: {:ok, t()} | {:error, [error(), ...]}
```

The derived source a JSON document holds: read through `macaw_layers`' limits and scan
first, so input over a limit is refused before it is decoded (`limits:` tightens them),
then decoded and read by `new/1`.

# `identity_input`

```elixir
@spec identity_input(t()) :: name() | nil
```

The name of the input a `per` source's entities come from: the only input under
`:identity`, the one named under `{:identity, input}`; `nil` without `per` (spec/02 §1.3).

# `key`

```elixir
@spec key(name() | atom()) :: String.t() | nil
```

The key an input is read under (spec/02 §2.1): a short id, a source's name, or a path's
last name, its string.

    iex> Macaw.Data.Derived.key(["ops", "health"])
    "health"
    iex> Macaw.Data.Derived.key({:source, "vm.memory.total"})
    "vm.memory.total"

# `new`

```elixir
@spec new(map() | keyword()) :: {:ok, t()} | {:error, [error(), ...]}
```

A derived source of a keyword list, a map with atom keys, or its map form (spec/02
§1.5), checked alone (spec/02 §1.1): `{:ok, derived}`, or `{:error, errors}` with every
fault by path.

# `new!`

```elixir
@spec new!(map() | keyword()) :: t()
```

The derived source, or `ArgumentError` naming every fault.

# `per`

```elixir
@spec per(t(), [input()], (term() -> term())) ::
  {:ok, [map()]} | {:error, :nonconforming, {term(), term(), term()}}
```

A `per` source's batch over read inputs (spec/02 §1.3): `call` is the function applied to
one entity's argument (`argument/2`'s map, or the entity's rows alone when the source
has one input), called once per entity of the identity input in its order of first
appearance, a row with no identity dropped. Each answer, a row or a list of rows, is
given its entity's identity and the answers joined: `{:ok, rows}`, or
`{:error, :nonconforming, {index, field, reason}}` for an answer that is not rows or a
row naming another identity. The caller runs the whole batch as one bounded call.

# `resolve`

```elixir
@spec resolve([list()] | map(), list(), term()) :: {:ok, [String.t()]} | :error
```

What a reference names (spec/02 §3), given every definition's path (a list, or a map
keyed by path) and the scope it is read from: a short id from the inside out, a full
path exactly. `{:ok, path}` or `:error`; `{:source, name}` is never resolved.

    iex> paths = [["health"], ["ops", "health"], ["ops", "box", "load"]]
    iex> Macaw.Data.Derived.resolve(paths, ["ops", "box"], "health")
    {:ok, ["ops", "health"]}
    iex> Macaw.Data.Derived.resolve(paths, ["other"], :health)
    {:ok, ["health"]}
    iex> Macaw.Data.Derived.resolve(paths, [], ["ops", "box", "load"])
    {:ok, ["ops", "box", "load"]}
    iex> Macaw.Data.Derived.resolve(paths, ["ops"], {:source, "health"})
    :error

# `to_json`

```elixir
@spec to_json(t()) :: String.t()
```

The source's JSON form: `to_map/1`, encoded.

# `to_map`

```elixir
@spec to_map(t()) :: map()
```

The source's map form (spec/02 §1.5), JSON-ready with every name a string:
`new(to_map(derived)) == {:ok, derived}`.

# `validate`

```elixir
@spec validate(t(), keyword()) :: :ok | {:error, [error(), ...]}
```

`:ok`, or every fault of the source given what it reads (spec/02 §1.1): `inputs:`, a map
from an input's key (or name) to its `%Macaw.Data.Schema{}` where it is known, and
`functions:`, the host functions' registry (`Macaw.Data.Derived.Functions.registry/1`),
or `nil` to leave the name unchecked, and `operations:`, the registry a stored
function's `Macaw.Data.Ops` calls are held to (`Macaw.Data.Operations.registry/1`), or
`nil` to hold their names only to being literals (spec/04 §6.1).

# `window`

```elixir
@spec window([map()], window(), Macaw.Data.Schema.t()) :: [map()]
```

An input's rows in its window (spec/02 §2.2): `nil` every row; `%{last: n}` each
identity's newest `n` rows by the window field, else the newest `n`; a request the rows
its window, box, region and latest select, finished by `Macaw.Data.Source.Finish`.

    iex> schema = Macaw.Data.Schema.new!(%{t: %{type: :number, window: true}, v: %{type: :number}})
    iex> rows = for t <- 1..5, do: %{t: t, v: t * 10}
    iex> Macaw.Data.Derived.window(rows, %{last: 2}, schema)
    [%{t: 4, v: 40}, %{t: 5, v: 50}]
    iex> Macaw.Data.Derived.window(rows, %{window: [2, 3]}, schema)
    [%{t: 2, v: 20}, %{t: 3, v: 30}]

---

*Consult [api-reference.md](api-reference.md) for complete listing*