Where derived sources run (spec/02 §8): a value its owner holds and calls from its
own callbacks, as a Macaw.Data.Source.Push is held. One owner, and no second process
between the owner and its sources' rows. On a Macaw page the owner is macaw_compose's
runtime.
new/2 defines the sources (every input resolved, cycles refused by name, ranks
assigned). push/4 shows the engine its sources' latest deliveries: a source that
changed marks the derived sources that read it, and the first mark makes a pulse due
debounce: milliseconds later. run/2 runs a due pulse: the marked sources in rank
order, each at most once, every call bounded (Macaw.Data.Derived.Sandbox.run/3). A
source that fails keeps its last good output, and the pulse goes no further along it.
The engine never reads a clock: now is the owner's monotonic milliseconds.
iex> alias Macaw.Data.{Delivery, Derived, Schema}
iex> alias Macaw.Data.Derived.Engine
iex> count = Derived.new!(id: :count, from: [hosts: nil],
...> schema: %{n: %{type: :number}}, elixir: "fn %{hosts: rows} -> [%{n: Enum.count(rows)}] end")
iex> {:ok, engine} = Engine.new(%{["count"] => count}, sources: [:hosts], debounce: 0)
iex> hosts = Schema.new!(%{host: %{type: :category}})
iex> engine = Engine.push(engine, %{hosts: Delivery.new([%{host: "a"}, %{host: "b"}])}, %{hosts: hosts}, 0)
iex> engine = Engine.run(engine, 0)
iex> Delivery.rows(Engine.outputs(engine)[["count"]])
[%{"n" => 2}]
iex> Engine.status(engine)
%{["count"] => :live}Ported from visualize's Visualize.VirtualSource.Engine, without its variables (a
variable is an input, spec/02 §2.3) and with sources in place of feeds.
Summary
Types
A definition fault: the source's path, where in it, and why (spec/02 §8.1).
A source's fault (spec/02 §8.3): its kind and its detail, never a value.
A scope's path, [] the outermost (spec/02 §3).
Functions
The monotonic millisecond the next pulse is due, or nil (spec/02 §8.2).
Every failing derived source's fault by path, %{kind: kind, detail: detail}.
As new/2 with no options.
The engine over derived sources by path, defined (spec/02 §8.1): {:ok, engine}, or
{:error, errors}: every input that resolves to nothing, every host function the
registry lacks, then every cycle by name. Options (spec/02 §8.4): sources: (the names
of the sources the derived sources may read, []), functions: (the registry, %{},
default closed), packages: (the packages of operations a stored function may reach
besides macaw_data's own, [], default closed, spec/04 §6.1), debounce:
(milliseconds, 250), and max_reductions:, timeout: and max_heap: (each call's
bounds, tighten-only).
Every derived source's last good output by path, a %Macaw.Data.Delivery{} naming its
output schema's version.
The sources' latest deliveries and their schemas, each by name, shown to the engine at
the owner's monotonic millisecond now (spec/02 §8.2): every derived source that reads
a source whose delivery is not the one last seen is marked, and the first mark makes the
next pulse due debounce later.
What a name read from a scope names (spec/02 §8.4): {:derived, path} for a derived
source, resolved from the inside out, else {:source, name} for a source the engine may
read, else :error.
The due pulse run at now (spec/02 §8.2): the marked sources in rank order, each at
most once; a changed answer marks its readers, which run later in the same pulse. The
engine is returned as it is when no pulse is due.
How many times each derived source has run, by path.
Every derived source's output schema, by path.
As source/3 with no options.
The derived source at path as a Macaw.Data.Source (spec/02 §8.5): a
%Macaw.Data.Derived.Output{} over its last good output and its output schema.
viewer_independent: is the owner's to declare (false): the engine sees its inputs'
deliveries, not their descriptions.
The sources some derived source reads, sorted.
Every derived source's state by path (spec/02 §8.3): :loading before its first
answer, :live after, and {:down, {:derived, id, kind}} while it fails.
Types
A definition fault: the source's path, where in it, and why (spec/02 §8.1).
A source's fault (spec/02 §8.3): its kind and its detail, never a value.
@type path() :: [String.t(), ...]
A scope's path, [] the outermost (spec/02 §3).
@type t() :: %Macaw.Data.Derived.Engine{ arities: term(), cache: term(), checked: term(), compiled: term(), debounce: term(), derived: term(), due: term(), faults: term(), functions: term(), inputs: term(), marked: term(), operations: term(), order: term(), outputs: term(), ranks: term(), readers: term(), run_opts: term(), runs: term(), seen: term(), source_schemas: term(), sources: term() }
Functions
The monotonic millisecond the next pulse is due, or nil (spec/02 §8.2).
Every failing derived source's fault by path, %{kind: kind, detail: detail}.
@spec new(%{required(list()) => Macaw.Data.Derived.t()}) :: {:ok, t()} | {:error, [error(), ...]}
As new/2 with no options.
@spec new(%{required(list()) => Macaw.Data.Derived.t()}, keyword()) :: {:ok, t()} | {:error, [error(), ...]}
The engine over derived sources by path, defined (spec/02 §8.1): {:ok, engine}, or
{:error, errors}: every input that resolves to nothing, every host function the
registry lacks, then every cycle by name. Options (spec/02 §8.4): sources: (the names
of the sources the derived sources may read, []), functions: (the registry, %{},
default closed), packages: (the packages of operations a stored function may reach
besides macaw_data's own, [], default closed, spec/04 §6.1), debounce:
(milliseconds, 250), and max_reductions:, timeout: and max_heap: (each call's
bounds, tighten-only).
@spec outputs(t()) :: %{required(path()) => Macaw.Data.Delivery.t()}
Every derived source's last good output by path, a %Macaw.Data.Delivery{} naming its
output schema's version.
The sources' latest deliveries and their schemas, each by name, shown to the engine at
the owner's monotonic millisecond now (spec/02 §8.2): every derived source that reads
a source whose delivery is not the one last seen is marked, and the first mark makes the
next pulse due debounce later.
What a name read from a scope names (spec/02 §8.4): {:derived, path} for a derived
source, resolved from the inside out, else {:source, name} for a source the engine may
read, else :error.
The due pulse run at now (spec/02 §8.2): the marked sources in rank order, each at
most once; a changed answer marks its readers, which run later in the same pulse. The
engine is returned as it is when no pulse is due.
@spec runs(t()) :: %{required(path()) => non_neg_integer()}
How many times each derived source has run, by path.
@spec schemas(t()) :: %{required(path()) => Macaw.Data.Schema.t()}
Every derived source's output schema, by path.
@spec source(t(), list()) :: Macaw.Data.Derived.Output.t()
As source/3 with no options.
@spec source(t(), list(), keyword()) :: Macaw.Data.Derived.Output.t()
The derived source at path as a Macaw.Data.Source (spec/02 §8.5): a
%Macaw.Data.Derived.Output{} over its last good output and its output schema.
viewer_independent: is the owner's to declare (false): the engine sees its inputs'
deliveries, not their descriptions.
The sources some derived source reads, sorted.
Every derived source's state by path (spec/02 §8.3): :loading before its first
answer, :live after, and {:down, {:derived, id, kind}} while it fails.