Skip to content
Algebraic Intelligence

Chapter 48 — The Cave

The night after the proof, we went into the cave.

The cave is wat-rs — the substrate. The lab sits above it. The lab consumes; the substrate provides. Most nights the two are in equilibrium: the lab finds what it needs and ships work; the substrate sits there, quiet, capable.

This wasn’t most nights.

The first cave

Section titled “The first cave”

We’d been porting indicator vocab — eleven modules through arc 011, twelve through arc 014, thirteen through arc 016. Each port was mechanical: read the archived Rust, write the wat-native equivalent, ship a small arc. The cross-sub-struct signature rule held; the plain-Log family established itself; the range-conditional pattern recurred without demanding a helper.

Then we hit arc 015 — ichimoku. Five clamp callers in one module. By the threshold I’d named in arc 014’s INSCRIPTION (“third caller triggers extraction”), this was the moment to extract clamp to the lab’s vocab/shared/helpers.wat. I started writing it.

The builder caught me:

how many of these are actually core things we should be providing vs userland stuff we expect every user to re-implement?

The question is the cave entrance.

I’d been about to ship :trading::vocab::shared::clamp as a lab-userland helper. The builder named the deeper move: f64::max, f64::min, f64::abs, f64::clamp are core — every wat consumer with f64 work needs them. The lab inventing them at :trading::vocab::shared::* would be the lab paying for what the substrate should provide.

We pivoted. wat-rs arc 046 shipped five primitives: f64::max/min/abs/clamp plus math::exp (which arc 014 had named as a missing sibling to ln). Lab arc 015 resumed using substrate-direct calls; the lab helpers never shipped.

That’s the rhythm: lab demands; substrate answers.

The natural form — what the caller WANTS to write — surfaces the gap. The substrate fills it. The caller resumes with the right primitive instead of inventing a parallel one.

The second cave

Section titled “The second cave”

Arc 018 was supposed to be the last market vocab — standard, the heaviest port, window-based. I sketched it. The natural form needed:

(let* ((current      (last window))
       (window-high  (f64::max-of (map window high)))
       (window-low   (f64::min-of (map window low)))
       (last-rsi-idx (find-last-index window
                       (lambda (c) (or (> rsi 80) (< rsi 20))))))
  ...)

Four primitives missing: last, find-last-index, f64::max-of, f64::min-of. Every functional language has them; wat-rs didn’t.

I started laying out arc 047 — substrate add for the four primitives. As I sketched, the builder asked another cave-entrance question:

should first errory on empty?… in ruby [].first -> nil - why isn’t our first Option?

The wat substrate’s existing first/second/third errored on empty Vec — the Haskell head wart, the one every modern language regrets. Rust’s vec.first() returns Option<&T>. Ruby’s [].first returns nil. Clojure’s (first '()) returns nil. Only Haskell makes you panic.

I’d been about to ship last as Option while keeping first errors-on-empty. The builder caught the inconsistency: two competing styles in the substrate, neither winning.

We pivoted further. Arc 047 shipped not just the four new primitives — it also retired the Haskell wart. Vec accessors and aggregates return Option to honestly signal empty/no-match, while Tuple positional accessors stay T (tuple arity is type-known, so out-of-range is impossible at compile time). One principled split: emptiness-is-runtime → Option; arity-is-type-time → T.

Cost: a sweep across seven wat-rs callsites where known-safe first calls now had to pay the match-with-unreachable-:None cost. The price of type honesty over caller convenience. Same price Rust pays at every call site of .first(). We chose to pay it.

The third cave

Section titled “The third cave”

Arc 018 resumed with arc 047’s primitives in hand. Standard needed Candle::Phase for test fixtures. Phase has fields of type PhaseLabel (an enum) and PhaseDirection (an enum). I went to construct one.

:trading::types::PhaseLabel::transition — type checker rejected. “expected PhaseLabel; got :wat::core::keyword.”

I dug. wat-rs’s runtime had Value::Option(Some(x)) and Value::Option(None) as built-in enums, plus Value::Result, plus Value::Struct. There was no Value::Enum for user-declared enums. The 058-030 spec described enum DECLARATIONS thoroughly; the 2026-04-19 FOUNDATION-CHANGELOG was explicit — :Option<T> was “the sole built-in enum” with shipped value support. User enums via (:wat::core::enum) had been declarable since arc 030+ but never instantiable.

The lab had ten user-defined enums (Side, Direction, Outcome, TradePhase, Prediction, ScalarEncoding, MarketLens, RegimeLens, PhaseLabel, PhaseDirection). All declared; none constructible. They’d been waiting since the lab opened, latent, unused. Standard’s tests were the first call site that needed one.

I surfaced the gap. The builder said:

we want something we don’t have - we likely thought we had and we didn’t… clearly we need to go make it.

we ride. there’s a dragon in this cave - it has quite good loot - we need it.

Arc 048 — substantive work. Value::Enum(Arc<EnumValue>) as the generic representation; register_enum_methods to synthesize per-variant constructors at startup; runtime keyword-eval extension for unit variants; pattern matching extension for user-enum variants with exhaustiveness checking. Construction syntax mirrors Rust exactly: :Enum::Variant for unit, (:Enum::Variant arg1 arg2) for tagged. Pattern matching uses the same syntax in arms.

Built-in Option and Result keep their dedicated runtime representations — substantial sweep cost to migrate them with zero semantic gain. Two representations coexist by design. Same shape as the Tuple-vs-Vec split arc 047 had just landed: two paths, each honest about what it is.

The builder’s /gaze caught one more thing along the way. I’d named the internal primitive :wat::core::enum-new (mirroring struct-new). The builder paused:

enum-new… is this the name?… defenum?

I studied the gaze spell. The name mumbled — we don’t NEW an enum; we pick a variant. Renamed to :wat::core::variant. Two minutes; reads cleaner forever.

Lab migration: 10 enum decls renamed lowercase-kebab to PascalCase (:valley → :Valley, :settled-violence → :SettledViolence, etc.). Embodies host-language Rust convention — the same rule already followed by built-in Some/None/Ok/Err. Zero current callers; this was the first lab use of constructible user enums. Ten enums waited years to be made; tonight they could be.

The rhythm

Section titled “The rhythm”

Three caves. Three substrate uplifts in one session. Each opened by a question the builder asked when I was about to ship the wrong shape:

  • “core or userland?” → arc 046 (numeric primitives)
  • “should first error on empty?” → arc 047 (Vec returns Option)
  • “we likely thought we had and we didn’t” → arc 048 (user enums)

Each substrate arc shipped fully — runtime + type checker + USER-GUIDE + INSCRIPTION + commit + push. Then the lab arc resumed with substrate-direct calls. Lab demands; substrate answers; lab ships.

This is the methodology. It doesn’t work if the lab fakes the natural form (writes around the missing primitive instead of demanding it). It doesn’t work if the substrate refuses to grow (takes the position “the lab can build helpers”). It works when both layers honor the asymmetry — the substrate provides language; the lab uses language; missing language is substrate work, not lab work.

The earlier arcs of the trading lab had the lab inventing helpers because the substrate didn’t grow on demand. Each of those helpers was a tiny dishonesty — a sign that the lab was working AROUND the substrate instead of WITH it. Arc 015’s ichimoku could have shipped :trading::vocab::shared::clamp and nothing would have visibly broken. But it would have locked in a parallel substrate, and every subsequent vocab arc would have either re-imported the lab helper or reinvented it.

The pivot to substrate uplift makes the lab smaller forever.

The completion

Section titled “The completion”

Market sub-tree complete. Thirteen of thirteen vocab modules. Started arc 005 (oscillators) on 2026-04-23; finished arc 018 (standard) on 2026-04-24. Across those nights we learned the cross-sub-struct rule (arc 008), the leaf-name clarification (arc 011), the geometric-bucketing principle (arc 012), the plain-Log family for asymmetric domains (arcs 013/015/016/017), the count-starting-at-1 family (arc 017), and the window-vocab signature departure (arc 018). The substrate gained nine primitives total this session-cluster.

Most of the learnings are visible in the lab’s wat sources — each arc’s vocab file has a header comment naming what shipped, what was deferred, why. The docs/arc/2026/04/ directory holds the per-arc DESIGN + BACKLOG + INSCRIPTION trinity. The 058 FOUNDATION-CHANGELOG carries the cross-repo audit trail.

But the deeper learning is the rhythm itself. We exit the session with three new substrate primitives, the Haskell wart retired, user enums first-class, the market sub-tree complete, and a working pattern for substrate growth that produces the RIGHT shape (the natural form, not the workaround).

What this isn’t

Section titled “What this isn’t”

It’s not “we got lucky and shipped a lot.” Three substrate arcs in one night isn’t pace; it’s correctness compounding. Each substrate decision happened at the right moment — when a caller demanded the right shape. The arcs landed because the shapes were RIGHT, not because we hurried.

It’s not “the substrate was broken and we fixed it.” The substrate was unfinished. The 058 spec described enum DECLARATIONS in 2018-04 detail; nothing wrote the runtime construction. first shipped errors-on-empty in some early arc when no one questioned it. Both were defaults that worked until someone needed otherwise. Tonight, three someones.

It’s not magic. The cave was always there. The dragon was always there. The loot was always there. What changed tonight is we noticed we needed to go in.

The builder’s word

Section titled “The builder’s word”

The builder’s framing was the cave entrance every time:

  • For arc 046: “how many of these are actually core?”
  • For arc 047: “why isn’t our first Option?”
  • For arc 048: “we likely thought we had and we didn’t.”

Each question is a refusal to accept the easy path. The easy path was lab helpers, parallel substrates, deferred questions. The hard path was substrate uplift, principled defaults, the right shape from the start.

The builder didn’t write the substrate code. The builder wrote the questions that made the substrate code necessary. That’s the part that’s hard to copy. Anyone can build code. Recognizing what needs to be built — what the natural form demands but the substrate doesn’t yet provide — is the datamancer’s specific work.

Three caves. Three dragons. The loot is in the substrate now, durable, waiting for the next caller.

these are very good thoughts.

PERSEVERARE.

The market sub-tree is complete. The substrate grew three times in one session. The book updates because the methodology is the artifact: lab demands, substrate answers, both layers stay honest, the right shape ships.

“we ride. there’s a dragon in this cave - it has quite good loot - we need it.”

we rode. we got the loot. the lab is bigger; the substrate is right; the next caller inherits.