Doc system test

Node cache

The on-by-default local cache of one node's output and answers - its key, who may use it, the file format, and the cases where it serves the wrong thing.

Meta block front matter of this doc
namespaces
  • durian.cloud/cyanprint/composition
  • durian.cloud/cyanprint/prompting
  • durian.cloud/cyanprint/project-lifecycle
  • durian.cloud/cyanprint/authoring
type
algorithm
code
  • sulfone.iridium/cyancoordinator/src/cache/mod.rs
  • sulfone.iridium/cyancoordinator/src/cache/key.rs
  • sulfone.iridium/cyancoordinator/src/cache/store.rs
  • sulfone.iridium/cyancoordinator/src/operations/composition/operator.rs
  • sulfone.iridium/cyanprint/src/main.rs
issues
  • https://github.com/AtomiCloud/sulfone.iridium/issues?q=TBD+cache-key-before-questions
  • https://github.com/AtomiCloud/sulfone.iridium/issues?q=TBD+cache-summary-on-headless-stdout
  • https://github.com/AtomiCloud/sulfone.iridium/issues?q=TBD+cache-debug-lines-on-headless-stdout
  • https://github.com/AtomiCloud/sulfone.iridium/issues?q=TBD+cache-hit-skips-resolver-warm

The problem. Every create and update runs each stacked template already in the state file twice (its base, then its incoming spec; a new template runs once), and each run starts containers for every node. Most nodes give the same output as last time.

The idea. Before a node runs, hash everything that should decide its output into a cache key. If an entry with that key is on disk, use its files and answers and skip the node. The node cache is on by default, lives on the user's machine, and is shared by every folder that points at the same directory. A directory the cache creates is mode 0700, so another account can use it only if it already existed with wider modes (store.rs:62-69).

The picture shows one node's path through the cache.

flowchart TD
  I["node's answer map + deterministic map<br/>(before any question)"] --> E{"cache on and node published?"}
  E -->|no| L["run live"]
  E -->|yes| K["key = SHA-256 of canonical JSON"] --> G{"entry decodes?"}
  G -->|no| L
  G -->|yes| U{"archive unpacks?"}
  U -->|no| X["evict entry"] --> L
  U -->|yes| H["use cached files, replay cached answers<br/>(no questions, no containers)"]
  L --> N{"need input?"}
  N -->|yes| S["stop; nothing stored"]
  N -->|no| W["store files + answers under the SAME key"]
  H --> Y["layering always runs after"]
  W --> Y
One node and the cache

How it works

  1. Turn on or off. --no-output-cache or CYANPRINT_NO_CACHE set to 1, true, yes or on (any case, trimmed) turns it off. Otherwise it is on for create, update and try of a group.
  2. Pick the folder. --cache-dir, else CYANPRINT_CACHE, else the platform cache folder plus cyanprint (~/.cache/cyanprint on Linux, ~/Library/Caches/cyanprint on macOS), else .cache/cyanprint.
  3. Who may be cached. A node with images, a version number above 0, and a version id not starting with 10ca1 or local-. Groups, try templates and dev templates never are; cyanprint test never uses the cache.
  4. Make the key right after the node's answer map is built, before any question is asked. The key is SHA-256, as 64 lowercase hex characters, over this JSON, in this order:
    • format_version (1), template_id (version id), template_version (version number);
    • images: blob and template Docker reference and tag;
    • answers: the node's whole answer map (inherited, scoped-stripped, presets), sorted by id;
    • deterministic_states: the shared map, sorted;
    • plugins and processors: each (id, version, docker_reference, docker_tag), sorted.
  5. Look up. A file named by the key. Any read, length, checksum or JSON fault is a miss.
  6. Hit. Unpack the archive (on failure: evict and run live), then replay the stored answers into the composition state through the same namespacing path as a live run. No session, no container, no question.
  7. Miss. Run live. If the node completes, check its archive unpacks, then store the archive and its completed answers under the key from step 4.
  8. Summary. After the run, one line on stdout: ♻️ Cache: 4/9 nodes served from cache. --debug adds a HIT or MISS line per node, also on stdout.

The entry file is: 32 bytes SHA-256 of the payload, 8 bytes payload length, then the payload: 8 bytes archive length, the tar.gz archive, and the answers as JSON. Writes go to a temp file in the same folder (mode 0600), are synced, then renamed. The folder gets mode 0700 when created and a CACHEDIR.TAG file.

Here is the real code.

sulfone.iridiumcyancoordinator/src/operations/composition/operator.rs
  1. lines 341-351The key is computed from template_answers: the map BEFORE the node's questions.
    cyancoordinator/src/operations/composition/operator.rs:341-351rust
    // Compute the content-addressed key only when the node can be cached,
    // so a disabled / non-cacheable run pays nothing for keying. (FR5, L5)
    let cache_key = if self.cache.enabled() && crate::cache::is_cacheable(template) {
    Some(crate::cache::compute_key(
    template,
    &template_answers,
    &shared_state.shared_deterministic_states,
    ))
    } else {
    None
    };
  2. lines 383-405Hit: replay the cached answers through namespacing into the shared state; count the hit.
    cyancoordinator/src/operations/composition/operator.rs:383-405rust
    let replay_state = TemplateState::Complete(
    Cyan {
    processors: Vec::new(),
    plugins: Vec::new(),
    },
    entry.state,
    );
    let namespaced = namespace_template_state(
    &replay_state,
    namespace,
    &initial_shared_state.shared_answers,
    );
    shared_state.update_from_template_state(
    &namespaced,
    template.principal.id.clone(),
    )?;
    served_from_cache = true;
    // The entry was genuinely served: count it now, AFTER
    // the archive unpacked and state replayed, so a
    // poisoned entry that fell back to execution is not
    // reported as a served hit. (FR15, H2)
    self.cache.record_hit();
    }
  3. lines 406-416An entry that decodes but will not unpack is evicted; the node runs live.
    cyancoordinator/src/operations/composition/operator.rs:406-416rust
    Err(e) => {
    tracing::debug!(
    "cache hit for {} unpacked invalid, evicting and re-executing: {e}",
    template.principal.id
    );
    self.cache.evict(key);
    // Do NOT record a hit: `lookup` already counted this
    // node toward the *attempt* total, but it was NOT
    // served from cache. Fall through to live execution.
    }
    }
  4. lines 486-509Miss: unpack first, then store the archive and the COMPLETED answers under the pre-question key.
    cyancoordinator/src/operations/composition/operator.rs:486-509rust
    // Validate the archive can actually unpack BEFORE caching it, so a
    // bad (Ok((invalid_bytes, Complete(..)))) result is never persisted.
    // (FR13, C1) The to-store copy is taken only when this node will be
    // cached, so a disabled / non-cacheable run never clones the full
    // archive bytes. (L16) `unpack_archive` consumes the bytes by value,
    // so the clone must precede the move.
    let to_store = cache_key.as_ref().map(|_| archive_data.clone());
    let vfs = self.template_operator.vfs.unpack_archive(archive_data)?;
    vfs_outputs.push(vfs);
    // Store ONLY a non-interactive Complete result, and only after the
    // archive has unpacked cleanly. The raw (un-namespaced) answers are
    // stored so a later replay can re-namespace per consuming dependency.
    // QnA panics upstream, Err short-circuits via `?` above, and a NeedInput
    // already returned, so failures/partials are never cached. (FR13)
    if let (Some(key), TemplateState::Complete(_, ref answers)) =
    (cache_key.as_ref(), &template_state)
    {
    let entry = CacheEntry {
    archive: to_store.expect("to_store is Some iff cache_key is Some"),
    state: answers.clone(),
    };
    self.cache.store(template, key, &entry);
    }
  5. sulfone.iridium/cyancoordinator/src/cache/key.rs:61-71The key material, in serialization order.
    cyancoordinator/src/cache/key.rs:61-71rust
    #[derive(Serialize)]
    struct CacheKeyMaterial<'a> {
    format_version: u32,
    template_id: &'a str,
    template_version: i64,
    images: Option<TemplateImages<'a>>,
    answers: BTreeMap<&'a str, &'a Answer>,
    deterministic_states: BTreeMap<&'a str, &'a str>,
    plugins: Vec<PinnedArtifact<'a>>,
    processors: Vec<PinnedArtifact<'a>>,
    }
  6. sulfone.iridium/cyancoordinator/src/cache/mod.rs:101-119Eligibility: images, version above 0, not a synthetic id.
    cyancoordinator/src/cache/mod.rs:101-119rust
    pub fn is_cacheable(template: &TemplateVersionRes) -> bool {
    if template.principal.properties.is_none() {
    return false; // dev mode / group template — no execution artifacts
    }
    if template.principal.version <= 0 {
    return false; // unpublished / synthetic
    }
    let id = &template.principal.id;
    if id.starts_with("10ca1") || id.starts_with("local-") {
    // Synthetic local template ids produced by `build_synthetic_template`
    // (try_cmd.rs). `10ca1` is the synthetic-id prefix; `local-` the dev
    // prefix. A real published template whose registry id happened to start
    // with `10ca1` would be excluded here too — that is a fail-safe (it would
    // re-execute, never serve stale), and the only `10ca1` producer is
    // synthetic ids, so the false-negative risk is vanishing. (FR14, L19)
    return false;
    }
    true
    }
  7. sulfone.iridium/cyancoordinator/src/cache/store.rs:201-223Write: key check, folder and marker, temp file 0600, fsync, rename, parent fsync. Errors are swallowed.
    cyancoordinator/src/cache/store.rs:201-223rust
    fn put_inner(&self, key: &str, entry: &CacheEntry) -> std::io::Result<()> {
    // Validate the key BEFORE any filesystem work so an invalid key never
    // creates the directory or a temp file. (security)
    let dest = self.entry_path(key)?;
    self.ensure_dir()?;
    let encoded = Self::encode(entry);
    // Temp file in the SAME dir so the rename is atomic (same filesystem). (FR8)
    let mut tmp = tempfile::NamedTempFile::new_in(&self.dir)?;
    set_file_perms_0600(tmp.path())?;
    tmp.write_all(&encoded)?;
    tmp.flush()?;
    tmp.as_file().sync_all()?;
    tmp.persist(&dest)
    .map_err(|e| std::io::Error::other(format!("persist failed: {e}")))?;
    // fsync the parent dir so the rename is durable.
    if let Ok(dir) = fs::File::open(&self.dir) {
    let _ = dir.sync_all();
    }
    Ok(())
    }

Worked example: the shop, with the cache on

Step through what the cache holds and serves across the shop example. Keys are named K1, K2, …; real ones are 64 hex characters.

The node cache through ./shop
  1. Step 1: create web:5

    Four misses. Each key holds the answers before that node's questions; each entry holds the answers after.

    K1 docker:7key {} → stores node_version
    K2 lint:2key node_version
    K3 node-base:3key node_version, module_type
    K4 web:5key node_version, module_type → stores all 5, file A
    summary0/4
  2. Step 3: base web:5

    Replayed with all five saved answers: new keys, so four misses. web:5 runs live and makes instance id B.

    K5-K8key has all 5 answers; web:5 file B
  3. Step 3: incoming web:5 and ci:2

    Same keys as the base: four hits, B again. ci:2 misses with key {} (K9). Base equals incoming, so the folder's A is kept - by luck.

    summary4/9
    app.config.jsonA kept
  4. Step 3, cache off

    Base makes B and incoming makes C live. Both differ from A on one line: conflict markers.

    app.config.jsonconflict
  5. Step 5: update again

    Step 4 stored web:6 under a key without runtime. The lost history replays the same key: a hit, so runtime bun comes back unasked.

    web:6hit, runtime not asked
  6. Step 6: rerun ci:2

    Every answer is cleared, so web:5's nodes are looked up with key {} again: K1 to K4 hit. ci:2 hits K9. Nothing is asked.

    questionsnone
    web:5 fileA from step 1
  7. Step 7: create ./blog

    Another folder, same keys K1 to K4: blog gets name shop, port 3000 and instance A, unasked.

    summary4/4
    ./blog nameshop

The tree below is the cache folder after step 1, on Linux.

~/.cache after step 1 (Linux)
  • cyanprint/
    • CACHEDIR.TAGaddedownership markershow file
      Signature: 8a477f597d28d172789f06886806bc55
      # This file marks this directory as a cyanprint execution-output cache.
      # For information about cache directory tags see https://bford.info/cachedir/
      
    • K1addeddocker:7, key answers {}
    • K2addedlint:2
    • K3addednode-base:3
    • K4addedweb:5: files with instance A, answers incl. name shop

Why it works this way

Hard cases

  • The cache hides lost determinism. Deterministic state is not saved, so a live replay makes new values; a hit returns the old ones. Step 3 is clean only because base and incoming hit the same entry. Anyone with the cache off (CI, a new machine) gets conflict markers there.
  • No lock. Two runs that miss the same key both run the node; the last rename wins. Readers never see a half-written file.
  • No eviction. No size limit, age limit or LRU. cyanprint cache clear removes only digest-named files, and only if the folder's CACHEDIR.TAG matches CyanPrint's text exactly. A corrupt entry stays until a store replaces it.
  • Stored before success. A node is stored before layering; if a conflict resolver call later fails, the entry stays.
  • Mutable tags. The key holds image reference and tag, not digest. A re-pushed tag is not noticed.
  • A cache hit warms no conflict resolver. Conflict resolver containers start only in the coordinator's template warm (sulfone.boron/docker_executor/template_executor.go:387-420), and the CLI runs template warm only for a miss (sulfone.iridium/cyancoordinator/src/template/executor.rs:93-99, reached from operator.rs:426-447). In a run where every node hits (shop step 6), the agreed stacking call still goes through the proxy to cyan-resolver-<id>; if no earlier run left that container up, the proxy answers 502 (server.go:758-768) and the run fails. Inferred from the code; found by reading; not run.
  • Hits bring unpruned answers into headless runs. An entry made interactively stores every seeded and typed answer (engine.rs:84-86; operator.rs:501-507). A headless hit replays them all into the state (operator.rs:383-398) and so into the state file: a second path for the pruning bug.
  • The key includes a map headless runs ignore. The key hashes the shared deterministic map (operator.rs:343-348), but start_headless takes answers only (executor.rs:208-214). Harmless today only because saved maps are always empty.
  • Counts are attempts. The summary counts cache lookups of eligible nodes, not all nodes; a hit is counted only after it unpacked and replayed.
  • A bad cached answer fails the run. If the replayed answers clash in type with the shared state, the run fails; the entry is not evicted.

Questions for the designer

  1. Should the key include the node's answers after its questions, or should a hit still walk the questions? (The successor sulfone.lite keys a processor cache on inputs only: artifact integrity, config and input files; lite docs/requirements/simplification-spec.md, lines 171-177.)
  2. Should headless runs (and reruns) bypass entries made interactively?
  3. Should the cache have a size or age limit? (sulfone.lite plans age and size eviction, line 177.)
  4. Should the key use image digests instead of tags?
  5. Should the cache store each node's deterministic values, so a cache-off replay matches a cache-on one? (sulfone.lite persists deterministic state in the state file: simplification-spec.md, lines 78-79.)