Doc system test

Answer namespacing

When a tree has more than one node with images, question ids can be scoped by version id - how keys are classified, stripped, and kept global or scoped.

Meta block front matter of this doc
namespaces
  • durian.cloud/cyanprint/composition
  • durian.cloud/cyanprint/prompting
  • durian.cloud/cyanprint/registry
  • durian.cloud/cyanprint/project-lifecycle
type
algorithm
code
  • sulfone.iridium/cyancoordinator/src/operations/composition/operator.rs
issues
  • https://github.com/AtomiCloud/sulfone.iridium/issues?q=TBD+scoped-answers-lost-on-dependency-upgrade

The problem. Two nodes in one tree can ask a question with the same id: docker:7 and web:5 could both ask name. One flat answer map would give both the same answer, and a headless caller could not say which node a need_input question came from.

The idea. When a tree has more than one node with images, a question id can be written as <version id>/<question id>. A key that starts with a node's version id is scoped to that node; any other key is global and goes to every node.

The picture shows how one node's answer map is built from the shared map.

flowchart TD
  M["shared answer map"] --> C{"key starts with<br/>a known version id + /?"}
  C -->|"no: global"| G["copy as is"]
  C -->|"yes, this node's id"| S["strip the prefix,<br/>override the global"]
  C -->|"yes, another node's id"| X["leave out"]
  G --> A["this node's answers"]
  S --> A
  P["this node's presets"] -->|"only missing ids"| A
Building one node's answers when namespacing is on

How it works

Answer namespacing is decided once per composition run.

  1. On or off. Count the nodes with images (groups do not count). More than one: on. One: off, and every id stays raw, exactly as when that template runs alone.
  2. Known set. The namespaces are the version ids of the nodes with images. A version id containing / fails the run before that node runs.
  3. Classify each key. A key is scoped to node N if it starts with N/. A key that matches no known id is global, even if it contains / (a question id like org/project/name is global).
  4. Build the node's map. First every global key, as is. Then every key scoped to this node, with the prefix stripped, replacing a global of the same raw id. Keys scoped to other nodes are left out. Then preset answers fill only missing ids.
  5. Ask. The template script and the node cache see raw ids only.
  6. Need input. In headless mode, an unanswered question always comes out scoped: 7a90c2d4-1e3f-4b5a-8c6d-9e0f1a2b3c4d/node_version.
  7. Complete. Each answer the node hands back is re-keyed N/id only if the caller's original input had N/id. Every other answer (asked live, from a global, from a preset) stays raw, so it reaches every later node.

Here is the real code.

sulfone.iridiumcyancoordinator/src/operations/composition/operator.rs
  1. lines 220-238On when more than one node has images; the known set is their version ids.
    cyancoordinator/src/operations/composition/operator.rs:220-238rust
    let is_executable = |d: &ResolvedDependency| d.template.principal.properties.is_some();
    // A composition of more than one EXECUTABLE dependency can contain two templates
    // asking a question with the same raw id; namespace their ids by template id to
    // avoid collisions in the shared answer map. A single executable dependency has no
    // such risk, so its ids stay raw (existing create/update behavior unchanged).
    let multi_template = dependencies.iter().filter(|d| is_executable(d)).count() > 1;
    // Every executable dependency's template id is a potential `{ns}/…` namespace prefix
    // a caller-supplied answer can be scoped under. Classifying a shared-answer key
    // against this KNOWN set (rather than by the brittle "contains a `/`" shape) is
    // what lets a raw question id that legitimately contains `/` (e.g. an e2e fixture
    // id like `cyane2e/template1/name`) be routed as the GLOBAL answer it is, instead
    // of being silently dropped or misrouted. Every namespace id is guaranteed `/`-free
    // by the guard below, so a `{ns}/` prefix matches at most one namespace.
    let namespaces: Vec<String> = dependencies
    .iter()
    .filter(|d| is_executable(d))
    .map(|d| d.template.principal.id.clone())
    .collect();
  2. lines 276-282A version id with / cannot be a namespace: fail fast.
    cyancoordinator/src/operations/composition/operator.rs:276-282rust
    if multi_template && template_id.contains('/') {
    return Err(Box::new(std::io::Error::other(format!(
    "cannot namespace a multi-template composition: template id '{template_id}' \
    contains '/' which is the namespace separator"
    ))) as Box<dyn Error + Send>);
    }
    let namespace = multi_template.then_some(template_id.as_str());
  3. lines 309-332Globals first, then this node's scoped keys (stripped) override them; other nodes' keys are skipped.
    cyancoordinator/src/operations/composition/operator.rs:309-332rust
    match namespace {
    Some(ns) => {
    // Global answers first (belong to no namespace), so a dependency-scoped
    // answer below can override a global of the same raw id.
    for (key, answer) in &shared_state.shared_answers {
    if classify_namespace(key, &namespaces).is_none() {
    template_answers.insert(key.clone(), answer.clone());
    }
    }
    // This dependency's scoped answers override the globals (raw id restored).
    for (key, answer) in &shared_state.shared_answers {
    if classify_namespace(key, &namespaces) == Some(ns) {
    let raw_key = strip_namespace(ns, key)
    .expect("key classified under this namespace");
    template_answers.insert(raw_key, answer.clone());
    }
    }
    }
    None => {
    for (key, answer) in &shared_state.shared_answers {
    template_answers.insert(key.clone(), answer.clone());
    }
    }
    }
  4. lines 733-754strip_namespace and classify_namespace: a key is scoped only by a known id prefix.
    cyancoordinator/src/operations/composition/operator.rs:733-754rust
    fn strip_namespace(namespace: &str, namespaced_key: &str) -> Option<String> {
    let prefix = format!("{namespace}/");
    namespaced_key
    .strip_prefix(&prefix)
    .map(|rest| rest.to_string())
    }
    /// If `key` is scoped under one of the known dependency namespaces — i.e. it begins with
    /// `{ns}/` for some `ns` in `namespaces` — return that namespace; otherwise return `None`
    /// (the key is GLOBAL: it belongs to no dependency).
    ///
    /// This is the unambiguous replacement for the old "does the key contain a `/`?" routing
    /// test. A raw question id may itself legitimately contain `/` (e.g. an e2e fixture id
    /// `cyane2e/template1/name`); such a key matches no `{ns}/` prefix here, so it is correctly
    /// classified as global and routed to every dependency instead of being silently dropped.
    /// The match is unambiguous because every namespace id is guaranteed `/`-free by the guard
    /// in `execute_composition` (a `{ns}/` prefix can match at most one `/`-free `ns`).
    fn classify_namespace<'a>(key: &str, namespaces: &'a [String]) -> Option<&'a str> {
    namespaces
    .iter()
    .find_map(|ns| strip_namespace(ns, key).is_some().then_some(ns.as_str()))
    }
  5. lines 793-817NeedInput: the question id is always prefixed. Complete: an answer is prefixed only if the caller's input had that scoped key.
    cyancoordinator/src/operations/composition/operator.rs:793-817rust
    match state {
    TemplateState::NeedInput(question, det) => {
    TemplateState::NeedInput(rename_question(question, ns), det.clone())
    }
    TemplateState::Complete(cyan, answers) => {
    let namespaced: HashMap<String, Answer> = answers
    .iter()
    .map(|(k, v)| {
    let scoped = format!("{ns}/{k}");
    if supplied_answers.contains_key(&scoped) {
    // Caller targeted this dependency's answer explicitly — keep it
    // scoped so it cannot collide with a sibling's same-named answer.
    (scoped, v.clone())
    } else {
    // Global or derived answer — stays raw so it propagates to
    // sibling dependencies (flat-shared-state / cache replay).
    (k.clone(), v.clone())
    }
    })
    .collect();
    TemplateState::Complete(cyan.clone(), namespaced)
    }
    // QnA is never terminal; Err carries no ids. Clone unchanged.
    other => other.clone(),
    }

Worked example: a headless shop

Step through cyanprint create atomi/web:5 ./shop --headless --answers a.json with an empty cache, run again after each need_input. web:5's tree has four nodes with images, so namespacing is on.

Namespaced answers for web:5
  1. Run 1: no answers

    docker:7 runs first and reaches node_version. The run stops with exit 2; the folder it made is removed.

    a.json{}
    need_input id7a90c2d4-…/node_version
  2. Run 2: scoped answer

    docker:7 gets raw node_version 22. lint:2 and node-base:3 do not see it. node-base:3 gets module_type from its preset. web:5 reaches name.

    a.json7a90…/node_version: 22
    need_input ide5d07b3a-…/name
  3. Run 3: globals

    The caller adds name, port and features without a prefix. They are global: every node gets them, and only web:5 asks them.

    a.json7a90…/node_version, name, port, features
    outcomedone
  4. Saved answers

    node_version was targeted, so it stays scoped. The rest stay raw.

    history answers7a90…/node_version, module_type, name, port, features
  5. Later: update to web:6

    web:6's tree has docker:8 (8b11), not 7a90. The saved key matches no known id, so it is a global with the odd raw id 7a90…/node_version. docker:8 has no node_version.

    known ids8b11, 3c1f, b2e4, 1f6a
    resultnode_version asked again

The same shop made interactively saves node_version raw (step 1 of the shop example), so its update to web:6 reuses it.

Why it works this way

Hard cases

  • Namespacing does not isolate. An answer typed live into docker:7 stays raw and reaches every later node. If docker:7 also asked name (for the image), web:5 would reuse that answer and never ask its own name.
  • The format flips with the tree. ci:2 alone asks ci_provider; if a new ci version gained a dependency, the same question would come out as <id>/ci_provider.
  • Namespaces are version ids, so they expire. Scoped answers saved in history stop matching once that dependency moves to another version id. They are then sent as globals with a useless id, and the question is asked again (found while reading; not run).
  • Scoped wins over global for the same raw id, in any map order, because the two passes are separate.
  • Interactive and headless save different keys, but not always. A fresh interactive create has no scoped input, so what the user types is saved raw. An interactive update of a project whose saved history already holds scoped keys (made headless) seeds them; the completion re-scopes every answer whose <id>/<question> key was in that input (operator.rs:797-805), so those keys are saved scoped again.

Questions for the designer

  1. Should namespaces use the template name (atomi/docker) instead of the version id, so saved answers survive upgrades?
  2. Should an answer typed live into one node be kept from later nodes when namespacing is on?
  3. Should the need_input id format stay the same whether or not the tree has one node with images?
  4. Should a scoped key that matches no node be dropped (or warned about) instead of sent as a global?