Composition run
How one template version's whole dependency tree runs - flatten, then per node namespacing, presets, cache, live run and answer carry-over, then layering.
The problem. A template version is not one program but a dependency tree of them. Each node asks its own questions in its own container, and the user must get one folder's worth of files and one set of answers.
The idea. A composition run turns the tree into a run order, runs the nodes one after another while one composition state carries answers forward, then lays all outputs into one file set. The batch run calls it once per template spec. A stacked template already in the state file is run twice (its base, then its incoming spec); a template that is new to the project runs once.
The picture shows one composition run.
flowchart TD
T["template version + caller answers + deterministic map"] --> F["dependency flattening: run order"]
F --> N{next node}
N -->|"group (no images)"| G["add to order, skip"] --> N
N -->|"has images"| A["build its answers:<br/>namespacing, then presets"]
A --> K{"cache hit?"}
K -->|yes| H["cached files + answers"]
K -->|no| R["run live: warm, Q&A, build<br/>(opens a session)"]
R --> Q{"need input?"}
Q -->|yes| STOP["stop: empty file set,<br/>the question, the sessions"]
Q -->|no| S["store in cache"]
H --> C["carry answers into the state"]
S --> C --> N
N -->|"none left"| L["dependency layering"] --> OUT["one file set, final state,<br/>sessions, commands"]How it works
- Flatten. Dependency flattening fetches every node from the registry and returns the run order: children first, siblings by version id, root last, each with its preset answers.
- Start the state. A copy of the caller's answers and deterministic map. The clash list from any earlier run is cleared.
- Namespacing on or off. On when more than one node has images (answer namespacing).
- For each node, in order:
- Group: added to the order and skipped. It makes no files and opens no session.
- Answers: globals and this node's scoped answers from the state, then presets for ids still missing.
- Cache: if the node cache is on and the node is published, compute the key and look up. A hit gives files and answers with no container.
- Live: otherwise run the node on the coordinator: template warm and session warm in parallel, bootstrap in parallel with the question walk, then the build. The session id is kept for release (session release).
- Need input (headless only): stop here. No later node runs, no layering, an empty file set; the question goes back with every session so far.
- Store a completed live node in the cache, then carry its answers into the state (answer carry-over).
- Layer every output in run order (dependency layering), recording each file clash.
- Collect commands from every node in run order, groups included.
- Return the file set, the final state, the session ids and the commands.
Nodes run strictly one at a time; the only parallel work is inside one node. Each node's container work goes through the coordinator; the CLI holds all state.
Step through one live node as calls between the CLI and the coordinator (the coordinator side is in its own docs).
1/5In parallel. Session warm returns the session id.
Here is the real code of the loop.
- lines 591-604The entry: flatten, start the state, run the composition, collect commands.
cyancoordinator/src/operations/composition/operator.rs:591-604rust let dependencies = self.dependency_resolver.resolve_dependencies(template)?;let shared_state = CompositionState {shared_answers: answers.clone(),shared_deterministic_states: deterministic_states.clone(),execution_order: Vec::new(),need_input: None,};let (vfs, final_state, session_ids) =self.execute_composition(&dependencies, &shared_state, headless)?;let commands = Self::collect_commands(&dependencies);Ok((vfs, final_state, session_ids, commands))} - lines 195-211A copy of the state, an empty output list, the session guard, and a cleared clash list.
cyancoordinator/src/operations/composition/operator.rs:195-211rust let mut shared_state = initial_shared_state.clone();let mut vfs_outputs = Vec::new();// Coordinator sessions are allocated during each dependency's warm/bootstrap// (recorded as `actual_session_id`) and are normally returned to the caller for// release. But several fallible steps run AFTER a session is recorded and BEFORE// this vector reaches the caller — `update_from_template_state` (the need_input// branch and the completed branch), `unpack_archive`, and resolver/default VFS// layering. A `?` from any of them discards the locally accumulated sessions,// leaking the Boron sessions until server-side expiry. The guard releases every// accumulated session on ANY early return while armed; the success paths// `disarm()` it and move the ids out. This is the RAII fix for the recurring// composition-internal session-leak class — one mechanism covers every current// and future `?` between acquisition and return, instead of per-path patching.let mut sessions = CompositionSessionGuard::new(self.client.clone());// Clear previous conflictsself.file_conflicts.clear(); - lines 240-257Groups are only added to the order.
cyancoordinator/src/operations/composition/operator.rs:240-257rust for dep in dependencies {let template = &dep.template;// Check if template has execution artifacts (properties field)if template.principal.properties.is_none() {cprogress!(headless,"⏭️ Skipping template: {}/{} (v{}) - no execution artifacts (group template)",template.template.name,template.template.name, // TODO: Need usernametemplate.principal.version);// Update execution order tracking even for skipped templatesshared_state.execution_order.push(template.principal.id.clone());continue;} - lines 426-447A miss runs the node live: warm, questions, build, with this node's answers and the shared deterministic map.
cyancoordinator/src/operations/composition/operator.rs:426-447rust if !served_from_cache {// CACHE MISS: execute as today. (FR3) The "Executing" line lives// here (not before the cache check) so a cache HIT is never// reported as an execution; a non-cached run still emits it for// every node. Routed through `cprogress!` so under headless it is// suppressed (the single-JSON-on-stdout contract); interactive is// unchanged.cprogress!(headless,"🚀 Executing template: {}/{} (v{})",template.template.name,template.template.name, // TODO: Need usernametemplate.principal.version);let (archive_data, template_state, actual_session_id) =self.template_operator.template_executor.execute_template(template,&session_id,Some(&template_answers),Some(&shared_state.shared_deterministic_states),)?; - lines 466-484Need input: record the question and return at once with an empty file set and the sessions.
cyancoordinator/src/operations/composition/operator.rs:466-484rust let need_input = matches!(template_state,cyanprompt::domain::services::template::states::TemplateState::NeedInput(_, _));if need_input {let namespaced = namespace_template_state(&template_state,namespace,&initial_shared_state.shared_answers,);shared_state.update_from_template_state(&namespaced, template.principal.id.clone())?;// need_input is a legitimate terminal state, not an error: the caller// owns the session release from here, so disarm the guard and hand the// accumulated ids back. (`update_from_template_state` above can still// `?` — in that case the guard stays armed and releases the sessions.)let all_session_ids = sessions.disarm();return Ok((VirtualFileSystem::new(), shared_state, all_session_ids));} - lines 511-522A completed node's answers join the state.
cyancoordinator/src/operations/composition/operator.rs:511-522rust // Update shared state with results (caller-targeted answers namespaced// in a multi-template composition so they do not collide with another// dependency's; global/derived answers stay raw and propagate). The// "was this caller-targeted?" decision uses the immutable initial caller// input, not the evolving accumulator — see `namespace_template_state`.let namespaced = namespace_template_state(&template_state,namespace,&initial_shared_state.shared_answers,);shared_state.update_from_template_state(&namespaced, template.principal.id.clone())?; - lines 534-553After the loop: layer with resolver votes and keep the clashes.
cyancoordinator/src/operations/composition/operator.rs:534-553rust } else if let Some(ref client) = self.client {// Use resolver-aware layering// Vertical layering: collect resolvers from ALL templates in dependency treelet registry = Self::build_resolver_registry(dependencies);let template_infos = Self::build_template_infos(dependencies);let layerer =ResolverAwareLayerer::new(registry, template_infos, client.clone(), headless);let result = layerer.layer_merge(&vfs_outputs)?;// Track conflicts for state writingself.file_conflicts = layerer.get_conflicts();result} else {// Use default layering (LWW)self.vfs_layerer.layer_merge(&vfs_outputs)?};Ok((layered_vfs, shared_state, sessions.disarm())) - lines 699-711Commands from every node in run order.
cyancoordinator/src/operations/composition/operator.rs:699-711rust /// Collect commands from resolved dependencies in post-order./// Iterates over dependencies (already in post-order from resolve_dependencies),/// collects non-empty commands from each template, and flattens into a single vec.pub fn collect_commands(dependencies: &[ResolvedDependency]) -> Vec<String> {let mut commands = Vec::new();for dep in dependencies {let template_commands = &dep.template.commands;if !template_commands.is_empty() {commands.extend(template_commands.iter().cloned());}}commands}
Worked example: step 1 of the shop
Step through cyanprint create atomi/web:5 ./shop (interactive, cache on, empty cache), from the shop example.
- Flatten
Four nodes, all with images: namespacing is on.
order docker:7, lint:2, node-base:3, web:5state (empty) - docker:7
Key with no answers: miss. Runs live as session s1; node_version is asked (22). Stored as K1.
state node_versionsessions s1cache 0/1 - lint:2
Miss. Runs live (s2); asks nothing.
state node_versionsessions s1, s2cache 0/2 - node-base:3
Preset module_type esm fills the gap. Miss; runs live (s3); asks nothing.
state node_version, module_typesessions s1 to s3cache 0/3 - web:5
Miss; runs live (s4). name, port and features are asked; d.get makes instance id A.
state node_version, module_type, name, port, featuressessions s1 to s4cache 0/4 - Layer
package.json: no consensus, web:5's wins. .gitignore: all none, node-base:3's wins.
files 12 pathsclashes package.json, .gitignore - Return
The batch run merges the set into ./shop and saves the five answers; sessions are released at the end.
commands npm install (from node-base:3)
Step through the same run in headless mode with an empty answers file and an empty cache (with step-1 entries on the machine, docker:7's key hits and the whole tree is served from the cache instead).
- docker:7
Runs live (s1). node_version has no answer: need input.
need_input 7a90…/node_versionsessions s1 - Stop
lint:2, node-base:3 and web:5 never run; nothing is layered or cached.
file set emptyreturned question, s1
Why it is built this way
Hard cases
- A template already in the state file runs its whole tree twice per command (base and incoming); a new one (step 1 web:5, step 3 ci:2) runs once. With the cache off, a create into a folder with web and ci runs nine nodes live.
- Need input wastes work. Every node before the question has warmed, been asked, built and possibly been stored in the cache; the next round repeats it from the start.
- A tree of only groups makes an empty file set and a "No execution artifacts produced" line.
- The progress line names the template twice:
Executing template: web/web (v5), since the code has no user name at hand. - Commands come from every node, groups included, but only from the current specs: the base run's commands are dropped because they "would be duplicates or stale" (run.rs, lines 337-338).
- Clashes from this run are kept only until the next layering pass of the same command (template stacking replaces them).
Questions for the designer
- Should a headless need input keep the work of earlier nodes, instead of redoing it next round?
- Should independent nodes (no answer flows between them) run in parallel?