Batch run
The one run behind create and update - run every template spec twice, stack each list, merge once, write, save state, run commands - and how it fails safely.
The problem. Create and update differ only in which versions and answers they run. Both must rebuild the last run, make the new one, merge them with the folder, and clean up every container they started, even when something fails half way.
The idea. One function, batch_process, takes two lists of template specs (previous and current) and does the rest. Create and update only build the lists (create, update).
The sequence shows one batch run with two stacked templates, and the parties it talks to.
sequenceDiagram
participant CLI
participant Registry
participant Coord as Coordinator
participant Disk as Target folder
loop each previous spec, then each current spec
CLI->>Registry: get template user/name at version
CLI->>Coord: run its dependency tree (sessions opened)
Coord-->>CLI: one file set (or a pending question)
end
CLI->>Coord: stack previous sets (resolver calls if agreed)
CLI->>Coord: stack current sets (resolver calls if agreed)
CLI->>Disk: read local set
CLI->>CLI: three-way merge
CLI->>Disk: delete removed paths, write merged set
CLI->>Disk: append history entries (upgraded specs)
CLI-->>CLI: return sessions, clashes, commands, managed filesPhases
The code labels the phases MAP, LAYER and MERGE+WRITE. The table adds what each consumes and makes.
| Phase | Does | Makes |
|---|---|---|
| MAP (previous) | fetch and run each previous spec with saved answers | one file set per spec |
| MAP (current) | fetch and run each current spec; questions asked here | one file set per spec, final answers, commands, paths per template |
| LAYER | stack each list (template stacking) | merge base, incoming set |
| MERGE+WRITE | read the local set, merge, delete, write | the folder |
| save history | append a history entry for each upgraded or new spec | the state file on disk |
After it returns, create or update saves file_conflicts and managed_files, then runs post-template commands.
Stops and failures
A batch run ends in one of three ways. The guard objects make each one clean up the right things.
| End | When | What is cleaned |
|---|---|---|
| done | everything written | sessions released by the caller; folder kept |
| pending question (headless) | a spec reached a question with no supplied answer | nothing written; sessions handed to the caller; a folder this run created is removed |
| error | any step fails | sessions released by the guard on drop; a folder this run created is removed (headless) |
Here is the real code for the session guard and the early stop.
- lines 135-137One guard collects every coordinator session the run opens.
cyanprint/src/run.rs:135-137rust let mut prev_vfs_list = Vec::new();let mut session_guard =SessionCleanupGuard::new(|sid: &str| release_session(coord_client, sid), Vec::new()); - lines 152-170Each previous spec: run it, register its sessions; a pending question returns at once with the sessions handed over.
cyanprint/src/run.rs:152-170rust let (vfs, final_state, session_ids, _commands) = operator.execute_template(&template_res,&spec.answers,&spec.deterministic_states,headless,)?;session_guard.extend(session_ids);// Headless: a (re-created) prev template stopped on an unanswered question.// Surface it immediately; no files are written. Hand the sessions to the caller// (which cleans them at the headless boundary) by disarming via `take`.if let Some(question) = final_state.need_input {return Ok((session_guard.take(),Vec::new(),Vec::new(),HashMap::new(),Some(question),));} - lines 218-229Each current spec: keep its output, its own path list (before stacking), and its final answers.
cyanprint/src/run.rs:218-229rust curr_vfs_list.push(vfs);// Collect this template's normalized output paths from its own VFS (the active// set), before layering merges them into one.managed_by_template.insert(spec.key(),normalize_managed_paths(curr_vfs_list.last().unwrap()),);// Store the final answers for this template (includes Q&A answers)final_answers_map.insert(spec.key(), final_state.shared_answers);curr_template_res_list.push(template_res);// Collect commands from the full dependency tree (not just root templates)curr_resolved_commands.extend(commands); - lines 241-267Stack each list; one set is used as it is, none gives an empty set.
cyanprint/src/run.rs:241-267rust let prev_vfs = if prev_vfs_list.is_empty() {cyancoordinator::fs::VirtualFileSystem::new()} else if prev_vfs_list.len() == 1 {prev_vfs_list.into_iter().next().unwrap()} else {// Use resolver-aware horizontal layeringoperator.layer_merge_with_resolvers(&prev_vfs_list,&prev_template_res_list,coord_client,headless,)?};let curr_vfs = if curr_vfs_list.is_empty() {cyancoordinator::fs::VirtualFileSystem::new()} else if curr_vfs_list.len() == 1 {curr_vfs_list.into_iter().next().unwrap()} else {// Use resolver-aware horizontal layeringoperator.layer_merge_with_resolvers(&curr_vfs_list,&curr_template_res_list,coord_client,headless,)?}; - lines 290-327Append history for upgraded specs only; answers come from the run, not the spec.
cyanprint/src/run.rs:290-327rust // Save metadata for upgraded templates onlyif !upgraded_specs.is_empty() {crate::hprogress!(headless,"๐พ Saving template metadata for {} upgraded templates",upgraded_specs.len());for spec in upgraded_specs {let template_res = registry.get_template(spec.username.clone(),spec.template_name.clone(),Some(spec.version),)?;// Use final answers from execution (includes Q&A answers) if available,// otherwise fall back to spec.answerslet final_answers = final_answers_map.get(&spec.key()).cloned().unwrap_or_else(|| spec.answers.clone());let template_state = TemplateState::Complete(Cyan {processors: Vec::new(),plugins: Vec::new(),},final_answers,);operator.get_template_history().save_template_metadata(target_dir,&template_res,&template_state,&spec.username,)?;}} - lines 329-348All fallible work done: disarm the guard and hand sessions, clashes, commands and paths back.
cyanprint/src/run.rs:329-348rust // All fallible work is past โ disarm the guard and hand the sessions to the caller,// which cleans them at the headless boundary (a `?` from any operation above instead// drops the still-armed guard and releases them).let all_session_ids = session_guard.take();// Collect file conflicts from operator for state persistencelet file_conflicts = operator.get_file_conflicts().to_vec();// Use resolved commands from execute_template which includes the full dependency tree// (prev is just the 3-way-merge baseline; its commands would be duplicates or stale)let all_commands = curr_resolved_commands;crate::hprogress!(headless, "โ Batch process complete");Ok((all_session_ids,file_conflicts,all_commands,managed_by_template,None,)) - lines 434-494The guard: release every registered session on drop, unless take() handed them on.
cyanprint/src/run.rs:434-494rust /// RAII guard that releases coordinator sessions on `Drop` while armed.////// Coordinator sessions are acquired incrementally as templates execute, but several/// fallible operations (resolver-aware layering, file load, merge, deleted-file cleanup,/// `write_to_disk`, metadata + state save, post-template commands) run BETWEEN acquisition/// and the point the ids are handed to the headless cleanup boundary (`finish_headless`)./// A `?` in that window drops the local id vectors and returns `Err` before the boundary/// ever sees the ids โ leaking the sessions until the coordinator's own timeout. Patching/// each return site individually has historically missed paths (the same recurring class as/// the directory leak), so cleanup is bound to this guard's `Drop`: register each id as it/// is acquired and it is released best-effort on ANY early return. [`take`](Self::take)/// disarms and hands ownership of the ids back once they reach a caller that WILL clean them/// (a `done`/`need_input` boundary return), preventing a double release.////// Releasing a session on error is correct in BOTH headless and interactive modes (it is a/// coordinator HTTP call, not user-facing output), so โ unlike [`CreatedDirGuard`] โ this/// guard is NOT mode-gated; the interactive path is unaffected in its output while also no/// longer leaking sessions on these error paths.////// The release action is a closure (production passes [`release_session`] bound to the/// coordinator client) so the guard's DECISION โ which ids it releases, and that `take`/// disarms it โ is unit-testable with a recorder closure and no live coordinator.pub(crate) struct SessionCleanupGuard<F: FnMut(&str)> {release: F,session_ids: Vec<String>,armed: bool,}impl<F: FnMut(&str)> SessionCleanupGuard<F> {pub(crate) fn new(release: F, session_ids: Vec<String>) -> Self {Self {release,session_ids,armed: true,}}/// Register newly-acquired session ids with the guard so they are released if it drops/// while still armed.fn extend(&mut self, ids: impl IntoIterator<Item = String>) {self.session_ids.extend(ids);}/// Disarm and return the accumulated session ids. After this the caller owns cleanup/// (e.g. the headless boundary `finish_headless`), so the guard must not also release/// them.pub(crate) fn take(&mut self) -> Vec<String> {self.armed = false;std::mem::take(&mut self.session_ids)}}impl<F: FnMut(&str)> Drop for SessionCleanupGuard<F> {fn drop(&mut self) {if self.armed {for sid in &self.session_ids {(self.release)(sid);}}}}
Worked example: ./shop, step 3
Step through the batch run for cyanprint create atomi/ci:2 ./shop from the shop example.
- MAP previous
web:5 runs its four-node tree with saved answers. Cache miss (new key), live run, new UUID B.
sets base web:5sessions 4 - MAP current
web:5 again (cache hit, B), then ci:2 asks ci_provider.
sets base web:5 | web:5, ci:2sessions 5commands web tree + ci tree - LAYER
Previous has one set: used as is. Current has two: stacked; json-merger merges package.json; ci wins .editorconfig.
merge base web:5incoming web:5 + ci:2 - MERGE+WRITE
README.md keeps the user's edit; app.config.json keeps A because base and incoming are both B; ci files added.
deleted nonewritten merged set - Save
History entry for ci (version 2). Then clashes (.editorconfig, package.json) and managed files.
state file web [v5], ci [v2]
Why it works this way
Hard cases
- Commands come only from current specs, from every template version in each tree. Previous specs are only the merge base, so their commands would be stale or doubled.
- Paths per template are taken before stacking, so a template that lost a clash still lists the path.
- A pending question in a previous spec is possible: a rerun or a lost history can leave a previous spec without an answer it needs.
- A release that fails is ignored; the coordinator reclaims the session later. Cleanup never hides the run's own error.
- The cache line goes to stdout. With the cache on, the run prints
Cache: N/M nodes served from cachewith a plain print, even in headless mode, next to the JSON envelope.