Template stacking
How several templates installed into one folder are each replayed and laid on top of each other on the user's machine, oldest first.
The problem. A project often needs more than one template: a web app template, then a CI template, then a docs template. They are made by different people, know nothing of each other, and may write the same file.
The idea. Each cyanprint create into a folder that already has a state file adds one more stacked template.
On every run, every stacked template is replayed on its own, and their outputs are laid on top of each other, oldest first, using the same per-file rules as dependency layering.
This is client-side state applied to many templates: the stack exists only in the folder's state file. Nothing in the registry links two stacked templates.
The picture shows one run with two stacked templates.
flowchart TB
SF[".cyan_state.yaml<br/>atomi/web (1 Mar), atomi/ci (9 Mar)"] --> PS["previous specs<br/>web:5, ci:2 (saved)"]
SF --> CS["current specs<br/>web:6, ci:2 (update)"]
PS --> W5["replay web:5<br/>(its own tree, layered)"] & C2a["replay ci:2"]
CS --> W6["run web:6<br/>(its own tree, layered)"] & C2b["run ci:2"]
W5 --> SB["stack: web under ci"] --> B[merge base]
C2a --> SB
W6 --> SI["stack: web under ci"] --> I[incoming set]
C2b --> SI
B --> M((three-way merge))
I --> M
L[the folder] --> MHow it works
- Read the stack. Every template record with
active: truebecomes a template spec: its last history entry's version, answers, deterministic state and time. - Make two lists. Previous specs (what was run last time) and current specs (what runs now). Which versions and answers go in each depends on the command; see the table below.
- Order them. Each list is sorted by the time of the template's last history entry, oldest first, on
cyanprint updateand whencreateadds a new template. On a rerun or an upgrade done withcreate, the lists are not sorted. - Run each spec. Each spec runs its own whole dependency tree and comes back as one file set. Each spec opens its own coordinator sessions.
- Stack each list. If a list has more than one file set, they are layered per path:
- one writer: copied;
- two or more writers: each stacked template votes the conflict resolver it declares for that path, or none;
- all vote the same resolver with the same config: the resolver is called with every version;
- otherwise: the template laid last wins.
- Record. Every clash goes into
file_conflicts(only the last layering pass is kept), and each template's own output paths go into itsfileslist and themanaged_filesunion.
The lists per command are these.
| Command | Previous specs | Current specs |
|---|---|---|
create of a new template | every stacked template, saved | the same, plus the new one with no answers |
create of a stacked template at a new version (upgrade) | every stacked template, saved | the same, that one at the new version, saved answers |
create of a stacked template at its version (rerun) | every stacked template, saved | the same with every template's answers cleared |
update | every stacked template, saved | every one at its newest (or picked) version, saved answers |
Here is the real code for the order and the vote.
- lines 74-93One spec per active record, from its last history entry; the entry's time becomes installed_at.
cyanprint/src/update/spec.rs:74-93rust /// Read specs from .cyan_state.yaml (pure function)pub fn get(&self, state: &CyanState) -> Vec<TemplateSpec> {state.templates.iter().filter(|(_, s)| s.active).filter_map(|(key, s)| {let (username, template_name) = parse_template_key(key)?;let entry = s.history.last()?;Some(TemplateSpec::new(username,template_name,entry.version,entry.answers.clone(),entry.deterministic_states.clone(),entry.time,))}).collect()} - lines 164-167The only ordering rule: oldest installed_at first.
cyanprint/src/update/spec.rs:164-167rust /// Sort specs by installation time for consistent LWW orderingpub fn sort_specs(specs: &mut [TemplateSpec]) {specs.sort_by(|a, b| a.installed_at.cmp(&b.installed_at));} - sulfone.iridium/cyanprint/src/run.rs:618-620create sorts the specs it read from the state file.
cyanprint/src/run.rs:618-620rust // Build specs using composable primitiveslet mut prev_specs = manager.get(&state);sort_specs(&mut prev_specs); - sulfone.iridium/cyanprint/src/run.rs:644-681Upgrade-by-create rebuilds both lists from state.templates.iter(): a HashMap, never sorted.
cyanprint/src/run.rs:644-681rust TemplateUpdateType::UpgradeTemplate {previous_version,previous_answers,previous_states,} => {// Upgrade: prev uses old version, curr uses new versionlet target_key = format!("{}/{}", username, template.template.name);// Build prev_specs with the OLD version for the target templatelet prev: Vec<_> = state.templates.iter().filter(|(_, s)| s.active).filter_map(|(key, s)| {let (u, t) = parse_template_key(key)?;let entry = s.history.last()?;if key == &target_key {Some(TemplateSpec::new(u.to_string(),t.to_string(),previous_version,previous_answers.clone(),previous_states.clone(),entry.time,))} else {Some(TemplateSpec::new(u.to_string(),t.to_string(),entry.version,entry.answers.clone(),entry.deterministic_states.clone(),entry.time,))}}).collect(); - sulfone.iridium/cyanprint/src/run.rs:762-763Rerun: reset clears answers and deterministic state of every spec, not only the named one.
cyanprint/src/run.rs:762-763rust // Use manager.reset() to clear answers for curr_specslet curr = manager.reset(prev.clone()); - sulfone.iridium/cyanprint/src/update/orchestrator.rs:191-193update sorts both lists.
cyanprint/src/update/orchestrator.rs:191-193rust // Sort both lists by installation time for consistent LWW orderingsort_specs(&mut prev_specs);sort_specs(&mut curr_specs); - sulfone.iridium/cyancoordinator/src/operations/composition/operator.rs:632-651Stacking takes resolver votes from the stacked templates only, never from their dependencies.
cyancoordinator/src/operations/composition/operator.rs:632-651rust // Convert root templates to ResolvedDependency for helper functions// (preset_answers are not applicable for horizontal layering)let root_dependencies: Vec<ResolvedDependency> = root_templates.iter().map(|t| ResolvedDependency {template: t.clone(),preset_answers: HashMap::new(),}).collect();// Build resolver registry from ONLY root templates (horizontal layering scope)let registry = Self::build_resolver_registry(&root_dependencies);let template_infos = Self::build_template_infos(&root_dependencies);// Use resolver-aware layererlet layerer = ResolverAwareLayerer::new(registry, template_infos, client.clone(), headless);let result = layerer.layer_merge(vfs_list)?;// Track conflicts for state writingself.file_conflicts = layerer.get_conflicts();
Worked example: ./shop, step 3
Step through stacking ci:2 onto web:5 (step 3 of the shop example). The state shown is the incoming list.
- Read the stack
One record, atomi/web. ci is new, so it joins the current list with no answers and the time now.
previous web:5 (1 Mar)current web:5 (1 Mar), ci:2 (9 Mar) - Run each spec
web:5 runs its tree (docker:7, lint:2, node-base:3, web:5) and comes back as one set; ci:2 asks ci_provider and comes back as another.
sets web:5 set, ci:2 set - Path .editorconfig
Both write it. web:5 declares json-merger only for package.json; ci:2 the same. Both vote none.
votes none, nonerule all noneresult ci:2's file (laid last) - Path package.json
Both write it and both declare json-merger:1 with config arrays=concat for it.
votes json-merger, json-mergerrule agreedresult resolver merges both - Path .github/workflows/ci.yml
Only ci:2 writes it.
result copied - Record
file_conflicts gets .editorconfig and package.json. The clashes inside web:5's own tree from step 1 drop out of the record.
file_conflicts .editorconfig, package.json
Why it is built this way
Hard cases
- Every stacked template runs twice per command. Adding ci to a folder with five templates runs eleven template trees, each with its own sessions.
- Stack order is the last history time, not the install time. An upgrade adds a new history entry with the time of that run, so an upgraded template moves to the top of the stack on the next run.
cyanprint updatekeeps the old time for the run itself. - The stack is unsorted on rerun and upgrade-by-create. Two identical runs can lay web and ci in a different order, so a clash with no agreed resolver can flip.
- A rerun clears every template's answers, not only the named one's. With the node cache on, the empty-answer keys from the first runs hit and nothing is asked; with it off, every question of every stacked template is asked.
- Headless answers go only to the template being changed. A flat answer file is never used to answer another stacked template's same-named question.
- One writer that votes none breaks agreement. If ci:2 did not declare json-merger for
package.json, ci:2's wholepackage.jsonwould win and web's scripts would be gone. updatedrops the new history entries (see update), so the stack order and answers it should have saved are lost.
Questions for the designer
- Should the stack be ordered by first install time (the field is named
installed_at) or by last upgrade time (what the code reads)? - Should a rerun clear only the named template's answers?
- Should
create's upgrade and rerun paths sort the stack likeupdatedoes?