Conflict resolvers
One resolver artifact, called from two places with two different rules - the coordinator between processors, the CLI between template versions.
The problem. Many writers produce package.json: two processors inside one template version, four nodes of web:5's tree, and two stacked templates in ./shop.
Keeping only the last one throws away scripts and dependencies; the authors need a way to join the versions.
The idea. A conflict resolver is an artifact (a container on port 5553) that takes every version of one file and returns one. It is called from two places with two different rules: the coordinator's merger, between the processors of one template version, and the CLI, between template versions.
The picture shows the three clashes in the shop and who settles each.
flowchart TD
subgraph COORD["coordinator: merger container (one node's build)"]
P1["shop-render:1, invocation 0"] --> PM{"processor output merge"}
P2["shop-render:1, invocation 1"] --> PM
end
subgraph CLI["CLI"]
DL{"dependency layering<br/>(nodes of one tree)"}
TS{"template stacking<br/>(stacked templates)"}
end
PM -->|"one matching declaration"| R(("json-merger:1<br/>shared container :5553"))
DL -->|"all writers agree"| R
TS -->|"all writers agree"| R
PM -->|"web:5's file set"| DL
DL -->|"web:5's tree set"| TSThe two rules
| Coordinator (processor output merge) | CLI (dependency layering, stacking) | |
|---|---|---|
| A clash is | 2+ processors of one node write a path | 2+ nodes, or 2+ stacked templates, write a path |
| Declarations used | the node's own resolvers | each writer's own (stacking: stacked templates only) |
| Picked by | every declaration with a matching glob | each writer votes its first match; all must agree |
| No match | last processor wins | last layer wins |
| Two or more matches | the build fails | last layer wins, recorded as ambiguous |
| Some match, some not | (not a case: one list) | last layer wins, recorded as no consensus |
| Glob library | Go doublestar: * stops at / | Rust glob: * also matches / and dots |
| Bad glob in a declaration | the build fails (invalid pattern error) | never matches: that writer votes none |
| Conflict resolver call fails | the build fails | the run fails |
The processor output merge uses one author's declarations for one author's processors, so overlap is a mistake. Dependency layering and template stacking join several authors' files, so every author must consent (resolver agreement).
The request, the reply, the route
Both callers send the same body and expect the same reply:
{ "config": { "arrays": "concat" },"files": [{ "path": "package.json", "content": "{\"name\":\"shop\",\"type\":\"module\"}", "origin": { "template": "<id>", "layer": 0 } },{ "path": "package.json", "content": "{\"scripts\":{\"dev\":\"node src/server.ts\"}}", "origin": { "template": "<id>", "layer": 1 } } ] }
Reply, HTTP 200: { "path": "package.json", "content": "{β¦mergedβ¦}" }.
The fields mean different things at each place:
| Field | Coordinator | CLI |
|---|---|---|
origin.template | a processor version id | a template version id |
origin.layer | the processor's index in the Cyan | the run order index, or stack index |
config | the matching declaration's config | the first writer's declaration's config |
reply path | must equal the path, else the build fails | ignored |
The routes differ too:
| Coordinator | CLI | |
|---|---|---|
| Route | merger container to http://cyan-resolver-<id>:5553/api/resolve | CLI to POST /proxy/resolver/{id}/api/resolve on the coordinator, which forwards |
| Timeout | none set (Go default client) | proxy 30 s (timeout: 502); CLI client 600 s |
| Success | exactly 200 and a reply whose path equals the path (an empty 200 body decodes to path "" and fails the build) | any 2xx; else the body is read as problem details |
Conflict resolver containers are shared: no session, warmed with the template, named by version id only.
Here is the real code at both places.
- lines 100-116Coordinator: every declaration with a matching doublestar glob is a match.
docker_executor/merger.go:100-116go // findMatchingResolver finds resolvers whose file patterns match the given pathfunc findMatchingResolver(path string, resolvers []ResolverRes) ([]ResolverRes, error) {var matches []ResolverResfor _, resolver := range resolvers {for _, pattern := range resolver.Files {matched, err := doublestar.Match(pattern, path)if err != nil {return nil, fmt.Errorf("invalid glob pattern '%s' in resolver '%s': %w", pattern, resolver.ID, err)}if matched {matches = append(matches, resolver)break // One pattern match per resolver is enough}}}return matches, nil} - lines 419-438No match: copy the last processor's file, with its file mode.
docker_executor/merger.go:419-438go for _, conflictPath := range conflicts {versions := fileMap[conflictPath]// Match resolvers using doublestar.Match()matchingResolvers, err := findMatchingResolver(conflictPath, m.Template.Resolvers)if err != nil {return err}if len(matchingResolvers) == 0 {// LWW: use last versionfmt.Printf("Conflict detected for '%s': no resolver match, using last writer wins (layer %d)\n", conflictPath, versions[len(versions)-1].Layer)lastVersion := versions[len(versions)-1]destPath := filepath.Join(mergeDir, conflictPath)if err := os.MkdirAll(filepath.Dir(destPath), 0755); err != nil {return fmt.Errorf("failed to create directory for '%s': %w", conflictPath, err)}if err := copyFile(lastVersion.Path, destPath); err != nil {return fmt.Errorf("failed to copy file '%s': %w", conflictPath, err)} - lines 439-462One match: send all versions in processor order; the reply path must be the same path.
docker_executor/merger.go:439-462go } else if len(matchingResolvers) == 1 {// Call resolver with all versionsresolver := matchingResolvers[0]files, err := buildResolverFiles(conflictPath, versions)if err != nil {return err}request := ResolverRequest{Config: resolver.Config,Files: files,}fmt.Printf("Calling resolver '%s' for conflict '%s' with %d versions\n", resolver.ID, conflictPath, len(files))response, err := callResolver(resolver.ID, m.SessionId, request)if err != nil {return err}// Verify resolver returned the expected pathif response.Path != conflictPath {return fmt.Errorf("Resolver returned invalid path: expected '%s', got '%s'", conflictPath, response.Path)} - lines 485-488Two or more matches: fail, even for the same conflict resolver.
docker_executor/merger.go:485-488go } else {// Multiple resolvers match - ERRORreturn fmt.Errorf("Multiple resolvers match conflicting file '%s': [%s]. Template resolver configuration may be misconfigured.", conflictPath, strings.Join(getResolverIDs(matchingResolvers), ", "))} - lines 119-136origin.template is the processor id; origin.layer the processor index.
docker_executor/merger.go:119-136go func buildResolverFiles(relPath string, versions []processorFile) ([]ResolverFile, error) {var files []ResolverFilefor _, version := range versions {content, err := os.ReadFile(version.Path)if err != nil {return nil, fmt.Errorf("Failed to read file '%s': %w", relPath, err)}files = append(files, ResolverFile{Path: relPath,Content: string(content),Origin: ResolverOrigin{Template: version.Template,Layer: version.Layer,},})}return files, nil} - sulfone.boron/server.go:738-758Coordinator proxy for the CLI: forward to the shared container with a 30 s client; failure is 502.
server.go:738-758go endpoint := fmt.Sprintf("http://%s:%d/api/resolve", docker_executor.DockerContainerToString(d), docker_executor.ResolverPort)fmt.Println("π Upstream Endpoint:", endpoint)fmt.Println("π Start forwarding request...")reqBody, err := io.ReadAll(c.Request.Body)if err != nil {c.JSON(http.StatusBadRequest, ProblemDetails{Title: "Read request failed",Status: http.StatusBadRequest,Detail: "Failed read the initial request body",Type: "https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/400",TraceId: nil,Data: []string{err.Error()},})return}fmt.Println("π¦ Request Body Size:", len(reqBody), "bytes")client := &http.Client{Timeout: 30 * time.Second}resp, err := client.Post(endpoint, c.GetHeader("Content-Type"), bytes.NewBuffer(reqBody)) - sulfone.iridium/cyancoordinator/src/conflict_file_resolver/registry.rs:35-49CLI: each writer votes its first matching declaration.
cyancoordinator/src/conflict_file_resolver/registry.rs:35-49go pub fn get_resolver_choice(&self, template_id: &str, path: &str) -> ResolverChoice {let resolvers = match self.template_resolvers.get(template_id) {Some(r) => r,None => return ResolverChoice::None,};// Find the first resolver that matches the file pathfor resolver in resolvers {if self.matches_any_pattern(path, &resolver.file_patterns) {return ResolverChoice::Some(resolver.clone());}}ResolverChoice::None} - sulfone.iridium/cyancoordinator/src/conflict_file_resolver/consensus.rs:95-116CLI: unanimous agreement, else one of three last-layer-wins records.
cyancoordinator/src/conflict_file_resolver/consensus.rs:95-116go // Case 1: All variations agree on same resolverif without_resolver.is_empty() && resolver_groups.len() == 1 {let (_, group) = resolver_groups.iter().next().unwrap();if group.len() == with_resolver.len() {// All have the same resolverlet resolver = group[0].1.clone();return ConsensusResult::Agreed(resolver);}}// Case 2: All variations have no resolverif resolver_groups.is_empty() {return ConsensusResult::AllNone;}// Case 3: No consensus - some have resolver, some don'tif !with_resolver.is_empty() && !without_resolver.is_empty() {return ConsensusResult::NoConsensus {with_resolver,without_resolver,};} - sulfone.iridium/cyancoordinator/src/client.rs:181-201CLI: sanitise the id, POST to the proxy with a 600 s client.
cyancoordinator/src/client.rs:181-201go pub fn resolve_files(&self,cyan_id: &str,input: &ResolverInput,) -> Result<ResolverOutput, Box<dyn Error + Send>> {let host = (self.endpoint).to_string().to_owned();// Sanitize cyan_id to prevent path injectionlet sanitized_cyan_id: String = cyan_id.chars().filter(|c| c.is_alphanumeric() || *c == '-' || *c == '_').collect();let endpoint = host + "/proxy/resolver/" + &sanitized_cyan_id + "/api/resolve";let http_client = new_client()?;http_client.post(endpoint).json(input).send().map_err(|x| Box::new(x) as Box<dyn Error + Send>).and_then(|x| {if x.status().is_success() {x.json().map_err(|e| Box::new(e) as Box<dyn Error + Send>)
Worked example: package.json in the shop
web:5's build normally runs one processor, atomi/shop-render:1 (version id 6a2b8c10-5d4e-4f30-9a12-7b8c9d0e1f23), then the plugin atomi/shop-format:1, so its own build has no clash. Suppose its script returned shop-render:1 twice (two Cyan entries with different globs), and both invocations wrote package.json. web:5 declares json-merger:1 for package.json. Step through all three places one run touches.
- Inside web:5's build
Each invocation writes to its own folder. The merger finds two versions of package.json; exactly one declaration matches, so json-merger is called with both. Both versions carry origin.template 6a2bβ¦: the id cannot tell the two invocations apart, only layer can. shop-format:1 then runs on the merged folder.
caller coordinator mergerversions shop-render:1 (layer 0), shop-render:1 (layer 1)result one merged package.json - Variation: two declarations
If web:5 also declared json-merger for **/*.json with another config, two declarations would match: the build fails with Multiple resolvers match conflicting file 'package.json'.
result web:5 build error - web:5's tree (step 1)
docker:7, lint:2, node-base:3 and web:5 all wrote package.json. lint:2 declares nothing, so it votes none.
caller CLIvotes json-merger, none, json-merger, json-mergerresult web:5's file wins (lww_no_consensus) - Stacking web and ci (step 3)
web:5 and ci:2 both declare json-merger:1 with the same config: they agree. One call through the proxy; origin.template holds the template version ids.
caller CLI via proxyversions web:5 (layer 0), ci:2 (layer 1)result dev, test and ci scripts joined
The tree below shows package.json at the end of each stage.
- web5-tree/
package.jsonconflictstep 1: web:5's file won; eslint and docker:build lostshow file
{ "scripts": { "dev": "node src/server.ts", "test": "jest" } }
- shop/
package.jsonchangedstep 3: json-merger joined web:5 and ci:2show file
{ "scripts": { "dev": "node src/server.ts", "test": "jest", "ci": "npm test" } }
Why it is built this way
Hard cases
- One declaration, two meanings.
files: ["*.json"]matchessrc/data.jsonin the CLI but not in the merger, so the same template can merge a file in one place and keep the last version in the other. - Same conflict resolver twice is fatal only in the coordinator. Two declarations of json-merger with overlapping globs fail the build, but in the CLI the first one is simply the vote.
- Timeouts differ. A conflict resolver that takes 40 seconds works in the coordinator (no limit) but fails in the CLI (the proxy gives up at 30).
- Text only, everywhere. Both send file content as strings; binary files are not safe to resolve.
- Conflict order is hash order at both places, so conflict resolver calls for different paths come in no fixed order.
- A cache hit warms no conflict resolver. Conflict resolver containers are started only in the coordinator's template warm (sulfone.boron/docker_executor/template_executor.go:387-420), and the CLI warms a template only on a cache miss (sulfone.iridium/cyancoordinator/src/template/executor.rs:93-99). A run where every node hits still sends an agreed vote through the proxy; if no earlier run left
cyan-resolver-<id>up, the proxy answers 502 (server.go:758-768) and the run fails. Inferred from the code; found by reading; not run. - A bad glob has two outcomes. The merger returns an error and the build fails (merger.go:105-108, 423-426); the CLI treats the pattern as never matching (registry.rs:68-72), so the writer silently votes none.
- Identical versions still count as a clash at both places and still call the conflict resolver.
Questions for the designer
- Should both places use the same rule (one match vs unanimous vote), and the same glob library?
- What constraint put processor clashes in the coordinator and template clashes in the CLI?
- Should the proxy's 30 second limit be raised to match the CLI's 600, or made the same for the merger?
- Should
origin.templatesay what kind of id it holds? - Should the CLI and the merger treat a bad glob the same way (fail, or never match)?
- Should the proxy validate the id like the CLI does, and should it need authentication?
- Should a run whose every node hits the cache still start the conflict resolver containers it needs?