Action matrix: every ordinary action on every face, edge, vertex and body of the design corpus (#553)

apps/solidcraft-cli/tests/action_matrix.rs tries sketch, offset/angle/mid planes, press pull, extrude from a face, hole, draft, shell, fillet, chamfer, axes, points, planes, projection, move, copy, mirror and pattern on the final bodies of 104 designs (the old design files, the examples, the 76 reference recipes, the enclosure demos, the Pi case built through the UI, eight typical parts), through the commands and picks the desktop sends, each design in its own process, with an edit round on a sample of passes. tests/action_matrix/baseline.json is a ratchet: an action that passed there and fails now fails the test; newly passing ones are reported for cargo xtask action-matrix --bless. Every CI run checks a fixed sample; the nightly Action matrix job runs the whole matrix in four parts. 30,567 of 35,576 actions pass (85.9%); results and causes in docs/action-matrix.md, one ignored reproduction per open cause (#556-#568).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
bherbruckandClaude Opus 5.5 authored and GitHub committed 2026-10-13 02:41:17 -04:00
1 parent 9283d2b5bb
commit 736d9277b8
17 files changed
+14314

No files matched your search

+34
View File
@@ -24,6 +24,10 @@ commands:
replay the reference tests (tests/oracle/*/recipe.json), compare with each
case's expected.json and write docs/oracle.md (--cli: a solidcraft-cli
already built, instead of cargo run --release)
action-matrix [--bless] [--results FILE]
try every ordinary action on every face, edge, vertex and body of the design
corpus and compare with tests/action_matrix/baseline.json (--bless rewrites it;
--results keeps every outcome with its message). See docs/action-matrix.md
book [--no-build]
generate docs/book's command reference and screenshots, then mdbook build
into target/book
@@ -46,6 +50,9 @@ fn main() -> ExitCode {
Some("assets") => assets::run(&root()),
Some("features") => features::run(&root()),
Some("oracle") => cmd_oracle(rest.iter().position(|a| *a == "--cli").and_then(|i| rest.get(i + 1)).copied()),
Some("action-matrix") => {
cmd_action_matrix(rest.contains(&"--bless"), rest.iter().position(|a| *a == "--results").and_then(|i| rest.get(i + 1)).copied())
}
Some("book") => book::run(&root(), !rest.contains(&"--no-build")),
Some("licences") => licences::run(&root(), rest.contains(&"--check")),
Some("test") => {
@@ -95,6 +102,33 @@ pub fn metadata() -> Result<serde_json::Value, String> {
serde_json::from_slice(&out.stdout).map_err(|e| format!("cargo metadata: bad JSON: {e}"))
}
/// The full action matrix (apps/solidcraft-cli/tests/action_matrix.rs), with the ci profile.
fn cmd_action_matrix(bless: bool, results: Option<&str>) -> Result<(), String> {
let mut c = cargo();
c.args([
"test",
"--profile",
"ci",
"-p",
"solidcraft-cli",
"--test",
"action_matrix",
"action_matrix_all",
"--",
"--exact",
"--ignored",
"--nocapture",
]);
if bless {
c.env("ACTION_MATRIX_BLESS", "1");
}
if let Some(r) = results {
let p = std::path::absolute(r).map_err(|e| format!("{r}: {e}"))?;
c.env("ACTION_MATRIX_RESULTS", p);
}
run(c, "cargo test --profile ci -p solidcraft-cli --test action_matrix action_matrix_all")
}
fn cmd_layers() -> Result<(), String> {
let crates = layers::from_metadata(&metadata()?)?;
println!("Dependency layering (plan/architecture.md §3)\n");