diff --git a/AGENTS.md b/AGENTS.md index 525448b..71e32bb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -48,12 +48,16 @@ People trust ModelCraft with their models; a crash loses their work. **This outr HyperMesh, ANSA, Femap, Patran, ANSYS, …): no test data, measurements, decks, results or ids produced with them. Deck formats come from the vendors' publicly available reference manuals; we write our own descriptions and our own examples, never copy manual text or example decks. -- **Expected values are analytic.** Reference tests compare with closed-form solutions (patch - tests, beams, plates, Lamé cylinders, Kirsch plates, natural frequencies, wave and shock - solutions) and with mesh invariants (positive Jacobians, conserved volume, watertight - boundaries). A separate **benchmark** suite may compare with reference values published in - papers (lid-driven cavity, Schäfer–Turek, Turek–Hron, Taylor bar), each tagged with its source; - the analytic suite is the gate. Never values produced with a commercial product. +- **Verification and validation (docs/verification.md).** Every solver capability is checked + three ways, each case in `tests/oracle/` tagged with its source: **theory** (closed-form + solutions, patch tests, mesh invariants), **experiment** (measurements published in papers and + reports) and **published benchmarks** (reference results published by benchmark bodies and by + solver vendors' benchmark and verification manuals, e.g. NAFEMS, Abaqus). Owner decision + 2026-10-13: published vendor results may be used as reference values, cited per case. We never + run a commercial product, never copy its input files, text or figures: each case is our own + model built from the published description, and only the published numbers are used. Once our + result matches its reference within the stated tolerance, it is frozen as a **fixture** + (`fixture.json`), so any later change that moves it fails CI (fidelity never drifts silently). - **Algorithms come from papers.** Meshing and solver methods are implemented from published papers and theses, cited in the module docs and in `docs/meshing/literature.md`. - **Never read, copy or link GPL/LGPL/AGPL code**: TetGen, CGAL, Gmsh, Netgen, CalculiX, diff --git a/ROADMAP.md b/ROADMAP.md index d674d0c..fc57cd5 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -33,4 +33,5 @@ hex meshes today and it gives the best elements where stresses are read. | M12 CFD: lattice Boltzmann | D3Q19/D3Q27 on grids voxelised from the face-tagged surface; TRT, regularised and cumulant collision; interpolated walls, inlets and outlets; LES; octree refinement shared with the hex mesher; wgpu backend (deterministic per device, checked against the CPU) | planned | | M13 Fluid–structure interaction | Immersed-boundary coupling of the lattice Boltzmann fluid with the explicit or implicit structure; interface force transfer; sub-cycling; Aitken then interface quasi-Newton coupling; free-surface flow | planned | | M14 Blast and impact fluids | EOS-driven fluids in the explicit solver: ALE multi-material or SPH, with fluid–structure contact | planned | +| Continuous: verification and validation | Every capability checked against theory (closed forms), experiments (published measurements) and published benchmarks (NAFEMS, vendor benchmark and verification manuals such as Abaqus's), each case our own model in `tests/oracle/` with its cited reference and tolerance; `cargo xtask oracle` runs them and writes docs/oracle.md; once a case matches, our result is frozen as a fixture so fidelity cannot regress (docs/verification.md) | started | | Continuous: desktop and agents | Desktop app (viewport, model tree, commands, control channel: done; toolbar of every capability from the catalogue, ready or coming soon with its milestone, icons drawn with VectorCraft, docs/features.md: done), MCP server, browser build, command schema as the public API | started | diff --git a/docs/adr/0001-libraries.md b/docs/adr/0001-libraries.md index 445af4b..9f57042 100644 --- a/docs/adr/0001-libraries.md +++ b/docs/adr/0001-libraries.md @@ -89,8 +89,11 @@ dependency of `modelcraft-geometry` only behind its non-default `scripts` featur tooling (`modelcraft-testkit`) enables; the shipped library opens designs without it. **Commercial products** (Abaqus, Nastran, LS-DYNA, LS-PrePost, HyperMesh, ANSA, Femap, Patran, -ANSYS) are never run, scripted or used as a source of data. Deck formats come from the vendors' -public reference manuals, described in our own words with our own examples. +ANSYS) are never run or scripted, and their input files, text and figures are never copied. Deck +formats come from the vendors' public reference manuals, described in our own words with our own +examples. Published numerical results (benchmark and verification manuals) may serve as cited +reference values for our own models of the same problems (owner decision 2026-10-13; +docs/verification.md). ## Consequences diff --git a/docs/verification.md b/docs/verification.md new file mode 100644 index 0000000..e4f86ea --- /dev/null +++ b/docs/verification.md @@ -0,0 +1,29 @@ +# Verification and validation + +Every solver capability is checked against three kinds of reference, and every accepted result is +frozen so it cannot drift. + +| Kind | What it checks | Sources | +|---|---|---| +| Theory | The code solves the equations right | Closed-form solutions (beams, plates, Lamé, Kirsch, waves, shocks, Poiseuille…), patch tests, convergence rates, mesh invariants | +| Experiment | The model predicts reality | Measurements published in papers and reports (Taylor bar, cavity and cylinder flows, FSI experiments…) | +| Published benchmarks | Agreement with established codes on standard problems | NAFEMS benchmarks, vendor benchmark and verification manuals (e.g. Abaqus Benchmarks and Verification manuals), other published reference solutions | + +Rules: +- Each case is **our own model** of the problem, built from the published description (geometry, + material, loads, boundary conditions). We never run a commercial product and never copy its + input files, manual text or figures; only the published numbers are used, with a citation. +- Each case lives in `tests/oracle//`: + - `recipe.json` — the model as ModelCraft commands (or a SolidCraft recipe for geometry); + - `expected.json` — the reference values with `kind` (theory | experiment | benchmark), + `source` (citation: document, table or figure, page), `quantity`, `value`, `tolerance` and + the reason for that tolerance (mesh, measurement scatter, model differences); + - `fixture.json` — ModelCraft's accepted result, written by `cargo xtask oracle --bless ` + only after the case passes against `expected.json`. +- `cargo xtask oracle` runs every case, compares with `expected.json` (pass/fail with error) and + with `fixture.json` (any change beyond the fixture tolerance fails CI, even if the reference + still passes), and writes `docs/oracle.md` with the table of cases, sources, errors and status. +- A fixture is updated only deliberately (`--bless`), in a PR that says why the result moved + (a better element, a finer default mesh), never to hide a regression. +- The theory suite is the gate for every PR; experiment and benchmark cases run in CI too, with + their cited tolerances.