Boolean property tests: nightly only, and the cases they found as ignored tests (#1650)
Pushes to main skip boolean_props (random inputs); the nightly property-tests job runs it. The three shrunk cases are ignored tests with their issues (#1647, #1648, #1649), and proptest's failure file is named outright. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
db1fd39139
commit
a9beb437b3
5 files changed
+69
-9
No files matched your search
@@ -3,7 +3,8 @@
|
|||||||
|
|
||||||
[profile.ci]
|
[profile.ci]
|
||||||
# Heavy tests (one to four minutes each on a CI runner) that pull requests skip. They run on
|
# Heavy tests (one to four minutes each on a CI runner) that pull requests skip. They run on
|
||||||
# every push to main and nightly (`--ignore-default-filter`), and in `cargo xtask test`.
|
# every push to main and nightly (`--ignore-default-filter`), and in `cargo xtask test`. The
|
||||||
|
# boolean_props property tests run only nightly (ci.yml excludes them from pushes to main).
|
||||||
# Keep this list in sync with docs/book/src/dev/testing.md and CONTRIBUTING.md.
|
# Keep this list in sync with docs/book/src/dev/testing.md and CONTRIBUTING.md.
|
||||||
default-filter = """
|
default-filter = """
|
||||||
not (
|
not (
|
||||||
|
|||||||
@@ -229,7 +229,11 @@ jobs:
|
|||||||
run: |
|
run: |
|
||||||
# Pull requests skip the heavy tests (default-filter in .config/nextest.toml).
|
# Pull requests skip the heavy tests (default-filter in .config/nextest.toml).
|
||||||
all=()
|
all=()
|
||||||
if [ "$EVENT_NAME" != pull_request ]; then all=(--ignore-default-filter); fi
|
# The boolean property tests draw new random inputs every run; they run only in the
|
||||||
|
# nightly property-tests job, so a known kernel bug cannot turn main red at random.
|
||||||
|
if [ "$EVENT_NAME" != pull_request ]; then
|
||||||
|
all=(--ignore-default-filter -E 'not (package(solidcraft-kernel) & binary(boolean_props))')
|
||||||
|
fi
|
||||||
# Extract into ./target: tests that run a built binary (env!("CARGO_BIN_EXE_*")) have
|
# Extract into ./target: tests that run a built binary (env!("CARGO_BIN_EXE_*")) have
|
||||||
# the build job's path, which is the same checkout path here.
|
# the build job's path, which is the same checkout path here.
|
||||||
cargo nextest run --archive-file tests.tar.zst --workspace-remap . --extract-to . --extract-overwrite \
|
cargo nextest run --archive-file tests.tar.zst --workspace-remap . --extract-to . --extract-overwrite \
|
||||||
@@ -319,7 +323,8 @@ jobs:
|
|||||||
|
|
||||||
property-tests:
|
property-tests:
|
||||||
# The boolean property tests and the atomic command tests with many cases. Pull requests skip
|
# The boolean property tests and the atomic command tests with many cases. Pull requests skip
|
||||||
# them (.config/nextest.toml); pushes to main run them with small defaults (8 cases, seeds 0..4).
|
# them (.config/nextest.toml); pushes to main run atomic_commands with small defaults (seeds
|
||||||
|
# 0..4) and skip boolean_props, whose random inputs would fail main at random.
|
||||||
name: Property tests (many cases)
|
name: Property tests (many cases)
|
||||||
needs: build-tests
|
needs: build-tests
|
||||||
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
|
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
|
||||||
|
|||||||
+2
-1
@@ -181,7 +181,8 @@ Pull requests skip a few heavy tests (one to four minutes each), which run on ev
|
|||||||
`pattern_suppress_tests::enclosure_arc_extinction_keeps_a_real_sharp_inner_lip_after_resize_and_reopen`,
|
`pattern_suppress_tests::enclosure_arc_extinction_keeps_a_real_sharp_inner_lip_after_resize_and_reopen`,
|
||||||
`pattern_suppress_tests::enclosure_datums_follow_wall_height_and_board_depth_with_material_proofs`,
|
`pattern_suppress_tests::enclosure_datums_follow_wall_height_and_board_depth_with_material_proofs`,
|
||||||
`cmd::plastic::tests::enclosure_snap_fit_follows_resized_front_wall`, the `atomic_commands` contract tests
|
`cmd::plastic::tests::enclosure_snap_fit_follows_resized_front_wall`, the `atomic_commands` contract tests
|
||||||
- `solidcraft-kernel`: `tests::boolean_identity_on_tangent_primitives`, the `boolean_props` property tests
|
- `solidcraft-kernel`: `tests::boolean_identity_on_tangent_primitives`, and the `boolean_props` property tests
|
||||||
|
(these run only in the nightly "Property tests" job: their inputs are random, so main skips them too)
|
||||||
- `solidcraft-ui-egui` scenarios: `acceptance_sensor_enclosure`
|
- `solidcraft-ui-egui` scenarios: `acceptance_sensor_enclosure`
|
||||||
|
|
||||||
**The action matrix** tries every ordinary action on every face, edge, vertex and body of a
|
**The action matrix** tries every ordinary action on every face, edge, vertex and body of a
|
||||||
|
|||||||
@@ -4,10 +4,13 @@
|
|||||||
//! inputs, and when a case fails it shrinks the inputs to a small case that still fails. Paste
|
//! inputs, and when a case fails it shrinks the inputs to a small case that still fails. Paste
|
||||||
//! that case into a normal `#[test]` as a regression test.
|
//! that case into a normal `#[test]` as a regression test.
|
||||||
//!
|
//!
|
||||||
//! Case count: `PROPTEST_CASES` (default 8). Pull requests skip this file (.config/nextest.toml);
|
//! Case count: `PROPTEST_CASES` (default 8). Pull requests and pushes to main skip this file
|
||||||
//! pushes to main run it, and the nightly `property-tests` job uses more cases. A long local run:
|
//! (.config/nextest.toml, .github/workflows/ci.yml): its random inputs differ on every run, so a
|
||||||
|
//! known kernel bug would turn main red at random. The nightly `property-tests` job runs it with
|
||||||
|
//! more cases. A long local run:
|
||||||
//! `PROPTEST_CASES=2000 cargo test -p solidcraft-kernel --test boolean_props`.
|
//! `PROPTEST_CASES=2000 cargo test -p solidcraft-kernel --test boolean_props`.
|
||||||
//! proptest saves every failing seed in `tests/boolean_props.proptest-regressions`. Commit it.
|
//! proptest saves every failing seed in `tests/boolean_props.proptest-regressions`. Commit it, and
|
||||||
|
//! copy the shrunk case into a `#[test]` below (ignored, with its issue, until the kernel is fixed).
|
||||||
|
|
||||||
use proptest::prelude::*;
|
use proptest::prelude::*;
|
||||||
use solidcraft_geom::Vec3;
|
use solidcraft_geom::Vec3;
|
||||||
@@ -102,7 +105,15 @@ fn cfg(default: u32) -> ProptestConfig {
|
|||||||
// The env var overrides the default; proptest reads PROPTEST_CASES itself only when the
|
// The env var overrides the default; proptest reads PROPTEST_CASES itself only when the
|
||||||
// config does not set `cases`, so read it here.
|
// config does not set `cases`, so read it here.
|
||||||
let cases = std::env::var("PROPTEST_CASES").ok().and_then(|s| s.parse().ok()).unwrap_or(default);
|
let cases = std::env::var("PROPTEST_CASES").ok().and_then(|s| s.parse().ok()).unwrap_or(default);
|
||||||
ProptestConfig { cases, max_shrink_iters: 200, ..ProptestConfig::default() }
|
// Integration tests have no lib.rs or main.rs beside them, so proptest's default persistence
|
||||||
|
// cannot place its file; name it.
|
||||||
|
let file = concat!(env!("CARGO_MANIFEST_DIR"), "/tests/boolean_props.proptest-regressions");
|
||||||
|
ProptestConfig {
|
||||||
|
cases,
|
||||||
|
max_shrink_iters: 200,
|
||||||
|
failure_persistence: Some(Box::new(prop::test_runner::FileFailurePersistence::Direct(file))),
|
||||||
|
..ProptestConfig::default()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
proptest! {
|
proptest! {
|
||||||
@@ -218,3 +229,44 @@ proptest! {
|
|||||||
prop_assert!(close(run(&plate, &boss, BoolOp::Union)?, vp + vb, vp + vb), "plate + boss lost or gained volume");
|
prop_assert!(close(run(&plate, &boss, BoolOp::Union)?, vp + vb, vp + vb), "plate + boss lost or gained volume");
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Cases the property tests found, as plain tests. Ignored until the kernel bug is fixed: run them
|
||||||
|
// with `cargo test -p solidcraft-kernel --test boolean_props -- --ignored`.
|
||||||
|
|
||||||
|
fn check(r: Result<(), TestCaseError>) {
|
||||||
|
if let Err(e) = r {
|
||||||
|
panic!("{e}");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A box and a cylinder in general position: the intersect is refused as an implausible volume.
|
||||||
|
#[test]
|
||||||
|
#[ignore = "kernel bug, #1647"]
|
||||||
|
fn generic_box_cylinder_intersect() {
|
||||||
|
let a = Prim::Box { at: [11.483767761089219, 4.801612107504224, 3.6313476467611787], size: [4.789088988834864, 3.0, 10.419335961063835] };
|
||||||
|
let b = Prim::Cyl { at: [29.2480632732879, 8.3683289461485, 6.947922225984966], axis: 2, r: 13.502884497756853, h: 19.719218725477496 };
|
||||||
|
check(identity(&a, &b));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A sphere and a box on the grid: the union has a face that does not mesh.
|
||||||
|
#[test]
|
||||||
|
#[ignore = "kernel bug, #1648"]
|
||||||
|
fn grid_sphere_box_union_is_sound() {
|
||||||
|
let a = Prim::Sph { at: [2.5, 15.0, 27.5], r: 12.5 };
|
||||||
|
let b = Prim::Box { at: [0.0, 0.0, 17.5], size: [5.0, 7.5, 5.0] };
|
||||||
|
check(identity(&a, &b));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Two crossing cylinders: the union moved 1000 mm away loses about a tenth of its volume.
|
||||||
|
#[test]
|
||||||
|
#[ignore = "kernel bug, #1649"]
|
||||||
|
fn crossing_cylinders_union_moved_far() {
|
||||||
|
let a = Prim::Cyl { at: [17.5, 30.0, 7.5], axis: 0, r: 5.0, h: 10.0 }.body();
|
||||||
|
let b = Prim::Cyl { at: [17.5, 30.0, 15.0], axis: 1, r: 12.5, h: 5.0 }.body();
|
||||||
|
let t = Vec3::new(1000.0, -700.0, 300.0);
|
||||||
|
let mv = |x: &Body| transform(x, t, Vec3::ZERO, Vec3::Z, 0.0).expect("translate");
|
||||||
|
let s = vol(&a) + vol(&b);
|
||||||
|
let here = run(&a, &b, BoolOp::Union).unwrap();
|
||||||
|
let there = run(&mv(&a), &mv(&b), BoolOp::Union).unwrap();
|
||||||
|
assert!(close(here, there, s), "union at origin {here:.4}, moved by 1000 gives {there:.4}");
|
||||||
|
}
|
||||||
@@ -81,7 +81,8 @@ Pull requests skip a few heavy tests (each takes one to four minutes); they run
|
|||||||
`pattern_suppress_tests::enclosure_arc_extinction_keeps_a_real_sharp_inner_lip_after_resize_and_reopen`,
|
`pattern_suppress_tests::enclosure_arc_extinction_keeps_a_real_sharp_inner_lip_after_resize_and_reopen`,
|
||||||
`pattern_suppress_tests::enclosure_datums_follow_wall_height_and_board_depth_with_material_proofs`,
|
`pattern_suppress_tests::enclosure_datums_follow_wall_height_and_board_depth_with_material_proofs`,
|
||||||
`cmd::plastic::tests::enclosure_snap_fit_follows_resized_front_wall`, the `atomic_commands` contract tests
|
`cmd::plastic::tests::enclosure_snap_fit_follows_resized_front_wall`, the `atomic_commands` contract tests
|
||||||
- `solidcraft-kernel`: `tests::boolean_identity_on_tangent_primitives`, the `boolean_props` property tests
|
- `solidcraft-kernel`: `tests::boolean_identity_on_tangent_primitives`, and the `boolean_props` property tests
|
||||||
|
(these run only in the nightly "Property tests" job: their inputs are random, so main skips them too)
|
||||||
- `solidcraft-ui-egui` scenarios: `acceptance_sensor_enclosure`
|
- `solidcraft-ui-egui` scenarios: `acceptance_sensor_enclosure`
|
||||||
|
|
||||||
The **action matrix** (`apps/solidcraft-cli/tests/action_matrix.rs`) tries every ordinary action
|
The **action matrix** (`apps/solidcraft-cli/tests/action_matrix.rs`) tries every ordinary action
|
||||||
|
|||||||
Reference in new issue
Block a user