Appendix A: Technical Testing & Verification

This guide outlines the testing strategy, dependencies, and instructions for verifying the PyEQSP project and the eqsp package.

Testing Strategy

The eqsp package uses a Hybrid Testing Approach that integrates standard unit tests with doctest examples to ensure documentation and code remain in sync.

Test Categories

  1. Unit Tests (tests/src/test_*.py):

    • Verify core mathematical logic using static assertions.

    • Compare results against known-good values from the original MATLAB implementation.

  2. Mock Tests (tests/src/*_mock.py):

    • Verify library interaction (e.g., Mayavi/Matplotlib) without needing a display.

    • Check if the correct arguments are passed to the plotting engines.

  3. Extra Tests (tests/src/*_extra.py):

    • Introspective integration tests.

    • Use non-interactive backends (like Matplotlib’s Agg) to verify that the rendered plot objects contain the expected mathematical labels and data properties.

  4. Doctests:

    • Live-tested examples inside module docstrings.

    • Ensures that every >>> example in the documentation is verified during test runs.

Dependencies

Install development dependencies via pip:

pip install pytest coverage

Running Tests

Project-Wide Run

We use a two-tier automated verification system:

  1. Pre-commit Hooks (Local): Automated checks run on every git commit to catch linting errors, documentation typos, and broken links before they leave your machine.

    pre-commit install  # Run once to set up
pre-commit run --all-files  # Run manually to verify everything

  2. Unified Verification Script (Global): The verify_all.py script (located in validation/) is the definitive project-wide entry point.

    • Pull Requests: Every PR must pass all pre-commit hooks and python3 validation/verify_all.py (Ruff, Pylint, Pytest).

    • Environment Orchestration: Use --venv DIR to activate a specific environment and --uninstall to remove any existing pyeqsp package before running the suite. This prevents local source shadowing by stale site-packages.

    • CI Pipeline: GitHub Actions runs validation/verify_all.py across Python 3.11–3.13.

The orchestrated script enforces a Zero-Warning Policy for the Sphinx documentation build (make html SPHINXOPTS="-W"), ensuring that no orphaned pages or malformed Table of Contents entries reach production.

Granular Control

You can run tests at three levels of granularity:

  1. By Module (Bridge): Runs both manual unit tests and bridged doctests for that module.

    pytest tests/src/test_point_set_props.py

  2. By File (Direct): Runs only the doctests localized in that source file.

    python3 -m eqsp.point_set_props -v

  3. Interactive Inspection: To visually verify 2D or 3D output on your local machine:

    python3 tests/src/inspect_illustrations.py
python3 tests/src/inspect_visualizations.py

Visual Verification (Thesis Examples)

The examples/phd-thesis/ directory contains high-fidelity scripts that reproduce figures from the canonical PhD thesis. Use these to verify that the library’s results match the originally published data:

cd examples/phd-thesis
# Run a numerical plot (Agg backend, saves PNG)
python3 fig_4_2_min_dist_s2.py --upper-bound 5000

> **Note:** The virtual environment configuration (`VENV`) used for automated and manual testing was specific to a standardized Linux build (e.g. Kubuntu 25.10). For full 3D functionality, ensure that your environment has **Mayavi** and its dependencies installed.

For instructions on running these scripts and a full mapping of scripts to thesis figures, see [Thesis Research Reproduction](../user/phd-thesis-examples.md).

## Code Coverage

To generate a full coverage report, use the provided helper script:

```bash
python3 tests/run_coverage.py

This will run all tests (including doctests) and produce a summary in the terminal. The project maintains a strict benchmark of 100% coverage. A comprehensive pragma audit was conducted in 0.99 Beta to ensure that all testable code paths are exercised, and that existing pragmas are only applied to truly unreachable or environment-dependent code.

Detailed results are saved in tests/results/run_coverage.log.

Private Implementation Tests

By default, the coverage script strictly excludes private implementation tests and internal doctests to maintain a clear boundary between the Public API and internal performance optimizations.

To include these high-fidelity tests in the coverage report, use the --include-private flag (benchmark: 100% coverage):

python3 tests/run_coverage.py --include-private

Detailed results are saved in tests/results/run_coverage_include_private.log.

The private testing suite includes:

  • tests/src/test_private_histograms.py: Verifies vectorized region lookup logic on S².

  • tests/src/test_private_partitions.py: Bridge test for eqsp._private._partitions doctests.

  • tests/src/test_private_region_props.py: Bridge test for eqsp._private._region_props doctests.

These tests ensure that internal mathematics optimizations (such as vectorized colatitude lookups) match the reference MATLAB logic with high precision.

Diagnostic Tool Validation

To maintain the quality of the project’s prevention mechanisms, the scripts in validation/ are verified via:

  • Doctests: Every diagnostic script includes embedded examples covering its core parsing and regex logic.

  • Orthography Scanning: quality_check.py includes a specialized module to enforce the Australian -ize English standard, ensuring consistent Oxford spelling across all public prose. It also enforces canonical terminology (e.g., ensuring “N-sphere” rather than “Nrd-sphere”) and catches positional-only argument violations in doc examples.

  • Structural Integrity: The suite verifies that ruff.toml maintains its flat-format compatibility and ensures that all # pragma: no cover exclusions are effectively attached to statements.

  • Unit Tests (tests/src/test_maintenance_scripts.py): A dedicated suite that checks the functional I/O behaviour by mocking the repository filesystem. This ensures that tools like check_links.py and quality_check.py accurately identify and report errors in real-world scenarios.

All diagnostic scripts use internal environment isolation (via sys.path) and headless Matplotlib configuration to ensure they run consistently across diverse build environments without interfering with global system state or requiring a display.

Environment Isolation: The validation/verify_all.py script prepends the current Python executable’s directory to the system PATH. This ensures that subprocesses (like make and sphinx-build) use the tools and packages from the active virtual environment, preventing conflicts with system-wide Python installations.

Performance Benchmarking

The benchmarks/ directory contains scripts to verify the algorithmic complexity and execution speed of core functions.

Running the Suite

To run all system benchmarks and generate a summary report:

# Standard partitions
python3 benchmarks/run_benchmarks.py

# Symmetric (even-collar) partitions
python3 benchmarks/run_benchmarks_even.py

Results and Logging

The runner saves individual results for each benchmark in a standardized format:

  • Standard Summary: benchmarks/results/run_benchmarks.log

  • Symmetric Summary: benchmarks/results/run_benchmarks_even.log

  • Standard Individual Logs: benchmarks/results/benchmark_*.log (e.g., benchmark_eq_regions.log)

  • Symmetric Individual Logs: benchmarks/results/benchmark_*_even.log (e.g., benchmark_eq_regions_even.log)

Thesis Benchmark (Section 3.10.2)

The script benchmarks/src/benchmark_eq_regions.py specifically replicates the “Running time” benchmark from Section 3.10.2 of the thesis. It verifies the \(O(N^{0.6})\) scaling behaviour.

To run it independently with progress tracking:

python3 benchmarks/src/benchmark_eq_regions.py --show-progress

Code Quality

The project uses ruff and pylint to maintain high code quality standards.

Ruff (Style and Formatting)

Ruff handles fast linting and automatic formatting:

ruff check .
ruff format .

Configuration Compatibility

The ruff.toml file uses a flat configuration format (omitting the [lint] section) to ensure compatibility across all project environments. This allows the same configuration to be parsed by both:

  • Modern Ruff (0.15.x+) in the main .venv.

  • Legacy Ruff (0.0.291) in specialized environments like .venv_sys, where version constraints are imposed by system-managed plugins (e.g., python-lsp-ruff).

Important

Newer Ruff versions will issue a deprecation warning about top-level settings, but they remain functional. This approach avoids breaking IDE integration in restricted environments.

Pylint (Deep Static Analysis)

Pylint is used for deep semantic analysis. The configuration is refined to allow standard mathematical notation (including variable names like N_values, Ns, Phi) while enforcing strict code quality across the entire repository. The project baseline is a 10.00/10 rating:

pylint eqsp benchmarks examples tests validation release  # Project-wide scan

Vale (Optional Prose Linting)

The project includes a .vale.ini configuration for optional prose linting. Unlike ruff and pylint, vale is not part of the mandatory validation/verify_all.py suite and is not required for standard verification. It is intended for manual documentation audits.