Development Tooling Architecture
| Status | Applies to | Owner |
|---|---|---|
| Pre-release draft | current tooling architecture | Development tooling maintainers |
The top-level tooling/ package is development-only. It is excluded from the
installed Python package and cannot define public application behavior.
Production REGENIE options and execution remain in src/g, crates/, and the
TOML-backed g regenie --config flow.
Boundaries
The maintained tooling layers are:
tooling.cli: Hydra entrypoints and campaign orchestration;tooling.benchmark: benchmark implementations and the shared native lifecycle harness;tooling.profile_deep: typed deep-profile models, budget/config policy, profiler wrappers, diagnostics, and reports;tooling.common: production command/config rendering, subprocess handling, cache snapshots, downloads, registry dispatch, and artifact contracts;tooling.configs: saved dataset, machine, telemetry, workload, and complete workflow configurations;tooling.data,tooling.debug,tooling.performance, andtooling.server: narrow domain implementations.
The canonical shared contracts are:
tooling.common.g_regeniefor generated production TOML andg regenie --configcommand vectors;tooling.benchmark.native_lifecyclefor fresh, discarded-warm, hot, telemetry, cache, and completed-output evidence;tooling.common.commandsfor shell-free subprocesses, logs, timeouts, and redacted environment capture;tooling.common.jax_cachefor CPU feature-aware cache paths and fingerprints;tooling.common.reportsandtooling.common.artifact_formatfor versioned machine-readable evidence;tooling.common.registryfor groupedtool.namedispatch.
Benchmark and profiler modules should reuse these boundaries instead of building production argument vectors, output discovery, cache logic, or report dictionaries independently.
Deep-Profile Ownership
tooling.cli.profile_regenie2_deep adapts Hydra configuration and starts the
campaign through tooling.profile_deep.runner. The CLI remains the maintained
entrypoint for all saved profile configurations.
The profile package owns its implementations directly:
baseline,candidates, andtrialsconstruct matched workloads, select tuning candidates, and keep measured trials separate from diagnostics;applicationandexecutionconstruct isolated children, validate completed output evidence, and manage logged subprocesses and timeouts;toolsandprofilersdetect profiler availability and capture diagnostics;diagnostics,reports, andartifactsinterpret native evidence and write the existing summary and versioned artifact contracts;planningbuilds dry-run evidence without executing workloads;config,budget,commands,environment,jax_cache, andmodelsown their respective configuration, policy, transport, and evidence structures.
Profile implementation modules import their owners directly. They do not import the CLI or expose forwarding aliases for its former private helpers. Tests patch the module that owns the operation being exercised.
Configuration Ownership
Every maintained Hydra entrypoint selects an explicit saved config. Workflow
identity and defaults live in tooling/configs/; Justfile recipes only select
those configs and forward intentional overrides. Hydra must keep
hydra.job.chdir: false so repository-relative inputs remain stable.
There is no generic context object or generic sweep framework. An entrypoint parses only the fields it owns into a typed arguments dataclass. Reusable machine and dataset values come from Hydra config groups, while each campaign owns its valid tuning dimensions and validation.
Production Invocation
Development tools may invoke production in two supported ways:
- a fresh subprocess running
g regenie --config <generated.toml>; - a same-process call to
g._core.cli.run(["regenie", "--config", ...])when measuring an already-loaded native lifecycle.
Both use tooling.common.g_regenie to generate the same application config.
Legacy Python orchestration APIs, native PyClasses created only for benchmarks,
alternate output formats, and post-run finalization are not compatibility
surfaces.
Evidence Ownership
Headline timing and diagnostics are different lifecycles. Production headline runs use telemetry off. Profile runs use a distinct output root and may collect native stage summaries or profiler artifacts; they never contribute to the headline elapsed-time distribution.
The shared lifecycle helper owns exact cache-tree snapshots and validates successful manifests, exact committed chunk coverage, safe and complete Parquet part sets, row counts, schemas, metadata, content hashes, and telemetry evidence. Persistent JAX caches are campaign-owned, must start in the state declared by the workflow, and are snapshotted before and after every lifecycle whose cache semantics matter.
Tooling Artifact Format v1 is an internal development evidence format. Its version and the benchmark-, matrix-, and deep-profile-specific evidence schema versions are intentionally independent from public pre-release product contracts. Public option, output-manifest, telemetry, and profile-summary contracts remain at version 0 until release; do not reset internal tooling evidence versions when changing those product contracts.
The canonical bundle may include artifact_manifest.json, report.json,
events.jsonl, metrics.jsonl, commands/commands.jsonl, resolved Hydra/tool
configuration, and summary.md. Validate a completed bundle with:
uv run --no-sync python -m tooling.cli.schema_check \
--config-name schema_check tool.path=<artifact-directory>
Execution Policy
Dry plans and small report checks may run on the login node. CPU compilation,
large tests, and Criterion belong on an exclusive CPU allocation. GPU benchmark
and profiler runs must be serialized on landau through the Justfile SLURM
wrappers.
The matrix workflows replace handwritten application command recipes. Bounded execution requires a prepared bounded BGEN fixture because the production CLI does not expose a variant-limit option. Criterion targets run directly by package and bench name; they are not embedded in the GPU profiler orchestrator.
See Development Tooling Guide for commands and evidence details.