Journey 13 — Scale and HPC gate

solvjump run examples/satellite_comm_study_quick.yaml
# standard: satellite_comm_study_standard.yaml (2048 pts)
# full: satellite_comm_study.yaml (16384 pts)

Tier: quick | 512 scenarios | 6 h @ 5 min steps

-- Act 1 - Ops cadence --
  -> 72 x 300 s (5-minute planning steps). 50 s micro-steps trigger srk_fallback and erase the speed story — do not use them for this demo.

-- Act 2 - Can SolvJump replace HPC? --
  -> No. It gates the cluster: explore the ODE envelope on the ground, queue only finalists for high-fidelity confirmation.

-- Act 3 - Run the comm-window study --
  -> Study file: satellite_comm_study_quick.yaml
  _section: Study definition
  model: <product-root>\python\examples\satellite_comm_model.py:make_rhs
  dynamics_params: inj_error_deg in [-30, 30], drag_scale in [0.5, 2]
  y0: [0.0, 500.0]
  t_span: [0.0, 21600.0]
  mode: auto
  calibration_seeds: 24
  scoring_params: gs_lon_deg in [-180, 180]
  jump_lags: [0.1, 0.5, 1.0, 300.0]
  safety_margin: 2.0
  _section: Operator routing, calibration & sweep (3 steps)
  model: <product-root>\python\examples\satellite_comm_model.py:make_rhs
  dim: 2
  stiffness_nom: S0
  horizon: [0.0, 21600.0]
  problem_set_operators: 256
  -> flame_propagation | dim=2 | stiffness=S2 | dynamics=unknown
  log: Jump recommendation: confidence=recalibrate_required, cache_ref=(none), mode=auto
  seeds: 24
  reason: no exact cache match for this model
  -> Collecting trajectory training pairs: jump lag=300.0s, SolvTune arm=solvsrk, objective=balanced, target chain=72
  log: SRK collection done - 144 train pairs from 3 trajectories (0 failed, need >=8 train)
  log: Fitted nominal+guard kernels - train median SCD=8.01, holdout median SCD=7.12 (n=8/8)
  log: Rollout validation - SCD by chain depth [1:7.12, 10:5.55, 50:4.49, 72:4.52], max_valid_chain=72 (short_chain)
  log: Accuracy gate passed (operator <operator_id>)
  method: grid
  grid_points: 512
  steps: 72
  dt: 300.0
  compare_brute_srk: True
  finalists: 500
  -> Sweep finished - example params {'inj_error_deg': 4.285714285714285, 'drag_scale': 0.5, 'gs_lon_deg': -180.0}, final state ~ [23.9546, 497.3764]
  log: SRK timing sample: 3 point(s) integrated for comparison
  log: Horizon aligned: 72 jumps x 300.0 s = 21600.0 s (jump_lag matches dt)
  log: Jump pool (NOT SRK truth): top 500 of 512 by jump metrics -> finalists_top500.csv
  log: #1 comm window: gs_lon=-77 deg, inj_error=-21.4 deg, drag_scale=0.71 -> pass in 0 min, peak el=89.9 deg
  srk_sample_n: 50
  full_grid_truth: True
  min_spearman: 0.85
  -> Full-grid SRK truth: 512/512 rankable in 275.9 s
  _section: SRK ground truth (authoritative top-N)
  grid_points: 512
  srk_ranked: 512
  export: finalists_ground_truth_top100.csv
  srk_best_jump_rank: 1
  -> SRK #1: gs_lon=-77 deg, window=0 s
  log: SRK top-10 in jump top-500: 10/10 (100% recall)
  log: SRK top-50 in jump top-500: 50/50 (100% recall)
  log: SRK top-100 in jump top-500: 100/100 (100% recall)
  _section: Rank verification (jump vs full SRK)
  verified_rows: 50/512 jump-ranked rows
  spearman_rho: 1.000 (gate >= 0.85)
  srk_verify_time: 24.8 s
  best_srk_jump_rank: #1 (regret 0)
  verdict: PASS — safe to cite finalist ranking
  -> top-10 overlap: 10/10 (100%)
  log: top-50 overlap: 50/50 (100%)
  log: comm_ok agreement (jump vs SRK): 100%
  log: Report: <session-dir>\rank_verification_report.json
  log: SRK shortlist (verified ranking for HPC handoff): <session-dir>\srk_shortlist.csv
  log: SRK ground truth (authoritative): <session-dir>\finalists_ground_truth_top100.csv
  log: Authoritative deliverable: finalists_ground_truth_top100.csv (SRK ground truth — use this for HPC handoff)

-- Act 4 - Extrapolate to cluster scale --
  -> Using measured per-point cost — not a fake multi-hour local run.
  _section: HPC gate (ground SolvJump before you queue)
  anchor: Measured on this laptop: 512 pts, 527 ms/pt brute, 1.9 ms/pt jump
  cluster_exploration: 100,000 scenarios (single revision)
  brute_srk_single_machine: 14.6 h
  solvjump_single_machine: 3.3 min
  design_revisions: 20 program iterations
  brute_revisions_total: 292.6 h
  solvjump_revisions_total: 1.1 h
  finalists_file: <session-dir>\finalists_ground_truth_top100.csv
  finalists_in_file: 100 ranked rows (top 500 requested)
  _section: What SolvJump does for HPC (honest)
  log: SolvJump does not replace your cluster solver or CFD/FEA jobs. It runs on an engineer workstation and searches your ODE parameter envelope fast.
  log: Workflow: SolvJump sweep on the ground, rank with terminal_metrics(), write finalists_ground_truth_top100.csv — queue those rows to HPC for high-fidelity confirmation.
  log: Using SRK ground-truth finalists (authoritative ranking).
  hpc_jobs_without_gate: 100,000 full jobs @ 0.05 core-h each = 5,000 core-hours
  hpc_jobs_with_gate: 500 finalist jobs = 25 core-hours
  core_hours_saved: 4,975 core-hours not queued (exploration on ground)
  -> Brute SRK across 20 revisions at cluster scale is 292.6 h on one machine — that is the 'hours/days' waste when teams re-run full grids every time a bound moves.
  log: SolvJump helps HPC by shrinking what you submit — not by magically making cluster jobs milliseconds. Golden SolvSRK + your HPC toolchain still prove finalists.

-- Act 5 - Recommendation --
  -> SolvJump on the laptop; HPC for rows in the finalists CSV, not 100k exploration jobs. Optional: --tier standard / full. Next: Journey 14 maps remaining Python exports.
  _section: Why run SolvJump here?
  log: Search station longitude, injection error, and drag for a downlink window — one command, no custom solver loop.
  log: Screen 512 six-hour propagations in ~6.8 s instead of full SolvSRK on every grid point.
  log: About 270× faster than full SolvSRK on every grid point (jump sweep vs brute estimate).
  log: Leave with a ranked list of comm windows — pick a ground station and orbit correction before the pass closes.
  log: Sweep ground-station longitude in scoring only; dynamics runs are not repeated when you shift lon across the grid.
  log: Jump ranking matches full SolvSRK on a sample — safe to cite the order in a review.
  log: Quickly generates a verified parameter shortlist for HPC handoff, avoiding full brute-force sweeps on the cluster.
  log: Same workflow scales to 2k–16k scenarios with --tier standard or full.
  _section: Why run SolvJump here?
  log: Estimate cluster cost from a laptop anchor sweep before queueing 100k-point jobs.
  log: Explore the ODE envelope on the ground; queue HPC only for finalists worth confirming.
  log: Avoid burning core-hours on parameter regions you could have ruled out locally.
  log: About 270× faster than full SolvSRK on every grid point (jump sweep vs brute estimate).

-- Run complete --
  session_id: <session_id>
  results_folder: <session-dir>
  points_completed: 512 ok, 0 skipped (of 512)
  fast_sweep: 997.8 ms
  timing_sample: 1579.9 ms (3 full-SolvSRK spot checks)
  vs_full_SolvSRK_on_every_study_point: ~4.5 min for all 512 points in this study (from 3 spot checks — not a larger search)
  speedup: ~270.2× vs full SolvSRK on every grid point
  receipt_calibration: <session-dir>\calibration_receipt.json
  receipt_sweep: <session-dir>\sweep_receipt.json
  receipt_jump_pool: <session-dir>\finalists_top500.csv
  receipt_rank_verification: <session-dir>\rank_verification_report.json
  receipt_srk_shortlist: <session-dir>\srk_shortlist.csv
  receipt_finalists: <session-dir>\finalists_ground_truth_top100.csv
  -- Operator setup --
  accuracy_check: excellent quality
  operator_id: <operator_id>
  fit_time: 5.8 s (one-time before the sweep)
  comm_criteria: Elevation >= 10 deg within 2 h of plan time
  windows_found: 512/512 station-orbit combinations
  no_window: 0
  -- Sweep results --
  completed: 512 parameter sets
  skipped: 0
  -- How long did this take? --
  search_method: fast jump sweep
  grid_size: 512
  operator_fit: 5.8 s
  fast_sweep: 998 ms
  this_run_total: 6.8 s
  vs_full_SolvSRK_on_every_study_point: ~4.5 min for all 512 points in this study (extrapolated from 3 spot checks — not a larger search)
  speedup_vs_full_grid: ~270.2× (jump sweep vs brute estimate)
  -- Batch economics (ops week (5 re-sweeps)) --
  scenario: 5 re-sweeps x 512 points = 2,560 full integrates
  brute_srk_total: 22.5 min
  solvjump_total: 10.8 s
  marginal_re_sweep: 998 ms
  -- Act 4 - Extrapolate to cluster scale --
  -- HPC gate (ground SolvJump before you queue) --
  anchor: Measured on this laptop: 512 pts, 527 ms/pt brute, 1.9 ms/pt jump
  cluster_exploration: 100,000 scenarios (single revision)
  brute_srk_single_machine: 14.6 h
  solvjump_single_machine: 3.3 min
  design_revisions: 20 program iterations
  brute_revisions_total: 292.6 h
  solvjump_revisions_total: 1.1 h
  finalists_file: <session-dir>\finalists_ground_truth_top100.csv
  finalists_in_file: 100 ranked rows (top 500 requested)
  -- What SolvJump does for HPC (honest) --
  hpc_jobs_without_gate: 100,000 full jobs @ 0.05 core-h each = 5,000 core-hours
  hpc_jobs_with_gate: 500 finalist jobs = 25 core-hours
  core_hours_saved: 4,975 core-hours not queued (exploration on ground)
  -- Act 5 - Recommendation --
  -- Why run SolvJump here? --
  -- Why run SolvJump here? --
  -- Run complete --
  session_id: <session_id>
  results_folder: <session-dir>
  points_completed: 512 ok, 0 skipped (of 512)
  fast_sweep: 997.8 ms
  timing_sample: 1579.9 ms (3 full-SolvSRK spot checks)
  vs_full_SolvSRK_on_every_study_point: ~4.5 min for all 512 points in this study (from 3 spot checks — not a larger search)
  speedup: ~270.2× vs full SolvSRK on every grid point
  receipt_calibration: <session-dir>\calibration_receipt.json
  receipt_sweep: <session-dir>\sweep_receipt.json
  receipt_jump_pool: <session-dir>\finalists_top500.csv
  receipt_rank_verification: <session-dir>\rank_verification_report.json
  receipt_srk_shortlist: <session-dir>\srk_shortlist.csv
  receipt_finalists: <session-dir>\finalists_ground_truth_top100.csv

cd python
# Guided tour — narrated scenario walkthrough (recommended first run):
python examples/journeys/13_scale_and_hpc.py --tier quick
 
# Production one-liner — same API call you ship:
solvjump run examples/satellite_comm_study_quick.yaml