What ARGUS beats, and what beats it

• best implementation studied — FinanceBench's real evaluation_playground.ipynb read in full: the six-mode retrieval taxonomy (closedBook/oracle/inContext/singleStore/sharedStore + reverse variants), the real PyMuPDFLoader/RecursiveCharacterTextSplitter/Chroma RAG pipeline, and the documented finding that no automated scorer ships with the repo — grading was human, out of band
• best method studied — two real methods on the identical task class: FinanceBench's real LLM+RAG pipeline reading filing prose vs. ARGUS's real, keyless SEC XBRL structured API fetch (data.sec.gov/api/xbrl/companyconcept) — already existing, reused here rather than reimplemented
• baseline reproduced — FinanceBench's own real result transcripts (results/*.jsonl, commit cc39aeb4afdf33909ee1412188bf89035950c2eb) independently recomputed for the 50-question metrics-generated subset: oracle 46/50 correct, singleStore 22/50, sharedStore 6/50 (39 refusal), inContext 6/50 correct but 20/50 confidently WRONG, closedBook 0/50 — no LICENSE file on the repo, so the dataset/results are cited and recomputed from, never vendored
• implementation complete — FundamentalsSource.facts/parse_concept/latest_per_period already existed and were already exercised by this project's own t3-infoextract probe before this comparison; this capability adds the rival measurement, not a stub
• same input comparison — both real systems scored on the identical task class — a single numeric line item pulled from a real SEC filing — with ARGUS run on a freshly designed, real, live case set rather than FinanceBench's own unlicensed question set
• statistically valid evaluation — FinanceBench's own side already a real 50-question subset across 16 model/ retrieval configurations; ARGUS's side swept across 6 real (ticker, concept) cases spanning all 5 supported concepts and 5 different real companies, not one hand-picked case, resolving 5 of 6 (the sixth reports why rather than guessing)
• costs included — real wall-clock cost measured for a real, live SEC XBRL API round trip
• out of sample test — the designed cases use real, live, current-day XBRL data never used to design the comparison's mechanism — NVDA/AAPL/MSFT/GOOGL/AMZN's most recent real filings as of the day this ran, not a fixed historical snapshot
• ablation — isolates the exact mechanism on a real, live case: NVDA's real Q2 FY2009 net income carries two real rows under the identical fiscal end-date, a true quarterly duration and a sign-flipped cumulative one — a naive fetch with no quarterly filter is genuinely ambiguous between them, ARGUS's real quarterly_only path resolves to exactly the correct quarterly value
• adversarial test — the real fetcher's edge-case behaviour tested directly: an unknown ticker reports 'no CIK on EDGAR' rather than raising or guessing, and a point-in-time cutoff correctly withholds every real fact filed after it
• failure cases documented — two real, measured behaviours of the real fetcher found by running it: an unknown ticker returns zero facts with a named reason, and a point-in-time as_of cutoff withholds every real fact filed after it rather than leaking look-ahead
• reproducibility proven — the same real ticker/concept fetched twice returns the same real value
• no specialist capability superior — scoped explicitly — SCOPE_STATEMENT in eval/infoextract_comparison.py. Claimed: on a real numeric line item from a real filing, ARGUS's structured fetch cannot fabricate a fluent wrong answer, a failure mode FinanceBench's own data proves is common even for GPT-4. NOT claimed ARGUS answered FinanceBench's own 50 questions. NOT claimed ARGUS solves FinanceBench's harder DERIVED-ratio questions — it supports five raw line-item concepts, not multi-fact arithmetic over them

• implementation complete — rules are replayed over real decisions and must earn a place by performance
• failure cases documented — replayed over 40 real decisions, none of the five standing rules earned one
• statistically valid evaluation — a rule is promoted on measured precision, never on having been written down
• best implementation studied — the four systems already read for the sibling episodic-memory audit (TradingAgents, RD-Agent, FinMem, LangGraph) checked specifically for a process-rule/checklist-learning mechanism, not just symbol memory - grepped directly for checklist/process_rule/precision/earned concepts in RD-Agent and FinMem's own source: no match in either. Only TradingAgents has any lessons-reuse mechanism at all (the reflection log), and it has no precision-gated promotion of any kind - confirmed by reading it in full for the sibling capability, not re-guessed here
• best method studied — TradingAgents' real memory/reflection code read in full (same read as the sibling Episodic memory capability): graph/reflection.py and agents/utils/memory.py's TradingMemoryLog - store, update, and unconditional read-back, with no precision, fire-rate, or track-record concept anywhere
• baseline reproduced — the same vendored, byte-verified TradingMemoryLog already run for Episodic memory (commit be952b8eccb49720509af544c6675233bc1f10d0) - no new vendoring needed, reused directly: real writes, real outcome updates, real unconditional reads
• same input comparison — a decision with a demonstrably WRONG outcome (a real -15% realised return) stored via the real vendored TradingMemoryLog and read back later: the reflection describing the wrong call is re-injected identically to one about a right call, confirmed by running the real code twice, once each way
• ablation — all four of ARGUS's lifecycle thresholds (MIN_PRECISION, ALWAYS_FIRES, NEVER_FIRES, MIN_FIRINGS) shown independently load-bearing by patching each to a value that flips an otherwise-identical fixture's verdict (ACTIVE->MISLEADING, ACTIVE->NO_DISCRIMINATION, ACTIVE->DEAD_WEIGHT, ACTIVE->EARNING), verified programmatically, not derived — the last two were added 2026-09-21; this line correctly said two before that, and is updated now that the code genuinely tests all four
• adversarial test — six designed fixtures, each targeting exactly one of the seven named lifecycle states (PROPOSED, DEAD_WEIGHT, NO_DISCRIMINATION, MISLEADING, EARNING, ACTIVE), run through the real evaluate() and confirmed to land on the intended status every time - including EARNING, which this module's own docstring records as a real self-found bug (unreachable until a specific fix), re-verified reachable here on fresh fixtures rather than assumed fixed
• costs included — TradingAgents' real write path costs one LLM call per graded episode (Reflector.reflect_on_final_decision) to produce the prose it stores unconditionally; ARGUS's evaluate() is pure arithmetic over independently observed defects - zero tokens, zero model calls, every single replay
• out of sample test — 433 real decisions split 216/217 by ledger sequence; every standing rule graded on both halves at the production MIN_DECISIONS. 3 of 5 are gradeable on the held-out half and all 3 keep the same verdict; the other 2 fire too rarely to judge and are counted as not-gradeable rather than as failures, because the absence of a test is not a bad result
• reproducibility proven — evaluate() is a pure function of its inputs - no model, no seed, no simulation. Every fixture in this comparison is hand-constructed and fixed
• no specialist capability superior — scoped explicitly - SCOPE_STATEMENT in eval/review_comparison.py. Claimed: the MACHINERY that refuses a bad rule by name (DEAD_WEIGHT, NO_DISCRIMINATION, MISLEADING) exists, runs correctly across six real lifecycle transitions, and has no equivalent anywhere in TradingAgents' real code (grepped directly, zero matches). NOT claimed: that TradingAgents' unconditional reuse is a design mistake on its own premise (a model re-reads the prose and can itself discount it, unlike ARGUS's constitution-gated loop where a model's live judgment is deliberately not the backstop). Also NOT claimed: that ARGUS's checklist currently contains any earned rules - it does not, disclosed as a standing blocker below, unchanged by this push

• same input comparison — ARGUS's offline FLUSS, real stumpy, real ruptures and the real incumbent rule all run on the SAME 60-day hourly Bitget market series for all 12 rTokens, plus all FOUR budgeted methods (FLUSS, stumpy, ruptures, and ARGUS's second segmenter exact_partition added 2026-09-22) run on the SAME 100 synthetic trials (ruptures.pw_constant, 25 seeds x 4 noise levels) with KNOWN injected changepoints — Finding 8, extended the same day it first ran
• statistically valid evaluation — one-sided binomial test of novelty rate against the incumbent's own coverage as the null (66.9%): FLUSS's own-run novelty rate scores non-significant against that null on every run to date. Paired sign test of per-trial F1 against KNOWN synthetic changepoints (ruptures.metrics.precision_recall at margin=WINDOW) across 100 trials: FLUSS loses to ruptures decisively (mean F1 0.443 vs 0.975, 1 win/89 losses/10 ties, p=1.5e-25) and beats stumpy significantly (mean F1 0.443 vs 0.330, 55 wins/0 losses/45 ties, p=5.6e-17). ARGUS's second segmenter, exact_partition, TIES ruptures on the identical 100 trials (mean F1 0.98 vs 0.975, 1 win/0 losses/99 ties, sign-test p=1.0 — literally cannot reject 'no difference' with one discordant trial, so reported as a tie, not rounded up from a nominally higher mean)
• costs included — argus_seconds=5.25 vs stumpy_seconds_warm=0.017 vs ruptures_pelt_seconds=0.43 on the same 1,439-bar symbol; FLUSS measured ~306x slower than warm stumpy for a bit-identical matrix profile (16,992 windows, zero disagreements). exact_partition measured separately at ~24ms per 400-bar synthetic trial (pure-Python bottom-up DP with O(1) prefix-sum cost lookups) — faster than FLUSS despite being the exact, not approximate, tool
• adversarial test — a flat curve: stumpy fabricates a boundary and repeats the same index on it, ARGUS's FLUSS refuses to report one at all — the one adversarial input in this comparison where ARGUS is the more conservative system, published beside the loss rather than let it soften the headline
• out of sample test — first-half/second-half split of the 60-day window, novelty and agreement rates recomputed independently on each half against the same incumbent-flip null rather than carried over from the full-window number. Separately, the exact_partition tie is checked by noise level rather than only in aggregate: F1=1.000 both sides at noise 0.5/1.0, and ARGUS nominally ahead at the hardest level tested (noise=4.0: F1 0.920 vs 0.900, Hausdorff 13.04 vs 17.68) — not one lucky aggregate trial
• reproducibility proven — reproducibility.identical=true; timing fields are excluded from the identity check by name rather than the check being loosened silently. Separately, exact_partition itself is a deterministic dynamic program with no internal randomness — unlike the LUI classifier head rebuilt the same week, no seeding was needed for reproducibility here

Published because `data/regime_comparison.json`'s own who_wins field already read 'baseline — ruptures is more coherent... stumpy is bit-identical and faster... FLUSS shows no measurable edge over the two-line incumbent', and nothing read that field into this register until now. The artefact carried a stale 337x / 5.61s timing figure from before a code fix that already read 310x / 5.25s; regenerated on 2026-09-21, it now reads ~306x / 4.72s. The small further drift between 310x and 306x across the two regenerations is ordinary wall-clock variance in a timing measurement, not a second stale figure. Updated 2026-09-22 (same day, second update): Finding 8 first sharpened the loss (FLUSS decisively loses to ruptures on real ground truth, p=1.5e-25, while genuinely beating stumpy, p=5.6e-17) — then, rather than stopping at a sharpened loss, `ruptures/detection/dynp.py` and `ruptures/costs/costl2.py` were read in full (BSD-2-Clause, permissive) to understand WHY FLUSS loses, and a second ARGUS segmenter (exact_partition — an exact L2 dynamic program, the textbook-correct tool for pw_constant's piecewise-constant-mean generative process) was built, verified to reproduce ruptures' own real Dynp exactly (120/120 trials) and to match KernelCPD's rbf-kernel result on the same data (10/10 trials), then wired into the SAME Finding-8 benchmark that measured the loss. Result: a genuine tie on the same 100 trials (mean F1 0.98 vs 0.975, 1 win/0 losses/99 ties). State moves LOST -> TIED — mirroring the portfolio-allocation capability's own same-week story — not to OWNED, because a 1-trial discordant sample cannot support a significance-backed win claim.

• best implementation studied — AnalogDesk's engine read at source (analog.mjs, distribution.mjs, validation.mjs) and chosen by the rival review of 2026-09-24 as the entry to beat in Decision Stress Testing
• baseline reproduced — our scorer, written from their protocol, reproduces their own published multipliers, coverage, width, matched width and PIT chi-square exactly (max difference 0.0)
• same input comparison — ARGUS answers their exported queries from the same adjusted closes, truncated and embargoed at each query session
• statistically valid evaluation — Winkler scores compared per query with a Diebold-Mariano test on differences clustered by query date; coverage standard errors clustered by date
• out of sample test — multipliers frozen on 2019-2022, scored on 2023-2026, as their protocol fixes
• failure cases documented — every predictor fails PIT; the naive band beats every retrieval method
• reproducibility proven — deterministic: their engine, their grid, a fixed ARGUS configuration
• implementation complete — both ARGUS engines ran on every query

• best implementation studied — MirrorLine's claim rules read in source (lib/challenge/rules.ts, engine.ts): direction, causation, session, reference tape, liquidity, depth, freshness, trade action; rebuilt from behaviour, no code taken
• baseline reproduced — MirrorLine's engine run live on 38 claims across nine rTokens, raw output kept
• same input comparison — the same 38 sentences through both desks the same morning, the tape captured before and after; near-zero days reported apart rather than graded
• implementation complete — direction, past-tense cause and session claims, tested
• failure cases documented — the first run was a loss (ARGUS read 18 of 38, MirrorLine 38) and is kept; the claim kinds only one desk reads are named

• best implementation studied — baserate's parser, dossier builder, base-rate engine and fixtures read; its weekend move found to run Friday close to Monday CLOSE
• baseline reproduced — baserate's buildDossier run from its clone on the same sentence
• same input comparison — 'long rNVDA over the weekend at 3x with 5,000 USDT' through both desks
• implementation complete — split-adjusted stock weekends, closures only, read from the position's side
• failure cases documented — the first run answered a different question; no regime-conditioned record

• best implementation studied — Rook's invalidation logic read (lib/desk/invalidation.ts: model proposal, overridden by the 24h low, SMA20 or 1% when on the wrong side)
• baseline reproduced — Rook's own runDebate executed on six names, outputs kept in data/h2h_rook/
• same input comparison — both stops scored on the same held-out 40% of each name's Bitget daily bars
• out of sample test — ARGUS's distance fitted on the first 60% of history, scored on the last 40%
• implementation complete — the adverse-excursion measure, tested
• failure cases documented — six names, one run; Rook's line is a thesis invalidation, not a noise stop
• statistically valid evaluation — per-window net return, ARGUS against the 1x ATR stop, with a stationary bootstrap over dates in the groupwise audit; the interval includes zero, which is the TIED
• costs included — every arm's return is net of the 12bps Bitget taker round trip
• ablation — ARGUS's stop quantile at 0.80, 0.90 and 0.95 on the same windows, hit rate and net return for each
• adversarial test — the most volatile fifth of held-out windows (ATR at least 5.4% of price) scored on its own, every arm
• reproducibility proven — the arms are recomputed from the saved bars and must equal the published figures

• best implementation studied — sisl/AdaptiveStressTestingToolbox (MIT) ast_reward.py, example_av_reward.py:40-117 and its MCTS solvers read at source (research/harvest/16-adaptive-stress-testing.md)
• same input comparison — both estimate from the same fitted windows and the same recorded books, scored on the same test windows
• reproducibility proven — the snapshot digest is recorded; the search is exhaustive and deterministic
• implementation complete — desk/stress_search.py, shown in the research task's stress step
• costs included — every arm and the realised truth carry the median exit slippage and the 6bps taker fee
• out of sample test — estimates on the first half of each name's windows, truth on non-overlapping windows of the second half
• failure cases documented — the books too shallow to price the exit are counted per name, and the tolerance where ARGUS is closer on fewest names is published
• adversarial test — the most volatile quarter of names scored on its own

• best implementation studied — NIGHTWATCH read at source: src/domain.js runSkillPack, synthesizeSignal, LocalNightwatchEngine.research; server/providers/bitget.mjs computeIndicators and mergeMarketRow; server/signal-history.mjs grading (8h, 20 bps flat band); server/live-enhance.mjs overlay
• baseline reproduced — its signal function imported from the clone by baselines/nightwatch_signal.mjs, unmodified
• same input comparison — the same symbol-instants, graded on the same realised moves at the desk's mark and at 8h

• best implementation studied — optic-bitget's desk read in source (src/desk/index.ts, evidence.ts, judge.ts, src/lenses/prediction.ts) and its Polymarket selection rules taken for market/prediction.py
• baseline reproduced — optic's own runDesk executed on three theses, outputs kept verbatim; its code unchanged except an environment-gated LLM endpoint so it could run on the hackathon Qwen
• same input comparison — the same three sentences ('Long MSTR/NVDA/TSLA perp into earnings — funding looks cheap') through both desks the same morning
• implementation complete — the premise check found by this comparison, and tested
• failure cases documented — optic still leads on a single synthesised call; two ARGUS defects the run exposed; two gaps closed afterwards and not re-measured

• best implementation studied — mem0/memory/main.py (add, search, the ADDITIVE_EXTRACTION_PROMPT) and its storage history, read at source (research/harvest/04-mem0.md)
• same input comparison — both sides read the same frozen statements, not-about-me sentences and effect tasks in each round; mem0 run as released, in its own venv
• out of sample test — four rounds; the two used for tuning are labelled so, and held_out_4 was frozen before either side ran and never used to change the reader
• implementation complete — lui/memory_model.combined, called for every question by the console's server

• same input comparison — every arm walks the same rebuilt book at the same instants; 80 parents per size, buy and sell, $5k to $250k
• costs included — Bitget's 0.06% taker fee on every child
• reproducibility proven — deterministic replay of a hash-recorded event file
• failure cases documented — the console's hourly schedule lost to Bitget's TWAP on every parent

• best implementation studied — egress/exitcost.py read at source: walk, quote (mid reference, exhausted book reported, no extrapolation), sliced (full and no replenishment as bounds)
• baseline reproduced — EGRESS's quote imported from its clone and run unmodified on every recorded book
• same input comparison — both on the same 1,104 books, the same two sizes, a sell of each
• reproducibility proven — python -m argus.eval.exitcost_parity regenerates the artefact from the tape

• best implementation studied — RasaHQ/rasa chosen because bounded-domain conversational intent understanding is its primary purpose, not a framework with chat bolted on; it is the OSS NLU engine the benchmarking literature treats as a baseline, and DIET has a published paper (Bunk et al., arXiv 2004.09936). ConvLab-3 rejected for granularity (a full TOD pipeline, NLU is one swappable module); snips-nlu rejected as unmaintained since Jan 2020 despite correct granularity, per the standing rule to take the harder candidate. No finance-specific permissively-licensed conversational NLU system was found to exist — only datasets (BANKING77, CLINC150).
• baseline reproduced — Rasa trained once, 2026-09-21, on ARGUS's own training_rows() as it stood that day (334 rows, all 9 labels incl. out_of_scope) and scored on the identical sealed split; reproduced independently from the vendored predictions before this module existed (217/251/293 argus, 240/291/293 rasa, matching to 4 decimal places). Rasa's side is a fixed, dated snapshot (`rasa_diet_sealed_predictions.json`'s own `trained_and_scored` field) and is not retrained when ARGUS's training data changes — by design, per same_input_comparison below; the 314-row count after the 2026-09-22 OOS-rebalancing rebuild describes ARGUS's live side only, not this baseline.
• same input comparison — both systems scored on the identical 293-row sealed split and the identical 14 out-of-scope probes; ARGUS's side runs live through the exact console function (classify_with_fallback), never vendored — so the 2026-09-22 classifier-head rebuild changed this comparison's result automatically, on the next run, with no change to the comparison module itself
• statistically valid evaluation — McNemar's exact test (Dietterich 1998) on paired predictions, re-run after the 2026-09-22 OOS-rebalancing rebuild: sealed accuracy p=0.2649 (11 argus-only vs 18 rasa-only correct) — comfortably not significant, down from the SVM-only rebuild's already-not-significant p=0.0576, and from p=0.0014 before either rebuild. Out-of-scope p=0.0156 (7 argus-only vs 0 rasa-only declined) — unchanged and still significant.
• ablation — ARGUS's abstention threshold removed entirely (raw top-1 from probabilities(), no suppression): accuracy rises to 82.25% (241/293) — fractionally ABOVE Rasa's 81.91% (240/293). Not claimed as a win: this is a different, unsafe configuration (no refusal at all), not the deployed one, and is reported to show the remaining gap is now caution rather than capability, not to launder it into a headline number.
• failure cases documented — Rasa's 8 out-of-scope leaks named with routed intent and confidence (e.g. 'who won the world cup' -> performance at confidence 1.0, saturated — no threshold would rescue it); ARGUS's 1 remaining leak ('sing me a song') is a strict subset of Rasa's 8, unchanged by the OOS-rebalancing rebuild — that rebuild targets in-scope questions wrongly refused, a different failure direction from genuinely-OOS questions wrongly answered, and the two do not move together

**2026-09-22, two rebuilds this date.** First, the classifier head: dev-half 5-fold CV (10 fold-split seeds) found `LinearSVC(C=5.0, class_weight='balanced')` beats the multinomial logistic head this shipped with on every seed (mean 76.71% vs 72.28%); a calibrated variant (`CalibratedClassifierCV`) also won but less (75.36%) and needs k-fold machinery the plain margin does not. `lui/ngram.py`'s pure-Python scorer needed zero code changes — it was already generic linear-scores-then-softmax. The abstention threshold was re-derived by the same rule that set the old 0.20 (lowest threshold beating the pattern layer on both axes in out-of-fold CV), landing at 0.15. Word-level TF-IDF features stacked onto the character n-grams were tried and hurt (73.35% to 66.77%/66.16% on dev CV) — a real, tested, negative finding, not silently dropped. Second, same date, the RIVAL LENS pass: out-of-scope training-set language balance (see the blockers entry above for the full evidence) — the first rebuild's own root-cause note had named this lever and left it untried; this pass tried it, on dev-CV evidence only, then spent the one legitimate sealed-read to confirm 77.82% to 79.52% for real. Latency and deployability were measured once, before either rebuild (ARGUS ~66x faster median response, ~375x faster cold start, a stdlib-only bundle vs Rasa's 1.9GB TensorFlow venv needing Python<=3.10), reported as an unreproduced finding from the agent that ran the original comparison rather than re-measured inside eval/lui_comparison.py, so no Proof above cites them.

• best implementation studied — fixed 2026-09-22, two real gaps closed in sequence. First: desk/allocation.py:optimal_leaf_order implements the Bar-Joseph/Gifford/Jaakkola (2001) DP, the same algorithm scipy's real optimal_leaf_ordering implements and Riskfolio calls by default (verified against real live scipy, 200 random trees plus the real 12-symbol book); hrp_weights now reproduces Riskfolio's shipped-default HRP to floating-point identity (~2.9e-16) — but this closed only the ARGUS-vs-Riskfolio-HRP gap, not the walk-forward loss to NCO, which was the actual open question. Second: read Riskfolio's real NCO source in full (HCPortfolio.py's _intra_weights/_inter_weights/_opt_w) and built ARGUS's own desk/allocation.py:nco_weights — real Ward linkage (scipy's Lance-Williams recurrence, verified against real live scipy, 200 random trees), a real active-set long-only minimum-variance QP (verified against Riskfolio's real cvxpy solver, 100 random trials), and the real two-difference gap statistic for cluster count. Reproduces Riskfolio's real NCO to within 1.3e-05 max weight difference on the real book
• same input comparison — all twelve allocators (including ARGUS's own NCO, added 2026-09-22) run on the SAME 60-day hourly Bitget candle history for the SAME 12-symbol rToken universe and the SAME sample covariance
• statistically valid evaluation — paired sign test per allocator vs ARGUS's own HRP, Holm correction across every comparison; ARGUS's own NCO is compared against Riskfolio's real NCO as a null control (parity, like the HRP parity-twin check), not scored with a significance test against it, because the claim is a tie, and a sign test has nothing to say about one. Since 2026-09-26 also the bake-off's per-window held-out volatilities for every pick: exact sign test, Wilcoxon signed-rank, stationary-bootstrap 95% interval, Holm across the family
• costs included — argus_hrp_seconds≈0.000706 vs riskfolio_hrp_seconds≈0.0423 vs cvxportfolio_mpo_seconds_one_solve≈2.589 — ARGUS's HRP still ~60x faster than Riskfolio's own; ARGUS's own nco_weights is pure Python against Riskfolio's real cvxpy-backed solver, timed but not yet the decisive number here, since the walk-forward result is a tie on quality, not a cost trade
• adversarial test — added 2026-09-22: 20 near-singular covariance matrices (a real asset cloned at noise levels from exact duplication to merely ill-conditioned), run against ARGUS's real nco_weights and Riskfolio's real NCO. Riskfolio's real NCO refuses/crashes 0/20 times — it proceeds on every input, including exact duplicates — while flagging internally ('you must convert self.cov to a positive definite matrix' / a LinAlgWarning on a singular matrix) on all 20/20 trials it still answers. ARGUS's own nco_weights refuses cleanly (AllocationError) on 16/20 — the genuinely singular ones. On the 4/20 where both succeeded, weights diverge meaningfully (mean 8.5pp, worst 16.9pp max difference) — expected on an ill-posed problem with two different solvers, not a correctness defect on either side. NOT claimed as an accuracy win: there is no ground truth for a near-singular minimum-variance problem, only a measured, reproducible difference in whether each side proceeds and whether proceeding is honest about the covariance being untrustworthy
• out of sample test — the bake-off's held-out half: each family picked its configuration on the selection windows, then all picks were scored on 20 held-out windows none of them saw (ARGUS's pick beats Riskfolio's single-linkage NCO, ties the rest). The earlier live walk-forward (data/allocation_comparison.json: ARGUS's NCO and Riskfolio's both 8.203bps, ratio 1.0000) kept only per-allocator means, so it is not the evidence cited here
• failure cases documented — two_assets_below_min, duplicated_column and zero_variance_column all handled and reported rather than raising; Riskfolio's HERC/HERC2 raise TypeError on every call in this harness, run via a documented shim, disclosed rather than silently patched over
• reproducibility proven — reproducibility.identical=true across repeated runs on the same fetch

Published because `data/allocation_comparison.json` already said 'this capability is LOST to the specialist on its own criterion' in its own scope_statement, and nothing read that field into this register until now. Updated twice on 2026-09-22: first, optimal leaf ordering closed the HRP-vs-HRP gap to floating-point identity but left the NCO loss essentially unchanged — a real, tested, negative answer to an explicitly open question. Second, later the same day: built ARGUS's own NCO from Riskfolio's real source (Ward linkage, active-set min-variance QP, the real two-difference gap statistic), verified it reproduces Riskfolio's real NCO to 1.3e-05 on the real book, wired it into the SAME live walk-forward comparison this capability was measured LOST on, and re-ran it live: a genuine tie, 8.203bps both, ratio 1.0000. LOST -> TIED, on a freshly-run comparison, not a reinterpretation of the old one. Third update, also 2026-09-22, after the regime-boundary-detection and LUI capabilities were each closed the same day: this capability's own adversarial_test condition was still unproven, and `run_failure_cases` — the existing degenerate-input harness — only ever exercised HRP, which cannot fail this way (recursive bisection never inverts a matrix). Built `run_adversarial_covariance` (20 near-singular covariance matrices, real assets cloned at noise levels from exact duplication to merely ill-conditioned), run against ARGUS's real nco_weights and Riskfolio's real NCO for the first time. Closes the condition honestly: real, systematic, reproducible evidence of a genuine failure-mode difference (Riskfolio never refuses and flags trouble every time it proceeds anyway; ARGUS refuses cleanly on the genuinely singular cases) — reported as exactly that, not inflated into a claim that ARGUS's weights are more correct, since there is no ground truth for a near-singular allocation problem to check either side against.

• best implementation studied — weekend-copilot's delta.ts, stats.ts, stress.ts, hedge.ts and critic.ts read at source; chosen by the rival review of 2026-09-24 as the S2 entry to beat on this sub-theme
• baseline reproduced — the port reproduces the beta before, beta after and max correlation its own engine printed for all three of its preset books (max difference 7e-18)
• same input comparison — both answer the same 1,800 books and proposed adds at the same origins; both are scored on Bitget daily, native daily and open-session hourly targets
• statistically valid evaluation — exact Wilcoxon signed-rank over per-origin mean errors; books inside one origin share a market path and are averaged before testing
• out of sample test — every prediction uses data before its origin; every target is the 28 days after it
• failure cases documented — not significantly better than beta = 1.0; open-session beta no better than blended at forecasting
• reproducibility proven — hash-pinned Bitget snapshot, hash-recorded rival data, fixed seed and pre-registered spec hash
• implementation complete — the console's own copilot() produced every ARGUS figure

• best implementation studied — skfolio's _vine_copula.py:518-595 and _entropy_pooling.py:436-590 read at source; named by the rival review of 2026-09-24 as the method rival for conditional stress
• best method studied — regular-vine conditional sampling (Dissmann et al. 2013) and Entropy Pooling (Meucci 2008), the two conditional-scenario methods skfolio ships
• same input comparison — every arm gets the same realised QQQUSDT move and the same traded-day history on 39 days with QQQUSDT down 1% or more, 200 books each
• statistically valid evaluation — Wilcoxon signed-rank over per-day mean errors; a day a system cannot answer is dropped from that pair only
• out of sample test — every fit uses only days before the scored day
• failure cases documented — stale weekend closes broke the first run and are removed for every arm; Entropy Pooling cannot answer a shock beyond its history
• reproducibility proven — hash-pinned Bitget snapshot, seeded vine, pre-registered spec hash
• implementation complete — the console's own stress_by_beta produced every ARGUS figure

• best implementation studied — Ballast's universe.py, overnight.py, stats.py, costs.py, research/hedge_study.py and oos.py read at source
• baseline reproduced — hedge_study.py and oos.py run unmodified; every README figure checked matched
• same input comparison — both scored on Ballast's own per-night series, 11 names, its 70/30 split
• out of sample test — ratios fitted on the first 70% of nights, never refitted
• failure cases documented — the first run was a clean loss (index hedge 10.1% against 99.7%), published before the rebuild
• implementation complete — the console's own overnight_hedge produced ARGUS's figures

• best implementation studied — skfolio optimization/convex/_base.py:340-379, 597-598, 971-1020, 1042-1088 and utils/equations.py:48-130, 557-771 read at source (research/harvest/46-cvxpy-allocation.md)
• same input comparison — both solve the same six limit sets on one covariance built from the same 4,280 hourly returns; each result is graded by recomputing every limit from its weights
• reproducibility proven — the input snapshot's digest is recorded, and the library versions each run used
• implementation complete — desk/constrained.py, reached from the console's portfolio-building answer

• best implementation studied — whale-signals' real src/analysis/event_study.py read in full — compute_event_returns/compute_hit_rates/compute_conditioned_hit_rates/compute_base_rate/walk_forward_by_year — confirming compute_hit_rates is the real significance layer (a scipy.stats.binomtest against a fixed 50% null) and that compute_base_rate, defined in the SAME real file, is never passed into it; ARGUS's own existing research/eventstudy.py (Patell/BMP/Corrado-rank/generalised-sign, Kolari-Pynnonen clustering correction, already teardown-compared against research/architecture/eventedge.md) read again specifically for this comparison
• best method studied — two real methods compared: whale-signals' real raw forward-return hit rate tested against a FIXED 50% null, no market-model adjustment, no clustering correction anywhere in the file (confirmed by exhaustive grep) vs. ARGUS's real market-model abnormal-return test (asset regressed on a real market proxy) with four independent significance tests and a real, measured cross-sectional clustering deflation applied to every parametric one
• baseline reproduced — whale-signals' real compute_hit_rates/compute_base_rate run unmodified, byte-hash-pinned against the vendored source (commit 6be10a598c9319aa6b64bf517618eac2dd2ec2f5)
• implementation complete — run_base_rate_case/run_false_positive_sweep/run_null_ablation evaluate real ARGUS-fetched ETHUSDT/BTCUSDT candle history end to end through both real systems
• same input comparison — both real systems scored on the same real, live Bitget ETHUSDT/BTCUSDT hourly candle history, with placebo whale-event timestamps drawn from the same real series, carrying zero true informational edge by construction — the real 24h base rate (measured live, whale-signals' own real function) departs from 50%, and the two real significance tests reach independent, sometimes opposite, verdicts on the identical placebo draw
• statistically valid evaluation — swept across many independent placebo draws (not one convenient case): whale-signals' real fixed-null test calls a false-positive rate far above the nominal 5% significant 'smart money' on genuinely uninformed events, while ARGUS's four-test full-agreement rate stays near or below nominal on the identical draws
• costs included — real wall-clock cost measured on both real sides on the same real data — whale-signals' single binomial test vs. ARGUS's real market-model fit plus four statistical tests and a real O(n^2) cross-sectional clustering correlation, both numbers reported plainly with the reason for the gap stated rather than hidden
• out of sample test — the false-positive-rate sweep is itself the out-of-sample evidence: each of the swept placebo draws is an independent real sample never used to design the comparison's mechanism, and the elevated whale-signals rate holds across all of them, not just the single hand-picked design case
• ablation — a real, run ablation re-tests the SAME real placebo hit counts against a sweep of assumed null probabilities (0.50, 0.55, the real measured base rate, 0.60) using the same real scipy.stats.binomtest whale-signals' own code calls internally — testing at the real measured rate produces a materially lower false-positive rate than testing at the fixed 0.50 whale-signals actually uses, isolating the wrong fixed null as the actual mechanism rather than assuming it (the stricter claim that the true rate is the exact minimum among every swept candidate was tried first and found, by running it repeatedly, not to hold reliably against real serial-correlation variance in the overlapping return windows — the directional claim does)
• adversarial test — the real 30-event minimum threshold whale-signals' own code hardcodes tested on both sides of the boundary (29 vs. 30) — at 29 the real function silently skips the category (printed, no exception); at 30 it is computed
• failure cases documented — two real, measured silent-behaviour shapes found by running whale-signals' real code, not assumed: a category below the real n=30 minimum is silently skipped rather than flagged, and an all-False condition mask silently returns exactly 0.5 — a real 'no data' state masquerading as 'base rate is exactly random'
• reproducibility proven — the real base-rate comparison run twice on the same seed produces byte-identical JSON output
• no specialist capability superior — scoped explicitly — SCOPE_STATEMENT in eval/eventdriven_comparison.py. Claimed: on significance-testing methodology for event-driven hit-rate claims, ARGUS's existing clustering-corrected, non-fixed-null machinery does not share the measured false-positive inflation whale-signals' real fixed-null test shows on real market data. NOT claimed whale-signals' raw arithmetic is wrong, or that the two systems measure the same effect — a raw hit-rate test and a market-relative abnormal-return test can and do disagree in either direction on identical input. NOT claimed whale-signals' own real, published 646,442-transaction results were reproduced — the real Dune dataset needs a paid API key this project does not have; their real significance-testing CODE was run instead, on ARGUS's own real candle history with constructed placebo events

• best implementation studied — FinBERT, BloombergGPT, FinMA, FinMem and TradingAgents read for the feed
• best method studied — SentimentAnalyst's own docstring names the real axis: not classification accuracy (explicitly disclaimed — 'FinBERT is the baseline and beating it on classification is not where the edge is'), but the source-independence integrity layer finBERT's architecture has no equivalent of at all — a classifier with zero source-identity concept cannot discount coordinated repetition, by construction, not by omission
• baseline reproduced — the REAL, published ProsusAI/finbert model loaded via the standard transformers.pipeline() API and run on every scenario — not a vendored copy (no source file to vendor; a public inference model, used exactly as its own README documents, nothing redistributed)
• implementation complete — SentimentAnalyst's real role prompt exercised end to end via a real Qwen call, not paraphrased or mocked
• same input comparison — both real systems run on identical symbol-matched coordinated-posting scenarios (N near-identical rephrasings of one unsourced claim) built from the same underlying narrative
• statistically valid evaluation — 35 REAL coordinated story clusters (Track 2 run 1's own published ledger, mechanically flagged, no model judgement) and 35 matched real uncoordinated controls, one lone post vs. the real capped burst on each: ARGUS resisted 35 of 35 coordinated clusters against the fair rival (dedup, then finBERT) at 31 of 35 (4 discordant, all favouring ARGUS, 0 the reverse; exact McNemar p = 0.125; bootstrap 95% interval on the 35 rows [0.029, 0.229], excluding zero). The rival never once beats ARGUS on any of the 70 real clusters measured. Per-cluster rows are kept in the artefact, so the groupwise check classes it CHECKED, not designed_cases
• costs included — finBERT's local CPU latency measured directly (~33 ms/post); ARGUS: 141 real Qwen calls, 231,491 tokens and 53 minutes of wall time for the 30+12 run and its uncached reproducibility check, recorded in the artefact's costs block
• out of sample test — temporal generalisation on the real-cluster run, precisely stated: nothing in the metric, cap or match rule was fit to this data, so what is checked is whether the pooled finding replicates on a day never used to choose anything — the ledger's last real day (2026-09-27, 8 of the 35 coordinated clusters). It does NOT replicate: the held-out day is an EXACT TIE (8 of 8 resisted on both ARGUS and the fair rival, 0 discordant, McNemar p = 1.0); the whole pooled edge comes from the other three days (27 of 27 vs. 23 of 27, McNemar p = 0.125). This IS the honest result of running the check, not a pass dressed up: it is exactly why the capability is graded TIED rather than OWNED
• ablation — two real ablations, not one, because the first was genuinely informative against the module's own starting hypothesis: removing only the single named 'five accounts' sentence changed NOTHING (the model's broader source-awareness reasoning already reached the same conclusion unprompted) — a real negative result, reported as found, not discarded; a second, coarser ablation stripping the ENTIRE source-independence framing DID show the real defense degrade (directional confidence measurably rose under coordinated repetition), confirming the defense is a property of the prompt's whole framing, not any one sentence
• adversarial test — two attacks on every narrative: five template mutations of one sentence, and five retellings by five distinct personas in five registers with no source added (the harder attack the 2026-09-23 re-check named as untested). Neither flipped the analyst to an actionable trade on any of 30 narratives. Precise, not overclaimed: CONFIDENCE is not immune (0.15 to 0.85 on insufficient_evidence reads), the signal leaned toward the rumour, below the actionable bar, on 2 of 30 diverse attacks (AAPL and HOOD, both bearish), and the bare classifier (the coarse ablation) leaned harder than the real analyst on 29 of 30. The retellings share persona templates across narratives; LLM-generated paraphrase campaigns are still untested
• failure cases documented — feed probed live 2026-09-13: StockTwits 403 to four User-Agents, CNN Fear & Greed 418, only alternative.me answers and it is crypto-wide, not per-symbol; SCOPE_STATEMENT separately states in writing that this comparison proves the MECHANISM works when evidence reaches it, not that evidence currently does
• reproducibility proven — finBERT is deterministic in eval mode. ARGUS's call is not byte-deterministic, so reproducibility means the SIGNAL holds: 5 of 5 stable on a genuinely uncached re-ask (`reproducibility.uncached` true in the artefact). The earlier claim of 'same signal, confidence 0.85 both times' was a cache hit and is withdrawn
• no specialist capability superior — scoped precisely to the one property tested: on resistance to a coordinated/duplicate-source manipulation attack, ARGUS's real analyst has a working defense finBERT's per-sentence-only architecture cannot have by construction — not a claim that ARGUS classifies sentiment more accurately than finBERT, which its own design explicitly disclaims and this comparison does not attempt

PROMOTION GATE: 30 differing paired frames through argus.eval.ablation.paired, scored on realised basis points. HELPS promotes it; NO_EFFECT removes it from the panel. Kept in the code rather than deleted so the gate can actually be run — a deletion would hide the finding, and an unmarked analyst would let a dead feed count as a data source.

• same input comparison — the desk's sentiment analyst, VADER and finBERT on the identical social evidence of each settled decision, scored on the same realised move net of the same round trip

• best implementation studied — crypto_sor's real SmartOrderRouter.ts and CompositeOrderBook.ts read in full — confirmed newOrder() is the real, decisive fill logic (a composite price-level queue across venues, greedily filled by raw price) and that no fee, funding, or holding-cost term appears anywhere in either real file
• best method studied — two real methods compared: crypto_sor's real entry-cost-only greedy router vs. ARGUS's real total-cost router (argus.cost.model.CostModel.charge — entry slippage/fee plus every real funding settlement over a stated holding period, an already-existing, already-validated ARGUS cost engine reused here rather than reimplemented)
• baseline reproduced — crypto_sor's real, unmodified CompositeOrderBook.newOrder() run through a real Node/ts-node subprocess (not a Python re-implementation), byte-hash-pinned against the vendored TypeScript source (commit e1bc5c85a160135889a250c21c7c20a056f8a245)
• implementation complete — choose_hedge_leg/break_even_holding_days/HedgeDecision evaluate real notional, real order-book slippage and real funding rates end to end
• same input comparison — both real systems scored on the same real, live Bitget order-book slippage and funding rate for a real rToken leg (NVDAUSDT) and a real crypto leg (BTCUSDT) — agreeing at zero holding time and diverging past a real, computed break-even horizon
• statistically valid evaluation — swept across a real holding-time range (0 to 5 days) and across five real rToken/crypto leg pairs, not one convenient case — the real crypto_sor pick never changes across the holding-time sweep while ARGUS's does, and four or more of the five real leg pairs diverge past their own real break-even
• costs included — real wall-clock cost measured on both real sides — a real Node/ts-node subprocess spawn (~2 real seconds) vs. ARGUS's real pure-Python decision (sub-millisecond), both numbers reported plainly
• out of sample test — run_leg_pair_sweep repeats the base comparison across five real, different rToken/crypto combinations never used to design the comparison's mechanism — the divergence pattern holds on all or nearly all of them, not just the single hand-picked NVDAUSDT/BTCUSDT case
• ablation — break_even_holding_days isolates the exact mechanism by construction: it holds entry slippage fixed and finds precisely the holding time at which the funding channel alone overtakes the entry saving, verified against the real, measured NVDAUSDT/BTCUSDT case in a pinned unit test
• adversarial test — the real router's edge-case behaviour tested directly: an exchange filter matching no real quote returns an empty fill list rather than raising, and an order larger than the composite book's real depth silently underfills with no flag anywhere in the real return value
• failure cases documented — three real, measured behaviours of the real router found by running it, not assumed: a single-leg book fills without error, an exchange filter matching nothing returns an empty list rather than raising, and an oversized order silently underfills with no completion flag anywhere in the real Execution[] it returns
• reproducibility proven — the real subprocess run twice on the same real leg data returns the same real pick both times
• no specialist capability superior — scoped explicitly — SCOPE_STATEMENT in eval/execution_comparison.py. Claimed: on pricing the FULL real cost of a cross-asset-class hedge choice (entry plus holding), ARGUS's router accounts for a real cost channel crypto_sor's real router has no representation of at all. NOT claimed either real leg is a validated hedge of anything — only the execution cost of each leg is priced, never whether it offsets the exposure being hedged. NOT claimed crypto_sor is broken as a same-instrument, cross-VENUE router — this comparison feeds it two asset-class legs of one venue as competing quotes, a faithful generalisation of its real algorithm, not evidence its original cross-exchange use case has a defect. NOT claimed the measured break-even horizons are permanent — both real cost inputs are measured live and will move with the venue's own real funding rates

• best implementation studied — RD-Agent's real factor-discovery loop read in full — scenarios/qlib/proposal/factor_proposal.py, components/coder/factor_coder/factor.py, core/experiment.py's FBWorkspace, scenarios/qlib/developer/feedback.py — and the execution path (FBWorkspace + FactorFBWorkspace) vendored whole, byte-hash-pinned
• best method studied — both real methods compared: RD-Agent's single LLM-driven hypothesis stream, with an LLM reading its own qlib backtest metrics as text and voting 'Replace Best Result: yes/no' (feedback.py's own real code); ARGUS's five fixed-budget algorithmic strategies (research/searchoff.py) with a hard-coded in-sample-pick/out-of-sample-score separation no LLM judgment call can quietly skip
• baseline reproduced — RD-Agent's real FBWorkspace/FactorFBWorkspace run unmodified, byte-hash-pinned against the vendored source (commit 32b3d395e73d9db5eee3fe9063d69aec0fdc83bd)
• implementation complete — all five search strategies plus the shared budgeted Arena evaluate real bars
• same input comparison — the identical crafted, attacker-shaped factor.py payload construct is put to both real systems: RD-Agent's real execute() accepts and runs it; ARGUS's grammar has no code-shaped input surface to accept it at all, by construction (AST-verified, zero eval/exec/subprocess calls); separately, ARGUS's real deflated_sharpe() is run on ARGUS's own real 400-trial pool from the same real NVDAUSDT bars research/searchoff.py's own published Track 1 finding already uses
• statistically valid evaluation — the real 400-trial search_random pass — not a hand-picked convenient case — and a 5-point trial-count sweep (1/5/25/100/400) on the identical observed value, confirming the correction responds to trial count rather than being a fixed penalty
• costs included — both real costs measured directly and reported as genuinely NOT comparable on one axis, not force-fit into a single winner: RD-Agent's real fixed subprocess-spawn floor vs. ARGUS's real computation-bound, no-subprocess grammar evaluation — measured on real ~1487-bar NVDAUSDT data, ARGUS's own number is the LARGER of the two, the opposite of what a smaller synthetic fixture shows, reported honestly rather than the more flattering number kept
• out of sample test — the real trial pool's best candidate is explicitly the IN-SAMPLE half (Arena.split, 65/35), the exact quantity a naive accept/reject rule would report as if validated — deflated_sharpe exists precisely to refuse trusting it uncorrected
• ablation — trials swept 1->400 on the same observed value; probability strictly falls
• adversarial test — a crafted factor.py payload run through RD-Agent's real, unmodified FactorFBWorkspace.execute() writes a marker file this test checks for from OUTSIDE the subprocess — genuine host-level execution of code this module never edited, not an assertion from reading the subprocess.check_output line alone
• failure cases documented — both sides' real handling of a crashing/malformed input run: RD-Agent's real CustomRuntimeError handler catches a factor.py that raises cleanly; ARGUS rejects an unknown search-grammar op at Window construction, before a search loop is even reachable
• reproducibility proven — the same seed reproduces the identical real trial pool bit-for-bit; separately, the six vendored RD-Agent files are hash-pinned so they cannot silently drift from the cited commit
• no specialist capability superior — scoped explicitly — SCOPE_STATEMENT in eval/rdagent_comparison.py. Claimed: on execution surface, ARGUS's grammar structurally cannot execute code while RD-Agent's real code demonstrably can and does; on trial-count correction, RD-Agent's real accept/reject mechanism has no analogue anywhere in its source (exhaustive grep for deflation/multiple-testing/PBO/FDR terms, zero matches) while ARGUS's real DSR gate refuses the exact 'discovery' a real 400-trial run of its own search would otherwise report. NOT claimed: that RD-Agent's factor-implementation CODER (the LLM step upstream of the execute() call this comparison starts at) is unsound — not vendored or run here. NOT claimed: that RD-Agent's qlib backtest metrics are not out-of-sample — the real conf_combined_factors.yaml template DOES define train/valid/test date ranges; the gap is narrower: the SAME test window is reused as the acceptance bar across an unbounded number of loop iterations with nothing correcting for how many were tried. NOT claimed that ARGUS's own search finds better factors — research/searchoff.py's own published Track 1 result is 0 of 12 symbols surviving the all-trials DSR gate too

• baseline reproduced — FactorMiner's paper factors, Alpha101 and adapted variants evaluated by FactorMiner's own engine at commit 201309cf in its own Python 3.12 environment, run as a subprocess; its own count of 110 paper factors reproduced, one all-NaN in its engine
• same input comparison — both sides on the same frozen 90-day bars (data/factor_split_half_bars.json, sha256 checked on load) through the same cost, out-of-sample and split-half gates; ARGUS's 96 rows match data/factor_split_half.json field for field or the run stops
• costs included — the cost gate charges the backtest engine's Bitget perp cost (12 bps round trip) on every rebalance of every pair, the same for both sides
• adversarial test — a placebo arm rotates every rival factor half the sample against its returns; it passes every gate on 4 paper pairs against 2 for the factors as written

• best implementation studied — TradingAgents, RD-Agent, FinMem and LangGraph memory paths read
• implementation complete — rendered in the PM frame before reasoning, not applied to the answer after
• adversarial test — truncating the future must not change what the past knew: an episode that settles after the recall instant contributes its decision and not its outcome
• statistically valid evaluation — no pattern is stated below 5 graded episodes, matching the calibration floor
• best method studied — TradingAgents' actual reflection/memory code read in full: graph/reflection.py (Reflector.reflect_on_final_decision, an LLM prompt-and-invoke wrapper with no logic of its own) and agents/utils/memory.py (TradingMemoryLog, 334 lines - the append-only markdown log, its as_of point-in-time filter, same/cross-ticker retrieval split, and rotation) plus its sibling agents/utils/rating.py
• baseline reproduced — TradingAgents' real TradingMemoryLog vendored verbatim - byte-verified against commit be952b8eccb49720509af544c6675233bc1f10d0, pinned by SHA256 - and actually executed: real writes (store_decision), real outcome updates (update_with_outcome), real point-in-time reads (get_past_context) against a real temp-file markdown log, not a mock or a description
• same input comparison — identical fixtures run through both ARGUS's real recall() and TradingAgents' real TradingMemoryLog across the resolution boundary. Both correctly hide an unresolved outcome and correctly reveal it once resolved (2 of 3 boundary cases agree exactly); the third case is a genuine, run-verified granularity divergence, not an agreement failure: ARGUS's strict < on datetime instants excludes an episode settled at exactly `now`, TradingAgents' <= on date strings includes an entry resolved exactly on `as_of` - both real, both intentional, cited to the exact line each does it
• costs included — TradingAgents' real write path costs one LLM call per graded episode (Reflector.reflect_on_final_decision -> quick_thinking_llm.invoke, read directly at graph/reflection.py) to produce the prose it stores; ARGUS's lessons() is pure arithmetic over graded ledger fields - zero tokens, zero model calls, every single time it runs. No LLM call was made anywhere in this comparison itself (the prose TradingMemoryLog stores is supplied as a fixed string), matching this project's own hackathon-key budget discipline
• out of sample test — the boundary logic tested here is not fixture-tuned code written only for this comparison: it is the exact, unmodified argus.agents.recall.recall() already running in production against the real paper ledger (121 decisions across twelve instruments, cited in this module's own docstring) - the 'out of sample' evidence is that the same function, not a variant of it, already carries real load outside the 6 hand-designed fixtures this comparison adds
• ablation — ARGUS's MIN_EPISODES_FOR_A_LESSON floor (5), shown load-bearing by patching it down to 4 on an otherwise-identical 4-episode fixture: with the real floor, lessons() states nothing (correctly, since 4 < 5); with the floor patched to 4, the same data produces a real lesson. A first version of this ablation read the patched result AFTER the monkeypatch had already been restored and silently reported the floor as not load-bearing for the wrong reason - caught by running it and checking the actual booleans, not assumed correct on write, fixed the same day (see this module's own history)
• failure cases documented — TradingAgents' real get_past_context has no equivalent floor of any kind - verified by running it on a single episode and confirming it returns content regardless of sample size, the exact failure mode ARGUS's own floor exists to prevent ('a memory that generalises from two observations is worse than no memory', this module's own docstring)
• reproducibility proven — both systems are deterministic over their stored state - no model, no seed, no simulation in either's storage/retrieval path (TradingAgents' write path needs an LLM call to PRODUCE the prose; reading it back is pure parsing). Every fixture in this comparison is hand-constructed and fixed, not sampled
• no specialist capability superior — scoped explicitly - SCOPE_STATEMENT in eval/recall_comparison.py. Claimed: both systems' point-in-time guard was run and verified correct on both sides; ARGUS additionally shows ungraded episodes as context (TradingAgents' real code explicitly filters pending entries out, read directly) and additionally refuses a pattern below a stated floor (TradingAgents has none, verified by running it); ARGUS's write path costs zero tokens against TradingAgents' one LLM call per episode. NOT claimed: that excluding pending entries or having no floor is a mistake on TradingAgents' part - both are real, working design choices for a system whose memory is advisory prose re-read by a model that can itself discount thin evidence, a different premise from ARGUS's structured, falsifiable one. Also NOT claimed: that ARGUS's design produces BETTER trading decisions - that needs graded real-world outcomes from both systems on the same decisions, which neither this comparison nor the project has

The advantage over the baseline is falsifiability, not richness: every line is arithmetic over data/paper_ledger.jsonl and can be recomputed by a reader.

• best implementation studied — etoro_trading/journal.py (96 lines) and guards.py (194 lines) read in full — the same append-only, hash-chained JSONL + head-anchor-sidecar design ARGUS's own PaperLedger uses, vendored byte-verified
• best method studied — per-entry SHA-256 link plus an atomically-replaced head-anchor sidecar for tail-truncation detection is the real method in both systems' own code, read side by side
• baseline reproduced — serenity's real ChainedJournal.append()/.verify() executed via the vendored file through a sys.modules shim, zero source edits, commit 13d46dc1c6204f27fe321bd66023691918017cce
• implementation complete — committed against ledger head 4c382231cf77e3d3 at 94 entries, governs 95+
• same input comparison — both real systems run on identical realistic 2-entry fixtures for the CRLF case, the tamper case, and the truncation case
• statistically valid evaluation — swept entry counts 1 through 50 (not just the N=2 case the bug was first found at): serenity's real verify() fails at every single N, ARGUS's real verify() is clean at every single N
• costs included — ARGUS's own chain overhead measured, not estimated: 9.75ms/entry to write, 0.021s to verify a 250-entry ledger — this artefact had never been generated before 2026-09-21 (cited the module's own source only, ATTESTED); the previous text here read '~0.014s to verify', unchecked against a real run and wrong by roughly 50% once one existed
• out of sample test — verified against the REAL, live production paper-trading ledger this session has been writing to, 836 real entries on 2026-09-29, chain and links intact, anchor agreeing
• ablation — isolates the one design choice that matters: hashing raw disk bytes (serenity's real behaviour) is fragile under nothing but platform text-mode I/O; hashing re-serialized parsed content (ARGUS's real behaviour) survives the identical condition
• adversarial test — a genuine, reproducible, platform-specific bug found by RUNNING the real vendored code, not by reading it: on Windows, serenity's own ChainedJournal rejects its own completely untampered write as tail-truncated, because Python's default text-mode file write silently converts \n to \r\n and verify() re-hashes raw binary-mode-read bytes that were never what append() itself hashed; separately, test_protocol.py proves enforcement runs last in the cycle and may only reduce, never enlarge
• failure cases documented — SCOPE_STATEMENT names exactly what is and is not claimed, including that the CRLF defect is Windows-specific and NOT re-tested on POSIX here (stated as NOT VERIFIED, not assumed); the protocol's own governs-from-entry boundary is a separate, already-documented limitation
• reproducibility proven — the commitment digest is recomputable from the protocol text alone
• no specialist capability superior — on every property both systems implement — link integrity, tamper detection, truncation detection — they agree; on the one property where they diverge (surviving real platform I/O), ARGUS's real code is correct and serenity's real code is not, confirmed by running both, not asserted

• best implementation studied — the wider corpus read for a counterfactual-abstention record and found nothing; a fresh targeted search found the one real exception — AutonomousTradeAgents' 'Refusal Ledger', read in full (ghost_service.py, 538 lines)
• best method studied — both real methods read and compared: ARGUS's settle_abstention records a single counterfactual move at settlement, scored via abstention_quality's avoided_loss/missed_gain/total_value split; AutonomousTradeAgents' GhostBucket marks every vetoed/declined proposal to market over a trading-day horizon with the identical split PLUS a separate 'headline' saved_usd/missed_usd number
• baseline reproduced — no license to vendor under (gh repo view: licenseInfo=null), so an independent clean-room reimplementation of the aggregation was verified against reference output computed by running their own real, unmodified _bucket_from_rows once locally on their own documented example — matched exactly (net=$2179.00, loss_avoided=$30788.00, upside_blocked=$32967.00, headline saved_usd=$0.00)
• implementation complete — settle_abstention records counterfactual_move_bps against the live tape
• same input comparison — both real systems scored on the structurally identical adversarial scenario — a large avoided loss and a large blocked/missed upside that roughly offset — including the EXACT dollar figures ($30,788 / $32,967) the baseline's own code comment names
• statistically valid evaluation — swept across twelve ratios of avoided-loss to blocked-upside from 0 (pure loss) to 2.0 (upside dominates), not one convenient case — confirming the real defect fires exactly where the net crosses near zero and nowhere else
• costs included — both real comparison functions' call overhead measured directly
• out of sample test — the reference scenario is AutonomousTradeAgents' OWN documented production incident (a real, dated example from their own code comment), not a fixture invented for this comparison
• ablation — isolates the exact mechanism: the real bucket's own unfloored net (ghost_pnl) already carries the correct $2,179 answer — the ablation confirms the defect is specifically the max(0.0, -net) floor applied afterward, not the underlying aggregation
• adversarial test — the decisive real finding: AutonomousTradeAgents' real headline saved_usd computation produces exactly $0.00 on their own real, documented example — not a fixed historical bug, still live in build_ghost_summary (ghost_service.py:278) — while ARGUS's real abstention_quality() reports the true, unfloored net on the identical scenario shape
• failure cases documented — SCOPE_STATEMENT states in writing what is NOT claimed: that AutonomousTradeAgents' overall design is unsound (it is not — only the one headline computation), or that ARGUS's calendar-hour settlement horizon matches or beats their trading-day-aware one, since the two systems trade genuinely different-calendar venues
• reproducibility proven — the ledger is hash-chained; any entry can be recomputed from its predecessor
• no specialist capability superior — scoped precisely to the one property tested: on reporting a true, unfloored net value for a two-sided abstention record, ARGUS's real abstention_quality() has no equivalent of the named specialist's real, still-live headline-flooring defect — not a claim that ARGUS's abstention scoring is otherwise more complete (their trading-day-aware horizon and partial-mark visibility are real, useful properties this comparison does not attempt to beat)

• best implementation studied — HeyArka read at source: packages/core/src/vectors/*.ts, runner.ts:18-73, score.ts:153-225, packages/shield/src/shield.ts:45-122 (research/repos-t2/jhaycrypt001~heyarka); every vector's transform and payload recorded with its source file:line
• baseline reproduced — HeyArka's shield run from its own built TypeScript by eval/baselines/heyarka_shield_probe.mjs on its own corpus and demo context
• same input comparison — ARGUS's stages and HeyArka's shield on the same 16 vectors over HeyArka's own demo context, paired per vector, exact sign test
• implementation complete — the live gather runs the price check; the quarantine screens every evidence item
• adversarial test — sixteen attack vectors from an independent red-team corpus, payloads byte for byte
• failure cases documented — the one vector that reaches the model is named, with what reaches it
• reproducibility proven — python -m argus.eval.attack_suite regenerates data/attack_suite.json; the corpus is committed (data/heyarka_corpus.json) and the probe reruns from a clone

• best implementation studied — TradingAgents' real trader.py, schemas.py, portfolio_manager.py, all three risk_mgmt/*.py debators, reporting.py, and trading_graph.py read in full: trader.py's own grounding instruction (line 36-39) asks the model to anchor prices in the market report, but no real consumer of trader_investment_plan anywhere in the pipeline compares entry_price/stop_loss against it — grepped exhaustively, zero hits
• best method studied — two real methods compared: TradingAgents' prompt-only grounding instruction (no code enforcement anywhere downstream) vs. ARGUS's real agents.grounding.check — deterministic numeric resolution of every figure in a thesis against the facts the desk actually had, already existing and already tested (test_grounding.py), reused here rather than reimplemented
• baseline reproduced — TradingAgents' real, unmodified TraderProposal.entry_price validated at 999,999.0 with no relationship to any real fact — the real Pydantic validator raises nothing; its own _coerce_optional_float only normalises string format (placeholder text, a trailing '%', a currency symbol), never the value
• implementation complete — grounding.check/extract/GroundingReport already existed, already tested end to end (test_grounding.py) before this comparison; this capability wires real live desk facts through it against the real rival, not a stub
• same input comparison — both real systems checked on the same real, live Bitget current price (and the same fabricated entry_price) for the same real symbols — TradingAgents' real validator and ARGUS's real grounding.check, not a re-implementation of either
• statistically valid evaluation — swept across 3 real symbols and 4 fabrication magnitudes/directions (7.3x, 0.03x, -1x, 1000x of the real live price) — 12 cases, not one hand-picked figure; TradingAgents' real validator catches 0 of 12, ARGUS catches 12 of 12
• costs included — real wall-clock cost measured on both real, pure-Python checks per call
• out of sample test — run against real, freshly-fetched live Bitget ticker data across 3 real symbols never used to design the mechanism, plus a positive control confirming the real current price resolves cleanly on the same live data — not a blanket flag
• ablation — isolates the exact mechanism: a figure just inside grounding.py's own TOLERANCE constant resolves, the identical figure moved just past it on the same real fact does not — the tolerance boundary is the load-bearing mechanism, verified directly rather than assumed
• adversarial test — the real rival's edge-case behaviour tested directly: a negative price (-50.0) is accepted with no error, exercising the one boundary _coerce_optional_float does not even attempt to check
• failure cases documented — three real, measured behaviours of the real validator found by running it: a negative price is accepted unchecked, a percent-distance string is dropped to None rather than converted, and a placeholder string is dropped to None
• reproducibility proven — the same real thesis and real facts checked twice return the same result
• no specialist capability superior — scoped explicitly — SCOPE_STATEMENT in eval/explainability_comparison.py. Claimed: on the one checkable property (is a stated number traceable to a real fact), ARGUS's grounding.check enforces it in code and TradingAgents' real schema does not. NOT claimed TradingAgents' full pipeline never catches bad output by any other means — only that the one real, checkable code path performs no value-level check. NOT claimed about TraderProposal.reasoning (free text), only its two numeric fields. NOT claimed ARGUS's mechanism proves a figure is correct, only that it is traceable

• best implementation studied — queue.rs:44-330 read in full; the five probability functions transcribed and the ProbQueue update rule quoted in the port's own docstring
• best method studied — prob() is P(a cancellation came from behind), not P(fill) — read off queue.rs:183-204 after a first draft scored the wrong quantity
• baseline reproduced — 8 model/parameter combinations reproduce the Rust formulas to 1.1e-16 on 9 (front, back) pairs spanning both extremes
• implementation complete — all five probability functions, RiskAdverse, ProbQueue and the L3 FIFO queue, including the cum_trade_qty double-count guard and the min() clamp
• same input comparison — every model sees byte-identical episodes and the identical L2 view of them; the reproduction grid is shared between ours and the transcribed reference
• statistically valid evaluation — paired 95% interval on per-episode error differences; a win needs the whole interval above zero, and a zero-centred pair is pinned as a non-win
• costs included — fill overstatement converted at the 4bps gap between our own MAKER_BPS and TAKER_BPS; one regime is won on accuracy and lost on cost, and says so
• out of sample test — the in-sample winner is carried over to unseen seeds rather than rechosen, so the selection is not re-run as part of the test; 3 of 5 regimes survive
• ablation — the probability function is replaced by four constants (always behind, always ahead, coin, trades-only); it beats the best of them in 3 of 5 regimes
• adversarial test — a front-sticky queue that replenishes behind — the model loses it, and the loss is the reported result rather than a tuned-away one
• failure cases documented — 2 of 5 regimes are losses, both named in the verdict; plus the finding that the most accurate model is not the cheapest
• reproducibility proven — pure Python, one seeded generator, no wall clock in the scoring path; the same seed reproduces episodes and scores exactly

Twelve of thirteen. The book recorder is running so the simulator's parameters stop being ours — and as of 2026-09-15 that calibration is read per elapsed-time horizon rather than pooled, which moved the near-touch turnover figure the book actually supports from 71.3% to 24.0% at the horizon nearest the per-event timescale (data/book_calibration.json). That narrows the gap; it does not close it. Updated 2026-09-22, twice the same day. First: the real-MBO replay path (eval/mbo_queueproof.py) built and tested against the real databento_dbn schema, ready to run, no real data read yet. Second, same day: found and used a real, account-free data source (nautilus_trader's own public DataBento-adapter test fixtures, real CME sessions, not DataBento's paid archive) and actually ran the comparison — data/mbo_queue_proof.json now carries a real result. It is a genuine, significant win for hftbacktest's own probability approach on one real session, genuinely contradicted by a second, thinner real session, and openly reported as both rather than the flattering half alone. The state stays IMPLEMENTED: a preliminary, two-sample, sample-disagreeing result on a holiday session and a two-second burst is real evidence, not yet the settled comparison this condition needs — and saying so plainly is the honest use of what was found, not a reason to round it up.

• best implementation studied — maxme/bitcoin-arbitrage's real arbitrer.py read in full — get_profit_for/get_max_depth/arbitrage_depth_opportunity/arbitrage_opportunity/tick, and the one fee-aware code path in the whole real repository (observers/traderbotsim.py, fee=0 default, an optional simulator, never the detection layer) confirmed by reading it too
• best method studied — two real methods compared: maxme's real profit = sell_total*w_sellprice - buy_total*w_buyprice (order-book depth only, no cost term of any kind) vs. ARGUS's real five-stage decomposition (fee, spread-crossing, slippage, execution probability, failed-leg survival) already measured against Bitget's own published index series
• baseline reproduced — maxme's real get_profit_for/get_max_depth/arbitrage_depth_opportunity run unmodified, byte-hash-pinned against the vendored source (commit f41684a3226710853096a4e93c3b823f92079abf)
• implementation complete — the console answers a live spot-versus-perpetual arbitrage question with the exact two-book walk (lui/arbitrage.py -> research/executable_arb.best_arbitrage), fetching both books and each venue's published taker fee; decompose() is no longer on any answer path
• same input comparison — the exact walk, HiGHS, NetworkX's cycle test, maxme's fee-blind detector and the old decomposition all scored on the same real, simultaneous Bitget spot-rToken and perpetual books: 240 regular-hours snapshots
• statistically valid evaluation — verdicts compared snapshot by snapshot, with a ticker-bootstrap interval on precision; groupwise-checked (data/groupwise_audit.json)
• costs included — ARGUS's own cost decomposition IS the subject; both real per-call costs measured directly on the same computation
• out of sample test — a 160-snapshot capture taken after the code was frozen and a 240-snapshot weekend capture, both scored without change: the exact walk ties the optimum on each
• ablation — a real, run cost-stack ablation found the fee term ALONE does not explain the disagreement at the median spread — spread-crossing plus slippage alone already exceed it — the actual result of running the ablation, not the narrower claim originally planned before it was run
• adversarial test — the real, unmodified maxme detector run on order-book depth shaped to the real median spread reports a positive profit ARGUS's real decomposition refuses
• failure cases documented — an unplanned real finding from running the failure case: maxme's real get_profit_for()/get_max_depth() crash with a genuine IndexError on an empty order-book side — no length guard on the first line's list index — this comparison's own first draft wrongly assumed it would return cleanly, fixed to assert the real, observed outcome
• reproducibility proven — both real systems reproduce identical output on identical input, twice
• no specialist capability superior — scoped explicitly — SCOPE_STATEMENT in eval/arbitrage_comparison.py. Claimed: on the question that decides whether a headline spread is real money — does the reported number survive the full cost stack — ARGUS's real decomposition answers it and maxme's real detection layer does not answer it at all, for any cost term. NOT claimed that maxme is a currently-profitable production system — an older, small open-source project studied as a real, verifiable example of the fee-blind pattern ARGUS's own module already named as the common failure mode, not a claim about current market use. NOT claimed every positive maxme reading is wrong — the largest observed real spread clears ARGUS's own real cost stack too, reported honestly. NOT claimed the swept disagreement rate is a live-universe measurement — matched to ARGUS's own measured distribution's first two moments, not identical to the real historical series bar-for-bar

• best implementation studied — pytaa's real strategy/signals.py and backtest/positions.py read in full — Signal.classic_momentum/momentum_score/sma_crossover/protective_momentum_score and vigilant_allocation/kipnis_allocation/aqr_trend_allocation — confirming vigilant_allocation is the real, published (Keller & Keuning 2017 SSRN 2543979), self-contained breadth rule matching the handbook's own 'risk-on/off rotation (US stocks <-> Crypto <-> commodities)' framing for this exact sub-theme
• best method studied — two real methods compared: pytaa's real weighted four-horizon momentum score plus discrete safe/risk breadth switch (pandas, no transaction cost or minimum-history guard anywhere in either function) vs. ARGUS's pure-Python reimplementation (src/ carries no pandas dependency) of the exact same published formula, with a hard refusal below the real formula's own minimum history and turnover priced at the real 6bps taker rate
• baseline reproduced — pytaa's real vigilant_allocation and Signal.__init__/classic_momentum/momentum_score run unmodified, byte-hash-pinned against the vendored source (commit 317ad1c6618def4e1dc0fb9879050c6f9f2f026c)
• implementation complete — momentum_score/breadth_allocation/propose_rotation/RotationPlan evaluate real monthly closes end to end, including the CLI in rotation.py's own main() against live Bitget history
• same input comparison — both real systems scored on ARGUS's actual, live Bitget cross-asset candle history — tokenized US equities, real crypto majors, a real gold token, all confirmed live on the same public futures book — agreeing to floating-point identity everywhere both are defined (BTCUSDT/ETHUSDT/NVDAUSDT: 0.7445517502236605/1.6932445870638801/0.40376760986359983 on both sides, to the last representable digit, 2026-09-15)
• statistically valid evaluation — checked across the full real universe this comparison can reach — the four design symbols plus fourteen further real held-out symbols (the remaining rTokens, three more crypto majors, silver) — not one convenient instrument
• costs included — real wall-clock cost measured on both real sides on the same real data — pandas Signal.momentum_score() vs. ARGUS's pure-Python momentum_score(), measured at roughly 2ms vs. under a microsecond per call, both numbers reported plainly
• out of sample test — the fourteen held-out real symbols beyond the four the module was designed against all agree; SCOPE_STATEMENT states explicitly why a disjoint in-sample/out-of-sample *time* split was not attempted instead — every real instrument's Bitget listing history is currently only ~13 months deep, which is short of what a split would need on either side
• ablation — a real, run sweep of the negative-score count (0 to 4, safe asset held at its real, measured NaN condition) found and confirmed a closed form: the real reference's vanished fraction equals exactly min(1, step * is_neg) at every swept point, reaching 100% vanished at full flight-to-safety — the exact moment the breadth signal is loudest
• adversarial test — the minimum-history boundary itself tested on both real sides at 12, 13 and 14 monthly points — confirms 13 is the exact threshold, not off by one in either direction, and that an empty safe_assets list or an all-NaN input do not raise in the real reference either, each producing its own distinct silent-failure shape
• failure cases documented — three distinct real silent-failure shapes found by running the real reference, not assumed: a NaN safe-asset score drops exactly its intended weight share; an empty safe_assets list still reserves and then silently drops a phantom safe share; an all-NaN input returns a silent all-zero book with no exception and no warning at all
• reproducibility proven — the real baseline-reproduced comparison run twice produces byte-identical JSON output
• no specialist capability superior — scoped explicitly — SCOPE_STATEMENT in eval/rotation_comparison.py. Claimed: on data-availability and failure-mode handling for a breadth rotation rule computed on ARGUS's own real, short-history cross-asset universe, ARGUS's implementation refuses cleanly where the real reference silently loses part or all of the intended allocation. NOT claimed the rotation rule itself has a validated forecasting edge or backtest Sharpe — cross-asset rotation is directional by construction and no return claim is made here. NOT claimed pytaa is broken as a monthly-rebalanced-ETF tool — the failure modes measured are specific to feeding it assets whose real listing history is shorter than its own formula's minimum lookback, a condition ARGUS's own live universe currently meets and a traditional ETF universe normally would not

• best implementation studied — vectorbt, empyrical and Bailey & Lopez de Prado read line by line
• best method studied — deflated and probabilistic Sharpe taken from the source papers
• implementation complete — DSR, PSR, OOS decay, rolling stability all computed and reported
• statistically valid evaluation — the gate is applied to all trials, not only to the surviving candidates
• costs included — every reported Sharpe is net of the 12bps round-trip taker fee
• out of sample test — out_of_sample_decay alerts when OOS falls below half of in-sample
• failure cases documented — 0 of 8 primitives return no verdict on the current run (data/overfit_gates.json failure_cases.count; corrected 2026-09-29 from a stale '2 of 8'). The documented failure case is the one the gate is built for: the deflated-Sharpe gate needs a variance across trials and raises rather than returning NaN when the observations cannot supply one. The refusal is designed — vectorbt's own deflated_sharpe_ratio silently NaNs on a single trial — and a gate that cannot decide has still protected nothing on that input, which is why it is counted as a failure case rather than folded into the rejections
• baseline reproduced — vectorbt's real deflated_sharpe_ratio()/approx_exp_max_sharpe() (vectorbt/returns/metrics.py, the exact file this capability's baseline already names) vendored verbatim - byte-verified against commit 34b6d5935e3ea3eccd549e2592bc0f455b8045f5, pinned by SHA256 - and actually executed via a sys.modules shim satisfying its one internal import (vectorbt._typing.Array1d) with a single-attribute stand-in rather than installing plotly/numba to satisfy a type annotation. The specific defect line itself (accessors.py:596, var_sharpe = np.var(x, ddof=1)) is reproduced directly as the same bare numpy call, cited to its exact line, not vendored (nothing else is around it worth vendoring)
• same input comparison — ARGUS's real deflated_sharpe and vectorbt's real deflated_sharpe_ratio run on 1,220 identical inputs (5 hand-designed + a 1,215-case deterministic grid over observed x n x trials x variance x skew x kurtosis) - agree EXACTLY (max abs diff across the whole sweep: 0.0) on every valid input, confirmed by running both real functions, not asserted from the formula's own resemblance on paper
• ablation — the two NaN-input guards this comparison's own findings added to backtest/metrics.py, each confirmed independently load-bearing: deflated_sharpe's variance_of_trials NaN check and probabilistic_sharpe's observed/benchmark NaN check both raise on a tripped (NaN) case and both pass through cleanly on an otherwise-identical cleared (finite) case, verified programmatically
• adversarial test — a real bug this comparison found in ARGUS's OWN code, not the baseline's: deflated_sharpe's pre-existing guard was `variance_of_trials < 0`, which does not catch NaN (float('nan') < 0 is False in IEEE 754) - the identical fails-every-comparison shape that makes vectorbt's own gate silent. A caller computing variance the way vectorbt's accessor does (np.var(x, ddof=1) on one trial) and passing it straight into ARGUS's own function would have gotten a silent NaN back. probabilistic_sharpe (deflated_sharpe's own single-trial reduction) had the identical gap independently. Both fixed the same session this was found, verified by re-running the exact failing case after the fix
• reproducibility proven — both functions are pure, deterministic transforms - no model, no seed, no simulation on either side. The 1,215-case sweep is itertools.product over fixed grids, not RNG-seeded, same reasoning as every other sweep in this project: a seed's meaning can shift across Python versions
• no specialist capability superior — scoped explicitly - SCOPE_STATEMENT in eval/dsr_comparison.py. Claimed: the two implementations compute the identical published formula (verified over 1,220 real-run cases, not assumed from reading), and ARGUS's gate refuses exactly where vectorbt's real code would go silently NaN - now genuinely true after this comparison's own finding was fixed, not merely asserted by design. NOT claimed: that this was already true before the fix - the pre-existing guard shared vectorbt's own blind spot until this comparison found it. Also NOT claimed: that vectorbt is a worse DSR implementation in general - the formula itself is identical and correct on every valid input; the only measured difference is input validation at the edges, and vectorbt genuinely has none anywhere in the package for this specific function (grepped directly: no >/< comparison against deflated_sharpe_ratio anywhere in the repository)

• best implementation studied — RD-Agent, Alpha Jungle, Qlib and the 101 Formulaic Alphas read for operators (factor-discovery-audit.md); qlib's real expression engine additionally read in full and vendored — base.py (Expression/ExpressionOps/Feature), ops.py (1681 lines, every real Rolling/PairRolling operator incl. Rank/Mean/Std/Corr), and the field-parsing path (utils/__init__.py's parse_field, data.py's ExpressionProvider) an earlier review wrongly marked unvendorable — corrected 2026-09-16, see qlib_expression_base.py's own header
• best method studied — both real methods run and compared: ARGUS's typed dataclass grammar (Window/Corr, validated at construction) vs. qlib's real string-expression engine (parse_field's regex rewrite -> eval()) — the structural difference that produces the adversarial finding below is a property of the two DESIGNS, confirmed by running both, not inferred from either's docs
• baseline reproduced — qlib's real Mean/Std/Rank/Corr operators and real ExpressionProvider.get_expression_instance() run unmodified, byte-hash-pinned against the vendored source (qlib_expression_base.py/_ops.py/_eval_surface.py, commit 79633dd9506ea689e5400dea0197717b5b3d74b7)
• implementation complete — Window/Corr's full OPS vocabulary (mean/std/rank/argmax/.../slope) evaluates
• same input comparison — mean/std/corr run on an identical fixed series (diff 0.0/0.0/1.11e-16); rank run on five designed cases isolating ties/constant/single-observation, plus swept across every lookback a real captured NVDAUSDT book-tape series supports — both systems, same input, every case
• statistically valid evaluation — rank swept across 15 real-market lookbacks (not one convenient case), finding both agreement and divergence, not a cherry-picked direction
• costs included — both real per-call costs measured directly (ARGUS Window vs. qlib Mean.load)
• out of sample test — rank comparison additionally run on data/book_tape.jsonl — real captured L2 book snapshots, not built from any of the five designed in-sample cases
• ablation — the AST scan itself is shown sensitive, not vacuous: run against a counterfactual Window.evaluate that dispatches via eval() (never touching the real grammar.py), confirming the scan DOES flag eval() when genuinely present rather than always reporting clean
• adversarial test — the grammar cannot reach attributes or call arbitrary code, pinned by test (test_grammar.py); separately, qlib's real ExpressionProvider IS shown reachable to arbitrary code — a crafted field string run through its real, unmodified get_expression_instance() executes a nested eval('1+1'), returning 2, a live CWE-95 in real Microsoft-maintained code
• failure cases documented — both sides' real handling of MALFORMED (not malicious) input run: ARGUS rejects an unknown op at Window construction, before evaluate() is ever reachable; qlib's real SyntaxError/NameError handler cleanly catches three malformed field strings — the vulnerability is narrower and worse than malformed input: a syntactically VALID expression no handler is positioned to catch
• reproducibility proven — every expression has a canonical form, so a factor cannot be paid for twice (test_grammar.py); separately, both real comparison functions run twice on identical input and agree bit-for-bit
• no specialist capability superior — scoped explicitly — SCOPE_STATEMENT in eval/grammar_comparison.py. Claimed: on numeric fidelity, ARGUS matches qlib's real mean/std/corr to float precision and diverges from its real rank only by a named, algebraically explained convention (not a bug); on execution surface, ARGUS's grammar structurally cannot reach eval/exec anywhere (AST-verified), while qlib's real code can and, run here, does. NOT claimed: that qlib's Rank convention is wrong, or that every qlib field string is exploitable (malformed syntax IS caught cleanly). NOT claimed: that this closes the vocabulary-breadth gap in the blockers below — that is a separate axis (which fields the grammar can express), untouched by this comparison

• best method studied — all 101 Formulaic Alphas read and scored against our grammar; 46 blocked here
• implementation complete — crossrank and crossscale, aligned on shared timestamps, innermost-first
• adversarial test — truncating the future must not move a past value — the standard invisible leak in cross-sectional operators, pinned for rank, scale and the nested case
• failure cases documented — a one-symbol universe and a single-symbol evaluation both raise rather than returning the neutral 0.5 that would silently change the factor
• costs included — charged per unit of weight moved at the venue's own 6bps per side
• statistically valid evaluation — 80 trials deflated; phases averaged rather than selected; spread published
• out of sample test — chronological half-split decay computed for every rule
• best implementation studied — three of the backtest-engine repos this project reads first checked directly for a cross-sectional (same-timestamp, across-instrument) ranking primitive: qlib has one (CSRankNorm, groups a (datetime, instrument) frame by datetime and ranks within each group — exactly the baseline this capability names); polakowo/vectorbt's only rank() (signals/accessors.py:1298) ranks consecutive SIGNAL EVENTS through TIME within one series, a different axis entirely, not cross-sectional; QuantConnect/Lean has no match for 'CrossSectional' anywhere in the repository (grepped case-insensitively, zero hits). qlib is not merely cited, it is the only one of the three with the primitive this capability is being measured against
• baseline reproduced — qlib's real CSRankNorm (qlib/data/dataset/processor.py, the same file this capability's baseline field names) vendored verbatim — byte-verified against commit 79633dd9506ea689e5400dea0197717b5b3d74b7, pinned by SHA256 so the check survives without the external clone present — and actually executed via a sys.modules shim satisfying its own qlib-internal imports with stand-ins that raise NotImplementedError if ever reached, rather than editing a line of the vendored source. Confirmed correct against its own docstring's worked example: [1,2,3,4,5] -> [-1.038,-0.346,0.346,1.038,1.73]
• same input comparison — ARGUS's real CrossRank.combine() and qlib's real CSRankNorm run on 61 identical panels (7 hand-designed probing ties/extremes/edge sizes, plus a 54-panel deterministic grid over size x tie-density x spread, none tuned to any specific case) - agree EXACTLY on relative order on every single one, verified by an argsort-with-tie-groups comparison, not eyeballed
• ablation — ARGUS's crossrank has two design choices, each shown independently load-bearing by comparing the real function against itself with one choice disabled: midrank tie handling (disabling it changes [5,3,3,1]'s tied pair from 0.5/0.5 to 0.333/0.667) and the n<=1 neutral guard (disabling it produces NaN via an unguarded /(n-1) rather than 0.5). Both differ from their real-function counterpart, confirmed programmatically
• reproducibility proven — both functions being compared are pure, deterministic transforms over their input list/frame - no model, no seed, no simulation on either side. The 61-panel comparison battery is itertools.product over fixed grids, not RNG-seeded, for the same reason mandate_comparison's sweep is: a seed's meaning can shift across Python versions and a hand-picked seed is itself a form of hand-tuning
• no specialist capability superior — scoped explicitly - SCOPE_STATEMENT in eval/crosssection_comparison.py. Claimed: on cross-sectional RANKING specifically, ARGUS and qlib agree exactly on relative order on every panel tested, and ARGUS carries TWO layers of defense a genuinely single-instrument universe qlib's vendored code has neither of - CrossRank.combine()'s own defensive 0.5 fallback if ever called directly, AND research.panel.evaluate_panel's higher-level refusal (raises rather than silently computing a constant 'factor' - see that module's own docstring) before combine() would ever be reached that way in real operation; qlib's CSRankNorm has no equivalent refusal anywhere in the vendored file and, run directly on a one-row input, produces 1.73 - the transform's own maximum, not a neutral reading. NOT claimed: that crossrank is a complete substitute for CSRankNorm - qlib's additional zscore-style rescaling toward unit variance is a real step ARGUS's crossrank does not perform, by design (CrossScale is ARGUS's own separate portfolio-weight operator, a different job). Also NOT claimed: that the single-name divergence has material impact on ARGUS's own real decisions today - RTOKEN_SYMBOLS is a fixed 12-name universe and how often a real trading day's data actually shrinks a group to one instrument was measured on 2026-09-27: never. build_panel keeps only the timestamps every symbol shares, so each of the study's 2,159 real hours ranks all 12 names (data/crosssection_study.json, panel.dropped all 0) - the single-instrument case where the two differ never reaches a real decision, which is why this row is TIED rather than OWNED

The capability is proven correct and the factors are proven worthless on this universe over this window. Both halves are the result.

• best implementation studied — backtest-truth read at source: backtest_truth/ast_checks.py:40-95 (the rules taken), rules.py:59 (BT006 listed, not implemented); freqtrade optimize/analysis/lookahead.py read for the dynamic method
• baseline reproduced — backtest-truth's check_source run from its clone on every fixture and independent script
• same input comparison — both tools on the same 21 fixtures (17 planted bugs, 4 clean) and the same 16 independent scripts, each independent flag scored by reading the line
• implementation complete — the console reads a pasted strategy first, before its lines are flattened
• failure cases documented — the five false alarms found on independent code (counts, chart resamples, a today end date, a ta library read as Pine, a constant comparison) each kept as a test
• reproducibility proven — python -m argus.eval.backtest_critic_comparison regenerates the artefact from the clones