{"by_state": {"lost": 8, "tied": 37, "implemented": 0, "owned": 6}, "capabilities": [{"name": "Deliberation priced as a trading cost", "state": "lost", "track": "Track 2", "subtheme": "Judged: Agent architecture quality", "rival": "HaoKang-Timmy/LatencySensitiveBench (arxiv 2505.19481, NeurIPS 2025)", "tested": "both real cost functions called on the identical delay values — including ARGUS's own three real, bake-off-measured THINKING_MS delays (3s/8s/40s), not synthetic numbers picked to favour either side", "conditions_met": 11, "artefacts": ["data/deliberation_comparison.json"], "tests": ["test_deliberation_comparison.py", "test_meta_pm.py", "test_cost.py"], "rerun": "python -m argus.eval.deliberation_comparison", "console": null}, {"name": "Abstention scored as a decision", "state": "tied", "track": "Track 2", "subtheme": "Judged: risk control layer effectiveness", "rival": "fd-shifts (c4467aec, Apache-2.0), the general selective-prediction evaluator, run unmodified on the same leans; and insaneamogh/AutonomousTradeAgents 'Ghost P&L' ledger, found 2026-09-16 by a fresh, targeted search…", "tested": "both real systems scored on the structurally identical adversarial scenario — a large avoided loss and a large blocked/missed upside that roughly offset — including the EXACT dollar figures ($30,788 / $32,967) the baseline's own code comment names", "conditions_met": 11, "artefacts": ["data/paper_ledger.jsonl"], "tests": ["test_abstention_comparison.py", "test_ledger.py"], "rerun": "python -m argus.eval.abstention_comparison", "console": {"question": "why did you do nothing all weekend", "book": ""}}, {"name": "Overfitting gates that raise instead of returning NaN", "state": "tied", "track": "Track 1", "subtheme": "Factor research (validation machinery)", "rival": "polakowo/vectorbt deflated_sharpe_ratio (accessors.py:596) - not itself gated anywhere in vectorbt, but silently NaN on a single trial (var_sharpe = np.var of one value, ddof=1); a caller filtering with a negated <…", "tested": "ARGUS's real deflated_sharpe and vectorbt's real deflated_sharpe_ratio run on 1,220 identical inputs (5 hand-designed + a 1,215-case deterministic grid over observed x n x trials x variance x skew x kurtosis) - agree EXACTLY (max abs diff across the whole sweep: 0.0) on every valid input, confirmed by running both…", "conditions_met": 11, "artefacts": ["data/track1_study.json", "data/overfit_gates.json"], "tests": ["test_metrics.py", "test_baselines.py", "test_dsr_comparison.py"], "rerun": "python -m argus.eval.dsr_comparison", "console": null}, {"name": "Typed factor grammar with no execution surface", "state": "tied", "track": "Track 1", "subtheme": "Factor research (validation machinery)", "rival": "Google CEL (cel-expr-python 0.1.3) for safe static typing, Polars 1.44.2 for evaluation speed, and microsoft/qlib's real expression engine (qlib/data/base.py, ops.py, data.py, utils) as the named rival", "tested": "mean/std/corr run on an identical fixed series (diff 0.0/0.0/1.11e-16); rank run on five designed cases isolating ties/constant/single-observation, plus swept across every lookback a real captured NVDAUSDT book-tape series supports — both systems, same input, every case", "conditions_met": 11, "artefacts": [], "tests": ["test_baselines.py", "test_grammar.py", "test_grammar_comparison.py"], "rerun": "python -m argus.eval.grammar_comparison", "console": null}, {"name": "Factor-discovery safety: no execution surface and trial-corrected selection, vs. RD-Agent", "state": "tied", "track": "Track 2", "subtheme": "Factor Discovery Agent", "rival": "microsoft/RD-Agent real factor-implementation pipeline (FBWorkspace, factor_coder) for the execution surface, and vectorbt's deflated_sharpe_ratio (Bailey and Lopez de Prado) for the trial correction", "tested": "the identical crafted, attacker-shaped factor.py payload construct is put to both real systems: RD-Agent's real execute() accepts and runs it;", "conditions_met": 11, "artefacts": [], "tests": ["test_baselines.py", "test_searchoff.py", "test_rdagent_comparison.py"], "rerun": "python -m argus.eval.rdagent_comparison", "console": null}, {"name": "Cross-sectional factor evaluation", "state": "tied", "track": "Track 1", "subtheme": "Factor research (validation machinery)", "rival": "microsoft/qlib, which groups a (datetime, instrument) frame by datetime and ranks within each group", "tested": "ARGUS's real CrossRank.combine() and qlib's real CSRankNorm run on 61 identical panels (7 hand-designed probing ties/extremes/edge sizes, plus a 54-panel deterministic grid over size x tie-density x spread, none tuned to any specific case) - agree EXACTLY on relative order on every single one, verified by an…", "conditions_met": 11, "artefacts": ["data/crosssection_study.json"], "tests": ["test_panel.py", "test_crosssection.py", "test_crosssection_comparison.py", "test_baselines.py"], "rerun": "python -m argus.eval.crosssection_comparison", "console": null}, {"name": "Cross-market cointegration with corrected multiple testing", "state": "lost", "track": "Track 1", "subtheme": "Cross-Market Correlation Strategies", "rival": "statsmodels adfuller; QuantConnect/Lean pearsonr pair ranking; FinceptTerminal", "tested": "ARGUS's real adf() and the real statsmodels adfuller run on identical series across three shapes (random walk, stationary AR(1), trending); the reimplemented and real FinceptTerminal formulas verified to agree exactly on identical input before either is used further;", "conditions_met": 11, "artefacts": [], "tests": ["test_baselines.py", "test_cointegration.py", "test_cointegration_comparison.py"], "rerun": "python -m argus.eval.cointegration_comparison", "console": null}, {"name": "Net executable arbitrage vs. a fee-blind detector", "state": "tied", "track": "Track 1", "subtheme": "Arbitrage", "rival": "maxme/bitcoin-arbitrage real profit-detection core (get_profit_for)", "tested": "the exact walk, HiGHS, NetworkX's cycle test, maxme's fee-blind detector and the old     decomposition all scored on the same real, simultaneous Bitget spot-rToken and perpetual     books: 240 regular-hours snapshots", "conditions_met": 11, "artefacts": ["data/general_arb_comparison.json"], "tests": ["test_baselines.py", "test_arbitrage_answer.py", "test_general_arb_comparison.py", "test_arbitrage_comparison.py"], "rerun": "python -m argus.eval.arbitrage_comparison", "console": null}, {"name": "Data-honest cross-asset breadth rotation vs. a silently-dropping reference", "state": "tied", "track": "Track 1", "subtheme": "Cross-Asset Allocation / Rotation", "rival": "pytaa real VAA breadth rule (vigilant_allocation) + Signal.momentum_score", "tested": "both real systems scored on ARGUS's actual, live Bitget cross-asset candle history — tokenized US equities, real crypto majors, a real gold token, all confirmed live on the same public futures book — agreeing to floating-point identity everywhere both are defined (BTCUSDT/ETHUSDT/NVDAUSDT:…", "conditions_met": 13, "artefacts": [], "tests": ["test_baselines.py", "test_rotation.py", "test_rotation_comparison.py"], "rerun": "python -m argus.eval.rotation_comparison", "console": null}, {"name": "Clustering-corrected, base-rate-honest event significance vs. a fixed-null test", "state": "tied", "track": "Track 2", "subtheme": "Event-Driven Agent", "rival": "whale-signals real fixed-null hit-rate test (compute_hit_rates/compute_base_rate)", "tested": "both real systems scored on the same real, live Bitget ETHUSDT/BTCUSDT hourly candle history, with placebo whale-event timestamps drawn from the same real series, carrying zero true informational edge by construction — the real 24h base rate (measured live, whale-signals' own real function) departs from 50%, and the…", "conditions_met": 13, "artefacts": [], "tests": ["test_baselines.py", "test_eventdriven_comparison.py"], "rerun": "python -m argus.eval.eventdriven_comparison", "console": {"question": "how does NVDA react to CPI", "book": ""}}, {"name": "Funding-aware cross-asset hedge routing vs. a fee-blind composite router", "state": "tied", "track": "Track 2", "subtheme": "Cross-Asset Execution Agent", "rival": "crypto_sor real composite order-book router (CompositeOrderBook.newOrder)", "tested": "both real systems scored on the same real, live Bitget order-book slippage and funding rate for a real rToken leg (NVDAUSDT) and a real crypto leg (BTCUSDT) — agreeing at zero holding time and diverging past a real, computed break-even horizon", "conditions_met": 11, "artefacts": [], "tests": ["test_baselines.py", "test_execution.py", "test_execution_comparison.py"], "rerun": "python -m argus.eval.execution_comparison", "console": {"question": "hedge my crypto after a macro shock", "book": "60% BTC, 40% ETH"}}, {"name": "Refusal-first earnings surprise ranking vs. a silently-exploding factor", "state": "owned", "track": "Track 2", "subtheme": "Earnings-Driven Trading Agent", "rival": "QuantConnect/Tutorials real SUE factor (FineSelectionAndSueSorting)", "tested": "every method on the same SEC filings: 7,258 filers across the design and holdout frames for pandas' period index and the positional pairing, and one seeded companyfacts sample of 309 filers for edgartools 5.59.1 run offline (data/general_sue_comparison.json, data/edgartools_real_check.json).", "conditions_met": 13, "artefacts": [], "tests": ["test_baselines.py", "test_sue.py", "test_earnings_comparison.py"], "rerun": "python -m argus.eval.earnings_comparison", "console": {"question": "did NVDA beat last quarter", "book": ""}}, {"name": "Per-profile mandate that changes the verdict", "state": "owned", "track": "Track 3", "subtheme": "Personalized Research Workbench", "rival": "hkuds/vibe-trading agent/src/live/enforcement.py:458-609, the only one of 16 systems with a per-user mandate that changes a verdict on identical state and is tested", "tested": "two profiles, one market frame, different verdicts (intra-ARGUS).", "conditions_met": 13, "artefacts": ["data/profile_divergence.json", "data/mandate_comparison.json"], "tests": ["test_mandate.py", "test_personalisation.py", "test_profilestudy.py", "test_mandate_comparison.py", "test_baselines.py"], "rerun": "python -m argus.eval.mandate_comparison", "console": {"question": "I'm an aggressive trader, should I buy 10% COIN", "book": ""}}, {"name": "Episodic memory across decisions", "state": "tied", "track": "Track 2", "subtheme": "Judged: Agent architecture quality", "rival": "TauricResearch/TradingAgents reflection log, which stores a model's prose lesson rather than a graded outcome", "tested": "identical fixtures run through both ARGUS's real recall() and TradingAgents' real TradingMemoryLog across the resolution boundary.", "conditions_met": 11, "artefacts": [], "tests": ["test_panel_parallel.py", "test_recall.py", "test_recall_comparison.py", "test_baselines.py"], "rerun": "python -m argus.eval.recall_comparison", "console": {"question": "why did you pass on NVDA", "book": ""}}, {"name": "Risk layer proved by domain sweep", "state": "owned", "track": "Track 2", "subtheme": "Judged: risk control layer effectiveness", "rival": "nautechsystems/nautilus_trader risk engine, QuantConnect brokerage models, freqtrade's MaxDrawdown/StoplossGuard/LowProfitPairs/CooldownPeriod protections", "tested": "ARGUS's circuit breaker and freqtrade's ported protections walked over the SAME real Bitget trade sequences at two scopes — per-symbol (compare_real_symbols) and whole-book (compare_combined_book)", "conditions_met": 13, "artefacts": ["data/risk_proof.json", "data/risk_layer_comparison.json"], "tests": ["test_freqtrade_baseline.py", "test_gate_ablation.py", "test_riskproof.py", "test_risk_layer_comparison.py"], "rerun": "python -m argus.eval.riskproof", "console": {"question": "what did the risk layer block", "book": ""}}, {"name": "Pre-registered trading protocol, hash-committed", "state": "tied", "track": "Track 2", "subtheme": "Judged: Agent architecture quality", "rival": "serenity-guardrails (Apache-2.0) hash-chained journal with a head anchor", "tested": "both real systems run on identical realistic 2-entry fixtures for the CRLF case, the tamper case, and the truncation case", "conditions_met": 11, "artefacts": ["data/protocol_commitments.jsonl", "data/journal_comparison.json"], "tests": ["test_journal_comparison.py", "test_protocol.py"], "rerun": "python -m argus.eval.journal_comparison", "console": {"question": "is the log tamper-evident", "book": ""}}, {"name": "Perception layer: what the desk can see", "state": "lost", "track": "Track 3", "subtheme": "Judged: data sources and Skill integration", "rival": "OpenBB-finance/OpenBB provider set; TauricResearch/TradingAgents feed list", "tested": "both real systems run on the identical failure shape — a core-equivalent source with every path broken, and an optional-equivalent source with its one path broken — TradingAgents raises on the former, ARGUS returns cleanly on both", "conditions_met": 11, "artefacts": ["data/desk_notes.jsonl", "data/perception_breadth.json", "data/feedlist_comparison.json", "data/skill_reliability.json"], "tests": ["test_feedlist_comparison.py", "test_evidence.py"], "rerun": "python -m argus.eval.feedlist_comparison", "console": {"question": "/status", "book": ""}}, {"name": "Sentiment integrity: resistance to coordinated posting, vs. finBERT", "state": "tied", "track": "Track 2", "subtheme": "Market Sentiment Agent", "rival": "ProsusAI/finBERT for classification; BloombergGPT and FinMA for the published bar", "tested": "both real systems run on identical symbol-matched coordinated-posting scenarios (N near-identical rephrasings of one unsourced claim) built from the same underlying narrative", "conditions_met": 13, "artefacts": ["data/sentiment_comparison.json", "data/sentiment_integrity_real.json"], "tests": ["test_sentiment_comparison.py"], "rerun": "python -m argus.eval.sentiment_comparison", "console": {"question": "is the hype on NVDA real", "book": ""}}, {"name": "Crowd sentiment classification, vs. RoB-RT on TweetEval", "state": "lost", "track": "Track 2", "subtheme": "Market Sentiment Agent", "rival": "RoB-RT (TweetEval, Barbieri et al. 2020), its published test predictions re-scored on the same tweets; finBERT and VADER beside it", "tested": "all 12,284 TweetEval sentiment test tweets, every scorer on the same items", "conditions_met": 7, "artefacts": ["data/sentiment_tweeteval.json"], "tests": ["test_sentiment_tweeteval.py"], "rerun": "", "console": null}, {"name": "Self-evolving review rules", "state": "tied", "track": "Track 3", "subtheme": "Review & Self-Evolution", "rival": "TauricResearch/TradingAgents reflection memory", "tested": "a decision with a demonstrably WRONG outcome (a real -15% realised return) stored via the real vendored TradingMemoryLog and read back later: the reflection describing the wrong call is re-injected identically to one about a right call, confirmed by running the real code twice, once each way", "conditions_met": 11, "artefacts": ["data/review_report.json", "data/review_oos.json"], "tests": ["test_review.py", "test_review_comparison.py", "test_baselines.py"], "rerun": "python -m argus.eval.review_comparison", "console": {"question": "what bad decision patterns do you have", "book": ""}}, {"name": "Path-shape matching with a calibrated null", "state": "lost", "track": "Track 3", "subtheme": "Decision Stress Testing", "rival": "TDAmeritrade/stumpy real matrix-profile distance and matrix-profile self-join", "tested": "both real distance functions fed the identical five scalar inputs (computed the ordinary way from the same z-normalised windows) on a general case and both constant-window edge cases; both real search functions run on the identical causally-truncated series for every swept query point", "conditions_met": 11, "artefacts": [], "tests": ["test_baselines.py", "test_shapematch.py", "test_shapematch_comparison.py"], "rerun": "python -m argus.eval.shapematch_comparison", "console": {"question": "has NVDA been here before", "book": ""}}, {"name": "Session-aware execution that refuses to solve through a boundary", "state": "lost", "track": "Track 3", "subtheme": "Execution Assistance", "rival": "nkaz001/hftbacktest queue model; Bitget's own instrument rules; Almgren & Chriss (2000) closed-form optimum vs nautechsystems/nautilus_trader's real TwapAlgorithm as the field's stated default", "tested": "trajectory() called twice on the IDENTICAL real ImpactParameters — once at the caller's risk_aversion, once at 0 — after finding that calling twap() directly scores the straight line under a fictional zero-impact market", "conditions_met": 13, "artefacts": ["data/schedule_comparison.json"], "tests": ["test_schedule.py", "test_guard.py", "test_schedule_comparison.py"], "rerun": "python -m argus.eval.schedule_comparison", "console": {"question": "how should I split a $250k order in NVDA", "book": ""}}, {"name": "Queue-position modelling ported from hftbacktest and measured against it", "state": "tied", "track": "Track 2", "subtheme": "Judged: execution realism behind the paper record", "rival": "nkaz001/hftbacktest, backtest/models/queue.rs", "tested": "every model sees byte-identical episodes and the identical L2 view of them; the reproduction grid is shared between ours and the transcribed reference", "conditions_met": 10, "artefacts": ["data/queue_proof.json"], "tests": ["test_queueproof.py", "test_queue.py"], "rerun": "", "console": null}, {"name": "Portfolio allocation, measured against Riskfolio-Lib's NCO", "state": "tied", "track": "Track 3", "subtheme": "Open Theme: portfolio tools", "rival": "dcajasn/Riskfolio-Lib 7.3.0 (BSD-3), NCO (Nested Clustered Optimization); cvxportfolio 1.5.1 (GPL-3.0) checked separately for the convex-rebalance path", "tested": "all twelve allocators (including ARGUS's own NCO, added 2026-09-22) run on the SAME 60-day hourly Bitget candle history for the SAME 12-symbol rToken universe and the SAME sample covariance", "conditions_met": 6, "artefacts": ["data/allocation_comparison.json", "data/nco_bakeoff.json"], "tests": ["test_allocation_comparison.py"], "rerun": "", "console": null}, {"name": "Regime-boundary detection, measured against stumpy FLUSS and ruptures", "state": "tied", "track": "Track 3", "subtheme": "Decision Stress Testing", "rival": "TDAmeritrade/stumpy (stump+fluss, real public API); deepcharles/ruptures (Pelt/KernelCPD/Dynp); the incumbent two-line volatility rule in strategies/track1_suite.py's own rotation_regime_switch", "tested": "ARGUS's offline FLUSS, real stumpy, real ruptures and the real incumbent rule all run on the SAME 60-day hourly Bitget market series for all 12 rTokens, plus all FOUR budgeted methods (FLUSS, stumpy, ruptures, and ARGUS's second segmenter exact_partition added 2026-09-22) run on the SAME 100 synthetic trials…", "conditions_met": 4, "artefacts": ["data/regime_comparison.json"], "tests": [], "rerun": "", "console": null}, {"name": "LUI intent routing, measured against Rasa's real DIET classifier", "state": "tied", "track": "Track 3", "subtheme": "Judged: LUI fluency", "rival": "RasaHQ/rasa 3.6.21 (Apache-2.0), DIET classifier, default pipeline config", "tested": "both systems scored on the identical 293-row sealed split and the identical 14 out-of-scope probes;", "conditions_met": 5, "artefacts": ["data/lui_comparison.json"], "tests": ["test_lui_comparison.py"], "rerun": "", "console": {"question": "wut abt nvda earnigns when", "book": ""}}, {"name": "Numeric decision grounding vs. TradingAgents' real, unchecked TraderProposal", "state": "tied", "track": "Track 2", "subtheme": "Judged: decision explainability", "rival": "TradingAgents' real, unmodified TraderProposal (agents/schemas.py)", "tested": "both real systems checked on the same real, live Bitget current price (and the same fabricated entry_price) for the same real symbols — TradingAgents' real validator and ARGUS's real grounding.check, not a re-implementation of either", "conditions_met": 11, "artefacts": [], "tests": ["test_explainability_comparison.py", "test_grounding.py"], "rerun": "python -m argus.eval.explainability_comparison", "console": {"question": "why did you pass on NVDA", "book": ""}}, {"name": "Structured filing extraction vs. FinanceBench's real, published LLM measurement", "state": "tied", "track": "Track 3", "subtheme": "Information Extraction & Signal Generation", "rival": "FinanceBench's own real, published metrics-generated results (Patronus AI, arXiv 2311.11944, 16 real model/retrieval-condition combinations graded Correct/Incorrect/Refusal by human review)", "tested": "both real systems scored on the identical task class — a single numeric line item pulled from a real SEC filing — with ARGUS run on a freshly designed, real, live case set rather than FinanceBench's own unlicensed question set", "conditions_met": 11, "artefacts": [], "tests": ["test_infoextract_comparison.py", "test_fundamentals.py"], "rerun": "python -m argus.eval.infoextract_comparison", "console": {"question": "what was NVDA's revenue last quarter", "book": ""}}, {"name": "Point-in-time correctness vs. OpenBB's real, ungated live-API agent", "state": "owned", "track": "Track 3", "subtheme": "Personalized Research Workbench", "rival": "OpenBB ODP (openbb_core, the platform openbb-mcp-server wraps and Agent Rita reads), with edgartools FactQuery.as_of as the best general-purpose tool and microsoft/qlib's PIT store as the category's source; OpenBB's…", "tested": "the identical scenario put to every real system: a research query anchored to a historical point in time.", "conditions_met": 13, "artefacts": ["data/pit_rivals.json"], "tests": ["test_workbench_comparison.py", "test_fundamentals.py", "test_pit.py"], "rerun": "python -m argus.eval.workbench_comparison", "console": {"question": "what was NVDA's net income as of 1 March 2026", "book": ""}}, {"name": "Market sentiment", "state": "tied", "track": "Track 2", "subtheme": "Market Sentiment Agent", "rival": "Santiment crowd-sentiment contrarian signals; Augmento; Bitget's own bitget-signal sentiment-analyst Skill; TauricResearch/TradingAgents v0.4 sentiment analyst; the S2 entry Jfash-cmd/sentinel-bitget-hackathon", "tested": "the desk's sentiment analyst, VADER and finBERT on the identical social evidence of each settled decision, scored on the same realised move net of the same round trip", "conditions_met": 1, "artefacts": ["data/sentiment_signal_comparison.json"], "tests": ["test_sentiment_signal_comparison.py"], "rerun": "python -m argus.eval.sentiment_signal_comparison", "console": {"question": "is the hype on NVDA real", "book": ""}}, {"name": "Factor Discovery Agent: hypothesis to tradable factor", "state": "tied", "track": "Track 2", "subtheme": "Factor Discovery Agent", "rival": "minihellboy/FactorMiner (arXiv 2602.14670); bigcan/sharpen Crucible; rookiewu417/FactorZen; QuantaAlpha/QuantaAlpha; microsoft/RD-Agent fin_factor", "tested": "both sides on the same frozen 90-day bars (data/factor_split_half_bars.json, sha256 checked on load) through the same cost, out-of-sample and split-half gates;", "conditions_met": 4, "artefacts": ["data/factor_quality_rivals.json"], "tests": ["test_factor_quality_rivals.py"], "rerun": "", "console": null}, {"name": "Analogue stress bands vs. AnalogDesk (S2) on its own pre-registered test", "state": "tied", "track": "Track 3", "subtheme": "Decision Stress Testing", "rival": "lixinde586-afk/analogdesk", "tested": "ARGUS answers their exported queries from the same adjusted closes, truncated and embargoed at each query session", "conditions_met": 6, "artefacts": ["data/analogstress_comparison.json"], "tests": [], "rerun": "python -m argus.eval.analogstress_comparison", "console": {"question": "has NVDA been here before", "book": ""}}, {"name": "Portfolio copilot: post-trade beta and co-movement vs. weekend-copilot (S2)", "state": "tied", "track": "Track 3", "subtheme": "Open Theme: portfolio tools", "rival": "PhiBao/weekend-copilot @461ad820 (MIT)", "tested": "both answer the same 1,800 books and proposed adds at the same origins; both are scored on Bitget daily, native daily and open-session hourly targets", "conditions_met": 6, "artefacts": ["data/copilot_rivals.json"], "tests": [], "rerun": "", "console": {"question": "I hold 60% NVDA and 40% AAPL, should I add 20% TSLA?", "book": ""}}, {"name": "Portfolio stress: the book's move when QQQ falls, vs. skfolio's vine copula and Entropy Pooling", "state": "tied", "track": "Track 3", "subtheme": "Open Theme: portfolio tools", "rival": "skfolio/skfolio 1.3.1 (BSD-3) VineCopula conditional sampling and EntropyPooling, run unmodified in their own interpreter on the same Bitget daily history", "tested": "every arm gets the same realised QQQUSDT move and the same traded-day history on 39 days with QQQUSDT down 1% or more, 200 books each", "conditions_met": 6, "artefacts": ["data/copilot_stress.json"], "tests": [], "rerun": "", "console": {"question": "I hold 60% NVDA and 40% AAPL, should I add 20% TSLA?", "book": ""}}, {"name": "Order splitting on realised cost vs. Bitget's own TWAP, on a full-depth replay", "state": "tied", "track": "Track 3", "subtheme": "Execution Assistance", "rival": "Bitget's native TWAP (equal market slices at a 60-second interval, its published spec), the even eight-slice TWAP every execution tool ships, and trading at once", "tested": "every arm walks the same rebuilt book at the same instants; 80 parents per size, buy and sell, $5k to $250k", "conditions_met": 4, "artefacts": ["data/execution_arena.json"], "tests": [], "rerun": "", "console": {"question": "how should I split a $100k order in NVDA", "book": ""}}, {"name": "Overnight hedge for an rToken holder vs. Ballast (S2)", "state": "tied", "track": "Track 3", "subtheme": "Open Theme: portfolio tools", "rival": "Ritapossible/Ballast @5cf6759 (MIT)", "tested": "both scored on Ballast's own per-night series, 11 names, its 70/30 split", "conditions_met": 6, "artefacts": ["data/copilot_hedge.json"], "tests": [], "rerun": "", "console": {"question": "I hold RNVDAUSDT, how do I protect it over the weekend?", "book": ""}}, {"name": "A trader's thesis tested on the same input as optic-bitget's desk", "state": "tied", "track": "Track 3", "subtheme": "Personalized Research Workbench", "rival": "neromtoobad/optic-bitget (Season 2, MIT): evidence table, Bull/Bear debate twice, Judge sampled three times, probability capped to 10-90%, run end to end", "tested": "the same three sentences ('Long MSTR/NVDA/TSLA perp into earnings — funding looks cheap') through both desks the same morning", "conditions_met": 5, "artefacts": ["data/h2h_optic/summary.json"], "tests": ["test_claim_check.py"], "rerun": "", "console": null}, {"name": "A trader's claims about the tape checked against it, measured against MirrorLine", "state": "tied", "track": "Track 3", "subtheme": "Decision Stress Testing", "rival": "PinnacleCryptNG/MirrorLine (Season 2, no licence file): its own challenge engine, getInterpretationChallenge, run from its clone on the same sentences", "tested": "the same 38 sentences through both desks the same morning, the tape captured before and after; near-zero days reported apart rather than graded", "conditions_met": 5, "artefacts": ["data/claimcheck_comparison.json"], "tests": ["test_claim_check.py"], "rerun": "python -m argus.eval.claimcheck_comparison", "console": null}, {"name": "A leveraged hold across the weekend, measured against baserate", "state": "tied", "track": "Track 3", "subtheme": "Decision Stress Testing", "rival": "Jayanng/baserate (Season 2, proprietary): its own trade parser and dossier builder on its pinned replay fixtures, 1,227 NVDA weekends", "tested": "'long rNVDA over the weekend at 3x with 5,000 USDT' through both desks", "conditions_met": 5, "artefacts": ["data/h2h_baserate/summary.json"], "tests": ["test_equity_history.py"], "rerun": "", "console": null}, {"name": "Where a stop sits in the noise, measured against Rook's invalidation price", "state": "tied", "track": "Track 3", "subtheme": "Decision Stress Testing", "rival": "iamsuperfly/Rook (Season 2, MIT): runDebate (bull/bear + judge) on the hackathon Qwen, long side, 24h, six names, from its clone", "tested": "both stops scored on the same held-out 40% of each name's Bitget daily bars", "conditions_met": 9, "artefacts": ["data/stopquality_comparison.json"], "tests": ["test_odds.py", "test_stopquality_comparison.py"], "rerun": "python -m argus.eval.stopquality_comparison", "console": null}, {"name": "Where a shut stock should open: the perpetual-implied open, against gloaming and nocturne", "state": "owned", "track": "Track 3", "subtheme": "Execution Assistance", "rival": "angelraph/gloaming (Season 2, MIT): overnight fair value from index futures, BTC/ETH and the dollar, its own model functions run from its clone; egbujor-emmanuel/nocturne (Season 2, MIT): the weekend reference price,…", "tested": "every candidate read at 09:00 New York from the same saved hourly bars and scored on the same Yahoo opens; nocturne's own rows with an ARGUS column", "conditions_met": 13, "artefacts": ["data/overnight_comparison.json", "data/void_comparison.json"], "tests": ["test_overnight_comparison.py"], "rerun": "python -m argus.eval.overnight_comparison", "console": null}, {"name": "A book under a trader's own limits (caps, groups, number of names) vs. skfolio MeanRisk", "state": "tied", "track": "Track 3", "subtheme": "Open Theme: portfolio tools", "rival": "skfolio 1.3.1 (BSD-3) MeanRisk, minimum variance, with its linear_constraints, cardinality and group_cardinalities, solved by CLARABEL and, for the count limits, by SCIP as its own docstring recommends; same covariance…", "tested": "both solve the same six limit sets on one covariance built from the same 4,280 hourly returns; each result is graded by recomputing every limit from its weights", "conditions_met": 4, "artefacts": ["data/constrained_comparison.json"], "tests": ["test_constrained.py"], "rerun": "python -m argus.eval.constrained_comparison", "console": null}, {"name": "The chance a position breaks its loss limit, searched over observed moves and recorded order books, vs. a normal-shock Monte Carlo", "state": "tied", "track": "Track 3", "subtheme": "Decision Stress Testing", "rival": "historical simulation, the general method (the breach frequency skfolio's and riskfolio's historical VaR are read from); and, as context, the normal-shock method of Open-Finance-Lab/AgenticTrading's…", "tested": "both estimate from the same fitted windows and the same recorded books, scored on the same test windows", "conditions_met": 7, "artefacts": ["data/stress_search_comparison.json"], "tests": ["test_stress_search.py"], "rerun": "python -m argus.eval.stress_search_comparison", "console": null}, {"name": "Prompt injection withheld on attacks it was never written against (NVIDIA garak)", "state": "lost", "track": "Track 2", "subtheme": "Judged: risk control layer effectiveness", "rival": "NVIDIA garak @8d1259ef (Apache-2.0): its latent-injection probes as development and its promptinject probes, built by its own resources/promptinject, as a held-out set scored once; the quarantine's own rules as they…", "tested": "the current rules and the frozen earlier rules score the same generated documents", "conditions_met": 5, "artefacts": ["data/garak_quarantine.json", "data/injection_classifier_rival.json"], "tests": ["test_garak_quarantine.py"], "rerun": "", "console": null}, {"name": "Research workbench data breadth vs. OpenBB's keyless providers", "state": "owned", "track": "Track 3", "subtheme": "Judged: data sources and Skill integration", "rival": "OpenBB-finance/OpenBB (commit 3e071fcc) with every keyless provider installed (cboe, finra, finviz, sec, tmx, yfinance, stockgrid, seeking_alpha, wsj), called through OpenBB's own obb interface by…", "tested": "the same underlyings, the same day, the same category mapping, for OpenBB, the desk and the workbench, on both days", "conditions_met": 13, "artefacts": ["data/openbb_breadth_raw.json", "data/perception_breadth.json"], "tests": ["test_perception_feeds.py", "test_perception_breadth.py"], "rerun": "python -m argus.eval.perception_breadth", "console": {"question": "/research", "book": ""}}, {"name": "Trader memory across sessions (what the trader says about themselves) vs. mem0", "state": "tied", "track": "Track 3", "subtheme": "Personalized Research Workbench", "rival": "mem0 (mem0ai, Apache-2.0), the most used open-source memory layer for LLM agents, run unmodified in its own environment with its default extraction prompt, Qwen as its model, a local sentence-transformers embedder and a…", "tested": "both sides read the same frozen statements, not-about-me sentences and effect tasks in each round; mem0 run as released, in its own venv", "conditions_met": 3, "artefacts": ["data/memory_comparison.json"], "tests": ["test_memory_comparison.py", "test_round12_reads.py"], "rerun": "python -m argus.eval.memory_comparison", "console": null}, {"name": "Attack corpus stopped before the model, vs. HeyArka's own shield on its own corpus", "state": "tied", "track": "Track 2", "subtheme": "Judged: risk control layer effectiveness", "rival": "HeyArka @8dbb967 (MIT): its @heyarka/core CORPUS of 16 vectors in six families and its @heyarka/shield pipeline (sanitize, provenance gate, corroboration gate, point-in-time guard), built from source and run on its own…", "tested": "ARGUS's stages and HeyArka's shield on the same 16 vectors over HeyArka's own demo context, paired per vector, exact sign test", "conditions_met": 7, "artefacts": ["data/heyarka_corpus.json", "data/heyarka_shield_probe.json", "data/attack_suite.json"], "tests": ["test_attack_suite.py"], "rerun": "", "console": null}, {"name": "Backtest critic, eight checks with quoted lines, vs. backtest-truth's static linter", "state": "tied", "track": "Track 1", "subtheme": "Factor research (validation machinery)", "rival": "MrMaca11an/backtest-truth (MIT), its static AST checks (backtest_truth/ast_checks.py check_source) run from its clone on identical code", "tested": "both tools on the same 21 fixtures (17 planted bugs, 4 clean) and the same 16 independent scripts, each independent flag scored by reading the line", "conditions_met": 6, "artefacts": ["data/backtest_critic_comparison.json"], "tests": ["test_backtest_critic.py"], "rerun": "python -m argus.eval.backtest_critic_comparison", "console": null}, {"name": "Volatility-targeted sizing (walk-forward GARCH) against fixed size, after fees", "state": "lost", "track": "Track 2", "subtheme": "Judged: risk control layer effectiveness", "rival": "Fixed size on the same signal (garchmethod's own demo, an EMA 9/21 crossover long/flat), against milesdeutscher/garchmethod's target / forecast sizing clipped to [0.25, 2.0], both through backtest/engine.py with…", "tested": "fixed, targeted and exposure-matched targeted arms on the same signal and closes", "conditions_met": 6, "artefacts": ["data/vol_target_comparison.json"], "tests": ["test_vol_target.py"], "rerun": "python -m argus.eval.vol_target_comparison", "console": null}, {"name": "Exit cost from the displayed book, vs. EGRESS's walk", "state": "tied", "track": "Track 3", "subtheme": "Execution Assistance", "rival": "Ritapossible/Egress egress/exitcost.py quote, run from its clone on the same 1,104 recorded Bitget order books (data/book_tape.jsonl) at $10,000 and $100,000", "tested": "both on the same 1,104 books, the same two sizes, a sell of each", "conditions_met": 4, "artefacts": ["data/exitcost_parity.json"], "tests": [], "rerun": "", "console": null}, {"name": "The desk's directional lean, vs. NIGHTWATCH AI's signal, on the same instants", "state": "tied", "track": "Track 3", "subtheme": "Decision Stress Testing", "rival": "drained69/nightwatchai LocalNightwatchEngine.research (five skill readers and synthesizeSignal), run unmodified from its clone on Bitget R-pair hourly candles closed before each instant and a Yahoo daily macro snapshot,…", "tested": "the same symbol-instants, graded on the same realised moves at the desk's mark and at 8h", "conditions_met": 3, "artefacts": ["data/nightwatch_comparison.json"], "tests": [], "rerun": "python -m argus.eval.nightwatch_comparison", "console": null}]}