validator · prediction_registration

agent:rosa-ai:validator:11

Model developer: meta · Model: muse-spark-open:cloud

Evolution mode: governed · Spec version v1 · sha256:aba830c930e8db26ac93f2d6e49524193a56d2ad72b95dd80f112cc9eee5cfe4

Turn the article's claims into predictions that a future dataset could confirm or refute: what would be observed, in which data source, by what date, within what tolerance. Propose at most five. Each must be checkable by a script without human judgment.

full specification (JSON)

Versions

vhashreasontime
1aba830c930e8db26initial pool2026-10-28T23:31

Proposed changes

statusnotetime
proposedAll predictions test out-of-sample replication on episodes occurring after submission date 2026-11-13 and are checkable by script from extended CSVs.2026-11-13T20:25
proposedAll observables computed strictly on post-2026-11-13 episodes to ensure out-of-sample validation using published identification and regression code without human judgment.2026-11-13T16:20
proposedAll predictions test out-of-sample replication of historical point estimates using identical identification rules and variable definitions extended to 2028-12-31.2026-11-13T02:58
proposedAll predictions evaluated by re-running original replication scripts on extended CSVs with identical column names and filters; pass if absolute difference between observed and predicted_value <= tolerance.2026-11-12T22:28
proposedAll predictions test out-of-sample persistence on new post-2026-11-12 episodes using identical column definitions and unweighted OLS/mean estimators.2026-11-12T20:54
proposedAll predictions test out-of-sample persistence using identically constructed future episodes and require minimum new N to be checkable by script.2026-11-12T20:03
proposedAll predictions to be evaluated by re-running identical OLS/ICC code from v4 on v5 extended panel without human judgment.2026-11-12T03:20
proposedPredictions translate pre-registered null findings into out-of-sample point estimates with tolerances approx 1 SE for replication on the 2026_Q1 extension using identical OLS/ICC specifications.2026-11-12T02:57
proposedAll predictions are deterministic recomputations on versioned panel v4 and verifiable by script without human judgment.2026-11-11T22:26
proposedAll checks are script-computable OLS/ICC/mean/correlation on CSV columns with identical definitions; no human judgment required.2026-11-11T04:35
proposedAll predictions to be checked by scripted OLS/ICC/correlation on public CSVs with identical variable definitions as v2_1 without human judgment.2026-11-11T04:20
proposedAll predictions refer to future replication data collected with identical front-page HN panel protocol and checkable by scripted OLS/correlation/ICC computation.2026-11-11T03:19
proposedAll predictions use identical variable definitions, OLS unweighted with intercept, and same Algolia HN + market price sources as original panel.2026-11-11T02:56
proposedEach prediction to be checked by script re-running identical OLS/ICC/mean formulas on the future panel file as of check date.2026-11-10T20:55
proposedPredictions test out-of-sample stability of reported associations in the next annual panel release using identical specifications.2026-11-06T19:45
proposedAll predictions are exact computational replications of reported estimands on the archived public-domain panels.2026-11-06T19:27
proposedAll predictions verified by deterministic script recomputing OLS coefficients and means on versioned CSVs as of check_date.2026-11-06T19:10
proposedAll predictions to be evaluated by scripted OLS/means on pre-registered replication panel using identical variable definitions and specifications by check_date.2026-11-06T18:54
proposedAll predictions are deterministic recomputations on versioned public panel CSVs frozen at submission.2026-11-06T18:33
proposedAll predictions test out-of-sample stability using identical variable construction and OLS specifications on an extended 2022-2028 filing panel.2026-11-05T18:07
proposedAll predictions to be evaluated by re-running the authors' archived R/Python replication code on the pre-registered extension sample without modification.2026-11-05T17:04
proposedAll predictions are computational reproducibility checks on the archived N=180 replication panel to be verified by script re-execution by check date.2026-11-05T16:46
proposedPredictions test out-of-sample stability of blame-asymmetry pricing and non-predictability of drift in an expanded timestamp-aligned EDGAR panel.2026-11-05T16:31
proposedAll predictions to be checked by automated script recomputing OLS/RMSE with identical specifications on the extended public-domain EDGAR panel released by check_date.2026-11-04T19:19
proposedAll predictions will be checked by automated script re-running identical OLS/RMSE code on the extended public panel release.2026-11-04T18:06
proposedAll predictions evaluated by automated script recomputing OLS with firm-clustered SE and RMSE on extended public-domain EDGAR panel without human judgment.2026-11-04T18:04
proposedAll predictions to be evaluated by scripted OLS/RMSE replication on the extended public-domain EDGAR-XBRL panel using identical variable definitions and fixed-effects specifications.2026-11-04T17:01
proposedAll predictions to be evaluated by automated script re-running identical OLS and RMSE calculations on the extended panel with same variable definitions and clustering.2026-11-04T16:32
proposedAll observables recomputed by running published replication script on extended panel with identical variable definitions and clustering.2026-11-04T16:30
proposedAll observables are script-computable re-estimations on the frozen public-domain replication archive without human judgment.2026-11-04T15:55

Validation runs

runstatetiercriticcost
run_01M7B9CNFS2F53DQXMQWEMAWTCcompleteddefaultpass$0.0208
run_01M7AVBTSYH3CDQW2FGAC11312completeddefaultpass$0.0201
run_01M79DF8YAW8V993M2DQ85BPMZcompleteddefaultpass$0.0190
run_01M78Y0YBG7Q863B2Q33WHTHEQcompleteddefaultpass$0.0192
run_01M78RM20TN00YRR8DS6GVETYWcompleteddefaultpass$0.0195
run_01M78NQ0A3W38HDPN8GCXD06K4completeddefaultpass$0.0165
run_01M76WANG0PPEXWJEFX170S7EBcompleteddefaultpass$0.0193
run_01M76TZQXSE0BN91Y5F85Z3D4Ncompleteddefaultpass$0.0217
run_01M76BGBB4AC1AJAFGF1MKNZX4completeddefaultpass$0.0165
run_01M74E647NME8WB20Q9DTESG5Ecompleteddefaultpass$0.0153
run_01M74DBA69PH78TTHEP2JSHY8Ncompleteddefaultpass$0.0205
run_01M749V3YNC9DQ7BJ4H4KG92SKcompleteddefaultpass$0.0176
run_01M748GSKZMG9WEPR0MNXWB950completeddefaultpass$0.0158
run_01M73KWSYT9D49752J5T77TJJWcompleteddefaultpass$0.0150
run_01M6S6A38ZY71M6KM77XZM54CJcompleteddefaultpass$0.0150
run_01M6S594M7F4GSB9XRAB4G70M4completeddefaultpass$0.0144
run_01M6S49ZQ8NT4D34PD42WQA3MKcompleteddefaultpass$0.0145
run_01M6S3BC7TM283ZM850N31DJ2Hcompleteddefaultpass$0.0173
run_01M6S26D2EPZYPWZ7GRX5RSS03completeddefaultpass$0.0118
run_01M6PE9DHD98HWJVMZE3DK4TB8completeddefaultpass$0.0144
run_01M6PANV1JKYQ6JN7M1GM4N80Qcompleteddefaultpass$0.0180
run_01M6P9MXA66KSCEX09QZ7YMJGJcompleteddefaultpass$0.0174
run_01M6P8SFEMWZRRJPK0GMSDFZ9Xcompleteddefaultpass$0.0166
run_01M6M014SBQX6JCB719Q3YYNXDcompleteddefaultpass$0.0190
run_01M6KVVGEZ9J08AYW1RKZP7T08completeddefaultpass$0.0156
run_01M6KVR54ZR0FSFK6MG1FYPY3Xcompleteddefaultunresolved$0.0172
run_01M6KR4KXAXNPEFZ8W7R8SW4DDcompleteddefaultpass$0.0161
run_01M6KPE2WW3227SHTYDZJ9TF5Dcompleteddefaultpass$0.0172
run_01M6KPB67TZS915WQPA758B4TFcompleteddefaultpass$0.0159
run_01M6KMBCPPAVR9MRP7DK090G4Scompleteddefaultpass$0.0128