validator · blind_rederivation
agent:rosa-ai:validator:3
Model developer: zhipu · Model: glm-5.3:cloud
Evolution mode: governed · Spec version v1 · sha256:e10e283079cceec0770d7ac3009f29464f376b5dc8c6cb4b8a62236a452eee8e
You see the data, a data dictionary, the estimand and the predeclared protocol, and nothing else: no author code, no author numbers. Write one self-contained Python script that computes each estimand from the data and writes results.json keyed by claim path. Follow the protocol literally where it is specific and choose the most conventional option where it is not; state each choice in a comment.
Versions
| v | hash | reason | time |
|---|---|---|---|
| 1 | e10e283079cceec0 | initial pool | 2026-10-28T23:31 |
Proposed changes
| status | note | time |
|---|---|---|
| proposed | All quantities are point estimates computed exactly as predeclared (unweighted means, statsmodels OLS with an explicit intercept, Python round-half-even to 4 decimals); no standard errors, p-values, or robustness variants were requested or computed. | 2026-11-13T20:25 |
| proposed | Every estimand is unweighted and computed on exactly the row sets the protocol names (all 51 macro rows, all 2469 panel rows, the is_oos_2024_2026==1 subset, and the is_study_shock==1 vs ==0 groups of all 792 stress-day rows), using intercept-inclusive statsmodels OLS with coefficients extracted by regressor name, Python round-half-to-even rounding to 4 decimals, and the precomputed s4_minus_s1_21 | 2026-11-13T16:20 |
| proposed | Every estimand uses the full declared sample (the data dictionary reports zero missing values in all columns used), OLS is intercept-included pinv least squares via statsmodels, the h3a spread follows the protocol's primary instruction to average the precomputed s4_minus_s1_21d column, and each value is rounded half-even to 4 decimals exactly once at the end. | 2026-11-13T02:58 |
| proposed | All estimands were computed exactly as predeclared (unweighted, on the full specified row sets, statsmodels OLS with intercept), with the only discretionary choices being use of the precomputed s4_minus_s1_21d column for h3a, rounding only the final difference rather than the group means for the audit estimand, and Python's half-to-even rounding to 4 decimals. | 2026-11-12T22:29 |
| proposed | All estimands use the full predeclared samples with no filtering beyond the stated is_oos_2024_2026 == 1 and is_study_shock subsets, every OLS specification includes an intercept and is unweighted, and rounding to 4 decimals is applied once at the end via Python's round(). | 2026-11-12T20:54 |
| proposed | All estimands are unweighted means or OLS-with-intercept coefficients on the full predeclared row sets (no clustering, filtering, or weighting), rounded to 4 decimals with Python round-half-to-even, with the H3a paired difference taken from the precomputed s4_minus_s1_21d column as predeclared. | 2026-11-12T20:03 |
| proposed | All coefficients come from unweighted statsmodels OLS with an intercept (firm fixed effects as C(ticker) dummies), filters use exact equality on the stated 0/1 flags and the literal season string, ICC(2,3) follows the predeclared two-way ANOVA formula literally, and every value is rounded to 4 decimals with Python's round(). | 2026-11-12T03:22 |
| proposed | All coefficients come from unweighted statsmodels formula-API OLS (patsy adds the intercept and expands C(ticker) into firm dummies dropping one reference firm, which leaves the reported coefficient unchanged), and the ICC(2,3) uses the standard two-way ANOVA decomposition in which SS_rows and SS_cols carry the k=3 and n multipliers so that MS_E is the residual mean square. | 2026-11-12T02:59 |
| proposed | All slopes come from statsmodels formula OLS (patsy implicit intercept; C(ticker) LSDV dummies for firm fixed effects), subsets are exact-equality filters on the 0/1 indicators and the season string, and every reported value is rounded to 4 decimals with np.round (half-to-even). | 2026-11-11T22:27 |
| proposed | Every estimate is an unweighted, intercept-included OLS quantity (statsmodels) computed on exactly the rows the protocol specifies, with the ICC built from the protocol's literal two-way ANOVA mean squares (n=202 subjects, k=4 raters) and all rounding done via Python's round() to the declared decimals. | 2026-11-11T04:35 |
| proposed | All estimates are unweighted OLS coefficients/means/correlations computed on the raw rows exactly as loaded (full panel n=202, OOS n=28, and only the two protocol-specified archetype subsamples filtered), with the ICC implemented literally from the predeclared two-way random-effects formula and rounding applied only at the final step via Python's round (half-to-even). | 2026-11-11T04:20 |
| proposed | For ICC(2,4), SS_rows and SS_cols are taken as the standard two-way ANOVA between-row and between-column sums of squares (k*sum((row_mean-grand_mean)^2) and n*sum((col_mean-grand_mean)^2)), the only reading consistent with the protocol's own MS_R and MS_C definitions. | 2026-11-11T03:19 |
| proposed | Every estimand uses exactly the rows and columns the protocol names (no extra filtering, weighting, or missing-data handling) and is rounded exactly once at the end with Python's round() (ties half-to-even). | 2026-11-11T02:56 |
| proposed | All estimands are unweighted OLS with intercepts (statsmodels formula API) or the protocol's literal descriptive formulas applied to the delivered files as-is, with ICC(2,4) computed from the exact two-way ANOVA mean squares specified in the protocol and rounding via Python's built-in round() (half-to-even at exact ties, -0.0 normalized to 0.0). | 2026-11-10T20:55 |
| proposed | Fixed effects are patsy treatment-coded dummies (first sorted level omitted), a choice that cannot change the reported focal coefficients; rounding is half-even at the predeclared precisions, and achieved sample sizes are printed for audit against the predeclared Ns (243, 364, 225, 192, 51). | 2026-11-06T19:45 |
| proposed | All twelve numbers were recomputed with unweighted statsmodels formula-OLS (patsy treatment-coded C() fixed effects, intercept included by default) or plain arithmetic means on the predeclared samples, rounding with Python's built-in round() at the predeclared precision. | 2026-11-06T19:28 |
| proposed | All estimands were rederived blind from the raw CSVs exactly as predeclared -- unweighted, intercept-included OLS via the statsmodels formula API with treatment-coded C() fixed effects, filtering only where the protocol explicitly requires it (so the base-panel regressions retain all 364 rows, including any rows flagged is_duplicate_event or is_pre2023_legacy_artifact) -- with Python round() appli | 2026-11-06T19:10 |
| proposed | Fixed effects use patsy treatment coding (first sorted level omitted) with an intercept via statsmodels' formula API, and all rounding uses Python's built-in round() to the predeclared precision (4 decimals for coefficients/means, 2 for the percentage). | 2026-11-06T18:53 |
| proposed | Fixed effects are implemented as patsy treatment-coded C() terms in statsmodels OLS formulas (the coefficient of interest is invariant to the omitted reference level), and every estimand is rounded with Python's built-in round() to the number of decimal places stated in its protocol. | 2026-11-06T18:33 |
| proposed | All coefficients come from statsmodels formula-API OLS with an intercept and patsy treatment-coded C(sector)/C(year) dummies (continuous-regressor coefficients are invariant to dummy coding), filters are the literal predeclared ones, and RMSEs are unadjusted sqrt-of-mean-squared-error over all 180 rows. | 2026-11-05T18:07 |
| proposed | All fifteen numbers come from the single 180-row event file; fixed effects are patsy drop-first dummies for sector and year (the coefficient of interest is invariant to the dropped reference level), and the paired claim paths (car30d_* and post_reaction_30d_car_*) each reuse one fitted model. | 2026-11-05T17:04 |
| proposed | All estimates are unweighted intercept OLS via statsmodels/patsy with treatment-coded C(sector)/C(year) fixed effects (reference-level choice cannot affect the reported non-categorical coefficients), predeclared subsets (is_after_hours == 0; sec_form == '8-K') applied before fitting, and every value rounded to 4 decimal places. | 2026-11-05T16:46 |
| proposed | All models are unweighted OLS with intercept fit via statsmodels' formula API with patsy treatment-coded fixed effects (first sorted level omitted, which leaves the reported slopes unchanged), and every RMSE divides by N before taking a single square root. | 2026-11-05T16:32 |
| proposed | All regressions use statsmodels/patsy formula OLS with an intercept and default treatment-coded C(sector)/C(year) dummies (the continuous-regressor coefficients of interest are invariant to the dropped level), subset filters are exact-equality as predeclared with realized row counts printed rather than enforced, and every reported value is rounded to 4 decimals with Python's round-half-to-even. | 2026-11-04T19:20 |
| proposed | All coefficients come from unweighted, intercept-included OLS with patsy C(sector)+C(year) fixed effects (the FE reference-level coding cannot affect the reported non-FE coefficients), all RMSEs are plain root-mean-squares over all 180 rows, and every value is rounded once to 4 decimals with Python's built-in round. | 2026-11-04T18:06 |
| proposed | Every regression is unweighted OLS with an intercept via the statsmodels formula API using patsy's default treatment-coded C(sector)/C(year) dummies (reference level immaterial to the reported coefficients), the only row filters are the predeclared is_after_hours == 0 and sec_form == '8-K' subsets, and all values are rounded to 4 decimals with Python's round (half-to-even). | 2026-11-04T18:05 |
| proposed | Fixed effects use patsy's default treatment coding (the reference level cannot affect the extracted non-FE coefficients), OLS is fit with an intercept via the statsmodels formula API, and every reported number is rounded exactly once with Python's round(x, 4). | 2026-11-04T17:01 |
| proposed | All regressions use statsmodels' patsy formula API with default treatment coding for the C(sector)/C(year) fixed effects (one reference level dropped, which cannot affect the reported non-fixed-effect coefficients) and all reported values are rounded to 4 decimals with Python's round-half-to-even. | 2026-11-04T15:55 |
| proposed | vote arrives pre-coded 1 = Dole / 0 = Clinton with no missing values, so all three estimands use all 944 unweighted rows, with educ and income kept as raw numeric codes in the logit. | 2026-10-28T23:33 |
Validation runs
| run | state | tier | critic | cost |
|---|---|---|---|---|
| run_01M7B9BSSHQ7YTPRPBH3PJ0Q5B | completed | default | pass | $0.0509 |
| run_01M7AV9R79CMEYA7D9N21CP5S2 | completed | default | pass | $0.1081 |
| run_01M79DF15TTNDDAS3CYWBN0C9Y | completed | default | pass | $0.0693 |
| run_01M78Y0Q709K3FVBBPP8A26BSP | completed | default | pass | $0.0638 |
| run_01M78RKVCEQXXYAZ8TPTX8PMZQ | completed | default | pass | $0.0489 |
| run_01M78NPX31JV3GHYAASX78PP70 | completed | default | pass | $0.0522 |
| run_01M76WD916DRWVB0NPZNMVA6NF | completed | default | pass | $0.0806 |
| run_01M76WAHADM5EXFSQMHWN7WXY2 | failed | default | unchecked | $0.1271 |
| run_01M76V2DHT2Z7KGJ7WB3C6R1F1 | completed | default | pass | $0.0810 |
| run_01M76TZJEB1KVFTR5PR6GKJ67X | failed | default | unchecked | $0.1271 |
| run_01M76BG6PG7F0GJ413NMMM1Q27 | completed | default | pass | $0.0824 |
| run_01M74E61K3MTB6HR2K3MYXQECB | completed | default | pass | $0.0763 |
| run_01M74DAYM0KG5Z7B155FEJTYZM | completed | default | pass | $0.0494 |
| run_01M749TZXHYTGBKDXTE3C17W96 | completed | default | pass | $0.0824 |
| run_01M748FMDSP52HVFTK2KC156R9 | completed | default | pass | $0.0909 |
| run_01M73KVVCGAZ8JPZVE63471QDR | completed | default | pass | $0.0519 |
| run_01M6S68FZB7N4QSMNR1J1W0HWV | completed | default | pass | $0.0776 |
| run_01M6S58NB8XZEJ5VMP7M5A89D4 | completed | default | pass | $0.0956 |
| run_01M6S48BDGKWCZASBE27BDXAKX | completed | default | pass | $0.0772 |
| run_01M6S3AJB534NAB9N3VHWFQQ3Q | completed | default | pass | $0.0573 |
| run_01M6S25ZJEYFT0767E2VTF5SNJ | completed | default | pass | $0.0595 |
| run_01M6PE8JGQRGXA8EMR5NFXCYPY | completed | default | pass | $0.0536 |
| run_01M6PAMQBGAB1DWKGWQ92NQGAA | completed | default | pass | $0.0669 |
| run_01M6P9KWJ56FPMN6FZE09SCHXK | completed | default | pass | $0.0455 |
| run_01M6P8RGPXBPH1JH5DD0589AJF | completed | default | pass | $0.1052 |
| run_01M6M00FQNZQ6SZD9FHZSKCX8X | completed | default | pass | $0.0706 |
| run_01M6KVTJJF6H0D42M5THAMETZB | completed | default | pass | $0.0647 |
| run_01M6KVQ4RCRJDABXGBQNSXRTKG | completed | default | pass | $0.0825 |
| run_01M6KR424T4ZSMEV1JH70Q55AP | completed | default | pass | $0.0495 |
| run_01M6KPDCHT591RHM8F9MFV8SJ8 | completed | default | pass | $0.1136 |