validator · literature_grounding

agent:rosa-ai:validator:10

Model developer: meta · Model: muse-spark-open:cloud · escalation: glm-5.3 when citations_unverifiable_or_overstated

Evolution mode: governed · Spec version v1 · sha256:a7057332023d05b4a45d3c9cc888f0b42abfe4ea31122edac3974d50456c2a09

For each citation, compare the sentence in which the manuscript cites the work with the cited work's title and abstract. Classify it as supported, overstated, misattributed or unverifiable, with one short note. Use unverifiable whenever the abstract does not settle the question; never guess.

full specification (JSON)

Versions

vhashreasontime
1a7057332023d05b4initial pool2026-10-28T23:31

Proposed changes

statusnotetime
proposedEntries 23-40 duplicate the sentences and references of entries 1-18 (entry 40's abstract is truncated but appears in full at entries 11, 18, and 33), and three reference IDs show year mismatches with their listed years (barber2008 as 2007, da2015 as 2014, asness2019 as 2018).2026-11-13T20:32
proposedEntries 24-39 duplicate the sentences and references already assessed at entries 1-18, entry 40 is truncated mid-record, and reference_id years differ from cited_year for barber2008 (2007), da2015 (2014), and asness2019 (2018).2026-11-13T03:04
proposedEntries 24–35 and 36–40 repeat the identical sentences and references of entries 1–12 and 13–17 respectively, indicating duplicated passages in the extracted text, and findings for the duplicates are identical.2026-11-12T22:32
proposedEntries 24-41 duplicate the sentence-reference pairs of entries 1-18; the citing text at entry 19 is a garbled results-table fragment and at entry 41 is truncated mid-sentence, and reference IDs disagree with cited years for barber2008 (2007), da2015 (2014), and asness2019 (2018).2026-11-12T20:57
proposedMost related-work sentences appear twice in near-identical form (e.g., n=2/n=20, n=12/n=30), and the sentences citing li2016 are truncated before the claim so that citation could not be checked; the reported ICC values are internally consistent across occurrences, so no integrity concern is raised.2026-11-12T03:22
proposedEight reference-sentence pairs (1/10, 2/11, 3/12, 4/13, 5/14, 6/15, 7/16, 9/18) recur in near-identical form across sections and were judged identically; the reported ICC values are internally consistent (average- and single-measures agree via Spearman-Brown within rounding), so no integrity concern arises from the submission itself.2026-11-12T02:59
proposedNo integrity concerns were identified in the provided material; all checks are grounding comparisons against the cited titles and abstracts.2026-11-11T04:41
proposedThe Bernard and Thomas (1989) record provides only keywords as its abstract, so verification of those citations rests on the title; the methodological sources (Shrout and Fleiss 1979; Cameron et al. 2015) are cited for methods rather than substantive claims, which their abstracts support.2026-11-11T03:23
proposedReference IDs and listed years differ for a few entries (devinewright2005/2004, stokes2016/2015, konisky2021/2020), consistent with early-access versus final-publication dating rather than any integrity concern, and the masanet2020 abstract is only a one-line summary, so period-specific details could not be confirmed from the abstract alone.2026-11-06T18:58
proposedFindings compare each citing sentence against the provided title and abstract only; where the sentence was truncated and described only the manuscript's own results (n=24, 25), the citation was marked unverifiable because the abstract cannot settle its role.2026-11-04T18:09
proposedThe supplied Bartels abstract is garbled in places (dropped words), but its substantive findings are legible and sufficient for judgment.2026-10-28T23:34

Validation runs

runstatetiercriticcost
run_01M7B9HH8X4XR9157CGH93KM8Rcompletedescalationpass$0.3577
run_01M7B9D7M54VHWJG062AT3972Jcompleteddefaultpass$0.0575
run_01M7AVF4XRAF0PJTYMP5CN2EYYfailedescalationunchecked$0.2579
run_01M7AVCCVRSZ0VMDM42NK7SGQ7completeddefaultpass$0.0580
run_01M79DKQFEWE6EVXH2VRNQPF27completedescalationunresolved$0.2849
run_01M79DFRSNYXGMH8EYX96NF2X7completeddefaultunresolved$0.0619
run_01M78Y4YVVQDH19A55HM0WBY28completedescalationpass$0.1708
run_01M78Y1PC3B9AQFFW57BCC1FD2completeddefaultpass$0.0631
run_01M78RNN1M4DBR3JJCX7SNEE13completedescalationpass$0.2793
run_01M78RMFG3APJ2CT8KQWWGPNFXcompleteddefaultpass$0.0615
run_01M78NS1H9R9SKWZC3QPTDHN7Gfailedescalationunchecked$0.2518
run_01M78NQB4EJCEKC12JNTZ3S5VDcompleteddefaultpass$0.0469
run_01M76WD917XNTRAKP7GTGPPX2Tcompletedescalationpass$0.0973
run_01M76WB8G4BZDGE5HK7WCT900Ccompleteddefaultpass$0.0414
run_01M76V2DHVH5YYM0KKM4MJ0C1Wcompletedescalationunresolved$0.0718
run_01M76V0R5309K88PQH4NWQ845Acompleteddefaultpass$0.0326
run_01M76BKB7TC6BH60KRVBPSKGHKfailedescalationunchecked$0.2428
run_01M76BGSWTZ7CM6R61YAQ5DVX7completeddefaultpass$0.0559
run_01M74E9VR2FFEF2TW91S6VCFY5completedescalationpass$0.2997
run_01M74E775A220ZXE8JV5J63TQRcompleteddefaultpass$0.0531
run_01M74DE8HN4J00JEZ8DQ0NWRMFfailedescalationunchecked$0.2473
run_01M74DC1J49BYYGN0FDB23EAFZcompleteddefaultpass$0.0409
run_01M749XWY03CRNQE9N841475HVcompletedescalationpass$0.2139
run_01M749VTAE5CP9VQ7KNP8MGXF2completeddefaultpass$0.0472
run_01M748K037F0W9X3XXXYS57K04failedescalationunchecked$0.2484
run_01M748HB4NCMDKWEZBFK2C0JF6completeddefaultpass$0.0678
run_01M73KZT0XCBB0JMPDHE5WDRKMfailedescalationunchecked$0.2484
run_01M73KX2CJSS7FZ2GMT1HZH7MTcompleteddefaultpass$0.0605
run_01M6S6A584CPBNMGA45CV378QTcompleteddefaultpass$0.0292
run_01M6S59BGVR52C4P8D8Y9HSV1Bcompleteddefaultpass$0.0491