validator · literature_grounding
agent:rosa-ai:validator:10
모델 개발사: meta · 모델: muse-spark-open:cloud · escalation: glm-5.3 when citations_unverifiable_or_overstated
진화 모드: 관리형 · 명세 버전 v1 · sha256:a7057332023d05b4a45d3c9cc888f0b42abfe4ea31122edac3974d50456c2a09
For each citation, compare the sentence in which the manuscript cites the work with the cited work's title and abstract. Classify it as supported, overstated, misattributed or unverifiable, with one short note. Use unverifiable whenever the abstract does not settle the question; never guess.
Versions
| v | hash | reason | time |
|---|---|---|---|
| 1 | a7057332023d05b4 | initial pool | 2026-10-28T23:31 |
제안된 변경
| status | note | time |
|---|---|---|
| proposed | Entries 23-40 duplicate the sentences and references of entries 1-18 (entry 40's abstract is truncated but appears in full at entries 11, 18, and 33), and three reference IDs show year mismatches with their listed years (barber2008 as 2007, da2015 as 2014, asness2019 as 2018). | 2026-11-13T20:32 |
| proposed | Entries 24-39 duplicate the sentences and references already assessed at entries 1-18, entry 40 is truncated mid-record, and reference_id years differ from cited_year for barber2008 (2007), da2015 (2014), and asness2019 (2018). | 2026-11-13T03:04 |
| proposed | Entries 24–35 and 36–40 repeat the identical sentences and references of entries 1–12 and 13–17 respectively, indicating duplicated passages in the extracted text, and findings for the duplicates are identical. | 2026-11-12T22:32 |
| proposed | Entries 24-41 duplicate the sentence-reference pairs of entries 1-18; the citing text at entry 19 is a garbled results-table fragment and at entry 41 is truncated mid-sentence, and reference IDs disagree with cited years for barber2008 (2007), da2015 (2014), and asness2019 (2018). | 2026-11-12T20:57 |
| proposed | Most related-work sentences appear twice in near-identical form (e.g., n=2/n=20, n=12/n=30), and the sentences citing li2016 are truncated before the claim so that citation could not be checked; the reported ICC values are internally consistent across occurrences, so no integrity concern is raised. | 2026-11-12T03:22 |
| proposed | Eight reference-sentence pairs (1/10, 2/11, 3/12, 4/13, 5/14, 6/15, 7/16, 9/18) recur in near-identical form across sections and were judged identically; the reported ICC values are internally consistent (average- and single-measures agree via Spearman-Brown within rounding), so no integrity concern arises from the submission itself. | 2026-11-12T02:59 |
| proposed | No integrity concerns were identified in the provided material; all checks are grounding comparisons against the cited titles and abstracts. | 2026-11-11T04:41 |
| proposed | The Bernard and Thomas (1989) record provides only keywords as its abstract, so verification of those citations rests on the title; the methodological sources (Shrout and Fleiss 1979; Cameron et al. 2015) are cited for methods rather than substantive claims, which their abstracts support. | 2026-11-11T03:23 |
| proposed | Reference IDs and listed years differ for a few entries (devinewright2005/2004, stokes2016/2015, konisky2021/2020), consistent with early-access versus final-publication dating rather than any integrity concern, and the masanet2020 abstract is only a one-line summary, so period-specific details could not be confirmed from the abstract alone. | 2026-11-06T18:58 |
| proposed | Findings compare each citing sentence against the provided title and abstract only; where the sentence was truncated and described only the manuscript's own results (n=24, 25), the citation was marked unverifiable because the abstract cannot settle its role. | 2026-11-04T18:09 |
| proposed | The supplied Bartels abstract is garbled in places (dropped words), but its substantive findings are legible and sufficient for judgment. | 2026-10-28T23:34 |
검증 실행
| run | state | tier | critic | cost |
|---|---|---|---|---|
| run_01M7B9HH8X4XR9157CGH93KM8R | completed | escalation | pass | $0.3577 |
| run_01M7B9D7M54VHWJG062AT3972J | completed | default | pass | $0.0575 |
| run_01M7AVF4XRAF0PJTYMP5CN2EYY | failed | escalation | unchecked | $0.2579 |
| run_01M7AVCCVRSZ0VMDM42NK7SGQ7 | completed | default | pass | $0.0580 |
| run_01M79DKQFEWE6EVXH2VRNQPF27 | completed | escalation | unresolved | $0.2849 |
| run_01M79DFRSNYXGMH8EYX96NF2X7 | completed | default | unresolved | $0.0619 |
| run_01M78Y4YVVQDH19A55HM0WBY28 | completed | escalation | pass | $0.1708 |
| run_01M78Y1PC3B9AQFFW57BCC1FD2 | completed | default | pass | $0.0631 |
| run_01M78RNN1M4DBR3JJCX7SNEE13 | completed | escalation | pass | $0.2793 |
| run_01M78RMFG3APJ2CT8KQWWGPNFX | completed | default | pass | $0.0615 |
| run_01M78NS1H9R9SKWZC3QPTDHN7G | failed | escalation | unchecked | $0.2518 |
| run_01M78NQB4EJCEKC12JNTZ3S5VD | completed | default | pass | $0.0469 |
| run_01M76WD917XNTRAKP7GTGPPX2T | completed | escalation | pass | $0.0973 |
| run_01M76WB8G4BZDGE5HK7WCT900C | completed | default | pass | $0.0414 |
| run_01M76V2DHVH5YYM0KKM4MJ0C1W | completed | escalation | unresolved | $0.0718 |
| run_01M76V0R5309K88PQH4NWQ845A | completed | default | pass | $0.0326 |
| run_01M76BKB7TC6BH60KRVBPSKGHK | failed | escalation | unchecked | $0.2428 |
| run_01M76BGSWTZ7CM6R61YAQ5DVX7 | completed | default | pass | $0.0559 |
| run_01M74E9VR2FFEF2TW91S6VCFY5 | completed | escalation | pass | $0.2997 |
| run_01M74E775A220ZXE8JV5J63TQR | completed | default | pass | $0.0531 |
| run_01M74DE8HN4J00JEZ8DQ0NWRMF | failed | escalation | unchecked | $0.2473 |
| run_01M74DC1J49BYYGN0FDB23EAFZ | completed | default | pass | $0.0409 |
| run_01M749XWY03CRNQE9N841475HV | completed | escalation | pass | $0.2139 |
| run_01M749VTAE5CP9VQ7KNP8MGXF2 | completed | default | pass | $0.0472 |
| run_01M748K037F0W9X3XXXYS57K04 | failed | escalation | unchecked | $0.2484 |
| run_01M748HB4NCMDKWEZBFK2C0JF6 | completed | default | pass | $0.0678 |
| run_01M73KZT0XCBB0JMPDHE5WDRKM | failed | escalation | unchecked | $0.2484 |
| run_01M73KX2CJSS7FZ2GMT1HZH7MT | completed | default | pass | $0.0605 |
| run_01M6S6A584CPBNMGA45CV378QT | completed | default | pass | $0.0292 |
| run_01M6S59BGVR52C4P8D8Y9HSV1B | completed | default | pass | $0.0491 |