← 返回阅读笔记 / Back to reading note
来源:作者原文。2026-09-07抓取,原始行列,不是统一复跑成绩。
| Dataset | nar | qas | mul | mus | 2wiki | hot | news | gov | en.sum | en.qa | fin | legal | misc | ave. |
| LongBench | InfBench | UltraDomain | ||||||||||||
| Full | 21.4 | 39.4 | 51.5 | 28.2 | 38.1 | 48.1 | 24.9 | 32.6 | 13.0 | 15.2 | 47.8 | 46.5 | 48.7 | 35.0 |
| Mnference | 20.7 | 39.0 | 50.8 | 27.4 | 35.9 | 46.2 | 24.8 | 32.2 | 13.3 | 12.1 | 44.7 | 39.8 | 46.3 | 33.3 |
| SelfExtend | 19.6 | 37.8 | 47.4 | 22.7 | 37.2 | 42.0 | 21.4 | 29.1 | 11.1 | 9.3 | 41.2 | 37.9 | 34.1 | 30.1 |
| BGE-M3 | 20.3 | 33.0 | 44.3 | 21.1 | 35.4 | 42.1 | 17.7 | 19.8 | 9.6 | 16.3 | 41.7 | 41.2 | 43.7 | 29.7 |
| Stella-v5 | 13.7 | 32.4 | 43.5 | 21.0 | 35.6 | 40.6 | 20.3 | 18.2 | 10.0 | 19.5 | 42.8 | 35.1 | 43.9 | 29.0 |
| Jina-emb-v3 | 15.9 | 34.7 | 42.8 | 17.8 | 33.1 | 41.8 | 21.9 | 25.2 | 11.3 | 18.7 | 41.8 | 37.1 | 43.8 | 29.7 |
| GraphRAG | 16.2 | 36.3 | 45.4 | 19.3 | 37.5 | 38.0 | 18.4 | 25.6 | 10.8 | 13.5 | 39.9 | 39.6 | 41.7 | 29.4 |
| RQ-RAG | 19.6 | 34.1 | 46.5 | 21.9 | 36.1 | 41.7 | 20.1 | 18.6 | 10.4 | 16.1 | 41.8 | 40.9 | 43.2 | 30.1 |
| HyDE | 18.7 | 36.0 | 47.5 | 20.5 | 36.8 | 42.7 | - | - | - | 19.6 | 43.1 | 41.6 | 44.2 | - |
| MemoRAG | 27.5† | 43.9† | 52.2† | 33.9† | 54.1† | 54.8† | 26.3† | 32.9† | 15.7† | 22.9† | 51.5† | 51.0† | 55.6† | 40.2 |
| Query: What is the significance of the Outside Date mentioned in the agreement? Context: A Legal Contract (56.4K tokens) |
| Ground-truth target: The Outside Date is the deadline by which the Plan must become effective, or else the Agreement will terminate automatically. It is set as October 5, 2020, at 11:59 p.m. Eastern Time. |
| Standard RAG: The Outside Date is significant as it is a date where both parties have agreed in advance that if the merger or acquisition has not yet completed either side. It is set as October 5, 2020. (F1-Score: 0.36) |
| Clues #1: Definition of the “Outside Date” in the agreement Clues #2: “Outside Date” means October 5, 2020 at 11:59 p.m. Eastern Time. |
| MemoRAG: The Outside Date mentioned in the agreement is October 5, 2020, at 11:59 p.m. Eastern Time. It is a significant date in the context of the agreement because it is the deadline for the Plan to become effective. If the Plan has not become effective by this date, certain parties may have the right to terminate the agreement. (F1-Score: 0.83) |
| Dataset | Narrative | Qasper | MultiField | Hotpot | MuSiQue | 2Wiki |
| Num of Samples | 200 | 200 | 150 | 200 | 200 | 200 |
| Ave. Length | 18,409 | 3,619 | 4,559 | 9,151 | 11,214 | 4,887 |
| Metric | F1 | F1 | F1 | F1 | F1 | F1 |
| Dataset | GovReport | MultiNews | En.Sum | En.QA | Fin | Legal |
| Num of Samples | 200 | 200 | 103 | 351 | 345 | 438 |
| Ave. Length | 8,734 | 2,113 | 171,500 | 192,600 | 40,625 | 51,413 |
| Metric | Rouge-L | Rouge-L | F1 | Rouge-L | F1 | F1 |
| Dataset | Num | |||||
| Technology | 240 | 306,073 | 44,549 | 144029.7 | 14.4 | 40.2 |
| Biology | 220 | 257,644 | 39,218 | 125284.9 | 16.8 | 49.1 |
| Religion | 220 | 1,071,342 | 34,257 | 131424.8 | 17.4 | 54.2 |
| Fiction | 220 | 564,980 | 44,057 | 137689.7 | 16.2 | 43.6 |
| Psychology | 200 | 571,725 | 37,988 | 150119.5 | 16.7 | 46.5 |
| Music | 200 | 381,043 | 51,517 | 168672.9 | 17.5 | 49.7 |
| Art | 200 | 305,001 | 32,793 | 128961.2 | 17.8 | 52.2 |
| Philosophy | 200 | 678,553 | 38,729 | 135682.7 | 17.2 | 51.0 |
| Health | 180 | 289,258 | 50,600 | 135902.0 | 16.2 | 48.2 |
| History | 180 | 688,074 | 53,277 | 195265.0 | 17.9 | 51.0 |
| Literature | 180 | 534,836 | 33,043 | 129363.7 | 16.9 | 47.0 |
| Biography | 180 | 408,969 | 45,052 | 163522.3 | 18.0 | 52.0 |
| Politics | 180 | 387,157 | 49,853 | 139624.3 | 17.9 | 54.9 |
| Mathematics | 160 | 726,144 | 60,936 | 197924.6 | 16.7 | 47.6 |
| Physics | 160 | 226,811 | 36,717 | 105805.6 | 14.8 | 54.2 |
| Cooking | 120 | 466,885 | 58,360 | 156139.2 | 16.5 | 46.6 |
| Agriculture | 100 | 385,915 | 76,581 | 150969.6 | 15.6 | 45.9 |
| Computer | 100 | 437,070 | 51,704 | 215929.5 | 14.3 | 39.8 |
| Total | 3,240 | 1,071,342 | 32,793 | 150684.0 | 16.6 | 48.5 |