fedi-index:代理
模型作為代理:工具使用與多步驟任務——Arena 代理工作階段、τ²-bench 與 Terminal-Bench 測試結果。
| # | 等級 | 模型 | fedi-index | 百分位 | 人類偏好 | 解題能力 | 性價比 | 來源 | 分數拆解 |
|---|---|---|---|---|---|---|---|---|---|
| 1 | S | 1,530 ±9 | 100.0 | 1,559 | 1,469 | +40 | AT | › | |
| 2 | S | 1,506 ±7 | 99 | 1,522 | 1,468 | +73 | AT | › | |
| 3 | S | 1,502 ±8 | 98 | 1,520 | 1,461 | +65 | AT | › | |
| 4 | S | 1,500 ±3 | 96 | 1,513 | 1,466 | +108◆ | AT | › | |
| 5 | S | 1,494 ±9 | 95 | 1,507 | 1,465 | +49 | AT | › | |
| 6 | A | 1,492 ±8 | 94 | 1,506 | 1,463 | -24 | AT | › | |
| 7 | A | 1,491 ±6 | 93 | 1,504 | 1,460 | +9 | AT | › | |
| 8 | A | 1,487 ±7 | 91 | 1,500 | 1,455 | +42 | AT | › | |
| 9 | A | 1,486 ±6 | 90 | 1,500 | 1,453 | +92◆ | AT | › | |
| 10 | A | 1,482 ±4 | 89 | 1,491 | 1,460 | +60 | AT | › | |
| 11 | A | 1,481 ±8 | 88 | 1,489 | 1,462 | -28 | AT | › | |
| 12 | A | 1,477 ±3 | 86 | 1,487 | 1,452 | +98 | AT | › | |
| 13 | A | 1,476 ±4 | 85 | 1,487 | 1,448 | +47 | AT | › | |
| 14 | A | 1,468 ±5 | 84 | 1,468 | 1,469 | +44 | AτT | › | |
| 15 | A | 1,461 ±5 | 83 | 1,466 | 1,454 | +25 | AτT | › | |
| 16 | A | 1,461 ±5 | 82 | 1,469 | 1,443 | +5 | AT | › | |
| 17 | A | 1,461 ±4 | 80 | 1,476 | — | — | A | › | |
| 18 | B | 1,458 ±4 | 79 | 1,458 | 1,457 | +52 | AT | › | |
| 19 | B | 1,457 ±6 | 78 | 1,473 | — | — | A | › | |
| 20 | B | 1,456 ±4 | 76 | 1,457 | 1,454 | +36 | AτT | › | |
| 21 | B | 1,455 ±4 | 75 | 1,468 | 1,422 | — | AT | › | |
| 22 | B | 1,441 ±4 | 74 | 1,443 | 1,437 | +52 | AT | › | |
| 23 | B | 1,440 ±5 | 73 | 1,446 | 1,424 | +15 | AT | › | |
| 24 | B | 1,438 ±2 | 72 | 1,444 | 1,423 | +102 | AT | › | |
| 25 | B | 1,437 ±4 | 70 | 1,440 | 1,433 | +39 | AτT | › | |
| 26 | B | 1,437 ±5 | 69 | 1,439 | 1,432 | +49 | Aτ | › | |
| 27 | B | 1,437 ±3 | 68 | 1,440 | 1,428 | +65 | AT | › | |
| 28 | B | 1,435 ±3 | 67 | 1,445 | 1,410 | — | AT | › | |
| 29 | B | 1,435 ±3 | 65 | 1,447 | 1,404 | +101◆ | AT | › | |
| 30 | B | 1,434 ±4 | 64 | 1,438 | 1,426 | +104◆ | AT | › | |
| 31 | B | 1,433 ±4 | 63 | 1,447 | — | +30 | A | › | |
| 32 | B | 1,431 ±3 | 62 | 1,447 | 1,390 | +94◆ | AT | › | |
| 33 | B | 1,428 ±3 | 60 | 1,445 | 1,385 | +79 | AT | › | |
| 34 | B | 1,427 ±3 | 59 | 1,436 | 1,404 | +56 | AT | › | |
| 35 | B | 1,427 ±2 | 58 | 1,444 | 1,386 | +95 | AT | › | |
| 36 | B | 1,426 ±4 | 57 | 1,445 | 1,380 | +30 | AT | › | |
| 37 | B | 1,425 ±3 | 56 | 1,438 | 1,392 | — | AT | › | |
| 38 | B | 1,425 ±4 | 54 | 1,437 | 1,393 | +35 | AT | › | |
| 39 | B | 1,422 ±3 | 53 | 1,423 | 1,421 | +124◆ | AT | › | |
| 40 | B | 1,419 ±4 | 52 | 1,432 | 1,387 | +105◆ | AT | › | |
| 41 | B | 1,419 ±3 | 51 | 1,428 | 1,396 | +38 | AT | › | |
| 42 | C | 1,417 ±7 | 49 | — | 1,435 | +21 | τT | › | |
| 43 | C | 1,414 ±3 | 48 | 1,427 | 1,381 | +87 | AT | › | |
| 44 | C | 1,413 ±7 | 47 | — | 1,431 | +34 | τT | › | |
| 45 | C | 1,411 ±5 | 46 | 1,414 | 1,407 | +41 | AτT | › | |
| 46 | C | 1,411 ±6 | 44 | 1,423 | — | — | A | › | |
| 47 | C | 1,410 ±4 | 43 | 1,400 | 1,437 | +58 | Aτ | › | |
| 48 | C | 1,409 ±12 | 42 | — | 1,442 | +22 | τ | › | |
| 49 | C | 1,406 ±7 | 41 | 1,407 | 1,405 | +48 | AT | › | |
| 50 | C | 1,406 ±3 | 40 | 1,418 | — | — | A | › | |
| 51 | C | 1,404 ±6 | 38 | 1,417 | — | +35 | A | › | |
| 52 | C | 1,404 ±12 | 37 | — | 1,436 | +27 | τ | › | |
| 53 | C | 1,404 ±4 | 36 | 1,416 | — | +64 | A | › | |
| 54 | C | 1,403 ±12 | 35 | — | 1,435 | +35 | τ | › | |
| 55 | C | 1,396 ±11 | 33 | — | 1,426 | +1 | τ | › | |
| 56 | C | 1,391 ±6 | 32 | 1,403 | — | +46 | A | › | |
| 57 | C | 1,391 ±11 | 30 | — | 1,420 | +32 | τ | › | |
| 58 | C | 1,391 ±11 | 30 | — | 1,420 | +62 | τ | › | |
| 59 | C | 1,390 ±6 | 28 | 1,401 | — | +80 | A | › | |
| 60 | C | 1,389 ±11 | 27 | — | 1,417 | -8 | τ | › | |
| 61 | C | 1,388 ±4 | 26 | 1,399 | — | +63 | A | › | |
| 62 | D | 1,388 ±11 | 25 | — | 1,416 | -4 | τ | › | |
| 63 | D | 1,388 ±7 | 24 | 1,400 | — | +59 | A | › | |
| 64 | D | 1,388 ±12 | 22 | 1,401 | — | +42 | A | › | |
| 65 | D | 1,386 ±4 | 21 | 1,391 | 1,373 | +58 | AT | › | |
| 66 | D | 1,382 ±10 | 20 | — | 1,408 | +28 | τ | › | |
| 67 | D | 1,378 ±9 | 18 | — | 1,403 | -1 | τ | › | |
| 68 | D | 1,374 ±8 | 16 | — | 1,398 | +26 | τ | › | |
| 69 | D | 1,374 ±8 | 16 | — | 1,398 | +33 | τ | › | |
| 70 | D | 1,374 ±5 | 15 | 1,385 | — | — | A | › | |
| 71 | D | 1,370 ±5 | 14 | — | 1,396 | — | T | › | |
| 72 | D | 1,365 ±4 | 12 | — | 1,389 | +91 | T | › | |
| 73 | D | 1,364 ±8 | 11 | 1,375 | — | -20 | A | › | |
| 74 | D | 1,356 ±4 | 10 | — | 1,380 | — | T | › | |
| 75 | D | 1,353 ±4 | 9 | 1,362 | — | +48 | A | › | |
| 76 | D | 1,351 ±14 | 7 | 1,362 | — | +6 | A | › | |
| 77 | D | 1,350 ±4 | 6 | — | 1,372 | +43 | T | › | |
| 78 | D | 1,348 ±7 | 5 | 1,357 | — | +28 | A | › | |
| 79 | D | 1,347 ±9 | 4 | 1,356 | — | +50 | A | › | |
| 80 | D | 1,343 ±2 | 2 | — | 1,364 | +16 | T | › | |
| 81 | D | 1,341 ±1 | 1 | — | 1,362 | — | T | › | |
| 82 | D | 1,324 ±9 | 0 | 1,332 | — | — | A | › |
AArena(人類投票)TTerminal-Benchττ²-bench(代理) ◆ 帕累托前緣:沒有模型能同時更好又更便宜
fedi-index 是固定尺度上的 Elo 等值:在同一方法論版本內,各月份可互相比較。 等級 S–D:依分項內的百分位決定,且不會高於信賴區間所允許的等級(門檻見方法論)。 「暫定」——此處只有一個來源系列測量過該模型。 方法論