Jev-Omniのマルチモーダル判定とClefとの比較
音声、画像、日本語メール、動画の35件でJev-Omniを評価し、同じ25件でClefと比較しました。AIシステム開発に向けた選択肢判定の検証です。追加の動画実験と各入力への回答も掲載しています。
Jev-Omniは、文章、画像、音声、動画から、用意した選択肢を選ぶモデルです。質問と選択肢を渡すと、それぞれの確率を返します。
4種類の入力を同じモデルで扱えます。音声の内容を分類したり、画像に写った物を答えたり、メールに返信が必要かを判断したりできます。
音声10件、画像10件、架空の日本語メール10件、動画5件の計35件で試しました。そのうち画像・メール・動画の25件は、Clefとも比較しました。
Computer UseとBrowser Useの動画
Qwen3.8を主エージェントにして、decision modelを使った簡単なComputer Use(画面操作)とBrowser Use(ブラウザー操作)も試しました。Big Fiveの性格テストを題材にしています。
動画リンク: https://www.youtube.com/watch?v=Po3kZdTTMjw
結果の一覧
同じ入力と質問で、選択肢を元の順番と逆の順番で渡しました。並び方によって答えが変わるかを調べるためです。
「想定した答え」は、モデルの回答を見る前に、実験用の基準に沿って決めた答えです。表には、その答えと一致した数を示しています。
| 実験 | 入力数 | Jev-Omni | Clef |
|---|---|---|---|
| E1:4種類の入力 | 35 | 元の順番33/35、逆順31/35 | 対象外 |
| E2:共通の画像・メール・動画 | 25 | 元の順番23/25、逆順21/25。E1の回答を再利用 | 元の順番25/25、逆順25/25 |
| E3:元の動画で緊急停止を質問 | 5 | 集計対象6回答のうち6件一致。4回答は除外 | 同左 |
| E4:接近する場面の追加動画 | 5 | 集計対象2回答のうち2件一致。8回答は除外 | 同左 |
Jev-Omniは、元の順番で33/35(94.3%)、逆の順番で31/35(88.6%)が一致しました。画像I06とメールT01では、順番を変えると答えが変わりました。
この25件ではClefの成績が上でした。2回分を合わせると、Clefは50/50回答、Jev-Omniは44/50回答が一致しました。
動画の追加実験では、映像だけでは想定した答えを決められないものを、正答数の集計から外しました。
試し方
使用したモデル
| モデル | 使用した版 | 今回の役割 |
|---|---|---|
| Jev-Omni | akhilaaa3/Jev-Omni、revision c050d51354147985d13286cf4acf90f562f2c631 | 音声、画像、文章、動画の選択肢を判定 |
| Clef | simonlehmann/clef-NVFP4、revision 817ac58ad358f42489980ead62f8bf7cafc628c2 | 画像、文章、動画でJev-Omniと比較 |
4種類の入力には、同じ読み込み済みのJev-Omniモデルを使いました。画像・音声・動画はGateway経由で渡し、文章は共通の分類処理を直接呼び出しました。
Clefは、Qwen3.8-27Bをもとに作られたモデルです。今回は、NVFP4とFP8を組み合わせた非公式の量子化版を使いました。量子化は、モデルの数値を小さい形式で保存する方法です。上流Qwenのrevisionは記録されていません。
Clefには、最初の実験と同じ画像・メール・動画を、同じ説明、質問、選択肢とともに渡しました。入力ファイルのSHA-256と指示を照合しています。Jev-Omniは、最初の実験で保存した回答を使いました。音声は比較していません。
選択肢の順番と確率
生成した入力が実験用の条件を満たすかを確認してから、モデルに渡しました。
順番を変えても、確率は選択肢の名前で対応づけて比べています。本文では小数点以下6桁に丸め、全桁は各回答のJSONに残しています。
確率はモデルが返した値です。判断が正しい確率として確かめたものではありません。
実験データの作り方
| 入力 | 作成方法 | 記録された主要な設定 |
|---|---|---|
| 音声 | ResembleAI/chatterbox-turbo | 英語、付属の標準音声、声の複製なし。mono 24 kHz PCM16 WAVを全長で使用 |
| 画像 | ByteDance/SDXL-Lightning | sdxl_lightning_4step_unet.safetensors、4ステップ、guidance 0、1024×1024。生成PNGを品質90のJPEGに変換して評価 |
| メール | 実験用の架空の日本語メールを直接作成 | 生成モデルは使用していない |
| 動画 | MiniMaxAI/MiniMax-H3とAlibaba PDD8 | 記録されたAPI設定は9ステップ、flow shift 12、audio flow shift 3。短辺768、16:9、4秒を指定 |
PDD8で動画を生成しましたが、APIに指定したステップ数は9です。音声APIではseedが公開されていませんでした。画像・動画で指定したseedは生成記録にあります。
各入力のプロンプト、設定、作り直した記録は、生成記録のJSONと作成方法の説明にあります。
動画の読み込み方
評価には、生成した動画の先頭4秒を使いました。24 fps、96フレームのMP4です。生成元は107フレーム(4.458333秒)ですが、その末尾は使っていません。
| モデル | 使用フレーム | 前処理 |
|---|---|---|
| Jev-Omni | 16枚。0.0秒から3.75秒まで0.25秒間隔 | 長辺を最大768ピクセルに調整 |
| Clef | 8枚。source index [0,14,27,41,54,68,81,95] | 先頭・末尾を含む。事前resizeなしでdecodeし、Qwenの画像処理を適用 |
同じ動画ファイルでも、モデルが使うフレーム数と画像処理には違いがあります。処理時間は環境や計測範囲が異なり、各1回の記録しかないため、速さの比較には使っていません。
35件のデータで試しました
各種類10件、計40件を用意する予定でした。実験用の条件を満たしたのは、音声10件、画像10件、メール10件、動画5件です。用意できなかった動画5件は、不正解として数えていません。
| 入力 | 採用数 / 予定数 | Jev-Omni:元の順番 / 逆順 | Clef:元の順番 / 逆順 |
|---|---|---|---|
| 英語の音声 | 10/10 | 10/10・10/10 | 比較していない |
| 物の画像 | 10/10 | 10/10・9/10 | 10/10・10/10 |
| 日本語メール | 10/10 | 10/10・9/10 | 10/10・10/10 |
| 生成した動画 | 5/10 | 3/5・3/5 | 5/5・5/5 |
音声10件:苦情と通常の問い合わせ
5件は苦情、5件は通常の問い合わせです。不具合を訴え、対応を求める内容をComplaintとしました。購入前の確認などはOrdinary inquiryとしました。丁寧な言い方でも、不具合への対応を求めていれば苦情に含めます。
Jev-Omniには実際のWAVを渡しました。下の原稿は音声を生成するために書いたもので、書き起こしではありません。
音声の内容は、別の音声認識処理をCPUで動かして確認しました。人が聴いて確認した記録はありません。A01のarrange delivery付近は認識が不確定で、A07・A09はWhisper base.enで確認し直しています。詳しくは音声の対応表にあります。
10件とも、選択肢の順番を変えても想定した答えと一致しました。
A01:配達済みなのに未着
想定した答え:Complaint
配送の失敗を訴え、配達か返金を求めています。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.881104 | 0.999877 |
| Ordinary inquiry | 0.118896 | 0.000123 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A01.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A01.reverse
生成プロンプトと設定
The tracking page says my parcel was delivered yesterday, but it never arrived. I checked the front door and asked my neighbours. Please find out where it went and arrange delivery or a refund.
{
"fixture_id": "A01",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A01-attempt-01",
"selected_attempt_index": 1,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/asset.wav",
"sha256": "e800911c146952b239ca3cf79df4ed73e40a2806fdb2a721eed19f69854a8865",
"bytes": 510764
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/asset.wav",
"sha256": "e800911c146952b239ca3cf79df4ed73e40a2806fdb2a721eed19f69854a8865",
"bytes": 510764
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/receipt.json",
"sha256": "507ed6c07f101f640fe9e0f6731196b622c32ea7ce38e96dc448c46c27b68dde",
"bytes": 2776
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 10.64
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/generation.request.json",
"sha256": "ccdcdf5f85d9d2e35657d2db100f2453beefb6fce960f0f53835099aed6651f6",
"bytes": 257
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/generation.json",
"sha256": "ce478961f08d999125ba2f9ea00dd00c1a2a3bf5a42ceac898b1d1324110e704",
"bytes": 5426
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A02:二重請求の取り消し
想定した答え:Complaint
二重請求を訴え、余分な請求の取り消しを求めています。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.866133 | 0.999871 |
| Ordinary inquiry | 0.133868 | 0.000129 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A02.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A02.reverse
生成プロンプトと設定
I ordered one desk lamp, but my card was charged twice for the same order. The two payments have both posted. Could you please reverse the extra charge?
{
"fixture_id": "A02",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A02-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/asset.wav",
"sha256": "d008877d32667d9b9eb51bacaeb1eacb2ff7863fd337d5f9122b839cf94d0d71",
"bytes": 430124
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/asset.wav",
"sha256": "d008877d32667d9b9eb51bacaeb1eacb2ff7863fd337d5f9122b839cf94d0d71",
"bytes": 430124
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/receipt.json",
"sha256": "afd31a9007a91d50903474c922276bb8a198de2a6ba18f93118e055b1ee0bfd4",
"bytes": 1556
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 8.96
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/generation.request.json",
"sha256": "e4aa97475b89879b93f0dfd8e191401d07362fa60de7e25823cc26a9626dc44f",
"bytes": 216
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/generation.json",
"sha256": "bb5552a52f74fcb047a04b023b9f6d1aa94fda98996dbc45a97c98a0c90eec35",
"bytes": 5385
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A03:壊れたケトルの交換
想定した答え:Complaint
届いた商品が壊れていたと報告し、交換を求めています。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.759225 | 0.999659 |
| Ordinary inquiry | 0.240775 | 0.000341 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A03.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A03.reverse
生成プロンプトと設定
The kettle arrived this morning with a cracked handle, so I cannot use it safely. Please tell me how to get a replacement for the damaged one.
{
"fixture_id": "A03",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A03-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/asset.wav",
"sha256": "f67d95c676f356236d4bf8bb642ca4c7040e1bc0eeac6f7d4af36881b38080b5",
"bytes": 351404
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/asset.wav",
"sha256": "f67d95c676f356236d4bf8bb642ca4c7040e1bc0eeac6f7d4af36881b38080b5",
"bytes": 351404
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/receipt.json",
"sha256": "7b228f2ec3c6ddc09adc6a5f47cfc24df7bc3f872bb902343c1a52a6b2ab4439",
"bytes": 1529
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 7.32
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/generation.request.json",
"sha256": "351d1b49bee8ffb41e49b70baf60c1c209b65120f0ffcd2b9dda14c9c6c9f3a9",
"bytes": 206
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/generation.json",
"sha256": "5c14507ae0bd331d961bb9cd89f0e4d1fef308f0e4820dc6988068e2147caf4f",
"bytes": 5375
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A04:約束した折り返し電話がない
想定した答え:Complaint
丁寧な言い方ですが、約束した折り返し電話がないことへの対応を求めています。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.826151 | 0.999812 |
| Ordinary inquiry | 0.173849 | 0.000188 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A04.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A04.reverse
生成プロンプトと設定
Hello, I know your team is busy. You said someone would call me back by Tuesday, and it is Thursday now. I have not heard from anyone. Could you please arrange that callback?
{
"fixture_id": "A04",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A04-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/asset.wav",
"sha256": "6f7d5f65db14fbf9b88268e634b49757c76b61f358fc8796e5fcd830e651fdd7",
"bytes": 460844
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/asset.wav",
"sha256": "6f7d5f65db14fbf9b88268e634b49757c76b61f358fc8796e5fcd830e651fdd7",
"bytes": 460844
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/receipt.json",
"sha256": "999b90ab815a04a3b453eb1a8ef6451ca4f401fae6115801309d7ae971d7130f",
"bytes": 1605
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 9.6
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/generation.request.json",
"sha256": "da956535de808091ebfdffb4cf158914701162cb82d9fdc9e6329fc06176b74d",
"bytes": 238
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/generation.json",
"sha256": "58db357073faca8ba978fe23c40786744452c996bda4f373936f83bdde99e4d6",
"bytes": 5407
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A05:ノートの紙への不満
想定した答え:Complaint
使った商品の紙質に不満があり、対応を求めています。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.903379 | 0.999925 |
| Ordinary inquiry | 0.096621 | 0.000075 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A05.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A05.reverse
生成プロンプトと設定
I tried the new notebook this week. The pages are thinner than I expected, and ink shows through when I write. It is a little disappointing. Is there anything you can do about it?
{
"fixture_id": "A05",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A05-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/asset.wav",
"sha256": "c0d801a429dfd9312491b4be3c2423647045c2a6be03c8094fbea48faa45e463",
"bytes": 445484
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/asset.wav",
"sha256": "c0d801a429dfd9312491b4be3c2423647045c2a6be03c8094fbea48faa45e463",
"bytes": 445484
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/receipt.json",
"sha256": "c91492a815d9370c66c7a6c88685d09104f067a97d22149cbfe62044426d0124",
"bytes": 1615
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 9.28
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/generation.request.json",
"sha256": "29e760e36441683f98fe17f0283186931c6413930c83ff6fe3577f60ad60f3a7",
"bytes": 243
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/generation.json",
"sha256": "e8344046caf8278b5e6a0e04e41bd553cc6df993c17fe11656cd25bccd138b02",
"bytes": 5412
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A06:土曜日の開店時間
想定した答え:Ordinary inquiry
不具合は訴えず、営業時間を尋ねています。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.000253 | 0.018405 |
| Ordinary inquiry | 0.999747 | 0.981595 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A06.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A06.reverse
生成プロンプトと設定
Hi, what time does your shop open on Saturdays? I would like to visit before lunch.
{
"fixture_id": "A06",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A06-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/asset.wav",
"sha256": "6e237b86d8c3f23de45baa7b0d03875983515fca29e9844d6710f87a2b976607",
"bytes": 251564
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/asset.wav",
"sha256": "6e237b86d8c3f23de45baa7b0d03875983515fca29e9844d6710f87a2b976607",
"bytes": 251564
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/receipt.json",
"sha256": "4971da0cbd6bc846d5e5c9139a470fade40ff9b204e3e80c79c2f673a4b4da72",
"bytes": 1466
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 5.24
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/generation.request.json",
"sha256": "dc6150fb5c906e0fcd77bcc8e3f9439fe5f39c8980fed749b27de6e8b53fd1f3",
"bytes": 147
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/generation.json",
"sha256": "b79c742a5de60582587b9e23e2bc2874d025dc40c3d866d71560e57bf31cc189",
"bytes": 5316
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A07:購入前の配送時期の確認
想定した答え:Ordinary inquiry
購入前に、配送時期を尋ねています。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.000197 | 0.026156 |
| Ordinary inquiry | 0.999803 | 0.973844 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A07.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A07.reverse
生成プロンプトと設定
If I place an order for the blue chair today, when would you expect it to arrive? I have not ordered yet and am planning a room update.
{
"fixture_id": "A07",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A07-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/asset.wav",
"sha256": "2e4f53fe892d39b56c291904430db5e920cf35c37f45617b643840014f121735",
"bytes": 382124
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/asset.wav",
"sha256": "2e4f53fe892d39b56c291904430db5e920cf35c37f45617b643840014f121735",
"bytes": 382124
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/receipt.json",
"sha256": "ada91499e1f86c9478006cfe9ec3777769f90035b9d69ce6d1fdfa2dccf95bc4",
"bytes": 2280
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 7.96
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/generation.request.json",
"sha256": "ff038681f92f9c842a7403ce9772f6b4cb3b99d64e4c8d717f8e4b2f107982f9",
"bytes": 199
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/generation.json",
"sha256": "5df713a744a58368b1a4d8b9132ab1aeeb197fac4bb0c34ffebbc62aed91cda5",
"bytes": 5368
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A08:購入前の返品条件の確認
想定した答え:Ordinary inquiry
購入前に返品条件を尋ねています。商品やサービスの不具合は訴えていません。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.000404 | 0.054218 |
| Ordinary inquiry | 0.999596 | 0.945782 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A08.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A08.reverse
生成プロンプトと設定
Could you explain your return policy for an unopened item? I am comparing gift options and would like to know the return window before buying.
{
"fixture_id": "A08",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A08-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/asset.wav",
"sha256": "1cce8b3404b77b4da2ef342166209f8293ed97cf2bc931e13e430029f309a1f0",
"bytes": 389804
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/asset.wav",
"sha256": "1cce8b3404b77b4da2ef342166209f8293ed97cf2bc931e13e430029f309a1f0",
"bytes": 389804
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/receipt.json",
"sha256": "6c4354f781fe2a6c47dd5df69824132b19d70a14737ba92c76e55f846252e67c",
"bytes": 1561
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 8.12
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/generation.request.json",
"sha256": "d11f045befccd65cbdc20dc971a63a143717c615b6e6be0a7db5da7f91dbf852",
"bytes": 206
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/generation.json",
"sha256": "aa2f468e032a756233827f702b76185cd8a869473239cfac70e745ed27b5d5bb",
"bytes": 5375
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A09:期限内の注文の状況確認
想定した答え:Ordinary inquiry
約束した期限の前に、注文の状況を尋ねています。遅延は訴えていません。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.000496 | 0.072014 |
| Ordinary inquiry | 0.999504 | 0.927986 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A09.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A09.reverse
生成プロンプトと設定
My order is due to arrive next Friday, so it is still within the promised time. Could you tell me whether it has been packed yet? I just want to plan when to be home.
{
"fixture_id": "A09",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A09-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/asset.wav",
"sha256": "fa96dafcf7bf7950302b74fe4997ef448448da2be34bd2ff945538534454e350",
"bytes": 420524
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/asset.wav",
"sha256": "fa96dafcf7bf7950302b74fe4997ef448448da2be34bd2ff945538534454e350",
"bytes": 420524
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/receipt.json",
"sha256": "69525170da853cf10f4b3dffb6700dbb615a55bf3cf273b1d912b8f5a4a70fdb",
"bytes": 2358
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 8.76
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/generation.request.json",
"sha256": "c2906ff23971feed0497761e745388dde18d3d9d6f50c274f7866facb587632e",
"bytes": 230
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/generation.json",
"sha256": "3f3859c14454928fd187e7f8a60fbe81c3fe76239efec2cc511a6c4f8a7fb20f",
"bytes": 5399
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A10:満足している商品の使い方
想定した答え:Ordinary inquiry
商品に満足していると伝え、使い方を尋ねています。
質問と選択肢
モデルに伝えた説明
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
質問
Is the audible customer message a complaint or an ordinary inquiry?
元の順番
["Complaint", "Ordinary inquiry"]
逆順
["Ordinary inquiry", "Complaint"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 |
|---|---|---|
| Complaint | 0.000330 | 0.038683 |
| Ordinary inquiry | 0.999670 | 0.961316 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.A10.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.A10.reverse
生成プロンプトと設定
I am happy with the speaker and I am not complaining. I just need a little help pairing it with my tablet. Which button should I hold to start pairing?
{
"fixture_id": "A10",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A10-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/asset.wav",
"sha256": "442eddd9ccc75ba77235d0feb6cef7259201b61f05a542cdfa87fc2bc6e2633e",
"bytes": 374444
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/asset.wav",
"sha256": "442eddd9ccc75ba77235d0feb6cef7259201b61f05a542cdfa87fc2bc6e2633e",
"bytes": 374444
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/receipt.json",
"sha256": "527db193157f6a7cbe2c43112f1b771c75187410b012eb41ece1915064e89ae5",
"bytes": 1569
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 7.8
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/generation.request.json",
"sha256": "0b27bafa5a2b73cdd2a6628300ad10828a0b6b7ff05fd1407bad1ac20f41daef",
"bytes": 215
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/generation.json",
"sha256": "7fc58929b760c5c89f0a4942ac17500a3947c5a15fec5704c2a6159a80526419",
"bytes": 5384
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
画像10件:写っている物を選ぶ
画像の主な物体を、4つの選択肢から選ばせました。下の画像は、モデルに渡したJPEGです。
I06のザルでは、Jev-Omniの答えがColander(ザル)からSaucepan(鍋)に変わりました。Clefはどちらの順番でもColanderを選びました。
I01:リンゴ

想定した答え:Apple
果実の形、木質の軸、つやのある皮がリンゴの特徴に合っています。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Apple", "Pear", "Tomato", "Peach"]
逆順
["Peach", "Tomato", "Pear", "Apple"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Apple | Apple | 一致 / 一致 |
| Clef | Apple | Apple | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Apple | 0.999727 | 0.999759 | 0.986045 | 0.988375 |
| Pear | 0.000078 | 0.000190 | 0.005508 | 0.003795 |
| Tomato | 0.000022 | 0.000028 | 0.004290 | 0.003915 |
| Peach | 0.000173 | 0.000024 | 0.004158 | 0.003915 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I01.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I01.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I01.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I01.reverse
生成プロンプトと設定
A single red-green apple resting on a pale wooden kitchen counter near a window. Clearly show the apple rounded fruit body, shallow top dimple and short brown woody stem, smooth waxy skin; keep the whole fruit visible. Soft daylight, close product photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I01",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I01-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/asset.jpg",
"sha256": "4315bc149771bfd736a920262f0880ca714d6d6ac968e7f26c27e640b978638d",
"bytes": 87892
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/native.png",
"sha256": "61903b8cbea29daedff7915dd2270048a4b4a18d829ea3d6d579fd7283836c3d",
"bytes": 917357
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/receipt.json",
"sha256": "74c71f861337dc3579797b017c8e4b8cb9ab83ea7bc0b11437931d192206cb82",
"bytes": 1296
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100201,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100201,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100201,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/generation.request.json",
"sha256": "387292ebf2b24f3990c0aee84699ead4128f2d3018d877c14004a4e355fde2de",
"bytes": 564
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/generation.json",
"sha256": "bae00b6b89b26600141479b43bcabf35cf6c63126f0fe0ccde616887593c0618",
"bytes": 31259
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I02:陶器のティーポット

想定した答え:Ceramic teapot
陶器の本体に、ふた、注ぎ口、持ち手があります。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Electric kettle", "Ceramic teapot", "Watering can", "Moka pot"]
逆順
["Moka pot", "Watering can", "Ceramic teapot", "Electric kettle"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Ceramic teapot | Ceramic teapot | 一致 / 一致 |
| Clef | Ceramic teapot | Ceramic teapot | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Electric kettle | 0.000135 | 0.001426 | 0.009908 | 0.004438 |
| Ceramic teapot | 0.998972 | 0.995985 | 0.971916 | 0.980958 |
| Watering can | 0.000200 | 0.002410 | 0.014643 | 0.010983 |
| Moka pot | 0.000693 | 0.000179 | 0.003533 | 0.003622 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I02.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I02.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I02.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I02.reverse
生成プロンプトと設定
A single ivory glazed ceramic teapot on a dark dining table, three-quarter view. Clearly show its round ceramic body, removable lid with knob, curved pouring spout and loop handle, with no electrical cord or base. Natural afternoon light, realistic still life photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I02",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I02-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/asset.jpg",
"sha256": "fd04ead97f99532645ff4205bb5ef7a4a60f718bc924956165404a2e74447f72",
"bytes": 72057
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/native.png",
"sha256": "902e7bb898901204b5e9cb9352e3ebf5bc46cdc9f9927ebc70a1854faa4419c8",
"bytes": 893459
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/receipt.json",
"sha256": "92ae18ae18f11633424ee64e0283a87bd7fbb7a50d520805ff7db7f9b34dc7ef",
"bytes": 1279
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100202,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100202,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100202,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/generation.request.json",
"sha256": "f284a29e960718cd69c04732151f510046cb538b347e7b6579943d0a882fe212",
"bytes": 576
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/generation.json",
"sha256": "d8a3ff8a9ee5bf6db72e524bd2d28a40fce5e4317fd5e90c94552a2809e058c1",
"bytes": 31271
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I03:釘抜き付きハンマー

想定した答え:Claw hammer
平らな打撃面の反対側に、二股の釘抜きがあります。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Mallet", "Pickaxe", "Claw hammer", "Screwdriver"]
逆順
["Screwdriver", "Claw hammer", "Pickaxe", "Mallet"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Claw hammer | Claw hammer | 一致 / 一致 |
| Clef | Claw hammer | Claw hammer | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Mallet | 0.023393 | 0.002180 | 0.005112 | 0.008994 |
| Pickaxe | 0.056227 | 0.000686 | 0.002694 | 0.003991 |
| Claw hammer | 0.884709 | 0.996538 | 0.989583 | 0.984186 |
| Screwdriver | 0.035670 | 0.000596 | 0.002611 | 0.002830 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I03.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I03.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I03.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I03.reverse
生成プロンプトと設定
A single steel claw hammer with a wooden handle lying diagonally on a clean workshop bench. Clearly show one flat striking face and an opposite curved split claw, with the full handle visible. Detailed realistic photograph, side view. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I03",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I03-attempt-02",
"selected_attempt_index": 2,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/asset.jpg",
"sha256": "55f1e36f8b2b10df9e640424b3ac47b01d3609c709bc8f72c6a3fcbd57ab02a1",
"bytes": 175434
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/native.png",
"sha256": "ebbdd19af1be1b72f43124e8fbaa81aab0d6449fc05843fa5661a210be79e394",
"bytes": 1398723
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/receipt.json",
"sha256": "5038bb00814fc84d81f87addaa16961b53f6310d06c82664d0c50ac6fd32b4e3",
"bytes": 1296
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026102203,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026102203,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026102203,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/generation.request.json",
"sha256": "b4f8a36caea0535b95dd864466ecd567f0d305410cd87b5f4b0bc29a7ab9784f",
"bytes": 539
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/generation.json",
"sha256": "72b7323cbe39bc6aef373154aa5bf609ce0ee5d90910da8b0091dafa2a3ff96b",
"bytes": 31234
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I04:自転車

想定した答え:Bicycle
2つの車輪、フレーム、ペダル、チェーンがあり、モーターはありません。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Motorcycle", "Kick scooter", "Tricycle", "Bicycle"]
逆順
["Bicycle", "Tricycle", "Kick scooter", "Motorcycle"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Bicycle | Bicycle | 一致 / 一致 |
| Clef | Bicycle | Bicycle | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Motorcycle | 0.000013 | 0.000167 | 0.003824 | 0.004034 |
| Kick scooter | 0.000028 | 0.000022 | 0.003374 | 0.003560 |
| Tricycle | 0.000160 | 0.000073 | 0.004612 | 0.005344 |
| Bicycle | 0.999799 | 0.999738 | 0.988190 | 0.987062 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I04.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I04.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I04.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I04.reverse
生成プロンプトと設定
A single ordinary pedal bicycle parked beside a brick courtyard wall, entire bicycle visible from the side. Clearly show exactly two wheels, triangular frame, pedals and chain, saddle and handlebars, no motor. Overcast daylight, realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I04",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I04-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/asset.jpg",
"sha256": "0f07aec0990918bb711a076b15e87a3729dc7161818d1b8782b1e75f31c18a6b",
"bytes": 317955
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/native.png",
"sha256": "4fc3d4e678c178e1cfbd13680a4bdf4e21d6c86a6bc6e06402088d792884958c",
"bytes": 1805676
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/receipt.json",
"sha256": "c718e983d710a3e716d64878ba45d37245830843971c3d716769cb544a9637e3",
"bytes": 1309
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100204,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100204,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100204,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/generation.request.json",
"sha256": "f8faebb0705b5d97435eb0fa3e0bb73bcb83e7450bfafa614f02b6498348e35f",
"bytes": 555
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/generation.json",
"sha256": "fa20664a56043e029cd4390e77c7ef76f541a2362be361e15174ce55a35d674f",
"bytes": 31250
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I05:双眼鏡

想定した答え:Binoculars
2つの筒が中央でつながり、接眼部も2つあります。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Binoculars", "Camera", "VR goggles", "Telescope"]
逆順
["Telescope", "VR goggles", "Camera", "Binoculars"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Binoculars | Binoculars | 一致 / 一致 |
| Clef | Binoculars | Binoculars | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Binoculars | 0.999629 | 0.999916 | 0.987192 | 0.991572 |
| Camera | 0.000101 | 0.000057 | 0.004754 | 0.002273 |
| VR goggles | 0.000017 | 0.000019 | 0.003588 | 0.002742 |
| Telescope | 0.000253 | 0.000008 | 0.004466 | 0.003412 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I05.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I05.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I05.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I05.reverse
生成プロンプトと設定
A single black pair of binoculars on a picnic blanket, three-quarter view. Clearly show two parallel optical barrels joined by a central hinge and focus wheel, two front objective lenses and two rear eyepieces. Crisp outdoor photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I05",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I05-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/asset.jpg",
"sha256": "e3a5ebdc541e62cb65ebddf267205b9a7f581a7b9e671c978b7c0fea2efbf0ba",
"bytes": 153600
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/native.png",
"sha256": "3752690855410af9813e6f9474afda490e68a0d108d9024edf1bc70e3e2158f1",
"bytes": 1235886
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/receipt.json",
"sha256": "4d17193ec7704dd1b627f44902e1e49bce44e7336cc3b73f20475ca9c50f4f19",
"bytes": 1368
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100205,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100205,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100205,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/generation.request.json",
"sha256": "c87ac9cac8415fb843244998f6e6ca8d2f4f1077f59eec58c5901a0deb8189cf",
"bytes": 541
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/generation.json",
"sha256": "b55ffe7fa6424cd7e62c25b8d81e63bfd472396d25913f0e30e3a43b38577aab",
"bytes": 31236
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I06:ザル

想定した答え:Colander
穴のあるボウルに持ち手と脚があり、ザルの特徴に合っています。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Mixing bowl", "Colander", "Fine-mesh sieve", "Saucepan"]
逆順
["Saucepan", "Fine-mesh sieve", "Colander", "Mixing bowl"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Colander | Saucepan | 一致 / 不一致 |
| Clef | Colander | Colander | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Mixing bowl | 0.268925 | 0.103956 | 0.022317 | 0.018830 |
| Colander | 0.476610 | 0.213720 | 0.952654 | 0.950817 |
| Fine-mesh sieve | 0.019216 | 0.178570 | 0.009450 | 0.016360 |
| Saucepan | 0.235249 | 0.503754 | 0.015580 | 0.013993 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I06.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I06.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I06.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I06.reverse
生成プロンプトと設定
A single stainless steel colander standing on a kitchen worktop, empty and viewed slightly from above. Clearly show its bowl shape with discrete round drainage holes across the walls and base, two side handles and a foot ring; holes are punched metal rather than mesh. Realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I06",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I06-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/asset.jpg",
"sha256": "bf43d2e6faf01aa3fba5486488fa3d58541debe9931d43647bc034f6b0d9da3f",
"bytes": 132115
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/native.png",
"sha256": "dd3b2aa7ecde89a437b58482bdff672515c2e8e2344d1dff5d588d4a91fb05d5",
"bytes": 1128379
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/receipt.json",
"sha256": "cc8ed1d0054b9fccbda890e1492282a12beb8969c7ccbaae494f3851f916c38e",
"bytes": 1338
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100206,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100206,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100206,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/generation.request.json",
"sha256": "f41e493e7efe1e50c3805b88b09cc285072b90218e95a5c8514c849c7ee17f27",
"bytes": 595
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/generation.json",
"sha256": "7a08671a999d7d4b2c539884b3990f7aff418f7916eaf004afb3f4277d319727",
"bytes": 31290
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I07:磁気コンパス

想定した答え:Magnetic compass
向きを示す針とケースが見えます。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Wristwatch", "Barometer", "Magnetic compass", "Thermometer"]
逆順
["Thermometer", "Magnetic compass", "Barometer", "Wristwatch"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Magnetic compass | Magnetic compass | 一致 / 一致 |
| Clef | Magnetic compass | Magnetic compass | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Wristwatch | 0.005407 | 0.003093 | 0.002242 | 0.002847 |
| Barometer | 0.040501 | 0.001165 | 0.002386 | 0.003892 |
| Magnetic compass | 0.938139 | 0.994777 | 0.993330 | 0.990230 |
| Thermometer | 0.015954 | 0.000966 | 0.002041 | 0.003031 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I07.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I07.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I07.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I07.reverse
生成プロンプトと設定
A single open brass pocket magnetic compass on a flat rock outdoors. Clearly show a circular directional dial with simple untitled tick marks and a freely pivoting red-and-white needle pointing across it, a hinged cover and no watch hands, wrist strap or numeric readout. Close realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I07",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I07-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/asset.jpg",
"sha256": "7bd79a2659dac9281921a8c5304770abfecafb1b46c203d9eaf944edb130032b",
"bytes": 230111
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/native.png",
"sha256": "cc87cd16d34a666b2fba28d8240a43a7849015a6b8d96595528b0a056c9de999",
"bytes": 1569534
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/receipt.json",
"sha256": "e582bd37bc07321f79741b1162cc735ceda4f872873519738163a802132c5c4e",
"bytes": 1390
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100207,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100207,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100207,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/generation.request.json",
"sha256": "9b1f5fb4bcd85849b296eb85b2399dda65a126f3e90fa62463cbd219fa55222e",
"bytes": 604
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/generation.json",
"sha256": "5a960474c1be11cabad219fa0ab9c899a15b8a6408af6da2ce6c877645f8dd8c",
"bytes": 31299
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I08R:卓上扇風機

想定した答え:Electric desk fan
円形の保護枠の中に羽根があり、台座で支えられています。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Table lamp", "Loudspeaker", "Humidifier", "Electric desk fan"]
逆順
["Electric desk fan", "Humidifier", "Loudspeaker", "Table lamp"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Electric desk fan | Electric desk fan | 一致 / 一致 |
| Clef | Electric desk fan | Electric desk fan | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Table lamp | 0.000015 | 0.000289 | 0.001861 | 0.002577 |
| Loudspeaker | 0.000026 | 0.000035 | 0.001495 | 0.002346 |
| Humidifier | 0.000128 | 0.000158 | 0.002044 | 0.003108 |
| Electric desk fan | 0.999831 | 0.999517 | 0.994600 | 0.991969 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I08R.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I08R.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I08R.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I08R.reverse
生成プロンプトと設定
A single electric desk fan on a plain desk. Clearly visible rotor blades inside a round protective grille, supported by a standing pedestal. Realistic photograph, full object visible. No text, labels, logos, watermarks or other salient objects.
{
"fixture_id": "I08R",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I08R-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/asset.jpg",
"sha256": "9c455bfcb3271d0bf877ff52e6ad886b53190ec090cbedbad3c17a9643c1f284",
"bytes": 172973
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/native.png",
"sha256": "52b582b2de26dd7fdeab2751f20c287103ecbe6caa337fd8361939cfbb374b86",
"bytes": 1144635
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/receipt.json",
"sha256": "509d7010b76570c73a49b10c1af8ff686021281aa6ef6571913d68c0e9648520",
"bytes": 1290
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100308,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100308,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100308,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/generation.request.json",
"sha256": "28edd8216f0294a8cb1aa4727fd998cd2f4a3f6c17b5f7a3dcc2ee31cc00ee52",
"bytes": 412
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/generation.json",
"sha256": "e56b4e1cd6226e7ef135caed9e0312eb39c93ded3eb721f82d80aef27d3a4805",
"bytes": 30154
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input.",
"replacement_lineage": {
"replaces_fixture": "I08",
"original_target": "Winged corkscrew",
"new_target": "Electric desk fan",
"authorized_before_predictions": true,
"amendment": {
"path": "2026-10-02-jev-omni-40-synthetic/amendment.json",
"sha256": "294c7e0960d30a8615dd98f787dede79772bf4bf3b992b9917ad6473ee81cf42",
"bytes": 2274
},
"amendment_freeze": {
"path": "2026-10-02-jev-omni-40-synthetic/amendment.freeze.json",
"sha256": "4f52bdf1554f2aa9c06f64754f47c84b353cdb1747cacd537102223ae6e4d005",
"bytes": 225
},
"reason": "All three frozen winged-corkscrew attempts independently rejected for missing diagnostic helix/collar. User authorized one fixed everyday-object substitution before any classifier output; original protocol and rejected assets remain immutable."
}
}
I09:バドミントンのシャトル

想定した答え:Badminton shuttlecock
丸いコルクの底の周りに羽根が並んでいます。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Badminton shuttlecock", "Feather duster", "Table-tennis ball", "Spinning top"]
逆順
["Spinning top", "Table-tennis ball", "Feather duster", "Badminton shuttlecock"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Badminton shuttlecock | Badminton shuttlecock | 一致 / 一致 |
| Clef | Badminton shuttlecock | Badminton shuttlecock | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Badminton shuttlecock | 0.999583 | 0.999815 | 0.988556 | 0.990326 |
| Feather duster | 0.000123 | 0.000129 | 0.005436 | 0.004143 |
| Table-tennis ball | 0.000024 | 0.000040 | 0.003098 | 0.002938 |
| Spinning top | 0.000269 | 0.000016 | 0.002910 | 0.002593 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I09.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I09.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I09.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I09.reverse
生成プロンプトと設定
A single white feather badminton shuttlecock on a blue sports-court floor, close side view. Clearly show a small rounded cork base and an open tapered crown of individually visible feathers secured in a circular skirt. Entire object in focus, realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I09",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I09-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/asset.jpg",
"sha256": "7fcf2f7a9ef0967c693a2dbcebcad84f6b58700057f4f1dd3d3c0cb1752168d0",
"bytes": 113819
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/native.png",
"sha256": "8b2c8c9b0f017e862b387d6f28ea3f563ea1184e1fef170f0648319d5700453e",
"bytes": 1080435
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/receipt.json",
"sha256": "fd6d6b2ba11ebd6afc5e9f8f3577cf16bac6544b2cc15f6a17f879472b86c873",
"bytes": 1363
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100209,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100209,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100209,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/generation.request.json",
"sha256": "ba913aeea879195372a8e23a66207edd0f24c7fb70929ca21f396ff7d2df6c2a",
"bytes": 569
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/generation.json",
"sha256": "e5493587362c21658dc6cf800c492e2127701ccd5a973da29a620badbaa87550",
"bytes": 31264
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I10:テレビのリモコン

想定した答え:Television remote control
細長い本体にボタン、方向キー、赤外線の先端部があります。
質問と選択肢
モデルに伝えた説明
Identify the single principal foreground object using its visible shape and diagnostic features.
質問
Which option identifies the principal visible object?
元の順番
["Calculator", "Television remote control", "Mobile phone", "Game controller"]
逆順
["Game controller", "Mobile phone", "Television remote control", "Calculator"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Television remote control | Television remote control | 一致 / 一致 |
| Clef | Television remote control | Television remote control | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Calculator | 0.000131 | 0.001122 | 0.002589 | 0.003919 |
| Television remote control | 0.999112 | 0.996112 | 0.988725 | 0.989335 |
| Mobile phone | 0.000269 | 0.002537 | 0.003538 | 0.002164 |
| Game controller | 0.000488 | 0.000230 | 0.005148 | 0.004582 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.I10.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.I10.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.I10.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.I10.reverse
生成プロンプトと設定
A single matte black television remote control on a grey sofa cushion, full length visible from above. Clearly show an elongated narrow handheld body, many rubber buttons including a circular directional pad, colored buttons and a small infrared emitter at the top, with no screen or joysticks. Realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I10",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I10-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/asset.jpg",
"sha256": "1c1d8a0e11414eb2ecb0cded580f1200ed7a14bbe70fd1d845220e2723e290e4",
"bytes": 222750
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/native.png",
"sha256": "d4635642744dfd4845bb4b96dfc93a43219610025c30e401afd9b7120e27fd02",
"bytes": 1610855
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/receipt.json",
"sha256": "72bd07dc13b97009ed162c09b68ee69a14f4e9eb90bb42b7454b2c208d86c441",
"bytes": 1359
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100210,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100210,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100210,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/generation.request.json",
"sha256": "2fd12d6c46c75a127d85c881b212d8aa941f6287ee70a9c5b6f458b202e7dc57",
"bytes": 621
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/generation.json",
"sha256": "027e8ceeb27d464db91eb317ce194d493cf4a6a02014de4493f436306e60a5b2",
"bytes": 31316
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
日本語メール10件:返信が必要か
実験用に書いた架空の日本語メールを使いました。質問や対応の依頼が残っていればReply required、受領の連絡やお礼、解決済みの依頼だけならNo reply requiredとしました。
T01では、選択肢を逆の順番にすると、Jev-Omniの答えが「返信が必要」から「返信は不要」に変わりました。Clefはどちらでも「返信が必要」を選びました。残る9件は、両モデルとも両方の順番で想定した答えと一致しました。
T01:展示会の受付時間と予約
件名:展示会の受付時間について
架空ストア ご担当者様
来週の展示会への参加を検討しています。土曜日の受付は何時から何時まででしょうか。また、予約なしでも入場できますか。
お手数ですが、この二点について教えてください。
水野
想定した答え:Reply required
受付時間と入場方法について質問しています。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:展示会の受付時間について
架空ストア ご担当者様
来週の展示会への参加を検討しています。土曜日の受付は何時から何時まででしょうか。また、予約なしでも入場できますか。
お手数ですが、この二点について教えてください。
水野
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Reply required | No reply required | 一致 / 不一致 |
| Clef | Reply required | Reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.958538 | 0.086016 | 0.991015 | 0.991875 |
| No reply required | 0.041462 | 0.913984 | 0.008985 | 0.008125 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T01.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T01.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T01.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T01.reverse
生成プロンプトと設定
{
"fixture_id": "T01",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T01-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T01-attempt-00/asset.txt",
"sha256": "ad63ba8c1b406ac1cd643ed0a86b5bffa40f611b21fa927ea5c407d84f455470",
"bytes": 330
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T01-attempt-00/asset.txt",
"sha256": "ad63ba8c1b406ac1cd643ed0a86b5bffa40f611b21fa927ea5c407d84f455470",
"bytes": 330
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T01-attempt-00/receipt.json",
"sha256": "c4827fa675d2508e80031ee587a4c38318041ed4fffa343e8e5c5ee2a6915f92",
"bytes": 506
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T02:届いた商品への対応依頼
件名:届いた商品についてのお願い
架空ストア ご担当者様
本日届いた注文番号 F-204 のマグカップにひびがありました。交換を希望します。返送方法と交換品の発送予定をお知らせください。
写真は手元にありますので、必要でしたら送ります。よろしくお願いいたします。
藤井
想定した答え:Reply required
返品方法と交換時期を尋ねています。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:届いた商品についてのお願い
架空ストア ご担当者様
本日届いた注文番号 F-204 のマグカップにひびがありました。交換を希望します。返送方法と交換品の発送予定をお知らせください。
写真は手元にありますので、必要でしたら送ります。よろしくお願いいたします。
藤井
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Reply required | Reply required | 一致 / 一致 |
| Clef | Reply required | Reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.999157 | 0.999831 | 0.989672 | 0.990514 |
| No reply required | 0.000843 | 0.000169 | 0.010328 | 0.009486 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T02.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T02.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T02.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T02.reverse
生成プロンプトと設定
{
"fixture_id": "T02",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T02-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T02-attempt-00/asset.txt",
"sha256": "2f059c84917dcc914502603dbd2011b2b554e6420ffb22c51ac79eb2e532dbd2",
"bytes": 382
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T02-attempt-00/asset.txt",
"sha256": "2f059c84917dcc914502603dbd2011b2b554e6420ffb22c51ac79eb2e532dbd2",
"bytes": 382
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T02-attempt-00/receipt.json",
"sha256": "2c903d2a95fd4f5d20338f417232b6f68de96e46d31499d42bce2448001894c0",
"bytes": 533
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T03:見積書の送付方法の確認
件名:見積書の送付方法の確認
架空サービス ご担当者様
見積書を受け取りました。内容は社内で確認済みです。経理担当を宛先に追加してこの見積書を転送してもよろしいでしょうか。差し支えないか、ご承認をお願いします。
承認をいただくまで転送は控えます。
長谷川
想定した答え:Reply required
受領の連絡に加えて、承認を求める依頼が残っています。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:見積書の送付方法の確認
架空サービス ご担当者様
見積書を受け取りました。内容は社内で確認済みです。経理担当を宛先に追加してこの見積書を転送してもよろしいでしょうか。差し支えないか、ご承認をお願いします。
承認をいただくまで転送は控えます。
長谷川
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Reply required | Reply required | 一致 / 一致 |
| Clef | Reply required | Reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.997566 | 0.999620 | 0.990441 | 0.991084 |
| No reply required | 0.002434 | 0.000380 | 0.009559 | 0.008916 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T03.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T03.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T03.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T03.reverse
生成プロンプトと設定
{
"fixture_id": "T03",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T03-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T03-attempt-00/asset.txt",
"sha256": "63a2143131c028debd55fb558b52899e14f9db26fb6af773149016f819354446",
"bytes": 375
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T03-attempt-00/asset.txt",
"sha256": "63a2143131c028debd55fb558b52899e14f9db26fb6af773149016f819354446",
"bytes": 375
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T03-attempt-00/receipt.json",
"sha256": "46109acbe5f7a23b4d358dad9fd8d53d71aba62ec306ed9c8f4112991590a79e",
"bytes": 532
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T04:資料の到着と確認の依頼
件名:資料の到着と確認のお願い
架空サービス ご担当者様
資料を受け取りました。ありがとうございます。添付した申込内容を確認いただき、変更が必要な場合は明日正午までにご連絡ください。変更がない場合も、その旨を一言ご返信ください。
確認結果を待って手続きを進めます。
森
想定した答え:Reply required
修正が必要でも不要でも、返信するよう求めています。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:資料の到着と確認のお願い
架空サービス ご担当者様
資料を受け取りました。ありがとうございます。添付した申込内容を確認いただき、変更が必要な場合は明日正午までにご連絡ください。変更がない場合も、その旨を一言ご返信ください。
確認結果を待って手続きを進めます。
森
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Reply required | Reply required | 一致 / 一致 |
| Clef | Reply required | Reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.995930 | 0.998901 | 0.992062 | 0.993359 |
| No reply required | 0.004070 | 0.001099 | 0.007938 | 0.006641 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T04.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T04.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T04.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T04.reverse
生成プロンプトと設定
{
"fixture_id": "T04",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T04-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T04-attempt-00/asset.txt",
"sha256": "35636e02433c3e36c0a425444b54eb19c7a145eb933cbb59f56842303357d818",
"bytes": 396
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T04-attempt-00/asset.txt",
"sha256": "35636e02433c3e36c0a425444b54eb19c7a145eb933cbb59f56842303357d818",
"bytes": 396
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T04-attempt-00/receipt.json",
"sha256": "48b620cfe9a89fbae4de5763516572060d812261a2c78ef3bcbf3b148b6107e2",
"bytes": 530
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T05:登録情報についての新しい質問
件名:登録情報について
架空サービス ご担当者様
先日は丁寧なご案内をありがとうございました。
> 登録時の電話番号は変更できますか。
> 会員画面から変更できます。
電話番号の変更は完了しました。なお、今回の連絡先メールアドレスの変更はまだできていません。変更手順をこのメールへの返信で教えていただけますか。
佐々木
想定した答え:Reply required
引用した過去の問題は解決済みですが、メールアドレスについて新しい質問があります。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:登録情報について
架空サービス ご担当者様
先日は丁寧なご案内をありがとうございました。
> 登録時の電話番号は変更できますか。
> 会員画面から変更できます。
電話番号の変更は完了しました。なお、今回の連絡先メールアドレスの変更はまだできていません。変更手順をこのメールへの返信で教えていただけますか。
佐々木
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Reply required | Reply required | 一致 / 一致 |
| Clef | Reply required | Reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.999117 | 0.999869 | 0.988492 | 0.989672 |
| No reply required | 0.000883 | 0.000131 | 0.011508 | 0.010328 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T05.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T05.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T05.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T05.reverse
生成プロンプトと設定
{
"fixture_id": "T05",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T05-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T05-attempt-00/asset.txt",
"sha256": "2e581409133ffbdbe34e5f4bc9aa95178fa79a390163d4ec9fe0d33b830ca57b",
"bytes": 460
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T05-attempt-00/asset.txt",
"sha256": "2e581409133ffbdbe34e5f4bc9aa95178fa79a390163d4ec9fe0d33b830ca57b",
"bytes": 460
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T05-attempt-00/receipt.json",
"sha256": "767714b150d42dd5d1ecb3ee8913b5749f6767d1c917e73d84d5f1b372610bc8",
"bytes": 541
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T06:資料を受け取った連絡
件名:資料受領のご連絡
架空サービス ご担当者様
資料を確かに受け取りました。わかりやすいご説明をありがとうございます。まずは社内で共有いたします。
取り急ぎ、受領のお礼まで。
小林
想定した答え:No reply required
受領の連絡とお礼だけで、未回答の依頼はありません。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:資料受領のご連絡
架空サービス ご担当者様
資料を確かに受け取りました。わかりやすいご説明をありがとうございます。まずは社内で共有いたします。
取り急ぎ、受領のお礼まで。
小林
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | 一致 / 一致 |
| Clef | No reply required | No reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.000123 | 0.000588 | 0.004265 | 0.004265 |
| No reply required | 0.999877 | 0.999412 | 0.995735 | 0.995735 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T06.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T06.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T06.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T06.reverse
生成プロンプトと設定
{
"fixture_id": "T06",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T06-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T06-attempt-00/asset.txt",
"sha256": "c3f82c557a725293b36d64c3048ec0a6fa91847c7ea08060073fe35b0a843b77",
"bytes": 267
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T06-attempt-00/asset.txt",
"sha256": "c3f82c557a725293b36d64c3048ec0a6fa91847c7ea08060073fe35b0a843b77",
"bytes": 267
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T06-attempt-00/receipt.json",
"sha256": "2b16b862691f37e66412a3fe441629d3c88618817e319b026bc41656e7fcdfa9",
"bytes": 516
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T07:交換品の受領と解決
件名:交換品を受け取りました
架空ストア ご担当者様
交換品が届き、正常に使えることを確認しました。返送も完了し、今回の件はすべて解決しました。迅速なご対応に感謝いたします。
田中
想定した答え:No reply required
問題への対応は終わっており、新しい依頼はありません。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:交換品を受け取りました
架空ストア ご担当者様
交換品が届き、正常に使えることを確認しました。返送も完了し、今回の件はすべて解決しました。迅速なご対応に感謝いたします。
田中
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | 一致 / 一致 |
| Clef | No reply required | No reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.000163 | 0.000856 | 0.005140 | 0.004905 |
| No reply required | 0.999837 | 0.999144 | 0.994860 | 0.995095 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T07.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T07.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T07.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T07.reverse
生成プロンプトと設定
{
"fixture_id": "T07",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T07-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T07-attempt-00/asset.txt",
"sha256": "c272d354f44f37bfaa99e9a4e97c94687bf8121afe9f85783c5104f47f43c8fb",
"bytes": 265
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T07-attempt-00/asset.txt",
"sha256": "c272d354f44f37bfaa99e9a4e97c94687bf8121afe9f85783c5104f47f43c8fb",
"bytes": 265
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T07-attempt-00/receipt.json",
"sha256": "9b190814dea66a12494273606796632959f18167b557ed17c3ee4f8c55e64ba2",
"bytes": 516
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T08:担当者変更のお知らせ
件名:担当者変更のお知らせ
架空サービス ご担当者様
来月から弊社の窓口担当が吉田に変わります。連絡先一覧は本日、共有済みです。これは情報共有のみのお知らせで、返信は不要です。
引き続きよろしくお願いいたします。
中村
想定した答え:No reply required
情報を共有し、返信は不要と伝えています。質問もありません。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:担当者変更のお知らせ
架空サービス ご担当者様
来月から弊社の窓口担当が吉田に変わります。連絡先一覧は本日、共有済みです。これは情報共有のみのお知らせで、返信は不要です。
引き続きよろしくお願いいたします。
中村
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | 一致 / 一致 |
| Clef | No reply required | No reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.000156 | 0.000473 | 0.004039 | 0.003679 |
| No reply required | 0.999844 | 0.999527 | 0.995961 | 0.996322 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T08.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T08.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T08.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T08.reverse
生成プロンプトと設定
{
"fixture_id": "T08",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T08-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T08-attempt-00/asset.txt",
"sha256": "6f7cfde952fe36c696c72be7d0058c25e6d1d78be5dec5c77948ab4880fc10aa",
"bytes": 321
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T08-attempt-00/asset.txt",
"sha256": "6f7cfde952fe36c696c72be7d0058c25e6d1d78be5dec5c77948ab4880fc10aa",
"bytes": 321
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T08-attempt-00/receipt.json",
"sha256": "d8710369b006bc7aeaf228328884e4ca311ced837fb0d0eb60c2befdc21f728f",
"bytes": 514
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T09:合意済みの訪問日程
件名:訪問日程の共有
架空サービス ご担当者様
先ほどお電話で合意したとおり、訪問日時は十月十五日午後二時で確定しています。当日は受付で担当の山田をお呼びします。
このメールは確定済みの日程を記録するためのご連絡です。追加の確認事項はありません。
井上
想定した答え:No reply required
合意済みの日程を共有しており、新しい確認は求めていません。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:訪問日程の共有
架空サービス ご担当者様
先ほどお電話で合意したとおり、訪問日時は十月十五日午後二時で確定しています。当日は受付で担当の山田をお呼びします。
このメールは確定済みの日程を記録するためのご連絡です。追加の確認事項はありません。
井上
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | 一致 / 一致 |
| Clef | No reply required | No reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.000207 | 0.001001 | 0.005865 | 0.005911 |
| No reply required | 0.999793 | 0.999000 | 0.994135 | 0.994089 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T09.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T09.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T09.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T09.reverse
生成プロンプトと設定
{
"fixture_id": "T09",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T09-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T09-attempt-00/asset.txt",
"sha256": "e8bfa4730c00e17275008b83872f9b4f69b614dcaba8bd8dfc84d0d0c973d1c3",
"bytes": 372
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T09-attempt-00/asset.txt",
"sha256": "e8bfa4730c00e17275008b83872f9b4f69b614dcaba8bd8dfc84d0d0c973d1c3",
"bytes": 372
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T09-attempt-00/receipt.json",
"sha256": "ff4d8e0201cd7fdce5072ef78a2453fc6d61f80d3526730f8db72175e22cf52d",
"bytes": 522
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T10:会員情報の更新完了
件名:会員情報の更新完了
架空サービス ご担当者様
ご案内どおり操作し、会員情報の更新が完了しました。保存後の画面でも変更内容を確認できました。
> 住所の修正方法を教えていただけますか。
> 設定画面の「会員情報」から修正できます。
上記は以前のやり取りです。今回は完了のご報告のみで、新たな質問や依頼はありません。
加藤
想定した答え:No reply required
引用した過去の依頼は解決済みで、本文に新しい依頼はありません。
質問と選択肢
モデルに伝えた説明
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:会員情報の更新完了
架空サービス ご担当者様
ご案内どおり操作し、会員情報の更新が完了しました。保存後の画面でも変更内容を確認できました。
> 住所の修正方法を教えていただけますか。
> 設定画面の「会員情報」から修正できます。
上記は以前のやり取りです。今回は完了のご報告のみで、新たな質問や依頼はありません。
加藤
質問
Does this email require a human reply?
元の順番
["Reply required", "No reply required"]
逆順
["No reply required", "Reply required"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | 一致 / 一致 |
| Clef | No reply required | No reply required | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Reply required | 0.000116 | 0.000579 | 0.008916 | 0.007877 |
| No reply required | 0.999884 | 0.999421 | 0.991084 | 0.992123 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.T10.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.T10.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.T10.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.T10.reverse
生成プロンプトと設定
{
"fixture_id": "T10",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T10-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T10-attempt-00/asset.txt",
"sha256": "cde565a898ffffbca2c7f184efa9cf5dad3d8ceb889e680ca28b71731996a2cc",
"bytes": 469
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T10-attempt-00/asset.txt",
"sha256": "cde565a898ffffbca2c7f184efa9cf5dad3d8ceb889e680ca28b71731996a2cc",
"bytes": 469
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T10-attempt-00/receipt.json",
"sha256": "0aa2d45ab1b4bb5f44dc5ef5eb9d5a1ca1a9ef0b8383f7750766519a9f242b86",
"bytes": 557
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
元の動画5本:危険な場面が含まれるか
動画の中に、自車がすぐに対応する必要のある危険な場面が映っているかを質問しました。その後に回避できても、途中に危険があればDangerousとする基準です。
V01は子どもが車の進路に入る動画、V02は別の車が前方へ入る動画です。この2本はDangerous、残る3本はSafeを想定した答えにしました。V01・V02に衝突や負傷は映っていません。
Jev-Omniは、どちらの順番でも5本すべてでSafeを選びました。Dangerousを想定した2本は0/2、Safeを想定した3本は3/3が一致しました。Clefは5本とも一致しました。
ただし、動画には回避や停止まで含まれています。どの時点を見て「危険」と答えるべきかを決めにくく、今回の動画と質問は性能比較に不向きでした。実際の運転で危険を判断できるかは、この結果では分かりません。
V01:子どもが自車の進路へ入る
想定した答え:Dangerous
子どもが自車の進路に入り、すぐに対応が必要な場面です。
質問と選択肢
モデルに伝えた説明
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
質問
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
元の順番
["Dangerous", "Safe"]
逆順
["Safe", "Dangerous"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Safe | Safe | 不一致 / 不一致 |
| Clef | Dangerous | Dangerous | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Dangerous | 0.131173 | 0.437343 | 0.916708 | 0.934395 |
| Safe | 0.868827 | 0.562657 | 0.083292 | 0.065605 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.V01.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.V01.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.V01.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.V01.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. On a quiet residential street in daylight, ego car moves steadily forward at a plausible moderate speed. A clearly visible fictional child on the right sidewalk approaches the curb from 0.4 seconds. At 1.6 seconds the child steps from the curb into the ego lane a short distance ahead, continuing toward the lane center. Show converging paths and rapidly shrinking stopping gap by 2.5 seconds; ego brakes promptly. End while separated before impact.
{
"fixture_id": "V01",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V01-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/asset.mp4",
"sha256": "dcd7a01b8b25b24b46df0a9af7508a5b2443bc200225bdd9df4587a241f56014",
"bytes": 3691855
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/native.mp4",
"sha256": "2d902c8c5e945b74f787ffdcae22d2406cb0a2d8051c0e7bad527b0d72a81450",
"bytes": 2577780
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/receipt.json",
"sha256": "9148fd6d0fef2f3eac570f287fcda50ccf838c9d4060ddc74179c41085d05da9",
"bytes": 3576
},
"observed_media": {
"native": {
"sha256": "2d902c8c5e945b74f787ffdcae22d2406cb0a2d8051c0e7bad527b0d72a81450",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 2577780
},
"evaluated_transport": {
"sha256": "dcd7a01b8b25b24b46df0a9af7508a5b2443bc200225bdd9df4587a241f56014",
"bytes": 3691855,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026100201,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026100201
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026100201,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/generation.request.json",
"sha256": "6751480e8f8b19faa6579718fd1920fc9b55cad49bc752dfe947de109f7928a9",
"bytes": 850
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/generation.json",
"sha256": "5914ef666fdbded464bdda47e5ed043dcf01db66f9d4de839f39087fcc7fa662",
"bytes": 103820
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
V02:前方へ車が割り込む
想定した答え:Dangerous
別の車が自車の進路に入り、すぐに対応が必要な場面です。
質問と選択肢
モデルに伝えた説明
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
質問
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
元の順番
["Dangerous", "Safe"]
逆順
["Safe", "Dangerous"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Safe | Safe | 不一致 / 不一致 |
| Clef | Dangerous | Dangerous | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Dangerous | 0.110471 | 0.421552 | 0.551574 | 0.661048 |
| Safe | 0.889529 | 0.578448 | 0.448426 | 0.338952 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.V02.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.V02.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.V02.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.V02.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. On a daylight urban two-lane road, ego moves forward behind open road. A car in the adjacent right lane begins a clear leftward move at 0.8 seconds. At 1.4 seconds it cuts across the lane marking into the ego lane directly ahead with very short following distance; show rear vehicle size increasing and a closing gap. Ego brakes promptly. End before contact.
{
"fixture_id": "V02",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V02-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/asset.mp4",
"sha256": "4b23e1598cf11faf1711d2416d21b93531a38222dd23ee5d797290c3ba6cb96e",
"bytes": 2340360
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/native.mp4",
"sha256": "ea0bba64e4a7ae24d6ce6016847b2d2ade4c49e5d2c8c08f5d5c1b8923cae12d",
"bytes": 1892973
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/receipt.json",
"sha256": "2d4be3befc00dc0213186394d80f59436a7fdf7602be31162a046cbc52d8866d",
"bytes": 3509
},
"observed_media": {
"native": {
"sha256": "ea0bba64e4a7ae24d6ce6016847b2d2ade4c49e5d2c8c08f5d5c1b8923cae12d",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 1892973
},
"evaluated_transport": {
"sha256": "4b23e1598cf11faf1711d2416d21b93531a38222dd23ee5d797290c3ba6cb96e",
"bytes": 2340360,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026100202,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026100202
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026100202,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/generation.request.json",
"sha256": "c595d09476da1b19da583fcd5de702639e214134f56c199d0b2937a307a923cb",
"bytes": 759
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/generation.json",
"sha256": "aa5568995b80bee66ac6c44eefdfa63c7f5608b4b490c91d1f4bdaf73f4e8bb3",
"bytes": 103676
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
V08:距離を保って前を走る車
想定した答え:Safe
車間があり、すぐに対応が必要な危険は見えません。
質問と選択肢
モデルに伝えた説明
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
質問
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
元の順番
["Dangerous", "Safe"]
逆順
["Safe", "Dangerous"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Safe | Safe | 一致 / 一致 |
| Clef | Safe | Safe | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Dangerous | 0.071074 | 0.325950 | 0.008916 | 0.008413 |
| Safe | 0.928926 | 0.674050 | 0.991084 | 0.991587 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.V08.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.V08.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.V08.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.V08.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. Daylight dashcam on a straight road. Ego follows one car well ahead with clearly distant, nonclosing separation and substantial road visible between ego and the leading car at their modest steady speed. Both continue straight, leading car maintains nearly constant apparent size and relative position throughout all four seconds. Open road, no braking, no lane changes, no pedestrians.
{
"fixture_id": "V08",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V08-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/asset.mp4",
"sha256": "0522b5fc81629cb5dcc242a177d955feebe15f4b1805e17f952f9741c20e7f92",
"bytes": 1512796
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/native.mp4",
"sha256": "908544e0b7c40d48f741a03c6806d12d599151894e2c1d872c73343a2596e6e4",
"bytes": 1145176
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/receipt.json",
"sha256": "8dee295628be4cf88d390d6cf7bd4f5938681a290b4040cace4c5958492acd66",
"bytes": 3184
},
"observed_media": {
"native": {
"sha256": "908544e0b7c40d48f741a03c6806d12d599151894e2c1d872c73343a2596e6e4",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 1145176
},
"evaluated_transport": {
"sha256": "0522b5fc81629cb5dcc242a177d955feebe15f4b1805e17f952f9741c20e7f92",
"bytes": 1512796,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026100208,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026100208
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026100208,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/generation.request.json",
"sha256": "7da0f7da23db7d5286c724e892e8bf9d1e58118d533d80ce8845ccc57fb3c6e1",
"bytes": 786
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/generation.json",
"sha256": "2fe9db64ddae1a33bfc7780f487e7971392e78b13231887e4d5af210ea7efc04",
"bytes": 103604
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
V09:分離帯で隔てられた対向車
想定した答え:Safe
対向車との間には分離帯があり、自車の進路に入るものはありません。
質問と選択肢
モデルに伝えた説明
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
質問
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
元の順番
["Dangerous", "Safe"]
逆順
["Safe", "Dangerous"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Safe | Safe | 一致 / 一致 |
| Clef | Safe | Safe | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Dangerous | 0.039937 | 0.302391 | 0.007233 | 0.006641 |
| Safe | 0.960063 | 0.697609 | 0.992767 | 0.993359 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.V09.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.V09.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.V09.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.V09.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. Daylight dashcam in light rain, with small droplets near windshield edges but a clear central view. Ego moves steadily at a modest speed along an empty lane on a divided road with a continuous central barrier. Show generous forward sight distance, no standing water, no skidding, and no vehicles or people entering the lane for all four seconds.
{
"fixture_id": "V09",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V09-attempt-01",
"selected_attempt_index": 1,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/asset.mp4",
"sha256": "cdcce5e0b09700ed8b457ea81e5415c86720a889b47237d394e89276cfe72330",
"bytes": 1852141
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/native.mp4",
"sha256": "92e1afd4175c487dec865e0d32b90617e0e93e12e585d0530c93722f9ef0d17e",
"bytes": 1488990
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/receipt.json",
"sha256": "816a71502951f3d06ee034afffd46b34743493dd89ecd884270559a9c6ebe9bc",
"bytes": 3624
},
"observed_media": {
"native": {
"sha256": "92e1afd4175c487dec865e0d32b90617e0e93e12e585d0530c93722f9ef0d17e",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 1488990
},
"evaluated_transport": {
"sha256": "cdcce5e0b09700ed8b457ea81e5415c86720a889b47237d394e89276cfe72330",
"bytes": 1852141,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101209,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101209
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026101209,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/generation.request.json",
"sha256": "254de09d7582b9de7673f49339d52415789f404de4b6b7f845aa7faa0872c963",
"bytes": 746
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/generation.json",
"sha256": "8c5fc59eb20aa084866e5cdfe0449cd0b22a785e7b42bd9563aa92870a50261b",
"bytes": 103563
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
V10:分離された自転車レーン
想定した答え:Safe
自転車は分離されたレーンにあり、自車の進路には入っていません。
質問と選択肢
モデルに伝えた説明
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
質問
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
元の順番
["Dangerous", "Safe"]
逆順
["Safe", "Dangerous"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Safe | Safe | 一致 / 一致 |
| Clef | Safe | Safe | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Dangerous | 0.043043 | 0.262086 | 0.013428 | 0.012920 |
| Safe | 0.956957 | 0.737914 | 0.986572 | 0.987080 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E1_JEV_INITIAL.jev.V10.forward - Jev-Omni:逆順 JSON —
E1_JEV_INITIAL.jev.V10.reverse - Clef:元の順番 JSON —
E2_CLEF_COMMON.clef.V10.forward - Clef:逆順 JSON —
E2_CLEF_COMMON.clef.V10.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. Daylight dashcam with ego moving straight at steady modest speed. A cyclist travels in the same direction to the right, entirely in a protected bike lane separated by a raised continuous curb and regular bollards. Keep the cyclist, curb and bollards clearly visible; cyclist remains behind the separation for all four seconds. Ego lane remains open, no turns or crossings.
{
"fixture_id": "V10",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V10-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/asset.mp4",
"sha256": "5cf43c1116b9b095163b8f0df86f4b4865d5c27dae7f926e5e88c6d1dd94457e",
"bytes": 3608621
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/native.mp4",
"sha256": "cc00e7a6bb83ff19d457c576ada2da36be35190bfdfe87029dd0bd68484fc934",
"bytes": 2992586
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/receipt.json",
"sha256": "ac5999bac89cd93d9a5c5277b51a8f243a98499ecb31ee74e8ef74bce25b4199",
"bytes": 3318
},
"observed_media": {
"native": {
"sha256": "cc00e7a6bb83ff19d457c576ada2da36be35190bfdfe87029dd0bd68484fc934",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 2992586
},
"evaluated_transport": {
"sha256": "5cf43c1116b9b095163b8f0df86f4b4865d5c27dae7f926e5e88c6d1dd94457e",
"bytes": 3608621,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026100210,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026100210
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026100210,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/generation.request.json",
"sha256": "9f06d2a26666eefcd0837995375217486b70cb0201bc707f47f745257750b16d",
"bytes": 773
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/generation.json",
"sha256": "bfc3e83254a7327d7743f58a50537632c652ee10283eb1457569fd21655e4bca",
"bytes": 103605
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
元の5本で、緊急停止が必要かを調べました
同じ5本で、衝突を避けるために緊急停止が必要かを質問しました。ここでいう緊急停止は、すぐに強くブレーキをかけて停止することです。通常の減速やハンドル操作だけでは衝突を避けられない場合を指します。
質問だけでなく、説明、判定基準、選択肢も変えました。そのため、回答が変わっても、質問文だけが理由とはいえません。
V01・V02は、速度や残りの距離を映像から十分に読み取れず、緊急停止が必要かを決められませんでした。記録ではambiguousとし、集計から外しています。最初のDangerousという答えは、別の質問に対するものです。
| 想定した答え | 動画数 | 各モデルの回答数と扱い |
|---|---|---|
| Emergency stop needed | 0 | 0 |
| Emergency stop not needed | 3 | 6回答、すべて一致 |
| ambiguous | 2 | 4回答、正答数の集計から除外 |
各動画で選択肢の順番を変えて2回ずつ試しました。2つのモデルを合わせると、回答は20件です。動画は前節の5本と同じです。
V01:子どもが自車の進路へ入る
想定した答え:映像だけでは決められない(ambiguous)
子どもが進路に入るため、減速や停止は考えられます。ただし、速度や残りの距離が分からず、通常のブレーキで足りるかは決められません。衝突や明確な緊急ブレーキは映っていません。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| Clef | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.046034 | 0.357119 | 0.173849 | 0.346873 |
| Emergency stop not needed | 0.953966 | 0.642881 | 0.826151 | 0.653127 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.jev.V01.forward - Jev-Omni:逆順 JSON —
E3_EMERGENCY_ORIGINAL.jev.V01.reverse - Clef:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.clef.V01.forward - Clef:逆順 JSON —
E3_EMERGENCY_ORIGINAL.clef.V01.reverse
V02:前方へ車が割り込む
想定した答え:映像だけでは決められない(ambiguous)
右前方の車が徐々に自車の進路に入り、減速や車間の確保が必要になります。速度や距離が分からず、緊急停止が必要かは決められません。衝突は映っていません。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| Clef | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.012054 | 0.157896 | 0.032961 | 0.076961 |
| Emergency stop not needed | 0.987946 | 0.842104 | 0.967039 | 0.923039 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.jev.V02.forward - Jev-Omni:逆順 JSON —
E3_EMERGENCY_ORIGINAL.jev.V02.reverse - Clef:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.clef.V02.forward - Clef:逆順 JSON —
E3_EMERGENCY_ORIGINAL.clef.V02.reverse
V08:距離を保って前を走る車
想定した答え:Emergency stop not needed
前の車は車線内にあり、自車の進路は開いています。急な割り込みや障害物はなく、緊急停止が必要な場面は見えません。車間の調整が不要という意味ではありません。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 一致 / 一致 |
| Clef | Emergency stop not needed | Emergency stop not needed | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.013223 | 0.138520 | 0.006004 | 0.006004 |
| Emergency stop not needed | 0.986777 | 0.861480 | 0.993996 | 0.993996 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.jev.V08.forward - Jev-Omni:逆順 JSON —
E3_EMERGENCY_ORIGINAL.jev.V08.reverse - Clef:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.clef.V08.forward - Clef:逆順 JSON —
E3_EMERGENCY_ORIGINAL.clef.V08.reverse
V09:分離帯で隔てられた対向車
想定した答え:Emergency stop not needed
対向車はコンクリートの分離帯の向こうにあり、自車の進路には入っていません。濡れた路面や水滴だけでは、緊急停止が必要とはしていません。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 一致 / 一致 |
| Clef | Emergency stop not needed | Emergency stop not needed | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.005730 | 0.089773 | 0.005100 | 0.004982 |
| Emergency stop not needed | 0.994270 | 0.910227 | 0.994900 | 0.995018 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.jev.V09.forward - Jev-Omni:逆順 JSON —
E3_EMERGENCY_ORIGINAL.jev.V09.reverse - Clef:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.clef.V09.forward - Clef:逆順 JSON —
E3_EMERGENCY_ORIGINAL.clef.V09.reverse
V10:分離された自転車レーン
想定した答え:Emergency stop not needed
自転車は段差やポールで隔てられたレーンにとどまり、自車の進路は開いています。交差点での通常の減速や停止とは分けて判断しています。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 一致 / 一致 |
| Clef | Emergency stop not needed | Emergency stop not needed | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.006798 | 0.087720 | 0.007289 | 0.007877 |
| Emergency stop not needed | 0.993203 | 0.912280 | 0.992711 | 0.992123 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.jev.V10.forward - Jev-Omni:逆順 JSON —
E3_EMERGENCY_ORIGINAL.jev.V10.reverse - Clef:元の順番 JSON —
E3_EMERGENCY_ORIGINAL.clef.V10.forward - Clef:逆順 JSON —
E3_EMERGENCY_ORIGINAL.clef.V10.reverse
接近する場面の動画5本を追加しました
接触する場面を5種類考え、各2回、計10回生成しました。使う条件は、4秒の中に接近と明確な接触の両方が映っていることです。近づくだけの映像や、ボンネットで相手が隠れる映像は、接触を確認できたとは扱いませんでした。
10回とも条件を満たさず、使える接触動画は0本、接触動画としての分類も0回でした。そこで条件を変更し、各ケースで最後に生成したC01〜C05を、接近する場面として使いました。動画は作り直していません。
C01〜C04は映像だけでは答えを決められず、ambiguousとして集計から外しました。C05は「緊急停止は不要」としました。各モデルはC05の2回答とも一致し、C01〜C04の8回答は集計から外しています。
ClefはC02・C03で答えが変わりました
選択肢を逆の順番にすると、ClefはC02・C03で「緊急停止は不要」から「緊急停止が必要」に変わりました。Jev-Omniの答えは変わりませんでしたが、確率は変わりました。
| モデル / 動画 | 「必要」の確率:元の順番 | 「必要」の確率:逆順 |
|---|---|---|
| Clef / C02 | 0.389547 | 0.609524 |
| Clef / C03 | 0.397938 | 0.577495 |
| Jev-Omni / C02 | 0.008316 | 0.140749 |
| Jev-Omni / C03 | 0.017712 | 0.241847 |
この2本には想定した答えがないため、回答の変化を正解や不正解として数えていません。
C01:赤いセダンへの接近
想定した答え:映像だけでは決められない(ambiguous)
車間が縮まり、後部ランプも見えるため、減速は考えられます。ただし、接近する速度や停止までの距離が分からず、緊急停止が必要かは決められません。最後に近づいたことだけで、衝突とは判断していません。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| Clef | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.016029 | 0.178741 | 0.177241 | 0.405449 |
| Emergency stop not needed | 0.983971 | 0.821259 | 0.822759 | 0.594551 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C01.forward - Jev-Omni:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C01.reverse - Clef:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C01.forward - Clef:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C01.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. A red sedan is stopped in the middle of the ego lane on a straight level road. It is visible directly ahead from the first frame. The ego vehicle continues moving forward, the stopped sedan grows rapidly larger, and the gap closes continuously from 0.3 to 2.5 seconds. A clear pre-impact interval shows the blocked route and urgent opportunity to brake to a stop. At about 3.0 seconds the ego front bumper visibly contacts the sedan rear bumper; the sedan body jolts forward slightly while both intact bumpers visibly meet. Keep the scene continuous and contact visible before 3.5 seconds.
{
"fixture_id": "C01",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C01.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101401,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101401
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C01.attempt-1.generation.request.json",
"sha256": "46f8aae33f19a8fcdc691864f7c247918a2f5d5d0ca502c65362e629c5d50cf4",
"bytes": 1223
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C01.attempt-1/native.mp4",
"sha256": "e5e24e4e27ac1d22034c821d730701612f81a831cdebfb45c2dda3619f7b3d6a",
"bytes": 1589923
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C01/asset.mp4",
"sha256": "bf1bb41e1947614f6bb59ffedc492696177bc79220b6aabe76206cf7dafa764b",
"bytes": 2340198
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C01.attempt-1.final-inspected.json",
"sha256": "a22d2da04ddf086bebd7a39dcc0b2dfc97dafd80b2cd5c5b0138ee68d43305dd",
"bytes": 5392
},
"reference_classification": "ambiguous",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
C02:白いバンへの接近
想定した答え:映像だけでは決められない(ambiguous)
バンとの間隔は縮まっていますが、速度や停止までの距離が分かりません。通常の減速で足りるか、緊急停止が必要かは決められません。最後の位置や陰影だけで、衝突とは判断していません。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| Clef | Emergency stop not needed | Emergency stop needed | 除外 / 除外 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.008316 | 0.140749 | 0.389547 | 0.609524 |
| Emergency stop not needed | 0.991684 | 0.859251 | 0.610453 | 0.390476 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C02.forward - Jev-Omni:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C02.reverse - Clef:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C02.forward - Clef:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C02.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. An empty white van travels in the ego lane directly ahead from the first frame. At 0.4 seconds its brake lights illuminate and it rapidly slows to a stop. The ego vehicle keeps advancing, so the van enlarges and the visible following gap shrinks quickly over the next two seconds. Show a clear pre-impact interval during which hard braking to a stop is available and necessary to avoid the stationary van. At about 3.1 seconds the ego front bumper visibly contacts the van rear bumper and the van shifts forward slightly, with both vehicles retaining coherent bodies. All approach and contact are visible before 3.5 seconds.
{
"fixture_id": "C02",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C02.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101402,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101402
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C02.attempt-1.generation.request.json",
"sha256": "4d6e628c5ec435523792444799caee6c83eb25340a20a76a3cdf6aec3ba68618",
"bytes": 1258
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C02.attempt-1/native.mp4",
"sha256": "4c3661cc6e148509355e852519dba702cd7bf5bd2d903bcae71c35436bd06917",
"bytes": 1627265
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C02/asset.mp4",
"sha256": "ebebbf186ff2f12392950c6bcd0c96de9512af5cc8d346a0999903d34499155c",
"bytes": 2366486
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C02.attempt-1.final-inspected.json",
"sha256": "746c3e825014a9435051ec38e4be317d8d9681d6fb9bcff2f7e924fdf9290dd1",
"bytes": 5392
},
"reference_classification": "ambiguous",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
C03:進路を横切るピックアップトラック
想定した答え:映像だけでは決められない(ambiguous)
トラックが進路を横切っています。ただし、速度や停止距離が分からず、通常の減速やハンドル操作で足りるかは決められません。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| Clef | Emergency stop not needed | Emergency stop needed | 除外 / 除外 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.017712 | 0.241847 | 0.397938 | 0.577495 |
| Emergency stop not needed | 0.982288 | 0.758153 | 0.602062 | 0.422505 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C03.forward - Jev-Omni:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C03.reverse - Clef:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C03.forward - Clef:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C03.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. At a small daylight junction an empty blue pickup is visible ahead on the left from the first frame. From 0.4 to 1.3 seconds it turns across the road into the ego forward lane and then pauses broadside across that lane. The ego vehicle continues forward; the pickup side becomes rapidly larger and its blocking position stays visible for a sustained pre-impact interval, leaving a visible opportunity for very hard braking to a stop. At about 3.1 seconds the ego front bumper visibly contacts the pickup side panel; the pickup rocks or shifts slightly sideways at the contact point. Do not hide contact behind another vehicle or outside the frame. Contact must occur before 3.5 seconds.
{
"fixture_id": "C03",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C03.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101403,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101403
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C03.attempt-1.generation.request.json",
"sha256": "5a99783a84d0c32faaee64a72c35383afda49e627e832bdd8e9a3aeb7995b415",
"bytes": 1320
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C03.attempt-1/native.mp4",
"sha256": "c2776962ea247eeaaea6fcccb81f7c0deaa92e88ece6602313282dcbedd07fd6",
"bytes": 2696603
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C03/asset.mp4",
"sha256": "283c80fc7c43e96537981c374fea22369fd3cfaea6c06f6bbfc23b098ce0e656",
"bytes": 3819217
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C03.attempt-1.final-inspected.json",
"sha256": "f177fe1ee9a6c48fa618d806973ca58c2257f686f4e2d363b82aa6242924b295",
"bytes": 5392
},
"reference_classification": "ambiguous",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
C04:右から出てくるバン
想定した答え:映像だけでは決められない(ambiguous)
バンが右から出てきています。ただし、相対速度や停止距離が分からず、通常の減速やハンドル操作で足りるか、緊急停止が必要かは決められません。接触したとは判断していません。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| Clef | Emergency stop not needed | Emergency stop not needed | 除外 / 除外 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.018125 | 0.269326 | 0.175538 | 0.310694 |
| Emergency stop not needed | 0.981875 | 0.730674 | 0.824462 | 0.689306 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C04.forward - Jev-Omni:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C04.reverse - Clef:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C04.forward - Clef:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C04.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. An empty silver delivery van is visible at a right-side driveway from the first frame. Beginning at 0.4 seconds it reverses slowly out of the driveway, moves across the curb line, and stops with its rear half occupying the ego lane by 1.3 seconds. The ego vehicle continues advancing on the same straight route; the intrusion and rapidly shrinking gap remain visible before impact, providing a clear opportunity to brake hard to a stop. At about 3.0 seconds the ego front bumper visibly contacts the van rear-side corner and the van pivots slightly at the contact point. Keep coherent road geometry and directly visible vehicle boundaries. All evidence lies before 3.5 seconds.
{
"fixture_id": "C04",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C04.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101404,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101404
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C04.attempt-1.generation.request.json",
"sha256": "bc8932eb1f1d5fec94bc205a283f0a28aee0a2ad3401945d9ce03d57e547a75b",
"bytes": 1311
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C04.attempt-1/native.mp4",
"sha256": "b7b373ade4f6b7bb6e396a3230b9315eea77ce1a72fa70051f035fcc0222df02",
"bytes": 1942105
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C04/asset.mp4",
"sha256": "6f2d6fc6201a4866cdd10b2517e38c161382820b8214526f31fc26393c361959",
"bytes": 2403091
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C04.attempt-1.final-inspected.json",
"sha256": "58c5a633e04cfddd78f9ce593b4a2645d2b9ca40d51c2ec1ebb02cfb38724b5d",
"bytes": 5392
},
"reference_classification": "ambiguous",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
C05:黄色いトラックへの接近
想定した答え:Emergency stop not needed
トラックとの車間は徐々に縮まりますが、間隔は残っています。急に危険が生じたり、接触が迫ったりする様子はなく、通常の減速で車間を調整する場面としました。これは、映っている4秒の範囲での判断です。
質問と選択肢
モデルに伝えた説明
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
質問
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
元の順番
["Emergency stop needed", "Emergency stop not needed"]
逆順
["Emergency stop not needed", "Emergency stop needed"]
| モデル | 元の順番の回答 | 逆順の回答 | 想定回答との一致:元 / 逆 |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | 一致 / 一致 |
| Clef | Emergency stop not needed | Emergency stop not needed | 一致 / 一致 |
| 選択肢 | Jev-Omni:元 | Jev-Omni:逆 | Clef:元 | Clef:逆 |
|---|---|---|---|---|
| Emergency stop needed | 0.008985 | 0.148355 | 0.018055 | 0.035411 |
| Emergency stop not needed | 0.991015 | 0.851645 | 0.981945 | 0.964589 |
保存済み回答と実験ID
- Jev-Omni:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C05.forward - Jev-Omni:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C05.reverse - Clef:元の順番 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C05.forward - Clef:逆順 JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C05.reverse
生成プロンプトと設定
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. On a slight uphill straight road an empty yellow box truck is centered in the ego lane directly ahead from the first frame. At 0.5 seconds the truck begins rolling backward toward the advancing ego vehicle. Show its rear wheels moving backward, its rear body enlarging rapidly, and a continuously shrinking open gap from 0.5 through 2.5 seconds. The truck remains entirely in the ego route with a sustained visible opportunity for immediate very hard braking to stop before the gap closes. At about 3.1 seconds the ego front bumper visibly contacts the truck rear bumper; the truck rocks slightly and relative motion abruptly couples at directly visible bumper contact. Keep all approach and contact evidence before 3.5 seconds.
{
"fixture_id": "C05",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C05.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101405,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101405
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C05.attempt-1.generation.request.json",
"sha256": "64b1d072d10a35f37340c4705bb19ef220d0e7460c309af52fb38d91df142840",
"bytes": 1362
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C05.attempt-1/native.mp4",
"sha256": "55a1e1f01f4aa4ca5b1ccc631c036f9f50795a5726ad28cddfb44194961bbc79",
"bytes": 2434210
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C05/asset.mp4",
"sha256": "fa9964489e1e78799a4db1e48a4e3cdab456e5add9fd9a3921e6c2a11b8e96c1",
"bytes": 2953313
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C05.attempt-1.final-inspected.json",
"sha256": "673cb28357ea8619e8f86bd8429eb39c041e0efbf47ec905eb56b5b73e14cda4",
"bytes": 5392
},
"reference_classification": "not_needed",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
元の5本と合わせた結果
10本のうち、「緊急停止は不要」は4本、映像だけでは答えを決められないものは6本でした。「緊急停止が必要」と明確に判断できる動画は0本です。2つのモデルの回答は、元の20件と追加の20件を合わせて40件あります。
各モデルは集計対象の8回答すべてで一致し、残る12回答は除外しました。必要な場面の例がないため、必要なときに「必要」と答えられるかは、この結果では分かりません。記録のneeded_sensitivityもnullです。
データを作り直した記録
最初の動画生成は20回で、使えたのは5本でした。V03〜V07は用意できませんでした。条件を満たさなかった15回の生成結果は、モデルの回答の集計には含めていません。
| 埋まらなかった枠 | 予定した場面 |
|---|---|
| V03 | 停止車列に近づき、対応が必要になる場面 |
| V04 | 対向車が自車の車線へ入る場面 |
| V05 | ボールに続いて子どもが進路へ入る場面 |
| V06 | 柵によって道路から隔てられた子ども |
| V07 | 自車が最初から停止したまま横断を待つ場面 |
コルク抜きI08は3回生成しても条件を満たさず、分類させる前にI08Rの卓上扇風機へ変更しました。I03のハンマーも作り直しています。A01の最初の試行には採用記録がなく、使ったのはattempt-01です。記録のない採用欄はnullで、生成の成否を表す値ではありません。
下の表は、最初の46試行と、接触動画の10試行の記録です。全項目は生成記録のJSONにあります。不採用になった動画のファイルは含まれていません。
最初の46試行
| 試行 | 採用記録 | E1での使用 | 要求したseed |
|---|---|---|---|
A01-attempt-00 | 記録なし | 不使用 | 不明 |
A01-attempt-01 | 採用 | 使用 | 不明 |
A02-attempt-00 | 採用 | 使用 | 不明 |
A03-attempt-00 | 採用 | 使用 | 不明 |
A04-attempt-00 | 採用 | 使用 | 不明 |
A05-attempt-00 | 採用 | 使用 | 不明 |
A06-attempt-00 | 採用 | 使用 | 不明 |
A07-attempt-00 | 採用 | 使用 | 不明 |
A08-attempt-00 | 採用 | 使用 | 不明 |
A09-attempt-00 | 採用 | 使用 | 不明 |
A10-attempt-00 | 採用 | 使用 | 不明 |
I01-attempt-00 | 採用 | 使用 | 不明 |
I02-attempt-00 | 採用 | 使用 | 不明 |
I03-attempt-00 | 不採用 | 不使用 | 2026100203 |
I03-attempt-01 | 不採用 | 不使用 | 2026101203 |
I03-attempt-02 | 採用 | 使用 | 不明 |
I04-attempt-00 | 採用 | 使用 | 不明 |
I05-attempt-00 | 採用 | 使用 | 不明 |
I06-attempt-00 | 採用 | 使用 | 不明 |
I07-attempt-00 | 採用 | 使用 | 不明 |
I08-attempt-00 | 不採用 | 不使用 | 2026100208 |
I08-attempt-01 | 不採用 | 不使用 | 2026101208 |
I08-attempt-02 | 不採用 | 不使用 | 2026102208 |
I08R-attempt-00 | 採用 | 使用 | 不明 |
I09-attempt-00 | 採用 | 使用 | 不明 |
I10-attempt-00 | 採用 | 使用 | 不明 |
V01-attempt-00 | 採用 | 使用 | 不明 |
V02-attempt-00 | 採用 | 使用 | 不明 |
V03-attempt-00 | 不採用 | 不使用 | 2026100203 |
V03-attempt-01 | 不採用 | 不使用 | 2026101203 |
V03-attempt-02 | 不採用 | 不使用 | 2026102203 |
V04-attempt-00 | 不採用 | 不使用 | 2026100204 |
V04-attempt-01 | 不採用 | 不使用 | 2026101204 |
V04-attempt-02 | 不採用 | 不使用 | 2026102204 |
V05-attempt-00 | 不採用 | 不使用 | 2026100205 |
V05-attempt-01 | 不採用 | 不使用 | 2026101205 |
V05-attempt-02 | 不採用 | 不使用 | 2026102205 |
V06-attempt-00 | 不採用 | 不使用 | 2026100206 |
V06-attempt-01 | 不採用 | 不使用 | 2026101206 |
V07-attempt-00 | 不採用 | 不使用 | 2026100207 |
V07-attempt-01 | 不採用 | 不使用 | 2026101207 |
V07-attempt-02 | 不採用 | 不使用 | 2026102207 |
V08-attempt-00 | 採用 | 使用 | 不明 |
V09-attempt-00 | 不採用 | 不使用 | 2026100209 |
V09-attempt-01 | 採用 | 使用 | 不明 |
V10-attempt-00 | 採用 | 使用 | 不明 |
| 試行 | 接触動画としての採用 | E4の接近動画としての使用 |
|---|---|---|
C01.attempt-0 | 不採用 | 不使用 |
C01.attempt-1 | 不採用 | 使用 |
C02.attempt-0 | 不採用 | 不使用 |
C02.attempt-1 | 不採用 | 使用 |
C03.attempt-0 | 不採用 | 不使用 |
C03.attempt-1 | 不採用 | 使用 |
C04.attempt-0 | 不採用 | 不使用 |
C04.attempt-1 | 不採用 | 使用 |
C05.attempt-0 | 不採用 | 不使用 |
C05.attempt-1 | 不採用 | 使用 |
C01.attempt-0:不採用の理由と記録
{
"attempt_id": "C01.attempt-0",
"fixture_id": "C01",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C01.attempt-0.generation.request.json",
"sha256": "683c37a6abd38a90d35fbbb56422cded06ab78548d0d47cdc26092db04c0fbf7",
"bytes": 1223
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C01.attempt-0/review.json",
"sha256": "2bfabeeef18c76bc0724836694951ddfbc5c72b188c5ddd24c3693dcf25c2c6f",
"bytes": 17089
},
"reason": "Rejected by the frozen media gate: the actual four-second transport shows a continuous approach to a red sedan with a decreasing apparent gap, but no unmistakable ego-to-vehicle contact. In full-resolution late frames the lower sedan/bumper becomes hidden behind the ego hood; its visible body remains aligned and intact, with no directly observable boundary contact, deformation, deflection or coherent collision consequence. Close following and hood occlusion alone cannot establish an impact. The native tail does not establish contact either and cannot rescue transport acceptance.",
"reference_reason": "The visible closing conflict supports braking, especially in the late approach. However the actual clip does not establish contact, stopping/deceleration, calibrated ego speed or closing distance. It cannot reliably distinguish a necessary immediate very-hard stop from ordinary braking bringing the ego close behind the sedan. The reference therefore remains ambiguous; it does not assume a collision occurred, was unavoidable or would be prevented by a particular braking action."
}
C01.attempt-1:不採用の理由と記録
{
"attempt_id": "C01.attempt-1",
"fixture_id": "C01",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C01.attempt-1.generation.request.json",
"sha256": "46f8aae33f19a8fcdc691864f7c247918a2f5d5d0ca502c65362e629c5d50cf4",
"bytes": 1223
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C01.attempt-1/review.json",
"sha256": "276cff2dff398f71aa76290fdd6ee76378aeef04d8551529f91729f59f13f981",
"bytes": 15780
},
"reason": "Rejected by the frozen media gate: the red sedan is approached continuously and its lower rear becomes hidden behind the ego hood near the end, but no unmistakable ego-to-vehicle contact appears in actual transport frames 0–95. The aligned rear body stays intact and its final position stabilizes without clear touch, deformation, deflection or coherent collision consequence. Hood occlusion does not establish impact.",
"reference_reason": "Progressive closure and visible rear lamps support braking. Available stopping distance, calibrated closing speed and deceleration are unresolved; the sequence cannot reliably distinguish ordinary braking from a necessary immediate very-hard stop. Reference is ambiguous without assuming an impact from the close endpoint."
}
C02.attempt-0:不採用の理由と記録
{
"attempt_id": "C02.attempt-0",
"fixture_id": "C02",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C02.attempt-0.generation.request.json",
"sha256": "eb4100e61789fd6b98bf7dbcd6b29b41df77c353c24834296b577391835af930",
"bytes": 1258
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C02.attempt-0/review.json",
"sha256": "e06e7a592274a37288625de966bc96b13c6f84ca4044daca67b3b6b93222f430",
"bytes": 16579
},
"reason": "Rejected by the frozen media gate: the white van grows larger as the apparent following gap decreases, but actual transport frames 0–95 do not show unmistakable ego-to-vehicle contact. Full-resolution frames 72, 84, 90 and 95 show an intact aligned rear; the lower van becomes hidden by the ego hood and the upper rear becomes cropped. No visible boundary contact, deformation, deflection or coherent collision consequence establishes impact.",
"reference_reason": "The closing sequence supports braking, but the visible clip does not resolve ego speed, relative closing distance, available stopping distance or necessary braking intensity. Ordinary deceleration versus a necessary immediate very-hard stop cannot be distinguished reliably. Emergency-stop reference is ambiguous without assuming impact from hood occlusion."
}
C02.attempt-1:不採用の理由と記録
{
"attempt_id": "C02.attempt-1",
"fixture_id": "C02",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C02.attempt-1.generation.request.json",
"sha256": "4d6e628c5ec435523792444799caee6c83eb25340a20a76a3cdf6aec3ba68618",
"bytes": 1258
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C02.attempt-1/review.json",
"sha256": "b6b4899ea2a64cf42732b58e2dd53955eaffd68552aadf9504fb9796c67bae8b",
"bytes": 15699
},
"reason": "Rejected by the frozen media gate: white van approaches continuously and fills the viewpoint, with its lower rear hidden by ego hood late in actual transport frames 0–95. No unmistakable direct contact, abrupt deflection, coherent impact deformation or collision consequence is visible. Rear panel shading does not establish an impact, and close hood occlusion alone is insufficient.",
"reference_reason": "The van's shrinking apparent gap supports braking but does not resolve available stopping distance, closing speed or required intensity. The visible sequence cannot reliably distinguish ordinary deceleration from a necessary immediate very-hard stop. Reference remains ambiguous without treating the endpoint or panel shading as a collision."
}
C03.attempt-0:不採用の理由と記録
{
"attempt_id": "C03.attempt-0",
"fixture_id": "C03",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C03.attempt-0.generation.request.json",
"sha256": "d2b2ae1ba3c77550ad11d3fff82d9a908668f4a6cc35788253968180f7f0bf1b",
"bytes": 1320
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C03.attempt-0/review.json",
"sha256": "a1de42ec63c2da08961440fb4df4d0340ad05b6bd526355b6f8d0cb032236b11",
"bytes": 16524
},
"reason": "Rejected by the frozen media gate: a blue pickup crosses from the left across the forward route and grows closer, but actual transport frames 0–95 show no unmistakable ego-to-vehicle contact. The body remains intact and the lower pickup disappears behind the hood. Close proximity and occlusion do not establish an impact; no clear boundary touch, deformation, deflection or coherent collision consequence is visible.",
"reference_reason": "The visible crossing conflict calls for braking. The clip does not establish calibrated closing speed, stopping distance, deceleration or whether an immediate very-hard stop rather than ordinary braking was necessary and available. This reference is ambiguous; crossing and hood occlusion are not treated as proof of a collision or of a specific preventive action."
}
C03.attempt-1:不採用の理由と記録
{
"attempt_id": "C03.attempt-1",
"fixture_id": "C03",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C03.attempt-1.generation.request.json",
"sha256": "5a99783a84d0c32faaee64a72c35383afda49e627e832bdd8e9a3aeb7995b415",
"bytes": 1320
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C03.attempt-1/review.json",
"sha256": "17ba4958115c7c27bfde06a410e5ce4b424939e0d4c29cc793bd5aaca1f19fa3",
"bytes": 16297
},
"reason": "The pickup is continuously approached while crossing the ego route, but its lower side and the possible contact zone become hidden behind the ego hood. A light crease or scuff on the front door is visible late; without visible vehicle boundaries meeting or coherent impact motion, that change does not establish unmistakable contact within frames 0–95. The frozen media gate therefore fails.",
"reference_reason": "The crossing pickup creates a visible route conflict. The clip does not establish the speed, remaining stopping distance or urgency sufficiently to distinguish an immediate very hard stop from ordinary braking or steering. The possible late door crease cannot supply that prospective judgment; retain an ambiguous emergency-stop reference."
}
C04.attempt-0:不採用の理由と記録
{
"attempt_id": "C04.attempt-0",
"fixture_id": "C04",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C04.attempt-0.generation.request.json",
"sha256": "94c2c3dbf7fd72978bd3047f01957176da48202be3ef8828376a70212bfd91da",
"bytes": 1311
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C04.attempt-0/review.json",
"sha256": "b5df5ba40121a0ae518cb6a325634b49d9117b408f8fee7a2efa3c53911aff48",
"bytes": 16590
},
"reason": "Rejected by the frozen media gate: a white van extends from a driveway on the right across the forward route and becomes close, but actual transport frames 0–95 do not show unmistakable vehicle contact. The lower van becomes hood-occluded; visible body panels remain intact and its late position stabilizes. No directly observable touch, deformation, deflection or coherent impact consequence is established.",
"reference_reason": "The van obstructs the forward route and supports braking, but visible ego slowing and van motion do not resolve stopping distance or whether immediate very-hard stopping was required rather than ordinary braking. The close hood-occluded endpoint alone does not establish collision or the counterfactual efficacy of a stop. Reference remains ambiguous."
}
C04.attempt-1:不採用の理由と記録
{
"attempt_id": "C04.attempt-1",
"fixture_id": "C04",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C04.attempt-1.generation.request.json",
"sha256": "bc8932eb1f1d5fec94bc205a283f0a28aee0a2ad3401945d9ce03d57e547a75b",
"bytes": 1311
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C04.attempt-1/review.json",
"sha256": "1928136ebd43db05c67de369d3d3b00af14526e3b92cba2259d9ac8eaa0f025a",
"bytes": 16100
},
"reason": "A white van extends from the right driveway into the ego route during a continuous approach. Through actual transport frame95, the visible van and ego hood do not establish vehicle boundaries meeting, deformation, deflection or abrupt coupled impact motion. Close approach is insufficient for the frozen collision media gate.",
"reference_reason": "The emerging van creates a visible route conflict, but the clip does not establish relative speed, stopping distance or timing well enough to distinguish the need for immediate very hard stopping from ordinary braking or steering. Retain an ambiguous emergency-stop reference without inferring contact."
}
C05.attempt-0:不採用の理由と記録
{
"attempt_id": "C05.attempt-0",
"fixture_id": "C05",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C05.attempt-0.generation.request.json",
"sha256": "f67e1d3d25b668bb44c26bf34d350099f2eb4bcdf459c1876f726a086de3203c",
"bytes": 1362
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C05.attempt-0/review.json",
"sha256": "a741ecb2907b8c40740e66685fb354655b200181e1a7d472b5ff5da030fc85d7",
"bytes": 16478
},
"reason": "Rejected by the frozen media gate: the yellow box truck grows larger ahead, but an unobscured strip of roadway remains between its intact rear bumper and the ego hood through actual transport frame 95. Full-resolution 72/84/90/95 show a clear gap, with no visible touch, deformation, deflection or coherent impact consequence.",
"reference_reason": "The clip shows progressive closure to a truck without an abrupt visible obstruction or contact. Maintaining the gap may call for ordinary braking; calibrated speed, relative motion and stopping distance are not established well enough to determine necessary immediate very-hard stopping. Reference remains ambiguous instead of inferring urgency or rollback from the intended scenario."
}
C05.attempt-1:不採用の理由と記録
{
"attempt_id": "C05.attempt-1",
"fixture_id": "C05",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C05.attempt-1.generation.request.json",
"sha256": "64b1d072d10a35f37340c4705bb19ef220d0e7460c309af52fb38d91df142840",
"bytes": 1362
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C05.attempt-1/review.json",
"sha256": "747a0f0d7556565bc7813b32fe9f54a26931efdf4cf80b0e3066e0cf51050d97",
"bytes": 16302
},
"reason": "The yellow box truck is approached gradually, but a substantial visible road gap remains between its rear bumper and the ego hood throughout actual transport frames0–95. No vehicle boundaries meet and no impact motion or consequence occurs. This candidate fails the frozen collision media gate.",
"reference_reason": "The visible prefix shows gradual closure to a truck that remains ahead with a clear road gap, without sudden hazard onset, abrupt relative motion or a moment of visible imminent contact. Maintaining or restoring following distance through ordinary gradual braking is supported by this sequence; an immediate very hard stop is not visibly required. This qualitative not_needed annotation is confined to the visible prefix and is not inferred solely from the absence of collision. Rejected media remains ineligible for benchmark scoring."
}
実験データ
4つの実験で使った入力は40件、回答は160件です。Clefとの比較と、元の動画での緊急停止の実験は、最初の実験と同じ入力を使っています。
各回答のJSONには、選んだ答え、確率の全桁、実験ID、入力のハッシュがあります。
| ファイル | 内容 |
|---|---|
| E1_JEV_INITIAL.json | 35入力、70回答、未実行の動画5枠×2順序 |
| E2_CLEF_COMMON.json | Clefの50回答と、E1の比較対象ID |
| E3_EMERGENCY_ORIGINAL.json | 元の動画5本への20回答と、集計からの除外 |
| E4_NEAR_APPROACH_AMENDMENT.json | 追加動画5本への20回答と、集計からの除外 |
| GENERATOR_PROVENANCE.json | 40入力のプロンプト、seed、設定、生成履歴 |
| GENERATOR_PROVENANCE.md | 生成方法と記録の読み方 |
| AUDIO_EXPERIMENT_MAPPING.md | 原稿、保存済みASR、音声の採用理由と不確定な部分 |
| EXPORT_MANIFEST.json | 公開ファイルと元資料の対応、checksum、公開用の整理内容 |
| VALIDATION.json | 元の公開セットの数値、hash、意味対応、除外条件の検証記録 |
| SHA256SUMS.txt | 公開セット内の各ファイルのSHA-256 |
| README_PUBLIC_EXPORT.md | 公開セットの範囲と解釈上の注意 |
| 元の構成案 | 4種類の判定を中心にした英語の実験紹介 |
| 元の日本語動画原稿 | 実験結果を紹介する動画用の原稿 |
音声はWAV、画像はJPEG、動画は評価に使った4秒のMP4です。生成元の長い動画やPNG、モデルの重み、実行ログは含まれていません。
今回の結果について
今回の結果は、実験用に作った少数のデータによるものです。使いたい用途で同じように答えられるかは、その用途に合うデータで確かめる必要があります。
