Jev-Omni multimodal decisions and comparison with Clef
AI integration tests of Jev-Omni decisions on 35 audio, image, Japanese email, and video inputs, with a Clef comparison on the same 25 inputs. Includes additional video tests and saved answers.
Jev-Omni selects an answer from the choices you provide. It accepts text, images, audio, and video, and returns a probability for each choice.
The same model handles all 4 input types. You can ask it to classify a spoken message, identify an object in a photo, or decide whether an email needs a reply.
We tried 10 audio files, 10 images, 10 fictional Japanese emails, and 5 videos: 35 inputs in total. We also compared the 25 images, emails, and videos with Clef.
Computer Use and Browser Use video
We also tried simple Computer Use and Browser Use with decision models. Qwen3.8 acts as the main agent in this demo, which uses a Big Five personality test.
Video link: https://www.youtube.com/watch?v=Po3kZdTTMjw
Results
Each input was tested with the choices in original and reversed order. The counts show agreement with answers defined before inference. Reordering the choices does not add independent inputs.
| Experiment | Inputs | Jev-Omni | Clef |
|---|---|---|---|
| E1: 4 input types | 35 | Original order 33/35; reversed order 31/35 | Not included |
| E2: shared images, emails, and videos | 25 | Original order 23/25; reversed order 21/25. Reuses E1 responses | Original order 25/25; reversed order 25/25 |
| E3: emergency-stop question on the original videos | 5 | Agreement on all 6 of 6 scored responses; 4 responses excluded | Same |
| E4: additional near-approach videos | 5 | Agreement on all 2 of 2 scored responses; 8 responses excluded | Same |
Jev-Omni matched 33/35 (94.3%) in original order and 31/35 (88.6%) in reversed order. I06 and T01 changed answers when the choices were reversed. V01 and V02 missed the expected answer in both orders.
On the 25 shared inputs, Clef matched 50/50 responses and Jev-Omni matched 44/50 responses across both orders. Audio was not part of this comparison.
How we tested
Models
| Model | Version used | Role in this evaluation |
|---|---|---|
| Jev-Omni | akhilaaa3/Jev-Omni, revision c050d51354147985d13286cf4acf90f562f2c631 | Chooses answers for audio, images, text, and video |
| Clef | simonlehmann/clef-NVFP4, revision 817ac58ad358f42489980ead62f8bf7cafc628c2 | Compared with Jev-Omni on images, text, and video |
Jev-Omni returns the selected choice and a probability for each option. E1 used the same loaded model for all 4 input types. Media went through the Gateway; text called the shared classification function directly.
Clef is based on Qwen/Qwen3.8-27B. We used a community version that stores model values in the smaller NVFP4 and FP8 formats. The upstream Qwen revision was not recorded separately.
E2 reused Jev-Omni’s saved E1 responses after checking file hashes, descriptions, questions, and choices. Jev-Omni was not run again.
Expected answers and choice order
Scenarios, choices, and expected answers were set before inference. Generated inputs were checked against those criteria before acceptance.
Reversing the list changes positions, so we compare the meaning of the selected answer, such as Colander or Reply required. Probabilities are rounded to 6 decimal places here; the JSON retains full precision.
A high probability does not by itself show that an answer is correct.
Generating the inputs
| Input | Creation method | Main recorded settings |
|---|---|---|
| Audio | ResembleAI/chatterbox-turbo | English, supplied default voice, no voice cloning. Full-length mono 24 kHz PCM16 WAV |
| Images | ByteDance/SDXL-Lightning | sdxl_lightning_4step_unet.safetensors, 4 steps, guidance 0, 1024×1024. Generated PNGs converted to quality-90 JPEGs for evaluation |
| Emails | Fictional Japanese emails written directly for the experiment | No generation model used |
| Video | MiniMaxAI/MiniMax-H3 and Alibaba PDD8 | Recorded API settings: 9 steps, flow shift 12, audio flow shift 3. Requested short side 768, 16:9, duration 4 seconds |
PDD8 is the acceleration method, not an 8-step API setting. The audio API did not expose a seed.
The generation records and notes include prompts, revisions, settings, and retries.
Video frames
Each evaluated 4-second MP4 contains the first 96 frames at 24 fps. The source had 107 frames and lasted 4.458333 seconds; its final portion was not evaluated.
| Model | Frames used | Preprocessing |
|---|---|---|
| Jev-Omni | 16 frames, every 0.25 seconds from 0.0 to 3.75 seconds | Long side resized to a maximum of 768 pixels |
| Clef | 8 frames, source index [0,14,27,41,54,68,81,95] | Includes the first and last frames. Decoded without prior resizing, then processed by Qwen’s image processor |
The same MP4 therefore gives the models different frames and preprocessing. Timing was measured once in different environments, so it does not support a speed comparison.
The initial 35 inputs and the Clef comparison
We planned 40 inputs and accepted 35: 10 audio files, 10 images, 10 emails, and 5 videos. Five planned videos could not be prepared.
| Input | Accepted / planned | Jev-Omni: original / reversed order | Clef: original / reversed order |
|---|---|---|---|
| English audio | 10/10 | 10/10 · 10/10 | Not compared |
| Object images | 10/10 | 10/10 · 9/10 | 10/10 · 10/10 |
| Japanese emails | 10/10 | 10/10 · 9/10 | 10/10 · 10/10 |
| Generated videos | 5/10 | 3/5 · 3/5 | 5/5 · 5/5 |
10 audio inputs: complaints and ordinary inquiries
The 10 audio inputs contain 5 complaints and 5 ordinary inquiries. Complaint means a reported service or product failure with a request for action, even when politely worded. Ordinary inquiry covers factual or pre-purchase questions that do not claim a failure.
Jev-Omni received the WAV files. The scripts are generation inputs, not verified transcripts.
We checked the audio with a separate speech recognizer on the CPU. The records contain no human listening check. Recognition was uncertain near arrange delivery in A01, and A07 and A09 were rechecked with Whisper base.en. Details are in the audio mapping. All 10 inputs matched in both orders.
A01: Parcel marked delivered but missing
Frozen reference: Complaint. Asserts failed delivery and requests a remedy.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.881104 | 0.999877 |
| Ordinary inquiry | 0.118896 | 0.000123 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A01.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A01.reverse
Prompt and settings
The tracking page says my parcel was delivered yesterday, but it never arrived. I checked the front door and asked my neighbours. Please find out where it went and arrange delivery or a refund.
{
"fixture_id": "A01",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A01-attempt-01",
"selected_attempt_index": 1,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/asset.wav",
"sha256": "e800911c146952b239ca3cf79df4ed73e40a2806fdb2a721eed19f69854a8865",
"bytes": 510764
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/asset.wav",
"sha256": "e800911c146952b239ca3cf79df4ed73e40a2806fdb2a721eed19f69854a8865",
"bytes": 510764
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/receipt.json",
"sha256": "507ed6c07f101f640fe9e0f6731196b622c32ea7ce38e96dc448c46c27b68dde",
"bytes": 2776
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 10.64
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/generation.request.json",
"sha256": "ccdcdf5f85d9d2e35657d2db100f2453beefb6fce960f0f53835099aed6651f6",
"bytes": 257
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A01-attempt-01/generation.json",
"sha256": "ce478961f08d999125ba2f9ea00dd00c1a2a3bf5a42ceac898b1d1324110e704",
"bytes": 5426
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A02: Duplicate posted charge
Frozen reference: Complaint. Asserts duplicate posted charge and requests reversal.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.866133 | 0.999871 |
| Ordinary inquiry | 0.133868 | 0.000129 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A02.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A02.reverse
Prompt and settings
I ordered one desk lamp, but my card was charged twice for the same order. The two payments have both posted. Could you please reverse the extra charge?
{
"fixture_id": "A02",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A02-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/asset.wav",
"sha256": "d008877d32667d9b9eb51bacaeb1eacb2ff7863fd337d5f9122b839cf94d0d71",
"bytes": 430124
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/asset.wav",
"sha256": "d008877d32667d9b9eb51bacaeb1eacb2ff7863fd337d5f9122b839cf94d0d71",
"bytes": 430124
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/receipt.json",
"sha256": "afd31a9007a91d50903474c922276bb8a198de2a6ba18f93118e055b1ee0bfd4",
"bytes": 1556
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 8.96
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/generation.request.json",
"sha256": "e4aa97475b89879b93f0dfd8e191401d07362fa60de7e25823cc26a9626dc44f",
"bytes": 216
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A02-attempt-00/generation.json",
"sha256": "bb5552a52f74fcb047a04b023b9f6d1aa94fda98996dbc45a97c98a0c90eec35",
"bytes": 5385
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A03: Replacement for a damaged kettle
Frozen reference: Complaint. Reports broken product and requests replacement.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.759225 | 0.999659 |
| Ordinary inquiry | 0.240775 | 0.000341 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A03.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A03.reverse
Prompt and settings
The kettle arrived this morning with a cracked handle, so I cannot use it safely. Please tell me how to get a replacement for the damaged one.
{
"fixture_id": "A03",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A03-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/asset.wav",
"sha256": "f67d95c676f356236d4bf8bb642ca4c7040e1bc0eeac6f7d4af36881b38080b5",
"bytes": 351404
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/asset.wav",
"sha256": "f67d95c676f356236d4bf8bb642ca4c7040e1bc0eeac6f7d4af36881b38080b5",
"bytes": 351404
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/receipt.json",
"sha256": "7b228f2ec3c6ddc09adc6a5f47cfc24df7bc3f872bb902343c1a52a6b2ab4439",
"bytes": 1529
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 7.32
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/generation.request.json",
"sha256": "351d1b49bee8ffb41e49b70baf60c1c209b65120f0ffcd2b9dda14c9c6c9f3a9",
"bytes": 206
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A03-attempt-00/generation.json",
"sha256": "5c14507ae0bd331d961bb9cd89f0e4d1fef308f0e4820dc6988068e2147caf4f",
"bytes": 5375
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A04: A promised callback has not arrived
Frozen reference: Complaint. Polite message asserts a missed promised callback and asks for remedy.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.826151 | 0.999812 |
| Ordinary inquiry | 0.173849 | 0.000188 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A04.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A04.reverse
Prompt and settings
Hello, I know your team is busy. You said someone would call me back by Tuesday, and it is Thursday now. I have not heard from anyone. Could you please arrange that callback?
{
"fixture_id": "A04",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A04-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/asset.wav",
"sha256": "6f7d5f65db14fbf9b88268e634b49757c76b61f358fc8796e5fcd830e651fdd7",
"bytes": 460844
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/asset.wav",
"sha256": "6f7d5f65db14fbf9b88268e634b49757c76b61f358fc8796e5fcd830e651fdd7",
"bytes": 460844
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/receipt.json",
"sha256": "999b90ab815a04a3b453eb1a8ef6451ca4f401fae6115801309d7ae971d7130f",
"bytes": 1605
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 9.6
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/generation.request.json",
"sha256": "da956535de808091ebfdffb4cf158914701162cb82d9fdc9e6329fc06176b74d",
"bytes": 238
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A04-attempt-00/generation.json",
"sha256": "58db357073faca8ba978fe23c40786744452c996bda4f373936f83bdde99e4d6",
"bytes": 5407
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A05: Disappointment with notebook paper
Frozen reference: Complaint. Mild dissatisfaction with an experienced outcome and request for remedy.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Complaint | Complaint | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.903379 | 0.999925 |
| Ordinary inquiry | 0.096621 | 0.000075 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A05.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A05.reverse
Prompt and settings
I tried the new notebook this week. The pages are thinner than I expected, and ink shows through when I write. It is a little disappointing. Is there anything you can do about it?
{
"fixture_id": "A05",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A05-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/asset.wav",
"sha256": "c0d801a429dfd9312491b4be3c2423647045c2a6be03c8094fbea48faa45e463",
"bytes": 445484
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/asset.wav",
"sha256": "c0d801a429dfd9312491b4be3c2423647045c2a6be03c8094fbea48faa45e463",
"bytes": 445484
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/receipt.json",
"sha256": "c91492a815d9370c66c7a6c88685d09104f067a97d22149cbfe62044426d0124",
"bytes": 1615
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 9.28
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/generation.request.json",
"sha256": "29e760e36441683f98fe17f0283186931c6413930c83ff6fe3577f60ad60f3a7",
"bytes": 243
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A05-attempt-00/generation.json",
"sha256": "e8344046caf8278b5e6a0e04e41bd553cc6df993c17fe11656cd25bccd138b02",
"bytes": 5412
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A06: Saturday opening hours
Frozen reference: Ordinary inquiry. Factual opening-hours question with no failure.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.000253 | 0.018405 |
| Ordinary inquiry | 0.999747 | 0.981595 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A06.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A06.reverse
Prompt and settings
Hi, what time does your shop open on Saturdays? I would like to visit before lunch.
{
"fixture_id": "A06",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A06-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/asset.wav",
"sha256": "6e237b86d8c3f23de45baa7b0d03875983515fca29e9844d6710f87a2b976607",
"bytes": 251564
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/asset.wav",
"sha256": "6e237b86d8c3f23de45baa7b0d03875983515fca29e9844d6710f87a2b976607",
"bytes": 251564
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/receipt.json",
"sha256": "4971da0cbd6bc846d5e5c9139a470fade40ff9b204e3e80c79c2f673a4b4da72",
"bytes": 1466
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 5.24
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/generation.request.json",
"sha256": "dc6150fb5c906e0fcd77bcc8e3f9439fe5f39c8980fed749b27de6e8b53fd1f3",
"bytes": 147
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A06-attempt-00/generation.json",
"sha256": "b79c742a5de60582587b9e23e2bc2874d025dc40c3d866d71560e57bf31cc189",
"bytes": 5316
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A07: Delivery estimate before ordering
Frozen reference: Ordinary inquiry. Prospective delivery question before purchase.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.000197 | 0.026156 |
| Ordinary inquiry | 0.999803 | 0.973844 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A07.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A07.reverse
Prompt and settings
If I place an order for the blue chair today, when would you expect it to arrive? I have not ordered yet and am planning a room update.
{
"fixture_id": "A07",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A07-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/asset.wav",
"sha256": "2e4f53fe892d39b56c291904430db5e920cf35c37f45617b643840014f121735",
"bytes": 382124
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/asset.wav",
"sha256": "2e4f53fe892d39b56c291904430db5e920cf35c37f45617b643840014f121735",
"bytes": 382124
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/receipt.json",
"sha256": "ada91499e1f86c9478006cfe9ec3777769f90035b9d69ce6d1fdfa2dccf95bc4",
"bytes": 2280
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 7.96
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/generation.request.json",
"sha256": "ff038681f92f9c842a7403ce9772f6b4cb3b99d64e4c8d717f8e4b2f107982f9",
"bytes": 199
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A07-attempt-00/generation.json",
"sha256": "5df713a744a58368b1a4d8b9132ab1aeeb197fac4bb0c34ffebbc62aed91cda5",
"bytes": 5368
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A08: Return policy before purchase
Frozen reference: Ordinary inquiry. Return-policy inquiry before purchase; no dissatisfaction or failed outcome.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.000404 | 0.054218 |
| Ordinary inquiry | 0.999596 | 0.945782 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A08.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A08.reverse
Prompt and settings
Could you explain your return policy for an unopened item? I am comparing gift options and would like to know the return window before buying.
{
"fixture_id": "A08",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A08-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/asset.wav",
"sha256": "1cce8b3404b77b4da2ef342166209f8293ed97cf2bc931e13e430029f309a1f0",
"bytes": 389804
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/asset.wav",
"sha256": "1cce8b3404b77b4da2ef342166209f8293ed97cf2bc931e13e430029f309a1f0",
"bytes": 389804
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/receipt.json",
"sha256": "6c4354f781fe2a6c47dd5df69824132b19d70a14737ba92c76e55f846252e67c",
"bytes": 1561
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 8.12
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/generation.request.json",
"sha256": "d11f045befccd65cbdc20dc971a63a143717c615b6e6be0a7db5da7f91dbf852",
"bytes": 206
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A08-attempt-00/generation.json",
"sha256": "aa2f468e032a756233827f702b76185cd8a869473239cfac70e745ed27b5d5bb",
"bytes": 5375
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A09: Order status before the promised date
Frozen reference: Ordinary inquiry. Status inquiry explicitly before the promised due date, with no asserted delay.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.000496 | 0.072014 |
| Ordinary inquiry | 0.999504 | 0.927986 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A09.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A09.reverse
Prompt and settings
My order is due to arrive next Friday, so it is still within the promised time. Could you tell me whether it has been packed yet? I just want to plan when to be home.
{
"fixture_id": "A09",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A09-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/asset.wav",
"sha256": "fa96dafcf7bf7950302b74fe4997ef448448da2be34bd2ff945538534454e350",
"bytes": 420524
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/asset.wav",
"sha256": "fa96dafcf7bf7950302b74fe4997ef448448da2be34bd2ff945538534454e350",
"bytes": 420524
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/receipt.json",
"sha256": "69525170da853cf10f4b3dffb6700dbb615a55bf3cf273b1d912b8f5a4a70fdb",
"bytes": 2358
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 8.76
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/generation.request.json",
"sha256": "c2906ff23971feed0497761e745388dde18d3d9d6f50c274f7866facb587632e",
"bytes": 230
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A09-attempt-00/generation.json",
"sha256": "3f3859c14454928fd187e7f8a60fbe81c3fe76239efec2cc511a6c4f8a7fb20f",
"bytes": 5399
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
A10: Pairing help from a satisfied customer
Frozen reference: Ordinary inquiry. Explicitly satisfied request for usage help without alleging defect.
State, question, and choices
state
Assess only the audible customer message. A complaint expresses dissatisfaction with a prior service or product outcome, an unmet obligation or error, or seeks a remedy for a failure. Ordinary inquiries ask factual, prospective, or status questions without asserting failure. Politeness does not remove a complaint.
question
Is the audible customer message a complaint or an ordinary inquiry?
Forward
["Complaint", "Ordinary inquiry"]
Reversed
["Ordinary inquiry", "Complaint"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Ordinary inquiry | Ordinary inquiry | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed |
|---|---|---|
| Complaint | 0.000330 | 0.038683 |
| Ordinary inquiry | 0.999670 | 0.961316 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.A10.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.A10.reverse
Prompt and settings
I am happy with the speaker and I am not complaining. I just need a little help pairing it with my tablet. Which button should I hold to start pairing?
{
"fixture_id": "A10",
"cohorts": [
"E1_JEV_INITIAL"
],
"modality": "audio",
"selected_attempt_id": "A10-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/asset.wav",
"sha256": "442eddd9ccc75ba77235d0feb6cef7259201b61f05a542cdfa87fc2bc6e2633e",
"bytes": 374444
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/asset.wav",
"sha256": "442eddd9ccc75ba77235d0feb6cef7259201b61f05a542cdfa87fc2bc6e2633e",
"bytes": 374444
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/receipt.json",
"sha256": "527db193157f6a7cbe2c43112f1b771c75187410b012eb41ece1915064e89ae5",
"bytes": 1569
},
"observed_media": {
"format": "PCM WAV",
"channels": 1,
"sample_rate_hz": 24000,
"sample_width_bytes": 2,
"duration_seconds": 7.8
},
"generator_id": "chatterbox_turbo",
"prompt_field": "input",
"seed": null,
"seed_status": "Not exposed by the installed speech API; no seed or deterministic regeneration claim.",
"submitted_settings": {
"language": "en"
},
"fixture_or_configured_settings": {
"language": "en",
"voice": "bundled default",
"voice_cloning": false,
"format": "PCM WAV",
"max_text_chars": 500,
"max_duration_seconds": 30,
"seed": null,
"seed_availability": "Not exposed through installed speech API; no reproducibility claim."
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/generation.request.json",
"sha256": "0b27bafa5a2b73cdd2a6628300ad10828a0b6b7ff05fd1407bad1ac20f41daef",
"bytes": 215
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/A10-attempt-00/generation.json",
"sha256": "7fc58929b760c5c89f0a4942ac17500a3947c5a15fec5704c2a6159a80526419",
"bytes": 5384
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
10 images: selecting the main object
Each image asks the model to identify the main foreground object from 4 choices. These are the JPEGs used for evaluation.
For colander I06, Jev-Omni changed from Colander to Saucepan when the choices were reversed. Clef chose Colander in both orders. I08R is a tabletop fan, substituted before inference after all 3 corkscrew attempts failed acceptance.
I01: Apple

Frozen reference: Apple. Visible apple body, woody stem and waxy skin distinguish nearby fruits.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Apple", "Pear", "Tomato", "Peach"]
Reversed
["Peach", "Tomato", "Pear", "Apple"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Apple | Apple | Match / Match |
| Clef | Apple | Apple | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Apple | 0.999727 | 0.999759 | 0.986045 | 0.988375 |
| Pear | 0.000078 | 0.000190 | 0.005508 | 0.003795 |
| Tomato | 0.000022 | 0.000028 | 0.004290 | 0.003915 |
| Peach | 0.000173 | 0.000024 | 0.004158 | 0.003915 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I01.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I01.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I01.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I01.reverse
Prompt and settings
A single red-green apple resting on a pale wooden kitchen counter near a window. Clearly show the apple rounded fruit body, shallow top dimple and short brown woody stem, smooth waxy skin; keep the whole fruit visible. Soft daylight, close product photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I01",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I01-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/asset.jpg",
"sha256": "4315bc149771bfd736a920262f0880ca714d6d6ac968e7f26c27e640b978638d",
"bytes": 87892
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/native.png",
"sha256": "61903b8cbea29daedff7915dd2270048a4b4a18d829ea3d6d579fd7283836c3d",
"bytes": 917357
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/receipt.json",
"sha256": "74c71f861337dc3579797b017c8e4b8cb9ab83ea7bc0b11437931d192206cb82",
"bytes": 1296
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100201,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100201,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100201,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/generation.request.json",
"sha256": "387292ebf2b24f3990c0aee84699ead4128f2d3018d877c14004a4e355fde2de",
"bytes": 564
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I01-attempt-00/generation.json",
"sha256": "bae00b6b89b26600141479b43bcabf35cf6c63126f0fe0ccde616887593c0618",
"bytes": 31259
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I02: Ceramic teapot

Frozen reference: Ceramic teapot. Ceramic body, lid, spout and handle distinguish kettle, can and moka pot.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Electric kettle", "Ceramic teapot", "Watering can", "Moka pot"]
Reversed
["Moka pot", "Watering can", "Ceramic teapot", "Electric kettle"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Ceramic teapot | Ceramic teapot | Match / Match |
| Clef | Ceramic teapot | Ceramic teapot | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Electric kettle | 0.000135 | 0.001426 | 0.009908 | 0.004438 |
| Ceramic teapot | 0.998972 | 0.995985 | 0.971916 | 0.980958 |
| Watering can | 0.000200 | 0.002410 | 0.014643 | 0.010983 |
| Moka pot | 0.000693 | 0.000179 | 0.003533 | 0.003622 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I02.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I02.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I02.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I02.reverse
Prompt and settings
A single ivory glazed ceramic teapot on a dark dining table, three-quarter view. Clearly show its round ceramic body, removable lid with knob, curved pouring spout and loop handle, with no electrical cord or base. Natural afternoon light, realistic still life photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I02",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I02-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/asset.jpg",
"sha256": "fd04ead97f99532645ff4205bb5ef7a4a60f718bc924956165404a2e74447f72",
"bytes": 72057
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/native.png",
"sha256": "902e7bb898901204b5e9cb9352e3ebf5bc46cdc9f9927ebc70a1854faa4419c8",
"bytes": 893459
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/receipt.json",
"sha256": "92ae18ae18f11633424ee64e0283a87bd7fbb7a50d520805ff7db7f9b34dc7ef",
"bytes": 1279
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100202,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100202,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100202,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/generation.request.json",
"sha256": "f284a29e960718cd69c04732151f510046cb538b347e7b6579943d0a882fe212",
"bytes": 576
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I02-attempt-00/generation.json",
"sha256": "d8a3ff8a9ee5bf6db72e524bd2d28a40fce5e4317fd5e90c94552a2809e058c1",
"bytes": 31271
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I03: Claw hammer

Frozen reference: Claw hammer. Flat strike face opposite split claw defines claw hammer.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Mallet", "Pickaxe", "Claw hammer", "Screwdriver"]
Reversed
["Screwdriver", "Claw hammer", "Pickaxe", "Mallet"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Claw hammer | Claw hammer | Match / Match |
| Clef | Claw hammer | Claw hammer | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Mallet | 0.023393 | 0.002180 | 0.005112 | 0.008994 |
| Pickaxe | 0.056227 | 0.000686 | 0.002694 | 0.003991 |
| Claw hammer | 0.884709 | 0.996538 | 0.989583 | 0.984186 |
| Screwdriver | 0.035670 | 0.000596 | 0.002611 | 0.002830 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I03.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I03.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I03.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I03.reverse
Prompt and settings
A single steel claw hammer with a wooden handle lying diagonally on a clean workshop bench. Clearly show one flat striking face and an opposite curved split claw, with the full handle visible. Detailed realistic photograph, side view. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I03",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I03-attempt-02",
"selected_attempt_index": 2,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/asset.jpg",
"sha256": "55f1e36f8b2b10df9e640424b3ac47b01d3609c709bc8f72c6a3fcbd57ab02a1",
"bytes": 175434
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/native.png",
"sha256": "ebbdd19af1be1b72f43124e8fbaa81aab0d6449fc05843fa5661a210be79e394",
"bytes": 1398723
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/receipt.json",
"sha256": "5038bb00814fc84d81f87addaa16961b53f6310d06c82664d0c50ac6fd32b4e3",
"bytes": 1296
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026102203,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026102203,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026102203,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/generation.request.json",
"sha256": "b4f8a36caea0535b95dd864466ecd567f0d305410cd87b5f4b0bc29a7ab9784f",
"bytes": 539
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I03-attempt-02/generation.json",
"sha256": "72b7323cbe39bc6aef373154aa5bf609ce0ee5d90910da8b0091dafa2a3ff96b",
"bytes": 31234
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I04: Bicycle

Frozen reference: Bicycle. Two wheels, frame, pedal chain and absence of motor define bicycle.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Motorcycle", "Kick scooter", "Tricycle", "Bicycle"]
Reversed
["Bicycle", "Tricycle", "Kick scooter", "Motorcycle"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Bicycle | Bicycle | Match / Match |
| Clef | Bicycle | Bicycle | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Motorcycle | 0.000013 | 0.000167 | 0.003824 | 0.004034 |
| Kick scooter | 0.000028 | 0.000022 | 0.003374 | 0.003560 |
| Tricycle | 0.000160 | 0.000073 | 0.004612 | 0.005344 |
| Bicycle | 0.999799 | 0.999738 | 0.988190 | 0.987062 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I04.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I04.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I04.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I04.reverse
Prompt and settings
A single ordinary pedal bicycle parked beside a brick courtyard wall, entire bicycle visible from the side. Clearly show exactly two wheels, triangular frame, pedals and chain, saddle and handlebars, no motor. Overcast daylight, realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I04",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I04-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/asset.jpg",
"sha256": "0f07aec0990918bb711a076b15e87a3729dc7161818d1b8782b1e75f31c18a6b",
"bytes": 317955
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/native.png",
"sha256": "4fc3d4e678c178e1cfbd13680a4bdf4e21d6c86a6bc6e06402088d792884958c",
"bytes": 1805676
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/receipt.json",
"sha256": "c718e983d710a3e716d64878ba45d37245830843971c3d716769cb544a9637e3",
"bytes": 1309
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100204,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100204,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100204,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/generation.request.json",
"sha256": "f8faebb0705b5d97435eb0fa3e0bb73bcb83e7450bfafa614f02b6498348e35f",
"bytes": 555
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I04-attempt-00/generation.json",
"sha256": "fa20664a56043e029cd4390e77c7ef76f541a2362be361e15174ce55a35d674f",
"bytes": 31250
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I05: Binoculars

Frozen reference: Binoculars. Paired barrels, hinge and paired eyepieces define binoculars.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Binoculars", "Camera", "VR goggles", "Telescope"]
Reversed
["Telescope", "VR goggles", "Camera", "Binoculars"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Binoculars | Binoculars | Match / Match |
| Clef | Binoculars | Binoculars | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Binoculars | 0.999629 | 0.999916 | 0.987192 | 0.991572 |
| Camera | 0.000101 | 0.000057 | 0.004754 | 0.002273 |
| VR goggles | 0.000017 | 0.000019 | 0.003588 | 0.002742 |
| Telescope | 0.000253 | 0.000008 | 0.004466 | 0.003412 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I05.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I05.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I05.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I05.reverse
Prompt and settings
A single black pair of binoculars on a picnic blanket, three-quarter view. Clearly show two parallel optical barrels joined by a central hinge and focus wheel, two front objective lenses and two rear eyepieces. Crisp outdoor photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I05",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I05-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/asset.jpg",
"sha256": "e3a5ebdc541e62cb65ebddf267205b9a7f581a7b9e671c978b7c0fea2efbf0ba",
"bytes": 153600
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/native.png",
"sha256": "3752690855410af9813e6f9474afda490e68a0d108d9024edf1bc70e3e2158f1",
"bytes": 1235886
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/receipt.json",
"sha256": "4d17193ec7704dd1b627f44902e1e49bce44e7336cc3b73f20475ca9c50f4f19",
"bytes": 1368
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100205,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100205,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100205,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/generation.request.json",
"sha256": "c87ac9cac8415fb843244998f6e6ca8d2f4f1077f59eec58c5901a0deb8189cf",
"bytes": 541
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I05-attempt-00/generation.json",
"sha256": "b55ffe7fa6424cd7e62c25b8d81e63bfd472396d25913f0e30e3a43b38577aab",
"bytes": 31236
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I06: Colander

Frozen reference: Colander. Perforated bowl with handles and foot ring distinguishes a colander.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Mixing bowl", "Colander", "Fine-mesh sieve", "Saucepan"]
Reversed
["Saucepan", "Fine-mesh sieve", "Colander", "Mixing bowl"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Colander | Saucepan | Match / Mismatch |
| Clef | Colander | Colander | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Mixing bowl | 0.268925 | 0.103956 | 0.022317 | 0.018830 |
| Colander | 0.476610 | 0.213720 | 0.952654 | 0.950817 |
| Fine-mesh sieve | 0.019216 | 0.178570 | 0.009450 | 0.016360 |
| Saucepan | 0.235249 | 0.503754 | 0.015580 | 0.013993 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I06.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I06.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I06.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I06.reverse
Prompt and settings
A single stainless steel colander standing on a kitchen worktop, empty and viewed slightly from above. Clearly show its bowl shape with discrete round drainage holes across the walls and base, two side handles and a foot ring; holes are punched metal rather than mesh. Realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I06",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I06-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/asset.jpg",
"sha256": "bf43d2e6faf01aa3fba5486488fa3d58541debe9931d43647bc034f6b0d9da3f",
"bytes": 132115
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/native.png",
"sha256": "dd3b2aa7ecde89a437b58482bdff672515c2e8e2344d1dff5d588d4a91fb05d5",
"bytes": 1128379
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/receipt.json",
"sha256": "cc8ed1d0054b9fccbda890e1492282a12beb8969c7ccbaae494f3851f916c38e",
"bytes": 1338
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100206,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100206,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100206,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/generation.request.json",
"sha256": "f41e493e7efe1e50c3805b88b09cc285072b90218e95a5c8514c849c7ee17f27",
"bytes": 595
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I06-attempt-00/generation.json",
"sha256": "7a08671a999d7d4b2c539884b3990f7aff418f7916eaf004afb3f4277d319727",
"bytes": 31290
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I07: Magnetic compass

Frozen reference: Magnetic compass. Pivoting directional needle and compass casing distinguish instruments.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Wristwatch", "Barometer", "Magnetic compass", "Thermometer"]
Reversed
["Thermometer", "Magnetic compass", "Barometer", "Wristwatch"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Magnetic compass | Magnetic compass | Match / Match |
| Clef | Magnetic compass | Magnetic compass | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Wristwatch | 0.005407 | 0.003093 | 0.002242 | 0.002847 |
| Barometer | 0.040501 | 0.001165 | 0.002386 | 0.003892 |
| Magnetic compass | 0.938139 | 0.994777 | 0.993330 | 0.990230 |
| Thermometer | 0.015954 | 0.000966 | 0.002041 | 0.003031 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I07.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I07.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I07.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I07.reverse
Prompt and settings
A single open brass pocket magnetic compass on a flat rock outdoors. Clearly show a circular directional dial with simple untitled tick marks and a freely pivoting red-and-white needle pointing across it, a hinged cover and no watch hands, wrist strap or numeric readout. Close realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I07",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I07-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/asset.jpg",
"sha256": "7bd79a2659dac9281921a8c5304770abfecafb1b46c203d9eaf944edb130032b",
"bytes": 230111
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/native.png",
"sha256": "cc87cd16d34a666b2fba28d8240a43a7849015a6b8d96595528b0a056c9de999",
"bytes": 1569534
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/receipt.json",
"sha256": "e582bd37bc07321f79741b1162cc735ceda4f872873519738163a802132c5c4e",
"bytes": 1390
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100207,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100207,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100207,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/generation.request.json",
"sha256": "9b1f5fb4bcd85849b296eb85b2399dda65a126f3e90fa62463cbd219fa55222e",
"bytes": 604
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I07-attempt-00/generation.json",
"sha256": "5a960474c1be11cabad219fa0ab9c899a15b8a6408af6da2ce6c877645f8dd8c",
"bytes": 31299
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I08R: Electric desk fan

Frozen reference: Electric desk fan. Visible rotor blades inside a round protective grille, supported by a standing pedestal, identify an electric desk fan.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Table lamp", "Loudspeaker", "Humidifier", "Electric desk fan"]
Reversed
["Electric desk fan", "Humidifier", "Loudspeaker", "Table lamp"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Electric desk fan | Electric desk fan | Match / Match |
| Clef | Electric desk fan | Electric desk fan | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Table lamp | 0.000015 | 0.000289 | 0.001861 | 0.002577 |
| Loudspeaker | 0.000026 | 0.000035 | 0.001495 | 0.002346 |
| Humidifier | 0.000128 | 0.000158 | 0.002044 | 0.003108 |
| Electric desk fan | 0.999831 | 0.999517 | 0.994600 | 0.991969 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I08R.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I08R.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I08R.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I08R.reverse
Prompt and settings
A single electric desk fan on a plain desk. Clearly visible rotor blades inside a round protective grille, supported by a standing pedestal. Realistic photograph, full object visible. No text, labels, logos, watermarks or other salient objects.
{
"fixture_id": "I08R",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I08R-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/asset.jpg",
"sha256": "9c455bfcb3271d0bf877ff52e6ad886b53190ec090cbedbad3c17a9643c1f284",
"bytes": 172973
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/native.png",
"sha256": "52b582b2de26dd7fdeab2751f20c287103ecbe6caa337fd8361939cfbb374b86",
"bytes": 1144635
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/receipt.json",
"sha256": "509d7010b76570c73a49b10c1af8ff686021281aa6ef6571913d68c0e9648520",
"bytes": 1290
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100308,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100308,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100308,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/generation.request.json",
"sha256": "28edd8216f0294a8cb1aa4727fd998cd2f4a3f6c17b5f7a3dcc2ee31cc00ee52",
"bytes": 412
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I08R-attempt-00/generation.json",
"sha256": "e56b4e1cd6226e7ef135caed9e0312eb39c93ded3eb721f82d80aef27d3a4805",
"bytes": 30154
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input.",
"replacement_lineage": {
"replaces_fixture": "I08",
"original_target": "Winged corkscrew",
"new_target": "Electric desk fan",
"authorized_before_predictions": true,
"amendment": {
"path": "2026-10-02-jev-omni-40-synthetic/amendment.json",
"sha256": "294c7e0960d30a8615dd98f787dede79772bf4bf3b992b9917ad6473ee81cf42",
"bytes": 2274
},
"amendment_freeze": {
"path": "2026-10-02-jev-omni-40-synthetic/amendment.freeze.json",
"sha256": "4f52bdf1554f2aa9c06f64754f47c84b353cdb1747cacd537102223ae6e4d005",
"bytes": 225
},
"reason": "All three frozen winged-corkscrew attempts independently rejected for missing diagnostic helix/collar. User authorized one fixed everyday-object substitution before any classifier output; original protocol and rejected assets remain immutable."
}
}
I09: Badminton shuttlecock

Frozen reference: Badminton shuttlecock. Feather skirt around rounded cork base defines shuttlecock.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Badminton shuttlecock", "Feather duster", "Table-tennis ball", "Spinning top"]
Reversed
["Spinning top", "Table-tennis ball", "Feather duster", "Badminton shuttlecock"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Badminton shuttlecock | Badminton shuttlecock | Match / Match |
| Clef | Badminton shuttlecock | Badminton shuttlecock | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Badminton shuttlecock | 0.999583 | 0.999815 | 0.988556 | 0.990326 |
| Feather duster | 0.000123 | 0.000129 | 0.005436 | 0.004143 |
| Table-tennis ball | 0.000024 | 0.000040 | 0.003098 | 0.002938 |
| Spinning top | 0.000269 | 0.000016 | 0.002910 | 0.002593 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I09.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I09.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I09.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I09.reverse
Prompt and settings
A single white feather badminton shuttlecock on a blue sports-court floor, close side view. Clearly show a small rounded cork base and an open tapered crown of individually visible feathers secured in a circular skirt. Entire object in focus, realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I09",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I09-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/asset.jpg",
"sha256": "7fcf2f7a9ef0967c693a2dbcebcad84f6b58700057f4f1dd3d3c0cb1752168d0",
"bytes": 113819
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/native.png",
"sha256": "8b2c8c9b0f017e862b387d6f28ea3f563ea1184e1fef170f0648319d5700453e",
"bytes": 1080435
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/receipt.json",
"sha256": "fd6d6b2ba11ebd6afc5e9f8f3577cf16bac6544b2cc15f6a17f879472b86c873",
"bytes": 1363
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100209,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100209,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100209,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/generation.request.json",
"sha256": "ba913aeea879195372a8e23a66207edd0f24c7fb70929ca21f396ff7d2df6c2a",
"bytes": 569
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I09-attempt-00/generation.json",
"sha256": "e5493587362c21658dc6cf800c492e2127701ccd5a973da29a620badbaa87550",
"bytes": 31264
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
I10: Television remote control

Frozen reference: Television remote control. Elongated button body, directional pad and infrared tip distinguish remote.
State, question, and choices
state
Identify the single principal foreground object using its visible shape and diagnostic features.
question
Which option identifies the principal visible object?
Forward
["Calculator", "Television remote control", "Mobile phone", "Game controller"]
Reversed
["Game controller", "Mobile phone", "Television remote control", "Calculator"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Television remote control | Television remote control | Match / Match |
| Clef | Television remote control | Television remote control | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Calculator | 0.000131 | 0.001122 | 0.002589 | 0.003919 |
| Television remote control | 0.999112 | 0.996112 | 0.988725 | 0.989335 |
| Mobile phone | 0.000269 | 0.002537 | 0.003538 | 0.002164 |
| Game controller | 0.000488 | 0.000230 | 0.005148 | 0.004582 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.I10.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.I10.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.I10.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.I10.reverse
Prompt and settings
A single matte black television remote control on a grey sofa cushion, full length visible from above. Clearly show an elongated narrow handheld body, many rubber buttons including a circular directional pad, colored buttons and a small infrared emitter at the top, with no screen or joysticks. Realistic photograph. Only this object is salient. No text, letters, words, captions, logos, labels, watermarks, comparison panels or other candidate objects.
{
"fixture_id": "I10",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "image",
"selected_attempt_id": "I10-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/asset.jpg",
"sha256": "1c1d8a0e11414eb2ecb0cded580f1200ed7a14bbe70fd1d845220e2723e290e4",
"bytes": 222750
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/native.png",
"sha256": "d4635642744dfd4845bb4b96dfc93a43219610025c30e401afd9b7120e27fd02",
"bytes": 1610855
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/receipt.json",
"sha256": "72bd07dc13b97009ed162c09b68ee69a14f4e9eb90bb42b7454b2c208d86c441",
"bytes": 1359
},
"observed_media": {
"native_width": 1024,
"native_height": 1024,
"dimension_source": "independent receipt when recorded; otherwise configured dimensions",
"evaluated_format": "JPEG",
"jpeg_quality": 90,
"native_format": "PNG"
},
"generator_id": "sdxl_lightning_4step",
"prompt_field": "prompt",
"seed": 2026100210,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"size": "1024x1024",
"seed": 2026100210,
"num_inference_steps": 4,
"guidance_scale": 0,
"response_format": "url"
},
"fixture_or_configured_settings": {
"format": "PNG",
"seed": 2026100210,
"steps": 4,
"width": 1024,
"height": 1024,
"guidance_scale": 0,
"scheduler": "EulerDiscreteScheduler",
"timestep_spacing": "trailing"
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/generation.request.json",
"sha256": "2fd12d6c46c75a127d85c881b212d8aa941f6287ee70a9c5b6f458b202e7dc57",
"bytes": 621
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/I10-attempt-00/generation.json",
"sha256": "027e8ceeb27d464db91eb317ce194d493cf4a6a02014de4493f436306e60a5b2",
"bytes": 31316
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
10 Japanese emails: is a human reply required?
Reply required means an unresolved request for an answer, action, approval, confirmation, or specified follow-up. Acknowledgments, thanks, and quotations of resolved requests are No reply required. All emails are fictional.
Jev-Omni changed T01 from Reply required to No reply required under reversal. Clef chose reply required in both orders. Both models matched the other 9 emails in both orders.
T01: Exhibition hours and reservations
件名:展示会の受付時間について
架空ストア ご担当者様
来週の展示会への参加を検討しています。土曜日の受付は何時から何時まででしょうか。また、予約なしでも入場できますか。
お手数ですが、この二点について教えてください。
水野
Frozen reference: Reply required. Current explicit questions about hours and entry require answers.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:展示会の受付時間について
架空ストア ご担当者様
来週の展示会への参加を検討しています。土曜日の受付は何時から何時まででしょうか。また、予約なしでも入場できますか。
お手数ですが、この二点について教えてください。
水野
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Reply required | No reply required | Match / Mismatch |
| Clef | Reply required | Reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.958538 | 0.086016 | 0.991015 | 0.991875 |
| No reply required | 0.041462 | 0.913984 | 0.008985 | 0.008125 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T01.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T01.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T01.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T01.reverse
Prompt and settings
{
"fixture_id": "T01",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T01-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T01-attempt-00/asset.txt",
"sha256": "ad63ba8c1b406ac1cd643ed0a86b5bffa40f611b21fa927ea5c407d84f455470",
"bytes": 330
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T01-attempt-00/asset.txt",
"sha256": "ad63ba8c1b406ac1cd643ed0a86b5bffa40f611b21fa927ea5c407d84f455470",
"bytes": 330
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T01-attempt-00/receipt.json",
"sha256": "c4827fa675d2508e80031ee587a4c38318041ed4fffa343e8e5c5ee2a6915f92",
"bytes": 506
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T02: Remedy request for a delivered item
件名:届いた商品についてのお願い
架空ストア ご担当者様
本日届いた注文番号 F-204 のマグカップにひびがありました。交換を希望します。返送方法と交換品の発送予定をお知らせください。
写真は手元にありますので、必要でしたら送ります。よろしくお願いいたします。
藤井
Frozen reference: Reply required. Current remedy request includes return instructions and replacement timing.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:届いた商品についてのお願い
架空ストア ご担当者様
本日届いた注文番号 F-204 のマグカップにひびがありました。交換を希望します。返送方法と交換品の発送予定をお知らせください。
写真は手元にありますので、必要でしたら送ります。よろしくお願いいたします。
藤井
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Reply required | Reply required | Match / Match |
| Clef | Reply required | Reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.999157 | 0.999831 | 0.989672 | 0.990514 |
| No reply required | 0.000843 | 0.000169 | 0.010328 | 0.009486 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T02.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T02.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T02.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T02.reverse
Prompt and settings
{
"fixture_id": "T02",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T02-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T02-attempt-00/asset.txt",
"sha256": "2f059c84917dcc914502603dbd2011b2b554e6420ffb22c51ac79eb2e532dbd2",
"bytes": 382
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T02-attempt-00/asset.txt",
"sha256": "2f059c84917dcc914502603dbd2011b2b554e6420ffb22c51ac79eb2e532dbd2",
"bytes": 382
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T02-attempt-00/receipt.json",
"sha256": "2c903d2a95fd4f5d20338f417232b6f68de96e46d31499d42bce2448001894c0",
"bytes": 533
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T03: Approval of a quotation delivery method
件名:見積書の送付方法の確認
架空サービス ご担当者様
見積書を受け取りました。内容は社内で確認済みです。経理担当を宛先に追加してこの見積書を転送してもよろしいでしょうか。差し支えないか、ご承認をお願いします。
承認をいただくまで転送は控えます。
長谷川
Frozen reference: Reply required. Explicit approval request remains outstanding despite acknowledgment of receipt.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:見積書の送付方法の確認
架空サービス ご担当者様
見積書を受け取りました。内容は社内で確認済みです。経理担当を宛先に追加してこの見積書を転送してもよろしいでしょうか。差し支えないか、ご承認をお願いします。
承認をいただくまで転送は控えます。
長谷川
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Reply required | Reply required | Match / Match |
| Clef | Reply required | Reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.997566 | 0.999620 | 0.990441 | 0.991084 |
| No reply required | 0.002434 | 0.000380 | 0.009559 | 0.008916 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T03.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T03.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T03.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T03.reverse
Prompt and settings
{
"fixture_id": "T03",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T03-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T03-attempt-00/asset.txt",
"sha256": "63a2143131c028debd55fb558b52899e14f9db26fb6af773149016f819354446",
"bytes": 375
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T03-attempt-00/asset.txt",
"sha256": "63a2143131c028debd55fb558b52899e14f9db26fb6af773149016f819354446",
"bytes": 375
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T03-attempt-00/receipt.json",
"sha256": "46109acbe5f7a23b4d358dad9fd8d53d71aba62ec306ed9c8f4112991590a79e",
"bytes": 532
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T04: Receipt of material with a follow-up request
件名:資料の到着と確認のお願い
架空サービス ご担当者様
資料を受け取りました。ありがとうございます。添付した申込内容を確認いただき、変更が必要な場合は明日正午までにご連絡ください。変更がない場合も、その旨を一言ご返信ください。
確認結果を待って手続きを進めます。
森
Frozen reference: Reply required. Conditional follow-up explicitly requires a response whether changes are needed or not.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:資料の到着と確認のお願い
架空サービス ご担当者様
資料を受け取りました。ありがとうございます。添付した申込内容を確認いただき、変更が必要な場合は明日正午までにご連絡ください。変更がない場合も、その旨を一言ご返信ください。
確認結果を待って手続きを進めます。
森
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Reply required | Reply required | Match / Match |
| Clef | Reply required | Reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.995930 | 0.998901 | 0.992062 | 0.993359 |
| No reply required | 0.004070 | 0.001099 | 0.007938 | 0.006641 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T04.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T04.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T04.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T04.reverse
Prompt and settings
{
"fixture_id": "T04",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T04-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T04-attempt-00/asset.txt",
"sha256": "35636e02433c3e36c0a425444b54eb19c7a145eb933cbb59f56842303357d818",
"bytes": 396
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T04-attempt-00/asset.txt",
"sha256": "35636e02433c3e36c0a425444b54eb19c7a145eb933cbb59f56842303357d818",
"bytes": 396
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T04-attempt-00/receipt.json",
"sha256": "48b620cfe9a89fbae4de5763516572060d812261a2c78ef3bcbf3b148b6107e2",
"bytes": 530
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T05: A new registration question in a resolved thread
件名:登録情報について
架空サービス ご担当者様
先日は丁寧なご案内をありがとうございました。
> 登録時の電話番号は変更できますか。
> 会員画面から変更できます。
電話番号の変更は完了しました。なお、今回の連絡先メールアドレスの変更はまだできていません。変更手順をこのメールへの返信で教えていただけますか。
佐々木
Frozen reference: Reply required. Old quoted issue is completed but an embedded current email-address question asks for a reply.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:登録情報について
架空サービス ご担当者様
先日は丁寧なご案内をありがとうございました。
> 登録時の電話番号は変更できますか。
> 会員画面から変更できます。
電話番号の変更は完了しました。なお、今回の連絡先メールアドレスの変更はまだできていません。変更手順をこのメールへの返信で教えていただけますか。
佐々木
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Reply required | Reply required | Match / Match |
| Clef | Reply required | Reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.999117 | 0.999869 | 0.988492 | 0.989672 |
| No reply required | 0.000883 | 0.000131 | 0.011508 | 0.010328 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T05.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T05.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T05.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T05.reverse
Prompt and settings
{
"fixture_id": "T05",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T05-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T05-attempt-00/asset.txt",
"sha256": "2e581409133ffbdbe34e5f4bc9aa95178fa79a390163d4ec9fe0d33b830ca57b",
"bytes": 460
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T05-attempt-00/asset.txt",
"sha256": "2e581409133ffbdbe34e5f4bc9aa95178fa79a390163d4ec9fe0d33b830ca57b",
"bytes": 460
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T05-attempt-00/receipt.json",
"sha256": "767714b150d42dd5d1ecb3ee8913b5749f6767d1c917e73d84d5f1b372610bc8",
"bytes": 541
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T06: Acknowledgment of received material
件名:資料受領のご連絡
架空サービス ご担当者様
資料を確かに受け取りました。わかりやすいご説明をありがとうございます。まずは社内で共有いたします。
取り急ぎ、受領のお礼まで。
小林
Frozen reference: No reply required. Acknowledgment and thanks only, with no outstanding request.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:資料受領のご連絡
架空サービス ご担当者様
資料を確かに受け取りました。わかりやすいご説明をありがとうございます。まずは社内で共有いたします。
取り急ぎ、受領のお礼まで。
小林
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | Match / Match |
| Clef | No reply required | No reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.000123 | 0.000588 | 0.004265 | 0.004265 |
| No reply required | 0.999877 | 0.999412 | 0.995735 | 0.995735 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T06.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T06.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T06.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T06.reverse
Prompt and settings
{
"fixture_id": "T06",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T06-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T06-attempt-00/asset.txt",
"sha256": "c3f82c557a725293b36d64c3048ec0a6fa91847c7ea08060073fe35b0a843b77",
"bytes": 267
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T06-attempt-00/asset.txt",
"sha256": "c3f82c557a725293b36d64c3048ec0a6fa91847c7ea08060073fe35b0a843b77",
"bytes": 267
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T06-attempt-00/receipt.json",
"sha256": "2b16b862691f37e66412a3fe441629d3c88618817e319b026bc41656e7fcdfa9",
"bytes": 516
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T07: Replacement received and issue resolved
件名:交換品を受け取りました
架空ストア ご担当者様
交換品が届き、正常に使えることを確認しました。返送も完了し、今回の件はすべて解決しました。迅速なご対応に感謝いたします。
田中
Frozen reference: No reply required. States issue fully resolved and returns completed; no new request.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:交換品を受け取りました
架空ストア ご担当者様
交換品が届き、正常に使えることを確認しました。返送も完了し、今回の件はすべて解決しました。迅速なご対応に感謝いたします。
田中
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | Match / Match |
| Clef | No reply required | No reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.000163 | 0.000856 | 0.005140 | 0.004905 |
| No reply required | 0.999837 | 0.999144 | 0.994860 | 0.995095 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T07.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T07.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T07.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T07.reverse
Prompt and settings
{
"fixture_id": "T07",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T07-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T07-attempt-00/asset.txt",
"sha256": "c272d354f44f37bfaa99e9a4e97c94687bf8121afe9f85783c5104f47f43c8fb",
"bytes": 265
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T07-attempt-00/asset.txt",
"sha256": "c272d354f44f37bfaa99e9a4e97c94687bf8121afe9f85783c5104f47f43c8fb",
"bytes": 265
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T07-attempt-00/receipt.json",
"sha256": "9b190814dea66a12494273606796632959f18167b557ed17c3ee4f8c55e64ba2",
"bytes": 516
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T08: Information about a contact change
件名:担当者変更のお知らせ
架空サービス ご担当者様
来月から弊社の窓口担当が吉田に変わります。連絡先一覧は本日、共有済みです。これは情報共有のみのお知らせで、返信は不要です。
引き続きよろしくお願いいたします。
中村
Frozen reference: No reply required. FYI explicitly states no reply and contains no conflicting question.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:担当者変更のお知らせ
架空サービス ご担当者様
来月から弊社の窓口担当が吉田に変わります。連絡先一覧は本日、共有済みです。これは情報共有のみのお知らせで、返信は不要です。
引き続きよろしくお願いいたします。
中村
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | Match / Match |
| Clef | No reply required | No reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.000156 | 0.000473 | 0.004039 | 0.003679 |
| No reply required | 0.999844 | 0.999527 | 0.995961 | 0.996322 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T08.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T08.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T08.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T08.reverse
Prompt and settings
{
"fixture_id": "T08",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T08-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T08-attempt-00/asset.txt",
"sha256": "6f7cfde952fe36c696c72be7d0058c25e6d1d78be5dec5c77948ab4880fc10aa",
"bytes": 321
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T08-attempt-00/asset.txt",
"sha256": "6f7cfde952fe36c696c72be7d0058c25e6d1d78be5dec5c77948ab4880fc10aa",
"bytes": 321
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T08-attempt-00/receipt.json",
"sha256": "d8710369b006bc7aeaf228328884e4ca311ced837fb0d0eb60c2befdc21f728f",
"bytes": 514
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T09: An already agreed visit schedule
件名:訪問日程の共有
架空サービス ご担当者様
先ほどお電話で合意したとおり、訪問日時は十月十五日午後二時で確定しています。当日は受付で担当の山田をお呼びします。
このメールは確定済みの日程を記録するためのご連絡です。追加の確認事項はありません。
井上
Frozen reference: No reply required. Records an already agreed scheduled fact and explicitly has no further confirmation request.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:訪問日程の共有
架空サービス ご担当者様
先ほどお電話で合意したとおり、訪問日時は十月十五日午後二時で確定しています。当日は受付で担当の山田をお呼びします。
このメールは確定済みの日程を記録するためのご連絡です。追加の確認事項はありません。
井上
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | Match / Match |
| Clef | No reply required | No reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.000207 | 0.001001 | 0.005865 | 0.005911 |
| No reply required | 0.999793 | 0.999000 | 0.994135 | 0.994089 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T09.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T09.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T09.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T09.reverse
Prompt and settings
{
"fixture_id": "T09",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T09-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T09-attempt-00/asset.txt",
"sha256": "e8bfa4730c00e17275008b83872f9b4f69b614dcaba8bd8dfc84d0d0c973d1c3",
"bytes": 372
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T09-attempt-00/asset.txt",
"sha256": "e8bfa4730c00e17275008b83872f9b4f69b614dcaba8bd8dfc84d0d0c973d1c3",
"bytes": 372
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T09-attempt-00/receipt.json",
"sha256": "ff4d8e0201cd7fdce5072ef78a2453fc6d61f80d3526730f8db72175e22cf52d",
"bytes": 522
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
T10: Membership information update completed
件名:会員情報の更新完了
架空サービス ご担当者様
ご案内どおり操作し、会員情報の更新が完了しました。保存後の画面でも変更内容を確認できました。
> 住所の修正方法を教えていただけますか。
> 設定画面の「会員情報」から修正できます。
上記は以前のやり取りです。今回は完了のご報告のみで、新たな質問や依頼はありません。
加藤
Frozen reference: No reply required. Quoted past request is resolved; current message expressly contains no new request.
State, question, and choices
state
Classify only the following fictional customer email. A reply is required for an outstanding explicit request for an answer, remedy, decision, approval, confirmation, or specified follow-up. Acknowledgments and information with no outstanding request do not require a reply. A quoted old request alone does not create a new obligation.
Email:
件名:会員情報の更新完了
架空サービス ご担当者様
ご案内どおり操作し、会員情報の更新が完了しました。保存後の画面でも変更内容を確認できました。
> 住所の修正方法を教えていただけますか。
> 設定画面の「会員情報」から修正できます。
上記は以前のやり取りです。今回は完了のご報告のみで、新たな質問や依頼はありません。
加藤
question
Does this email require a human reply?
Forward
["Reply required", "No reply required"]
Reversed
["No reply required", "Reply required"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | No reply required | No reply required | Match / Match |
| Clef | No reply required | No reply required | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Reply required | 0.000116 | 0.000579 | 0.008916 | 0.007877 |
| No reply required | 0.999884 | 0.999421 | 0.991084 | 0.992123 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.T10.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.T10.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.T10.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.T10.reverse
Prompt and settings
{
"fixture_id": "T10",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON"
],
"modality": "text",
"selected_attempt_id": "T10-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T10-attempt-00/asset.txt",
"sha256": "cde565a898ffffbca2c7f184efa9cf5dad3d8ceb889e680ca28b71731996a2cc",
"bytes": 469
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T10-attempt-00/asset.txt",
"sha256": "cde565a898ffffbca2c7f184efa9cf5dad3d8ceb889e680ca28b71731996a2cc",
"bytes": 469
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/T10-attempt-00/receipt.json",
"sha256": "0aa2d45ab1b4bb5f44dc5ef5eb9d5a1ca1a9ef0b8383f7750766519a9f242b86",
"bytes": 557
},
"generator_id": null,
"generation_method": "Original fictional Japanese email authored before predictions; no generative model used.",
"seed": null,
"settings": null
}
The original 5 videos: does a dangerous situation appear?
The question asks whether the video shows an immediate hazard in the ego vehicle’s path at any point. Later avoidance does not erase an earlier danger.
V01 shows a child entering the path; V02 shows a car cutting in ahead. Their expected answer was Dangerous, without requiring a visible collision or injury. V08, V09, and V10 were Safe.
Jev-Omni chose Safe for all 5 videos in both orders: 0/2 matches on Dangerous cases and 3/3 on Safe cases. Clef matched all 5.
The clips include avoidance and stopping, so it can be unclear which moment should count as dangerous. A better comparison needs footage that clearly shows the target event and a question suited to it. These 5 clips do not show whether a model can judge danger on real roads.
V01: A child enters the ego vehicle path
Frozen reference: Dangerous. Visible immediate path conflict.
State, question, and choices
state
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
question
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
Forward
["Dangerous", "Safe"]
Reversed
["Safe", "Dangerous"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Safe | Safe | Mismatch / Mismatch |
| Clef | Dangerous | Dangerous | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Dangerous | 0.131173 | 0.437343 | 0.916708 | 0.934395 |
| Safe | 0.868827 | 0.562657 | 0.083292 | 0.065605 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.V01.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.V01.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.V01.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.V01.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. On a quiet residential street in daylight, ego car moves steadily forward at a plausible moderate speed. A clearly visible fictional child on the right sidewalk approaches the curb from 0.4 seconds. At 1.6 seconds the child steps from the curb into the ego lane a short distance ahead, continuing toward the lane center. Show converging paths and rapidly shrinking stopping gap by 2.5 seconds; ego brakes promptly. End while separated before impact.
{
"fixture_id": "V01",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V01-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/asset.mp4",
"sha256": "dcd7a01b8b25b24b46df0a9af7508a5b2443bc200225bdd9df4587a241f56014",
"bytes": 3691855
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/native.mp4",
"sha256": "2d902c8c5e945b74f787ffdcae22d2406cb0a2d8051c0e7bad527b0d72a81450",
"bytes": 2577780
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/receipt.json",
"sha256": "9148fd6d0fef2f3eac570f287fcda50ccf838c9d4060ddc74179c41085d05da9",
"bytes": 3576
},
"observed_media": {
"native": {
"sha256": "2d902c8c5e945b74f787ffdcae22d2406cb0a2d8051c0e7bad527b0d72a81450",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 2577780
},
"evaluated_transport": {
"sha256": "dcd7a01b8b25b24b46df0a9af7508a5b2443bc200225bdd9df4587a241f56014",
"bytes": 3691855,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026100201,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026100201
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026100201,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/generation.request.json",
"sha256": "6751480e8f8b19faa6579718fd1920fc9b55cad49bc752dfe947de109f7928a9",
"bytes": 850
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V01-attempt-00/generation.json",
"sha256": "5914ef666fdbded464bdda47e5ed043dcf01db66f9d4de839f39087fcc7fa662",
"bytes": 103820
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
V02: A car moves into the forward path
Frozen reference: Dangerous. Visible immediate path conflict.
State, question, and choices
state
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
question
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
Forward
["Dangerous", "Safe"]
Reversed
["Safe", "Dangerous"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Safe | Safe | Mismatch / Mismatch |
| Clef | Dangerous | Dangerous | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Dangerous | 0.110471 | 0.421552 | 0.551574 | 0.661048 |
| Safe | 0.889529 | 0.578448 | 0.448426 | 0.338952 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.V02.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.V02.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.V02.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.V02.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. On a daylight urban two-lane road, ego moves forward behind open road. A car in the adjacent right lane begins a clear leftward move at 0.8 seconds. At 1.4 seconds it cuts across the lane marking into the ego lane directly ahead with very short following distance; show rear vehicle size increasing and a closing gap. Ego brakes promptly. End before contact.
{
"fixture_id": "V02",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V02-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/asset.mp4",
"sha256": "4b23e1598cf11faf1711d2416d21b93531a38222dd23ee5d797290c3ba6cb96e",
"bytes": 2340360
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/native.mp4",
"sha256": "ea0bba64e4a7ae24d6ce6016847b2d2ade4c49e5d2c8c08f5d5c1b8923cae12d",
"bytes": 1892973
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/receipt.json",
"sha256": "2d4be3befc00dc0213186394d80f59436a7fdf7602be31162a046cbc52d8866d",
"bytes": 3509
},
"observed_media": {
"native": {
"sha256": "ea0bba64e4a7ae24d6ce6016847b2d2ade4c49e5d2c8c08f5d5c1b8923cae12d",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 1892973
},
"evaluated_transport": {
"sha256": "4b23e1598cf11faf1711d2416d21b93531a38222dd23ee5d797290c3ba6cb96e",
"bytes": 2340360,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026100202,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026100202
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026100202,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/generation.request.json",
"sha256": "c595d09476da1b19da583fcd5de702639e214134f56c199d0b2937a307a923cb",
"bytes": 759
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V02-attempt-00/generation.json",
"sha256": "aa5568995b80bee66ac6c44eefdfa63c7f5608b4b490c91d1f4bdaf73f4e8bb3",
"bytes": 103676
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
V08: A lead car with visible separation
Frozen reference: Safe. Visible separation, adequate headway or stationary ego removes immediate conflict.
State, question, and choices
state
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
question
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
Forward
["Dangerous", "Safe"]
Reversed
["Safe", "Dangerous"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Safe | Safe | Match / Match |
| Clef | Safe | Safe | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Dangerous | 0.071074 | 0.325950 | 0.008916 | 0.008413 |
| Safe | 0.928926 | 0.674050 | 0.991084 | 0.991587 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.V08.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.V08.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.V08.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.V08.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. Daylight dashcam on a straight road. Ego follows one car well ahead with clearly distant, nonclosing separation and substantial road visible between ego and the leading car at their modest steady speed. Both continue straight, leading car maintains nearly constant apparent size and relative position throughout all four seconds. Open road, no braking, no lane changes, no pedestrians.
{
"fixture_id": "V08",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V08-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/asset.mp4",
"sha256": "0522b5fc81629cb5dcc242a177d955feebe15f4b1805e17f952f9741c20e7f92",
"bytes": 1512796
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/native.mp4",
"sha256": "908544e0b7c40d48f741a03c6806d12d599151894e2c1d872c73343a2596e6e4",
"bytes": 1145176
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/receipt.json",
"sha256": "8dee295628be4cf88d390d6cf7bd4f5938681a290b4040cace4c5958492acd66",
"bytes": 3184
},
"observed_media": {
"native": {
"sha256": "908544e0b7c40d48f741a03c6806d12d599151894e2c1d872c73343a2596e6e4",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 1145176
},
"evaluated_transport": {
"sha256": "0522b5fc81629cb5dcc242a177d955feebe15f4b1805e17f952f9741c20e7f92",
"bytes": 1512796,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026100208,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026100208
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026100208,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/generation.request.json",
"sha256": "7da0f7da23db7d5286c724e892e8bf9d1e58118d533d80ce8845ccc57fb3c6e1",
"bytes": 786
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V08-attempt-00/generation.json",
"sha256": "2fe9db64ddae1a33bfc7780f487e7971392e78b13231887e4d5af210ea7efc04",
"bytes": 103604
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
V09: Opposing traffic behind a concrete median
Frozen reference: Safe. Visible separation, adequate headway or stationary ego removes immediate conflict.
State, question, and choices
state
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
question
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
Forward
["Dangerous", "Safe"]
Reversed
["Safe", "Dangerous"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Safe | Safe | Match / Match |
| Clef | Safe | Safe | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Dangerous | 0.039937 | 0.302391 | 0.007233 | 0.006641 |
| Safe | 0.960063 | 0.697609 | 0.992767 | 0.993359 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.V09.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.V09.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.V09.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.V09.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. Daylight dashcam in light rain, with small droplets near windshield edges but a clear central view. Ego moves steadily at a modest speed along an empty lane on a divided road with a continuous central barrier. Show generous forward sight distance, no standing water, no skidding, and no vehicles or people entering the lane for all four seconds.
{
"fixture_id": "V09",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V09-attempt-01",
"selected_attempt_index": 1,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/asset.mp4",
"sha256": "cdcce5e0b09700ed8b457ea81e5415c86720a889b47237d394e89276cfe72330",
"bytes": 1852141
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/native.mp4",
"sha256": "92e1afd4175c487dec865e0d32b90617e0e93e12e585d0530c93722f9ef0d17e",
"bytes": 1488990
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/receipt.json",
"sha256": "816a71502951f3d06ee034afffd46b34743493dd89ecd884270559a9c6ebe9bc",
"bytes": 3624
},
"observed_media": {
"native": {
"sha256": "92e1afd4175c487dec865e0d32b90617e0e93e12e585d0530c93722f9ef0d17e",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 1488990
},
"evaluated_transport": {
"sha256": "cdcce5e0b09700ed8b457ea81e5415c86720a889b47237d394e89276cfe72330",
"bytes": 1852141,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101209,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101209
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026101209,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/generation.request.json",
"sha256": "254de09d7582b9de7673f49339d52415789f404de4b6b7f845aa7faa0872c963",
"bytes": 746
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V09-attempt-01/generation.json",
"sha256": "8c5fc59eb20aa084866e5cdfe0449cd0b22a785e7b42bd9563aa92870a50261b",
"bytes": 103563
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
V10: A cyclist in a separate lane
Frozen reference: Safe. Visible separation, adequate headway or stationary ego removes immediate conflict.
State, question, and choices
state
Assess only the visible dashcam scene. Dangerous means a visible immediate conflict with the ego vehicle travel path requires prompt braking or evasion. Safe means visible separation, adequate headway, or a stopped ego vehicle removes immediate conflict. The presence of a child, rain, or another vehicle alone does not establish danger.
question
Does any interval in this clip show an immediate dangerous conflict with the ego vehicle travel path, or is the situation safe throughout? Later successful braking does not erase an earlier dangerous conflict.
Forward
["Dangerous", "Safe"]
Reversed
["Safe", "Dangerous"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Safe | Safe | Match / Match |
| Clef | Safe | Safe | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Dangerous | 0.043043 | 0.262086 | 0.013428 | 0.012920 |
| Safe | 0.956957 | 0.737914 | 0.986572 | 0.987080 |
Saved responses
- Jev-Omni forward JSON —
E1_JEV_INITIAL.jev.V10.forward - Jev-Omni reversed JSON —
E1_JEV_INITIAL.jev.V10.reverse - Clef forward JSON —
E2_CLEF_COMMON.clef.V10.forward - Clef reversed JSON —
E2_CLEF_COMMON.clef.V10.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam clip, fixed camera inside ego car, coherent continuous shot, no cuts, no text overlays, labels, captions, logos or watermarks. Physically consistent vehicles, road geometry and actor motion. Non-graphic: no injury, no collision, no impact. Daylight dashcam with ego moving straight at steady modest speed. A cyclist travels in the same direction to the right, entirely in a protected bike lane separated by a raised continuous curb and regular bollards. Keep the cyclist, curb and bollards clearly visible; cyclist remains behind the separation for all four seconds. Ego lane remains open, no turns or crossings.
{
"fixture_id": "V10",
"cohorts": [
"E1_JEV_INITIAL",
"E2_CLEF_COMMON",
"E3_EMERGENCY_ORIGINAL"
],
"modality": "video",
"selected_attempt_id": "V10-attempt-00",
"selected_attempt_index": 0,
"evaluated_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/asset.mp4",
"sha256": "5cf43c1116b9b095163b8f0df86f4b4865d5c27dae7f926e5e88c6d1dd94457e",
"bytes": 3608621
},
"native_asset": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/native.mp4",
"sha256": "cc00e7a6bb83ff19d457c576ada2da36be35190bfdfe87029dd0bd68484fc934",
"bytes": 2992586
},
"acceptance_receipt": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/receipt.json",
"sha256": "ac5999bac89cd93d9a5c5277b51a8f243a98499ecb31ee74e8ef74bce25b4199",
"bytes": 3318
},
"observed_media": {
"native": {
"sha256": "cc00e7a6bb83ff19d457c576ada2da36be35190bfdfe87029dd0bd68484fc934",
"duration_seconds": 4.458333,
"width": 1344,
"height": 768,
"fps": "24/1",
"codec": "h264",
"frames": 107,
"bytes": 2992586
},
"evaluated_transport": {
"sha256": "5cf43c1116b9b095163b8f0df86f4b4865d5c27dae7f926e5e88c6d1dd94457e",
"bytes": 3608621,
"duration_seconds": 4,
"frames": 96,
"fps": 24
}
},
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026100210,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026100210
},
"fixture_or_configured_settings": {
"format": "MP4",
"seed": 2026100210,
"width": 1344,
"height": 768,
"duration_seconds": 4,
"fps": 24,
"api_steps": 9,
"short_edge": 768,
"aspect_ratio": "16:9",
"flow_shift_high": 12,
"flow_shift_low": 3
},
"generation_request": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/generation.request.json",
"sha256": "9f06d2a26666eefcd0837995375217486b70cb0201bc707f47f745257750b16d",
"bytes": 773
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-jev-omni-40-synthetic/attempts/V10-attempt-00/generation.json",
"sha256": "bfc3e83254a7327d7743f58a50537632c652ee10283eb1457569fd21655e4bca",
"bytes": 103605
},
"request_vs_configuration_notice": "Submitted settings and recorded runtime/fixture settings are separate evidence; a prepared generation record alone is not proof of execution. The selected asset and pre-inference acceptance receipt establish the retained input."
}
Asking about emergency stopping in the original 5 videos
We then asked whether the same 5 videos required an emergency stop to avoid a collision. This means immediate, hard braking to a stop when ordinary deceleration or steering would not suffice.
The description, criteria, and choices changed along with the question, so any response changes cannot be attributed to wording alone.
V01 and V02 were ambiguous: the footage did not establish speed, remaining distance, or deceleration well enough. This differs from the earlier Dangerous/Safe judgment.
| Expected answer | Videos | Responses per model and treatment |
|---|---|---|
| Emergency stop needed | 0 | 0 |
| Emergency stop not needed | 3 | 6 responses, all in agreement |
| ambiguous | 2 | 4 responses, excluded from agreement counts |
E3 contains 20 responses: 2 models × 5 original videos × 2 orders.
V01: A child enters the ego vehicle path
Frozen reference: ambiguous. A child moves from near the right curb into the forward roadway while the ego viewpoint advances. The conflict supports slowing or stopping, but the child is visible over multiple seconds and the clip does not establish ego speed, remaining distance, or deceleration accurately enough to distinguish immediate very hard stopping from ordinary braking. No collision or verified emergency braking is visible.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Clef | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.046034 | 0.357119 | 0.173849 | 0.346873 |
| Emergency stop not needed | 0.953966 | 0.642881 | 0.826151 | 0.653127 |
Saved responses
- Jev-Omni forward JSON —
E3_EMERGENCY_ORIGINAL.jev.V01.forward - Jev-Omni reversed JSON —
E3_EMERGENCY_ORIGINAL.jev.V01.reverse - Clef forward JSON —
E3_EMERGENCY_ORIGINAL.clef.V01.forward - Clef reversed JSON —
E3_EMERGENCY_ORIGINAL.clef.V01.reverse
V02: A car moves into the forward path
Frozen reference: ambiguous. A white sedan ahead to the right moves left across the lane divider into the ego forward path as the apparent gap closes near an intersection. This supports braking and maintaining separation. Its motion is gradual in the visible sequence; the synthetic viewpoint and uncalibrated speed/gap do not establish that ordinary braking is insufficient or immediate very hard stopping is required. No collision is visible.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Clef | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.012054 | 0.157896 | 0.032961 | 0.076961 |
| Emergency stop not needed | 0.987946 | 0.842104 | 0.967039 | 0.923039 |
Saved responses
- Jev-Omni forward JSON —
E3_EMERGENCY_ORIGINAL.jev.V02.forward - Jev-Omni reversed JSON —
E3_EMERGENCY_ORIGINAL.jev.V02.reverse - Clef forward JSON —
E3_EMERGENCY_ORIGINAL.clef.V02.forward - Clef reversed JSON —
E3_EMERGENCY_ORIGINAL.clef.V02.reverse
V08: A lead car with visible separation
Frozen reference: Emergency stop not needed. The lead car remains ahead in its lane and the visible forward route stays open. There is no visible abrupt intrusion, stopped obstacle, or rapidly developing collision conflict requiring an immediate very hard stop. Following-distance adjustment may still be appropriate; exact speed and distance are not measurable here.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Match / Match |
| Clef | Emergency stop not needed | Emergency stop not needed | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.013223 | 0.138520 | 0.006004 | 0.006004 |
| Emergency stop not needed | 0.986777 | 0.861480 | 0.993996 | 0.993996 |
Saved responses
- Jev-Omni forward JSON —
E3_EMERGENCY_ORIGINAL.jev.V08.forward - Jev-Omni reversed JSON —
E3_EMERGENCY_ORIGINAL.jev.V08.reverse - Clef forward JSON —
E3_EMERGENCY_ORIGINAL.clef.V08.forward - Clef reversed JSON —
E3_EMERGENCY_ORIGINAL.clef.V08.reverse
V09: Opposing traffic behind a concrete median
Frozen reference: Emergency stop not needed. The ego forward lanes remain open throughout the inspected transport prefix. Passing opposing vehicles remain across a continuous concrete median, and no object enters the ego route. Wet-looking pavement and windshield droplets alone do not establish an imminent collision requiring emergency stopping.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Match / Match |
| Clef | Emergency stop not needed | Emergency stop not needed | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.005730 | 0.089773 | 0.005100 | 0.004982 |
| Emergency stop not needed | 0.994270 | 0.910227 | 0.994900 | 0.995018 |
Saved responses
- Jev-Omni forward JSON —
E3_EMERGENCY_ORIGINAL.jev.V09.forward - Jev-Omni reversed JSON —
E3_EMERGENCY_ORIGINAL.jev.V09.reverse - Clef forward JSON —
E3_EMERGENCY_ORIGINAL.clef.V09.forward - Clef reversed JSON —
E3_EMERGENCY_ORIGINAL.clef.V09.reverse
V10: A cyclist in a separate lane
Frozen reference: Emergency stop not needed. A cyclist stays to the right in a separate cycle lane behind the raised divider/posts while the ego forward lane remains open. No crossing into the ego route or other imminent collision conflict is visible. The approaching intersection/signals may call for ordinary slowing or stopping; that does not establish emergency braking to avoid a collision.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Match / Match |
| Clef | Emergency stop not needed | Emergency stop not needed | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.006798 | 0.087720 | 0.007289 | 0.007877 |
| Emergency stop not needed | 0.993203 | 0.912280 | 0.992711 | 0.992123 |
Saved responses
- Jev-Omni forward JSON —
E3_EMERGENCY_ORIGINAL.jev.V10.forward - Jev-Omni reversed JSON —
E3_EMERGENCY_ORIGINAL.jev.V10.reverse - Clef forward JSON —
E3_EMERGENCY_ORIGINAL.clef.V10.forward - Clef reversed JSON —
E3_EMERGENCY_ORIGINAL.clef.V10.reverse
Adding 5 near-approach videos
We generated 5 contact scenarios twice each, for 10 attempts. Acceptance required both an approach and clear contact within 4 seconds. An approach alone, or an object hidden by the hood, did not establish contact.
None met that criterion: 0 contact videos were accepted and 0 were classified as contact cases. We changed the criteria and used the final outputs C01–C05 as near-approach videos.
C01–C04 were ambiguous; C05 was Emergency stop not needed. Each model had 2 scored responses and 8 excluded responses. Both matched C05 in both orders.
Clef’s answers changed for C02 and C03
Clef changed C02 and C03 from “not needed” to “needed” when the choices were reversed. Jev-Omni kept its answers, but its probabilities changed.
| Model / video | Probability of “needed”: original order | Probability of “needed”: reversed order |
|---|---|---|
| Clef / C02 | 0.389547 | 0.609524 |
| Clef / C03 | 0.397938 | 0.577495 |
| Jev-Omni / C02 | 0.008316 | 0.140749 |
| Jev-Omni / C03 | 0.017712 | 0.241847 |
Both cases are ambiguous, so these changes are not counted as errors or successes.
C01: Approach to a red sedan
Frozen reference: ambiguous. Progressive closure and visible rear lamps support braking. Available stopping distance, calibrated closing speed and deceleration are unresolved; the sequence cannot reliably distinguish ordinary braking from a necessary immediate very-hard stop. Reference is ambiguous without assuming an impact from the close endpoint.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Clef | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.016029 | 0.178741 | 0.177241 | 0.405449 |
| Emergency stop not needed | 0.983971 | 0.821259 | 0.822759 | 0.594551 |
Saved responses
- Jev-Omni forward JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C01.forward - Jev-Omni reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C01.reverse - Clef forward JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C01.forward - Clef reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C01.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. A red sedan is stopped in the middle of the ego lane on a straight level road. It is visible directly ahead from the first frame. The ego vehicle continues moving forward, the stopped sedan grows rapidly larger, and the gap closes continuously from 0.3 to 2.5 seconds. A clear pre-impact interval shows the blocked route and urgent opportunity to brake to a stop. At about 3.0 seconds the ego front bumper visibly contacts the sedan rear bumper; the sedan body jolts forward slightly while both intact bumpers visibly meet. Keep the scene continuous and contact visible before 3.5 seconds.
{
"fixture_id": "C01",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C01.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101401,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101401
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C01.attempt-1.generation.request.json",
"sha256": "46f8aae33f19a8fcdc691864f7c247918a2f5d5d0ca502c65362e629c5d50cf4",
"bytes": 1223
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C01.attempt-1/native.mp4",
"sha256": "e5e24e4e27ac1d22034c821d730701612f81a831cdebfb45c2dda3619f7b3d6a",
"bytes": 1589923
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C01/asset.mp4",
"sha256": "bf1bb41e1947614f6bb59ffedc492696177bc79220b6aabe76206cf7dafa764b",
"bytes": 2340198
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C01.attempt-1.final-inspected.json",
"sha256": "a22d2da04ddf086bebd7a39dcc0b2dfc97dafd80b2cd5c5b0138ee68d43305dd",
"bytes": 5392
},
"reference_classification": "ambiguous",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
C02: Approach to a white van
Frozen reference: ambiguous. The van’s shrinking apparent gap supports braking but does not resolve available stopping distance, closing speed or required intensity. The visible sequence cannot reliably distinguish ordinary deceleration from a necessary immediate very-hard stop. Reference remains ambiguous without treating the endpoint or panel shading as a collision.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Clef | Emergency stop not needed | Emergency stop needed | Excluded / Excluded |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.008316 | 0.140749 | 0.389547 | 0.609524 |
| Emergency stop not needed | 0.991684 | 0.859251 | 0.610453 | 0.390476 |
Saved responses
- Jev-Omni forward JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C02.forward - Jev-Omni reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C02.reverse - Clef forward JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C02.forward - Clef reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C02.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. An empty white van travels in the ego lane directly ahead from the first frame. At 0.4 seconds its brake lights illuminate and it rapidly slows to a stop. The ego vehicle keeps advancing, so the van enlarges and the visible following gap shrinks quickly over the next two seconds. Show a clear pre-impact interval during which hard braking to a stop is available and necessary to avoid the stationary van. At about 3.1 seconds the ego front bumper visibly contacts the van rear bumper and the van shifts forward slightly, with both vehicles retaining coherent bodies. All approach and contact are visible before 3.5 seconds.
{
"fixture_id": "C02",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C02.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101402,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101402
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C02.attempt-1.generation.request.json",
"sha256": "4d6e628c5ec435523792444799caee6c83eb25340a20a76a3cdf6aec3ba68618",
"bytes": 1258
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C02.attempt-1/native.mp4",
"sha256": "4c3661cc6e148509355e852519dba702cd7bf5bd2d903bcae71c35436bd06917",
"bytes": 1627265
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C02/asset.mp4",
"sha256": "ebebbf186ff2f12392950c6bcd0c96de9512af5cc8d346a0999903d34499155c",
"bytes": 2366486
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C02.attempt-1.final-inspected.json",
"sha256": "746c3e825014a9435051ec38e4be317d8d9681d6fb9bcff2f7e924fdf9290dd1",
"bytes": 5392
},
"reference_classification": "ambiguous",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
C03: A pickup crosses the route
Frozen reference: ambiguous. The crossing pickup creates a visible route conflict. The clip does not establish the speed, remaining stopping distance or urgency sufficiently to distinguish an immediate very hard stop from ordinary braking or steering. The possible late door crease cannot supply that prospective judgment; retain an ambiguous emergency-stop reference.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Clef | Emergency stop not needed | Emergency stop needed | Excluded / Excluded |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.017712 | 0.241847 | 0.397938 | 0.577495 |
| Emergency stop not needed | 0.982288 | 0.758153 | 0.602062 | 0.422505 |
Saved responses
- Jev-Omni forward JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C03.forward - Jev-Omni reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C03.reverse - Clef forward JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C03.forward - Clef reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C03.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. At a small daylight junction an empty blue pickup is visible ahead on the left from the first frame. From 0.4 to 1.3 seconds it turns across the road into the ego forward lane and then pauses broadside across that lane. The ego vehicle continues forward; the pickup side becomes rapidly larger and its blocking position stays visible for a sustained pre-impact interval, leaving a visible opportunity for very hard braking to a stop. At about 3.1 seconds the ego front bumper visibly contacts the pickup side panel; the pickup rocks or shifts slightly sideways at the contact point. Do not hide contact behind another vehicle or outside the frame. Contact must occur before 3.5 seconds.
{
"fixture_id": "C03",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C03.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101403,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101403
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C03.attempt-1.generation.request.json",
"sha256": "5a99783a84d0c32faaee64a72c35383afda49e627e832bdd8e9a3aeb7995b415",
"bytes": 1320
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C03.attempt-1/native.mp4",
"sha256": "c2776962ea247eeaaea6fcccb81f7c0deaa92e88ece6602313282dcbedd07fd6",
"bytes": 2696603
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C03/asset.mp4",
"sha256": "283c80fc7c43e96537981c374fea22369fd3cfaea6c06f6bbfc23b098ce0e656",
"bytes": 3819217
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C03.attempt-1.final-inspected.json",
"sha256": "f177fe1ee9a6c48fa618d806973ca58c2257f686f4e2d363b82aa6242924b295",
"bytes": 5392
},
"reference_classification": "ambiguous",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
C04: A van emerges from the right
Frozen reference: ambiguous. The emerging van creates a visible route conflict, but the clip does not establish relative speed, stopping distance or timing well enough to distinguish the need for immediate very hard stopping from ordinary braking or steering. Retain an ambiguous emergency-stop reference without inferring contact.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Clef | Emergency stop not needed | Emergency stop not needed | Excluded / Excluded |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.018125 | 0.269326 | 0.175538 | 0.310694 |
| Emergency stop not needed | 0.981875 | 0.730674 | 0.824462 | 0.689306 |
Saved responses
- Jev-Omni forward JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C04.forward - Jev-Omni reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C04.reverse - Clef forward JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C04.forward - Clef reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C04.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. An empty silver delivery van is visible at a right-side driveway from the first frame. Beginning at 0.4 seconds it reverses slowly out of the driveway, moves across the curb line, and stops with its rear half occupying the ego lane by 1.3 seconds. The ego vehicle continues advancing on the same straight route; the intrusion and rapidly shrinking gap remain visible before impact, providing a clear opportunity to brake hard to a stop. At about 3.0 seconds the ego front bumper visibly contacts the van rear-side corner and the van pivots slightly at the contact point. Keep coherent road geometry and directly visible vehicle boundaries. All evidence lies before 3.5 seconds.
{
"fixture_id": "C04",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C04.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101404,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101404
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C04.attempt-1.generation.request.json",
"sha256": "bc8932eb1f1d5fec94bc205a283f0a28aee0a2ad3401945d9ce03d57e547a75b",
"bytes": 1311
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C04.attempt-1/native.mp4",
"sha256": "b7b373ade4f6b7bb6e396a3230b9315eea77ce1a72fa70051f035fcc0222df02",
"bytes": 1942105
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C04/asset.mp4",
"sha256": "6f2d6fc6201a4866cdd10b2517e38c161382820b8214526f31fc26393c361959",
"bytes": 2403091
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C04.attempt-1.final-inspected.json",
"sha256": "58c5a633e04cfddd78f9ce593b4a2645d2b9ca40d51c2ec1ebb02cfb38724b5d",
"bytes": 5392
},
"reference_classification": "ambiguous",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
C05: Approach to a yellow truck
Frozen reference: Emergency stop not needed. The visible prefix shows gradual closure to a truck that remains ahead with a clear road gap, without sudden hazard onset, abrupt relative motion or a moment of visible imminent contact. Maintaining or restoring following distance through ordinary gradual braking is supported by this sequence; an immediate very hard stop is not visibly required. This qualitative not_needed annotation is confined to the visible prefix and is not inferred solely from the absence of collision. Rejected media remains ineligible for benchmark scoring.
State, question, and choices
state
Judge only the visible driving sequence. An emergency stop means immediate, very hard braking intended to bring the ego vehicle to a stop to prevent an imminent collision; ordinary gradual braking or steering alone does not qualify. Consider the whole clip, including hazards later avoided. Do not assume a collision occurred or infer facts hidden outside the video.
question
At any point in this video, should the ego vehicle make an emergency stop to avoid a collision?
Forward
["Emergency stop needed", "Emergency stop not needed"]
Reversed
["Emergency stop not needed", "Emergency stop needed"]
| Model | Forward answer | Reversed answer | Reference match: forward / reversed |
|---|---|---|---|
| Jev-Omni | Emergency stop not needed | Emergency stop not needed | Match / Match |
| Clef | Emergency stop not needed | Emergency stop not needed | Match / Match |
| Choice | Jev-Omni forward | Jev-Omni reversed | Clef forward | Clef reversed |
|---|---|---|---|---|
| Emergency stop needed | 0.008985 | 0.148355 | 0.018055 | 0.035411 |
| Emergency stop not needed | 0.991015 | 0.851645 | 0.981945 | 0.964589 |
Saved responses
- Jev-Omni forward JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C05.forward - Jev-Omni reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.jev.C05.reverse - Clef forward JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C05.forward - Clef reversed JSON —
E4_NEAR_APPROACH_AMENDMENT.clef.C05.reverse
Prompt and settings
Four-second synthetic realistic forward-facing dashcam video, a fixed camera inside the ego vehicle, one continuous physically coherent shot, no cuts. Daylight on a closed road used only by empty fictional vehicles. No people, pedestrians, cyclists, visible occupants, injuries, blood, fire or graphic damage. No captions, text, labels, logos or watermarks. Show uninterrupted ego forward motion, visible road markings and changing vehicle separation before directly visible non-graphic vehicle-to-vehicle contact. On a slight uphill straight road an empty yellow box truck is centered in the ego lane directly ahead from the first frame. At 0.5 seconds the truck begins rolling backward toward the advancing ego vehicle. Show its rear wheels moving backward, its rear body enlarging rapidly, and a continuously shrinking open gap from 0.5 through 2.5 seconds. The truck remains entirely in the ego route with a sustained visible opportunity for immediate very hard braking to stop before the gap closes. At about 3.1 seconds the ego front bumper visibly contacts the truck rear bumper; the truck rocks slightly and relative motion abruptly couples at directly visible bumper contact. Keep all approach and contact evidence before 3.5 seconds.
{
"fixture_id": "C05",
"cohorts": [
"E4_NEAR_APPROACH_AMENDMENT"
],
"modality": "video",
"selected_attempt_id": "C05.attempt-1",
"selected_attempt_index": 1,
"generator_id": "minimax_h3_pdd8",
"prompt_field": "prompt",
"seed": 2026101405,
"seed_status": "requested; no cross-hardware determinism claimed",
"submitted_settings": {
"seconds": 4,
"aspect_ratio": "16:9",
"seed": 2026101405
},
"configured_request_controls": {
"num_inference_steps": 9,
"flow_shift": 12.0,
"audio_flow_shift": 3.0,
"target": {
"short_edge": 768,
"aspect_ratio": "16:9"
},
"duration_seconds": 4,
"fps": 24
},
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C05.attempt-1.generation.request.json",
"sha256": "64b1d072d10a35f37340c4705bb19ef220d0e7460c309af52fb38d91df142840",
"bytes": 1362
},
"recorded_runtime_identity_source": {
"path": "2026-10-02-clef-jev-collision/h3-loaded-identity.json",
"sha256": "8682073cf08b2dde8ac45bd592c47a9a1aaede7c5e9e577804a7e6eb003f11be",
"bytes": 4104
},
"native_asset": {
"path": "2026-10-02-clef-jev-collision/attempts/C05.attempt-1/native.mp4",
"sha256": "55a1e1f01f4aa4ca5b1ccc631c036f9f50795a5726ad28cddfb44194961bbc79",
"bytes": 2434210
},
"evaluated_asset": {
"path": "2026-10-02-clef-jev-near-approach/inputs/C05/asset.mp4",
"sha256": "fa9964489e1e78799a4db1e48a4e3cdab456e5add9fd9a3921e6c2a11b8e96c1",
"bytes": 2953313
},
"observed_media": {
"native_frame_count": 107,
"native_dimensions": null,
"native_dimensions_notice": "Selected final inspection does not record native dimensions; configured target is 1344x768. No new probe was run.",
"evaluated_transport_frames": 96,
"evaluated_transport_fps": 24,
"evaluated_transport_duration_seconds": 4
},
"inspection": {
"path": "2026-10-02-clef-jev-near-approach/references/C05.attempt-1.final-inspected.json",
"sha256": "673cb28357ea8619e8f86bd8429eb39c041e0efbf47ec905eb56b5b73e14cda4",
"bytes": 5392
},
"reference_classification": "not_needed",
"original_collision_media_accepted": false,
"selection_notice": "Selected final attempt index 1 for each slot under a separately frozen near-approach amendment; no result-based selection. The original prompt requested contact, but visible contact was not independently confirmed."
}
Combined counts for the original 5 and additional 5 videos
Across the 10 videos, 4 were “not needed,” 6 were ambiguous, and 0 were “needed.” E3 and E4 contain 40 responses in total: 2 models × 10 videos × 2 orders.
Each model matched all 8 scored responses; 12 were excluded. With no clear “needed” examples, emergency-stop detection sensitivity cannot be measured: needed_sensitivity = null.
Generation retries
Initial video generation took 20 attempts, with 5 accepted. V03–V07 remained unfilled. The 15 rejected generations are generation failures, not model errors.
| Unfilled slot | Planned scenario |
|---|---|
| V03 | Approaching a stopped queue of vehicles, requiring action |
| V04 | An oncoming vehicle entering the ego vehicle’s lane |
| V05 | A child following a ball into the vehicle’s path |
| V06 | A child separated from the road by a fence |
| V07 | The ego vehicle remains stopped from the start while waiting for someone to cross |
Corkscrew I08 failed all 3 attempts and was replaced by fan I08R. Hammer I03 also needed retries. A01-attempt-00 has no acceptance record; attempt-01 was used. null means the record is missing, not that generation did not occur.
The 46 initial attempts and 10 contact-video attempts follow. The generation records retain full prompts and source hashes. Rejected video binaries are not included.
The 46 initial attempt records
| Attempt | Acceptance receipt | Used in E1 | Requested seed |
|---|---|---|---|
A01-attempt-00 | No receipt | Not used | Unknown |
A01-attempt-01 | Accepted | Used | Unknown |
A02-attempt-00 | Accepted | Used | Unknown |
A03-attempt-00 | Accepted | Used | Unknown |
A04-attempt-00 | Accepted | Used | Unknown |
A05-attempt-00 | Accepted | Used | Unknown |
A06-attempt-00 | Accepted | Used | Unknown |
A07-attempt-00 | Accepted | Used | Unknown |
A08-attempt-00 | Accepted | Used | Unknown |
A09-attempt-00 | Accepted | Used | Unknown |
A10-attempt-00 | Accepted | Used | Unknown |
I01-attempt-00 | Accepted | Used | Unknown |
I02-attempt-00 | Accepted | Used | Unknown |
I03-attempt-00 | Rejected | Not used | 2026100203 |
I03-attempt-01 | Rejected | Not used | 2026101203 |
I03-attempt-02 | Accepted | Used | Unknown |
I04-attempt-00 | Accepted | Used | Unknown |
I05-attempt-00 | Accepted | Used | Unknown |
I06-attempt-00 | Accepted | Used | Unknown |
I07-attempt-00 | Accepted | Used | Unknown |
I08-attempt-00 | Rejected | Not used | 2026100208 |
I08-attempt-01 | Rejected | Not used | 2026101208 |
I08-attempt-02 | Rejected | Not used | 2026102208 |
I08R-attempt-00 | Accepted | Used | Unknown |
I09-attempt-00 | Accepted | Used | Unknown |
I10-attempt-00 | Accepted | Used | Unknown |
V01-attempt-00 | Accepted | Used | Unknown |
V02-attempt-00 | Accepted | Used | Unknown |
V03-attempt-00 | Rejected | Not used | 2026100203 |
V03-attempt-01 | Rejected | Not used | 2026101203 |
V03-attempt-02 | Rejected | Not used | 2026102203 |
V04-attempt-00 | Rejected | Not used | 2026100204 |
V04-attempt-01 | Rejected | Not used | 2026101204 |
V04-attempt-02 | Rejected | Not used | 2026102204 |
V05-attempt-00 | Rejected | Not used | 2026100205 |
V05-attempt-01 | Rejected | Not used | 2026101205 |
V05-attempt-02 | Rejected | Not used | 2026102205 |
V06-attempt-00 | Rejected | Not used | 2026100206 |
V06-attempt-01 | Rejected | Not used | 2026101206 |
V07-attempt-00 | Rejected | Not used | 2026100207 |
V07-attempt-01 | Rejected | Not used | 2026101207 |
V07-attempt-02 | Rejected | Not used | 2026102207 |
V08-attempt-00 | Accepted | Used | Unknown |
V09-attempt-00 | Rejected | Not used | 2026100209 |
V09-attempt-01 | Accepted | Used | Unknown |
V10-attempt-00 | Accepted | Used | Unknown |
| Attempt | Strict-contact acceptance | Used in E4 as near approach |
|---|---|---|
C01.attempt-0 | Rejected | Not used |
C01.attempt-1 | Rejected | Used |
C02.attempt-0 | Rejected | Not used |
C02.attempt-1 | Rejected | Used |
C03.attempt-0 | Rejected | Not used |
C03.attempt-1 | Rejected | Used |
C04.attempt-0 | Rejected | Not used |
C04.attempt-1 | Rejected | Used |
C05.attempt-0 | Rejected | Not used |
C05.attempt-1 | Rejected | Used |
C01.attempt-0: rejection reason and record
{
"attempt_id": "C01.attempt-0",
"fixture_id": "C01",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C01.attempt-0.generation.request.json",
"sha256": "683c37a6abd38a90d35fbbb56422cded06ab78548d0d47cdc26092db04c0fbf7",
"bytes": 1223
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C01.attempt-0/review.json",
"sha256": "2bfabeeef18c76bc0724836694951ddfbc5c72b188c5ddd24c3693dcf25c2c6f",
"bytes": 17089
},
"reason": "Rejected by the frozen media gate: the actual four-second transport shows a continuous approach to a red sedan with a decreasing apparent gap, but no unmistakable ego-to-vehicle contact. In full-resolution late frames the lower sedan/bumper becomes hidden behind the ego hood; its visible body remains aligned and intact, with no directly observable boundary contact, deformation, deflection or coherent collision consequence. Close following and hood occlusion alone cannot establish an impact. The native tail does not establish contact either and cannot rescue transport acceptance.",
"reference_reason": "The visible closing conflict supports braking, especially in the late approach. However the actual clip does not establish contact, stopping/deceleration, calibrated ego speed or closing distance. It cannot reliably distinguish a necessary immediate very-hard stop from ordinary braking bringing the ego close behind the sedan. The reference therefore remains ambiguous; it does not assume a collision occurred, was unavoidable or would be prevented by a particular braking action."
}
C01.attempt-1: rejection reason and record
{
"attempt_id": "C01.attempt-1",
"fixture_id": "C01",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C01.attempt-1.generation.request.json",
"sha256": "46f8aae33f19a8fcdc691864f7c247918a2f5d5d0ca502c65362e629c5d50cf4",
"bytes": 1223
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C01.attempt-1/review.json",
"sha256": "276cff2dff398f71aa76290fdd6ee76378aeef04d8551529f91729f59f13f981",
"bytes": 15780
},
"reason": "Rejected by the frozen media gate: the red sedan is approached continuously and its lower rear becomes hidden behind the ego hood near the end, but no unmistakable ego-to-vehicle contact appears in actual transport frames 0–95. The aligned rear body stays intact and its final position stabilizes without clear touch, deformation, deflection or coherent collision consequence. Hood occlusion does not establish impact.",
"reference_reason": "Progressive closure and visible rear lamps support braking. Available stopping distance, calibrated closing speed and deceleration are unresolved; the sequence cannot reliably distinguish ordinary braking from a necessary immediate very-hard stop. Reference is ambiguous without assuming an impact from the close endpoint."
}
C02.attempt-0: rejection reason and record
{
"attempt_id": "C02.attempt-0",
"fixture_id": "C02",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C02.attempt-0.generation.request.json",
"sha256": "eb4100e61789fd6b98bf7dbcd6b29b41df77c353c24834296b577391835af930",
"bytes": 1258
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C02.attempt-0/review.json",
"sha256": "e06e7a592274a37288625de966bc96b13c6f84ca4044daca67b3b6b93222f430",
"bytes": 16579
},
"reason": "Rejected by the frozen media gate: the white van grows larger as the apparent following gap decreases, but actual transport frames 0–95 do not show unmistakable ego-to-vehicle contact. Full-resolution frames 72, 84, 90 and 95 show an intact aligned rear; the lower van becomes hidden by the ego hood and the upper rear becomes cropped. No visible boundary contact, deformation, deflection or coherent collision consequence establishes impact.",
"reference_reason": "The closing sequence supports braking, but the visible clip does not resolve ego speed, relative closing distance, available stopping distance or necessary braking intensity. Ordinary deceleration versus a necessary immediate very-hard stop cannot be distinguished reliably. Emergency-stop reference is ambiguous without assuming impact from hood occlusion."
}
C02.attempt-1: rejection reason and record
{
"attempt_id": "C02.attempt-1",
"fixture_id": "C02",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C02.attempt-1.generation.request.json",
"sha256": "4d6e628c5ec435523792444799caee6c83eb25340a20a76a3cdf6aec3ba68618",
"bytes": 1258
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C02.attempt-1/review.json",
"sha256": "b6b4899ea2a64cf42732b58e2dd53955eaffd68552aadf9504fb9796c67bae8b",
"bytes": 15699
},
"reason": "Rejected by the frozen media gate: white van approaches continuously and fills the viewpoint, with its lower rear hidden by ego hood late in actual transport frames 0–95. No unmistakable direct contact, abrupt deflection, coherent impact deformation or collision consequence is visible. Rear panel shading does not establish an impact, and close hood occlusion alone is insufficient.",
"reference_reason": "The van's shrinking apparent gap supports braking but does not resolve available stopping distance, closing speed or required intensity. The visible sequence cannot reliably distinguish ordinary deceleration from a necessary immediate very-hard stop. Reference remains ambiguous without treating the endpoint or panel shading as a collision."
}
C03.attempt-0: rejection reason and record
{
"attempt_id": "C03.attempt-0",
"fixture_id": "C03",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C03.attempt-0.generation.request.json",
"sha256": "d2b2ae1ba3c77550ad11d3fff82d9a908668f4a6cc35788253968180f7f0bf1b",
"bytes": 1320
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C03.attempt-0/review.json",
"sha256": "a1de42ec63c2da08961440fb4df4d0340ad05b6bd526355b6f8d0cb032236b11",
"bytes": 16524
},
"reason": "Rejected by the frozen media gate: a blue pickup crosses from the left across the forward route and grows closer, but actual transport frames 0–95 show no unmistakable ego-to-vehicle contact. The body remains intact and the lower pickup disappears behind the hood. Close proximity and occlusion do not establish an impact; no clear boundary touch, deformation, deflection or coherent collision consequence is visible.",
"reference_reason": "The visible crossing conflict calls for braking. The clip does not establish calibrated closing speed, stopping distance, deceleration or whether an immediate very-hard stop rather than ordinary braking was necessary and available. This reference is ambiguous; crossing and hood occlusion are not treated as proof of a collision or of a specific preventive action."
}
C03.attempt-1: rejection reason and record
{
"attempt_id": "C03.attempt-1",
"fixture_id": "C03",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C03.attempt-1.generation.request.json",
"sha256": "5a99783a84d0c32faaee64a72c35383afda49e627e832bdd8e9a3aeb7995b415",
"bytes": 1320
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C03.attempt-1/review.json",
"sha256": "17ba4958115c7c27bfde06a410e5ce4b424939e0d4c29cc793bd5aaca1f19fa3",
"bytes": 16297
},
"reason": "The pickup is continuously approached while crossing the ego route, but its lower side and the possible contact zone become hidden behind the ego hood. A light crease or scuff on the front door is visible late; without visible vehicle boundaries meeting or coherent impact motion, that change does not establish unmistakable contact within frames 0–95. The frozen media gate therefore fails.",
"reference_reason": "The crossing pickup creates a visible route conflict. The clip does not establish the speed, remaining stopping distance or urgency sufficiently to distinguish an immediate very hard stop from ordinary braking or steering. The possible late door crease cannot supply that prospective judgment; retain an ambiguous emergency-stop reference."
}
C04.attempt-0: rejection reason and record
{
"attempt_id": "C04.attempt-0",
"fixture_id": "C04",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C04.attempt-0.generation.request.json",
"sha256": "94c2c3dbf7fd72978bd3047f01957176da48202be3ef8828376a70212bfd91da",
"bytes": 1311
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C04.attempt-0/review.json",
"sha256": "b5df5ba40121a0ae518cb6a325634b49d9117b408f8fee7a2efa3c53911aff48",
"bytes": 16590
},
"reason": "Rejected by the frozen media gate: a white van extends from a driveway on the right across the forward route and becomes close, but actual transport frames 0–95 do not show unmistakable vehicle contact. The lower van becomes hood-occluded; visible body panels remain intact and its late position stabilizes. No directly observable touch, deformation, deflection or coherent impact consequence is established.",
"reference_reason": "The van obstructs the forward route and supports braking, but visible ego slowing and van motion do not resolve stopping distance or whether immediate very-hard stopping was required rather than ordinary braking. The close hood-occluded endpoint alone does not establish collision or the counterfactual efficacy of a stop. Reference remains ambiguous."
}
C04.attempt-1: rejection reason and record
{
"attempt_id": "C04.attempt-1",
"fixture_id": "C04",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C04.attempt-1.generation.request.json",
"sha256": "bc8932eb1f1d5fec94bc205a283f0a28aee0a2ad3401945d9ce03d57e547a75b",
"bytes": 1311
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C04.attempt-1/review.json",
"sha256": "1928136ebd43db05c67de369d3d3b00af14526e3b92cba2259d9ac8eaa0f025a",
"bytes": 16100
},
"reason": "A white van extends from the right driveway into the ego route during a continuous approach. Through actual transport frame95, the visible van and ego hood do not establish vehicle boundaries meeting, deformation, deflection or abrupt coupled impact motion. Close approach is insufficient for the frozen collision media gate.",
"reference_reason": "The emerging van creates a visible route conflict, but the clip does not establish relative speed, stopping distance or timing well enough to distinguish the need for immediate very hard stopping from ordinary braking or steering. Retain an ambiguous emergency-stop reference without inferring contact."
}
C05.attempt-0: rejection reason and record
{
"attempt_id": "C05.attempt-0",
"fixture_id": "C05",
"attempt_index": 0,
"selected_for_E4": false,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C05.attempt-0.generation.request.json",
"sha256": "f67e1d3d25b668bb44c26bf34d350099f2eb4bcdf459c1876f726a086de3203c",
"bytes": 1362
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C05.attempt-0/review.json",
"sha256": "a741ecb2907b8c40740e66685fb354655b200181e1a7d472b5ff5da030fc85d7",
"bytes": 16478
},
"reason": "Rejected by the frozen media gate: the yellow box truck grows larger ahead, but an unobscured strip of roadway remains between its intact rear bumper and the ego hood through actual transport frame 95. Full-resolution 72/84/90/95 show a clear gap, with no visible touch, deformation, deflection or coherent impact consequence.",
"reference_reason": "The clip shows progressive closure to a truck without an abrupt visible obstruction or contact. Maintaining the gap may call for ordinary braking; calibrated speed, relative motion and stopping distance are not established well enough to determine necessary immediate very-hard stopping. Reference remains ambiguous instead of inferring urgency or rollback from the intended scenario."
}
C05.attempt-1: rejection reason and record
{
"attempt_id": "C05.attempt-1",
"fixture_id": "C05",
"attempt_index": 1,
"selected_for_E4": true,
"generation_request": {
"path": "2026-10-02-clef-jev-collision/C05.attempt-1.generation.request.json",
"sha256": "64b1d072d10a35f37340c4705bb19ef220d0e7460c309af52fb38d91df142840",
"bytes": 1362
},
"strict_contact_media_accepted": false,
"strict_contact_score_eligible": false,
"review": {
"path": "2026-10-02-clef-jev-collision/attempts/C05.attempt-1/review.json",
"sha256": "747a0f0d7556565bc7813b32fe9f54a26931efdf4cf80b0e3066e0cf51050d97",
"bytes": 16302
},
"reason": "The yellow box truck is approached gradually, but a substantial visible road gap remains between its rear bumper and the ego hood throughout actual transport frames0–95. No vehicle boundaries meet and no impact motion or consequence occurs. This candidate fails the frozen collision media gate.",
"reference_reason": "The visible prefix shows gradual closure to a truck that remains ahead with a clear road gap, without sudden hazard onset, abrupt relative motion or a moment of visible imminent contact. Maintaining or restoring following distance through ordinary gradual braking is supported by this sequence; an immediate very hard stop is not visibly required. This qualitative not_needed annotation is confined to the visible prefix and is not inferred solely from the absence of collision. Rejected media remains ineligible for benchmark scoring."
}
Experiment data
The 4 experiments cover 40 unique inputs and 160 completed decisions. E1 and E2 share inputs, E3 reuses the original 5 videos, and E4 adds 5 new videos.
Saved JSON includes selected positions, confidence, full-precision choice probabilities, request hashes, and experiment IDs.
| File | Contents |
|---|---|
| E1_JEV_INITIAL.json | 35 inputs, 70 responses, and 5 unexecuted video slots × 2 orders |
| E2_CLEF_COMMON.json | 50 Clef responses and IDs for the E1 comparison baseline |
| E3_EMERGENCY_ORIGINAL.json | 20 responses to the original 5 videos and scoring exclusions |
| E4_NEAR_APPROACH_AMENDMENT.json | 20 responses to the additional 5 videos and scoring exclusions |
| GENERATOR_PROVENANCE.json | Prompts, seeds, settings, and generation history for 40 inputs |
| GENERATOR_PROVENANCE.md | Generation methods and how to read the records |
| AUDIO_EXPERIMENT_MAPPING.md | Scripts, saved ASR, audio acceptance reasons, and uncertainties |
| EXPORT_MANIFEST.json | Mapping of public files to source material, checksums, and public-export organization |
| VALIDATION.json | Verification records for counts, hashes, semantic mappings, and exclusions in the original public bundle |
| SHA256SUMS.txt | SHA-256 for each file in the public bundle |
| README_PUBLIC_EXPORT.md | Scope of the public bundle and interpretation caveats |
| Original article outline | English experiment introduction centered on decisions across 4 input types |
| Original Japanese video script | Script for a video presenting the experiment results |
What this test shows
Jev-Omni offers one model for trying decisions across audio, images, text, and video. Clef scored higher on the shared image, email, and video inputs.
Reversing the choices changed Jev-Omni’s answers for I06 and T01 and Clef’s answers for C02 and C03.
These results come from a small set of generated inputs. The tasks, languages, and number of choices differ by input type. Check your own tasks with suitable data before choosing a model.
