{
  "id": "robotics-grasp-outcome-v1",
  "title": "Can a learned world predict whether a grasp lifts the object?",
  "status": "registered",
  "registered_at": "2026-10-16T03:02:05.492927+00:00",
  "variants": [
    "edge",
    "nano"
  ],
  "models": [
    "cosmos3-edge-robot-action",
    "cosmos3-nano-robot-action",
    "frozen-image"
  ],
  "seeds": [
    202610160,
    202610161,
    202610162
  ],
  "cases": [
    {
      "id": "scene-202610150-center-close",
      "group": "scene-202610150",
      "path": "scene-202610150-center-close/input.json",
      "sha256": "77d49fc6766d235a43475acf51599bf9836f77e95a0e7b45e596e6e79017ac2b"
    },
    {
      "id": "scene-202610150-left-miss",
      "group": "scene-202610150",
      "path": "scene-202610150-left-miss/input.json",
      "sha256": "bab4e1de74af9e518f2efd56018d5d3aa3acdeff1664fa0b264f527136de350f"
    },
    {
      "id": "scene-202610150-right-miss",
      "group": "scene-202610150",
      "path": "scene-202610150-right-miss/input.json",
      "sha256": "7f3768d729806397e66298e30b0bd7653a46ce872c04e3fb7acbc388623bf23d"
    },
    {
      "id": "scene-202610150-center-open",
      "group": "scene-202610150",
      "path": "scene-202610150-center-open/input.json",
      "sha256": "3cb3877eb90c3a8e0ca62f9253ded677bf3b2fe9208926ca0637dcfeb597fe0a"
    },
    {
      "id": "scene-202610151-center-close",
      "group": "scene-202610151",
      "path": "scene-202610151-center-close/input.json",
      "sha256": "cd571c7f2dec3ba7e02bfa1cbe9ee7f5e9e948227c0c52b05eb43208c464e6fc"
    },
    {
      "id": "scene-202610151-left-miss",
      "group": "scene-202610151",
      "path": "scene-202610151-left-miss/input.json",
      "sha256": "6185663093b9611544ad0a189c8a6fbf18f799de0a9a56bcd05fe469f44a7678"
    },
    {
      "id": "scene-202610151-right-miss",
      "group": "scene-202610151",
      "path": "scene-202610151-right-miss/input.json",
      "sha256": "99f0b6a6827c8dc1a888b6f7c6eeb17969ca0d853a95c95f00a9d99d582605bc"
    },
    {
      "id": "scene-202610151-center-open",
      "group": "scene-202610151",
      "path": "scene-202610151-center-open/input.json",
      "sha256": "edf55f5b6b682012849b2ede2f9088dadf76603110051bb5c04674bbf03ef584"
    },
    {
      "id": "scene-202610152-center-close",
      "group": "scene-202610152",
      "path": "scene-202610152-center-close/input.json",
      "sha256": "9d8007df651ad2162d277fb2e80091038ff6560a2d9f149a6cfc80c71e7eeb34"
    },
    {
      "id": "scene-202610152-left-miss",
      "group": "scene-202610152",
      "path": "scene-202610152-left-miss/input.json",
      "sha256": "17a4c4ae11d5ef4afc669b2f920f64a476fcb65382c2132eb74fe44ad05cf3a9"
    },
    {
      "id": "scene-202610152-right-miss",
      "group": "scene-202610152",
      "path": "scene-202610152-right-miss/input.json",
      "sha256": "25e26f11070997b38c7700f8f5cf01c69fd82ab7644d3846a17fb0070888555e"
    },
    {
      "id": "scene-202610152-center-open",
      "group": "scene-202610152",
      "path": "scene-202610152-center-open/input.json",
      "sha256": "b76e2cacbb347bb3551ff62469f7229e1a7a0af611b04c947467814b2e072e09"
    },
    {
      "id": "scene-202610153-center-close",
      "group": "scene-202610153",
      "path": "scene-202610153-center-close/input.json",
      "sha256": "6dd4f1809427e51bd0bcac8cb20b4bf935939582b64e914909fe3d114a4db65e"
    },
    {
      "id": "scene-202610153-left-miss",
      "group": "scene-202610153",
      "path": "scene-202610153-left-miss/input.json",
      "sha256": "33fb127e7d8bda5259cc6b7fa7feeb09a6b5fb750610501f6b126b742449180b"
    },
    {
      "id": "scene-202610153-right-miss",
      "group": "scene-202610153",
      "path": "scene-202610153-right-miss/input.json",
      "sha256": "499bfb6b1d3110143f751c6672c323dca1cb69ccd253bd7ba950e6f34271b91e"
    },
    {
      "id": "scene-202610153-center-open",
      "group": "scene-202610153",
      "path": "scene-202610153-center-open/input.json",
      "sha256": "0418eefa4213585c6d3bc5505cd9009a024bd9f5e9ccb0cd616682e2b4baddb2"
    },
    {
      "id": "scene-202610154-center-close",
      "group": "scene-202610154",
      "path": "scene-202610154-center-close/input.json",
      "sha256": "0aafae2c3ec3f2cbac89e27758ae3da0a4b96ff7312dcb48aecfcd34385bda62"
    },
    {
      "id": "scene-202610154-left-miss",
      "group": "scene-202610154",
      "path": "scene-202610154-left-miss/input.json",
      "sha256": "7bbf0b74b95d11b9e7bd0827aadc3b71619128b947ddb0d8125db069d3f5f460"
    },
    {
      "id": "scene-202610154-right-miss",
      "group": "scene-202610154",
      "path": "scene-202610154-right-miss/input.json",
      "sha256": "6f47efe341175adc5ffdf69d4634dbdaa0ab5ab4a44c87596f03d3f9bb84f4ab"
    },
    {
      "id": "scene-202610154-center-open",
      "group": "scene-202610154",
      "path": "scene-202610154-center-open/input.json",
      "sha256": "ae1153b4fea3ebcd5461abeaf34cc0d18b6416b87d76ba003257cdd2a32c53b1"
    },
    {
      "id": "scene-202610155-center-close",
      "group": "scene-202610155",
      "path": "scene-202610155-center-close/input.json",
      "sha256": "87fd81cb244c1562b871a5bfb39401e82cae6b9f461d0049bde17b2bedaaf899"
    },
    {
      "id": "scene-202610155-left-miss",
      "group": "scene-202610155",
      "path": "scene-202610155-left-miss/input.json",
      "sha256": "fb2a516d0854066a9ff8862ebf68306d0440555079a98933c4c79d25b8bab7e4"
    },
    {
      "id": "scene-202610155-right-miss",
      "group": "scene-202610155",
      "path": "scene-202610155-right-miss/input.json",
      "sha256": "635415707d3f883437b17004d459b26162bb682b6eb77caca13a963fc8cc33af"
    },
    {
      "id": "scene-202610155-center-open",
      "group": "scene-202610155",
      "path": "scene-202610155-center-open/input.json",
      "sha256": "440c42a574ac907cf2542c56f24f11b0a96de855fb272e2645b9c459a5e14dbc"
    },
    {
      "id": "scene-202610156-center-close",
      "group": "scene-202610156",
      "path": "scene-202610156-center-close/input.json",
      "sha256": "5ff1509863132061acd6e27a9b115f55946f2255d7b0f212356f0cbf73ba408b"
    },
    {
      "id": "scene-202610156-left-miss",
      "group": "scene-202610156",
      "path": "scene-202610156-left-miss/input.json",
      "sha256": "8ea17896768e623fc7d30b4db4a1855dc828e9a522b63514be313918e6400d75"
    },
    {
      "id": "scene-202610156-right-miss",
      "group": "scene-202610156",
      "path": "scene-202610156-right-miss/input.json",
      "sha256": "eb477c8a3081fd038fecc54a93a871c2557971e17e534e65bc9999843139366a"
    },
    {
      "id": "scene-202610156-center-open",
      "group": "scene-202610156",
      "path": "scene-202610156-center-open/input.json",
      "sha256": "9c8dedf3804d3d7c5ae84d711494c58bb471bb7c8c16e10963aee3f0975cc766"
    },
    {
      "id": "scene-202610157-center-close",
      "group": "scene-202610157",
      "path": "scene-202610157-center-close/input.json",
      "sha256": "1426c343da946d8b6c7e81e2a55b6ea3203a98a7a6b851223e98289c2906c8c2"
    },
    {
      "id": "scene-202610157-left-miss",
      "group": "scene-202610157",
      "path": "scene-202610157-left-miss/input.json",
      "sha256": "4b7e9dff452634835d8405ff661a9ae16c4eaba776bf2b53c7bc21ba0f855fff"
    },
    {
      "id": "scene-202610157-right-miss",
      "group": "scene-202610157",
      "path": "scene-202610157-right-miss/input.json",
      "sha256": "43da7eb3b0f19a77db79be96cc8a8103e19053c8ef3b846b82d5f18932363b65"
    },
    {
      "id": "scene-202610157-center-open",
      "group": "scene-202610157",
      "path": "scene-202610157-center-open/input.json",
      "sha256": "d6ffe8d1993f223a8474c8de599e00ef1fcd01f444dc515a00867792a9ad48f1"
    }
  ],
  "reference": {
    "index_sha256": "126bfbadccae0b06f2c1a9abf8ed9d124e643fad1b8a71b423a6da370f79302d",
    "environment": "widowx_carrot_on_plate",
    "simply_claim": "Simulation-only sustained carrot lift, not the original put-on-plate success predicate",
    "simpler_revision": "eddb5690749a1e34267807c0f7b5eede323918d7",
    "maniskill_revision": "ef7a4d4fdf4b69f2c2154db5b15b9ac8dfe10682",
    "policy": "Fixed 48-step approach, descend, close-or-stay-open, lift and hold; 8 scene seeds x 4 interventions.",
    "target": "Minimum object-height increase over final five frames > 0.02 m. All 17 positive references exceed 0.04 m; 15 negatives.",
    "intervention_labels": "left-miss and right-miss are historical file IDs for +/-0.04 m lateral offsets, not assumed failures."
  },
  "conditioning": "Same initial 320x256 image, task text, realized TCP trajectory and commanded binary gripper state. No later reference images, object poses, labels or outcomes supplied to the model.",
  "scope": "Predict object-contact/lift outcomes along a supplied robot trajectory; not actuator simulation, closed-loop policy ranking or sim-to-real validation.",
  "sampling": {
    "steps": 30,
    "guidance": 1,
    "flow_shift": 10,
    "dtype": "bfloat16",
    "chunks": 3,
    "transitions_per_chunk": 16,
    "frames": 49,
    "fps": 5,
    "chunk_seed": "base seed + chunk index modulo 2^32",
    "reset": "Initial reference image only at start; subsequent chunks use the last generated frame."
  },
  "metrics": [
    "outcome-agreement-percent",
    "visible-outcome-percent",
    "false-success-percent",
    "missed-success-percent",
    "generation-seconds"
  ],
  "metric_definition": "Judge S/F/U from eight fixed frames. S = sustained visible lift in final five frames; F = visible failure; U = insufficient visual evidence. Agreement/error percentages exclude U and report visible coverage. False-success denominator is reference failures; missed-success denominator is reference successes. No overall composite.",
  "evaluator": {
    "model": "gpt-6-astra",
    "script_sha256": "c5f8fc5bcf761726ee485df32d49214e58c97aa4f9de5ca307119297bc09c518",
    "score_script_sha256": "559984ecaa62aec1329a80750692f24223f2fec43e8a69557d9560075fc08b36",
    "validation_sha256": "79d5e44ea4ab7755cb6bde765fbd526f31eaf482d3c9eca5e39ee753bd79ae32",
    "validation": "31/32 reference outcomes correct, one false success; all four generated connector checks agree with visual inspection. Provider alias is not immutable.",
    "budget_usd": 20
  },
  "uncertainty": "Equal weight per scene; 2,000 scene-cluster bootstrap draws, seed 20261015. Three generation seeds within scenes are repetitions, not independent scenes. Paired differences include only jointly scored attempts.",
  "failure_policy": "One registered attempt per model/case/seed. Retain every error and partial artifact; no hidden retries or removal. Unstarted attempts after setup/deadline failure are explicit unavailable runs, never zero scores.",
  "baseline": "Frozen initial image for all 49 frames, explicitly non-generative; deterministic visual failure label, generation cost/latency inapplicable. Baseline checks whether predicting no lift beats a learned world.",
  "example_selection": "First registered seed; all cases selectable in input order without selecting by score.",
  "limitations": [
    "Eight simulated scenes, one object, one robot and a fixed scripted controller; diagnostic generalization is limited.",
    "Two configurations of the same NVIDIA model family, not broad provider coverage.",
    "All 32 references were used to validate the evaluator. One scene was used for four unscored connector clips; full generation uses new seeds. No held-out-development claim.",
    "No model is given object truth, but the realized robot trajectory is itself supplied; this is not prediction from raw policy commands.",
    "VLM labels estimate visible outcomes and may share systematic bias. Unknowns and reference calibration errors are reported.",
    "Latency is warm offline generation on the recorded rented GPU; not network serving latency, real-time control latency or the vendors optimized throughput."
  ],
  "provenance": {
    "adapter_sha256": "5c2051e4d3f3a64a8d635ab9f1249f50088766ed66974e600ede0e54f0b9545e",
    "runner_sha256": "a03162203fc1741c8979c9ae4df729e19f3683446b63f1d916d769246f2f9a1f",
    "action_parity_sha256": "e3a84fcb2bcaa84e29344ea347184d996e0f875909cc8876af1583c5eb999acb",
    "coverage_audit_sha256": "54f692ba5476812d2b2c7b3e54afe65e4d9cdf3b68ade98e61c3b8dc1c271e6e"
  },
  "candidate_scope": "Cosmos-Predict2.5 remains a candidate for a separate compatibility-verified comparison; its continuous gripper/state interface is not interchangeable with Cosmos 3 command conditioning."
}