
Remotion To Hyperframes
- 207k installs
- 39.5k repo stars
- Updated August 5, 2026
- heygen-com/hyperframes
Remotion to HyperFrames is a skill for translating React-based Remotion video compositions into HTML-based HyperFrames compositions with GSAP timelines.
About
Remotion to HyperFrames translation skill that ports React-based Remotion compositions to HTML-based HyperFrames (GSAP) compositions. Enforces linting for blockers, maps Remotion APIs to HF equivalents, and includes SSIM-validated test corpus.
- Remotion API mapping to HyperFrames equivalents
- Automated lint detection for unsupported patterns
- SSIM-validated test corpus (T1-T4 tiers)
Remotion To Hyperframes by the numbers
- 207,370 all-time installs (skills.sh)
- +10,926 installs in the week ending Aug 4, 2026 (Skillselion tracking)
- Ranked #46 of 1,335 Generative Media skills by installs in the Skillselion catalog
- Security screen: LOW risk (skills.sh audit)
- Data as of Aug 5, 2026 (Skillselion catalog sync)
npx skills add https://github.com/heygen-com/hyperframes --skill remotion-to-hyperframesAdd your badge
Show developers this skill is listed on Skillselion. Paste this into your README.
| Installs | 207k |
|---|---|
| repo stars | ★ 39.5k |
| Security audit | 2 / 3 scanners passed |
| Last updated | August 5, 2026 |
| Repository | heygen-com/hyperframes ↗ |
How do you migrate Remotion compositions to HyperFrames?
Migrate video compositions from Remotion to HyperFrames HTML-based format.
Who is it for?
Software engineers migrating Remotion video projects to HyperFrames
Skip if: New HyperFrames videos, After Effects or Framer Motion sources, reverse export to Remotion, or Remotion mentions without an explicit migrate request.
When should I use this skill?
When user explicitly asks to port, convert, migrate, or translate a Remotion composition
What you get
HyperFrames index.html with GSAP timeline, SSIM-validated render diff, lint report, and TRANSLATION_NOTES.md documenting untranslated patterns.
- HyperFrames index.html
- TRANSLATION_NOTES.md
- SSIM diff report
By the numbers
- Tiered test corpus T1–T4 contains 4 fixtures total
- T4 escape-hatch lint validates 8 of 8 cases
- Documents ~80% mechanical translation coverage for typical Remotion compositions
Files
Remotion to HyperFrames
Overview
Translate Remotion (React-based) video compositions into HyperFrames (HTML + GSAP) compositions. Most Remotion idioms have direct HyperFrames equivalents — the translation is mechanical for ~80% of typical compositions. This skill encodes the mapping and guards against the lossy 20% by refusing to translate patterns that don't fit HF's seek-driven model and recommending the runtime interop pattern from PR #214 instead.
The skill ships with a tiered test corpus (T1–T4, 4 fixtures total) that grades translations against measured SSIM thresholds. Don't translate without running the eval — a translation that "looks right" but renders 0.05 SSIM lower than the validated baseline is silently wrong.
When to use
Use this skill ONLY when the user explicitly asks to migrate from Remotion. Example trigger phrases:
- "port my Remotion project to HyperFrames"
- "convert this Remotion code to HyperFrames"
- "migrate from Remotion"
- "translate this Remotion comp"
- "rewrite this as HyperFrames HTML"
Do NOT use this skill when:
- (a) The user is authoring a new HyperFrames composition, even if they have or are A/B-testing a similar Remotion video.
- (b) The user mentions Remotion in passing without asking for migration.
- (c) The user shares Remotion code as reference material rather than asking for a translation.
- (d) The user asks for "the same video as my Remotion one" without explicitly asking to migrate the source — treat that as a fresh HyperFrames build.
When in doubt, default to authoring a native HyperFrames composition with the hyperframes skill instead.
Workflow
Step 1: Lint the source
Run `scripts/lint_source.py` over the Remotion source directory. The lint detects patterns that can't translate cleanly:
- Blockers (refuse + recommend interop):
useState,useReducer,useEffect/useLayoutEffectwith non-empty deps, asynccalculateMetadata, third-party React UI libraries (MUI, Chakra, Mantine, antd, shadcn, Radix, NextUI). - Warnings (translate after dropping the construct):
@remotion/lambdaconfig,delayRender,useCallback,useMemo, custom hooks. - Info (translate with note):
staticFile,interpolateColors.
If any blocker fires, stop. Read `references/escape-hatch.md` and surface the recommendation message. Warnings don't stop translation — drop the offending construct in step 3 and note the gap in TRANSLATION_NOTES.md. @remotion/lambda config is the canonical warning case: the skill drops the import + renderMediaOnLambda(...) calls but translates the rest of the composition.
Step 2: Plan the translation
Read `references/api-map.md` — the index of every Remotion API and its HF equivalent or per-topic reference. Identify which topic references you'll need based on what the source uses:
| Source contains | Load reference |
|---|---|
Composition, defaultProps, schema, calculateMetadata | `parameters.md` |
Sequence, Series, Loop, AbsoluteFill, Freeze | `sequencing.md` |
useCurrentFrame, interpolate, spring, Easing, interpolateColors | `timing.md` |
Audio, Video, Img, IFrame, staticFile, delayRender | `media.md` |
TransitionSeries, @remotion/transitions | `transitions.md` |
@remotion/lottie | `lottie.md` |
@remotion/google-fonts/<Family>, Font.loadFont, @font-face | `fonts.md` |
Don't load all of them — load only what the specific source needs.
Step 3: Generate the HF composition
Emit index.html with:
- Root
<div id="stage">carrying the composition'sdata-composition-id,data-start="0",data-duration(in seconds),data-fps,data-width,data-height, plus onedata-*per scalar prop. - A flat list of scene divs with
data-start/data-duration/data-track-index. - Inline
<style>for layout; CSS sets thefromstate of every animated property. - A single
<script>tag at the bottom containing one pausedgsap.timeline({paused: true}). Every RemotionuseCurrentFrame()derivation becomes a tween on this timeline at the right offset. window.__timelines["<composition-id>"] = tl;registers the timeline with HF's runtime.
Custom React subcomponents inline as repeated HTML using the prop interface as the template (see `parameters.md` for the per-instance data-* pattern).
Step 4: Validate
Run the eval harness — `references/eval.md` for the full guide. Quick path:
# Render Remotion baseline (after npm install in the fixture)
cd remotion-src && npx remotion render <CompositionId> out/baseline.mp4
# Render HF translation
cd ../hf-src && npx hyperframes render --output ../hf.mp4
# SSIM diff
../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diffThreshold: ~0.02 below p05 of the source's complexity tier (see eval.md's validated thresholds table). If the diff fails, run `scripts/frame_strip.sh` to see _which_ frames diverged, then re-read the relevant timing/sequencing/media reference.
Critical: both renders must use matching pixel format. Set Config.setVideoImageFormat("png") + Config.setColorSpace("bt709") in the Remotion source's remotion.config.ts — otherwise the diff measures encoder differences (~0.05 SSIM hit), not translation fidelity.
Step 5: Document gaps
Anything that didn't translate cleanly (volume ramps dropped, custom presentations approximated, fonts substituted) gets a TRANSLATION_NOTES.md written next to the HF output. See `references/limitations.md` for the format.
What this skill explicitly does NOT do
- Translate React state machines. Compositions that drive animation via
useState+useEffectare not deterministic frame-capture targets in HyperFrames' seek-driven model. Recommend the runtime interop pattern. - Run Remotion's render pipeline alongside HyperFrames. That's the runtime interop pattern from PR #214 — a separate solution for compositions that fail this skill's lint.
(@remotion/lambda is _not_ a blocker — Lambda config is deployment, not animation. The skill drops it as a warning and translates the rest. See `references/escape-hatch.md`.)
How to grade your own translation
Run the test corpus orchestrator:
./assets/test-corpus/run.shIt runs T1, T2, T3 (render + diff) and T4 (lint validation), prints a per-tier pass/fail table, and emits an aggregate JSON report. Use this to verify the skill is working end-to-end on a clean checkout — and as a regression check after editing any reference.
Validated baseline (as of 2026-04-27):
| Tier | Composition shape | Mean SSIM | Threshold |
|---|---|---|---|
| T1 | single-element fade-in | 0.974 | 0.95 |
| T2 | multi-scene + spring + audio + image | 0.985 | 0.95 |
| T3 | data-driven, custom subcomponents, count-up | 0.953 | 0.90 |
| T4 | escape-hatch (8 lint cases) | 8/8 pass | n/a |
run-report.json
#!/usr/bin/env bash
# run.sh — corpus orchestrator. Runs every tier and prints a pass/fail summary.
#
# Tiers 1-3: render Remotion baseline + HF translation, run SSIM diff,
# assert mean >= ssim_threshold from each fixture's expected.json.
# Tier 4: runs cases/validate.sh which lints each case and asserts against
# expected.json.
#
# Usage:
# ./run.sh run all tiers
# ./run.sh tier-1-title-card run a single tier
#
# Requirements:
# - ffmpeg, ffprobe, python3 on PATH
# - node 22 (for the HF CLI)
# - npm (for Remotion installs)
# - HF CLI built at packages/cli/dist/cli.js (run `bun run --filter @hyperframes/cli build`
# in the repo root if missing)
#
# Output:
# <fixture>/diff/summary.json per-fixture SSIM summary
# <fixture>/strip/strip.png per-fixture comparison strip (only on fail)
# ./run-report.json aggregate report
set -euo pipefail
THIS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SKILL_DIR="$(cd "$THIS_DIR/../.." && pwd)"
REPO_ROOT="$(cd "$SKILL_DIR/../.." && pwd)"
LINT="$SKILL_DIR/scripts/lint_source.py"
DIFF="$SKILL_DIR/scripts/render_diff.sh"
STRIP="$SKILL_DIR/scripts/frame_strip.sh"
HF_CLI="$REPO_ROOT/packages/cli/dist/cli.js"
REPORT="$THIS_DIR/run-report.json"
# Per-fixture results land here as one JSON file each, then the aggregator
# globs them. This is safer than building JSON via bash string concatenation
# (a fixture name containing a quote would break the previous approach).
RESULTS_DIR="$(mktemp -d)"
trap 'rm -rf "$RESULTS_DIR"' EXIT
# T4 is lint-only — no ffmpeg or HF CLI needed. Defer the render-tier
# toolchain checks until run_render_tier() actually runs, so
# `./run.sh tier-4-escape-hatch` works on a clean checkout.
require_render_tier_tools() {
if [[ ! -f "$HF_CLI" ]]; then
echo "error: HF CLI not built at $HF_CLI" >&2
echo " Run 'bun run --filter @hyperframes/cli build' in $REPO_ROOT" >&2
return 2
fi
if ! command -v ffmpeg >/dev/null 2>&1; then
echo "error: ffmpeg not on PATH" >&2
return 2
fi
return 0
}
# Write one fixture's result as a JSON file. Values are passed via argv so
# bash string interpolation can't corrupt the JSON or inject Python source.
write_result() {
local fixture_name="$1"
local status="$2"
shift 2
python3 - "$RESULTS_DIR/$fixture_name.json" "$fixture_name" "$status" "$@" <<'PY'
import json
import sys
out_path, fixture_name, status, *kvs = sys.argv[1:]
result = {"fixture": fixture_name, "status": status}
for i in range(0, len(kvs), 2):
k, v = kvs[i], kvs[i + 1]
try:
result[k] = float(v) if "." in v or v.lstrip("-").isdigit() else v
except ValueError:
result[k] = v
with open(out_path, "w") as f:
json.dump(result, f)
PY
}
# Read a top-level scalar value from a JSON file. Falls back to $3 if the
# key is missing (used to default composition_id for older fixtures).
read_json_value() {
local file="$1"
local key="$2"
local default="${3:-}"
python3 - "$file" "$key" "$default" <<'PY'
import json
import sys
path, key, default = sys.argv[1], sys.argv[2], sys.argv[3]
with open(path) as f:
data = json.load(f)
val = data.get(key, default)
print(val if val is not None else "")
PY
}
run_render_tier() {
local fixture_dir="$1"
local fixture_name
fixture_name=$(basename "$fixture_dir")
local expected="$fixture_dir/expected.json"
if ! require_render_tier_tools; then
echo " ⚠ $fixture_name: render toolchain unavailable, skipping"
write_result "$fixture_name" "skipped" reason "render toolchain unavailable"
return 0
fi
local threshold composition_id
threshold=$(read_json_value "$expected" "ssim_threshold")
composition_id=$(read_json_value "$expected" "composition_id" "Composition")
echo " ▶ $fixture_name (threshold $threshold, composition $composition_id)"
if [[ -x "$fixture_dir/setup.sh" ]]; then
"$fixture_dir/setup.sh" >/dev/null
fi
if ! python3 "$LINT" "$fixture_dir/remotion-src/src/" >/dev/null; then
echo " ✗ lint failed (blockers in Remotion source)"
write_result "$fixture_name" "fail" stage "lint"
return 0
fi
if [[ ! -d "$fixture_dir/remotion-src/node_modules" ]]; then
echo " ⏳ npm install (first run)"
(cd "$fixture_dir/remotion-src" && npm install --silent --no-progress >/dev/null 2>&1)
fi
echo " ⏳ render Remotion baseline"
if ! (cd "$fixture_dir/remotion-src" && \
npx --no-install remotion render "$composition_id" out/baseline.mp4 >/dev/null 2>&1); then
echo " ✗ Remotion render failed"
write_result "$fixture_name" "fail" stage "remotion-render"
return 0
fi
echo " ⏳ render HF translation"
if ! (cd "$fixture_dir" && \
node "$HF_CLI" render hf-src/ --output hf.mp4 --quiet >/dev/null 2>&1); then
echo " ✗ HF render failed"
write_result "$fixture_name" "fail" stage "hf-render"
return 0
fi
if R2HF_SSIM_THRESHOLD="$threshold" "$DIFF" \
"$fixture_dir/remotion-src/out/baseline.mp4" \
"$fixture_dir/hf.mp4" \
"$fixture_dir/diff" >/dev/null; then
local mean
mean=$(read_json_value "$fixture_dir/diff/summary.json" "mean")
echo " ✓ pass (mean SSIM $mean, threshold $threshold)"
write_result "$fixture_name" "pass" mean_ssim "$mean" threshold "$threshold"
else
local mean
mean=$(read_json_value "$fixture_dir/diff/summary.json" "mean")
echo " ✗ fail (mean SSIM $mean, threshold $threshold)"
"$STRIP" \
"$fixture_dir/remotion-src/out/baseline.mp4" \
"$fixture_dir/hf.mp4" \
"$fixture_dir/strip" 8 >/dev/null
write_result "$fixture_name" "fail" stage "ssim" mean_ssim "$mean" threshold "$threshold"
fi
}
run_lint_tier() {
local fixture_dir="$1"
local fixture_name
fixture_name=$(basename "$fixture_dir")
echo " ▶ $fixture_name (lint-only)"
if "$fixture_dir/validate.sh" >/dev/null 2>&1; then
echo " ✓ pass (8/8 cases)"
write_result "$fixture_name" "pass" mode "lint"
else
echo " ✗ fail (some cases mismatched expected.json)"
write_result "$fixture_name" "fail" mode "lint"
fi
}
echo "remotion-to-hyperframes corpus run"
echo "=================================="
for tier in tier-1-title-card tier-2-multi-scene tier-3-data-driven; do
if [[ -n "${1:-}" && "$1" != "$tier" ]]; then
continue
fi
if [[ -d "$THIS_DIR/$tier" ]]; then
run_render_tier "$THIS_DIR/$tier"
fi
done
if [[ -z "${1:-}" || "$1" == "tier-4-escape-hatch" ]]; then
if [[ -d "$THIS_DIR/tier-4-escape-hatch" ]]; then
run_lint_tier "$THIS_DIR/tier-4-escape-hatch"
fi
fi
# Aggregate the per-fixture JSON files into one report.
#
# Skipped fixtures are *not* a pass — they mean a tier didn't run because
# tooling or fixtures were unavailable. The orchestrator exits non-zero on
# any skip so a clean checkout that lacks the HF CLI doesn't accidentally
# report "passed 1/4" (T4 alone) and look like the corpus is healthy.
#
# Single-tier mode (`./run.sh tier-N`) only writes a result file for the
# selected tier; tiers that weren't run aren't counted as skips.
python3 - "$RESULTS_DIR" "$REPORT" <<'PY'
import json
import sys
from pathlib import Path
results_dir, out_path = Path(sys.argv[1]), Path(sys.argv[2])
results = sorted(
(json.loads(p.read_text()) for p in results_dir.glob("*.json")),
key=lambda r: r["fixture"],
)
total = len(results)
passed = sum(1 for r in results if r["status"] == "pass")
failed = sum(1 for r in results if r["status"] == "fail")
skipped = sum(1 for r in results if r["status"] == "skipped")
report = {
"total": total,
"passed": passed,
"failed": failed,
"skipped": skipped,
"results": results,
}
out_path.write_text(json.dumps(report, indent=2))
print()
print("=" * 50)
print(f" passed {passed}/{total}, failed {failed}, skipped {skipped}")
print(f" report → {out_path}")
if skipped > 0:
skipped_fixtures = [r["fixture"] for r in results if r["status"] == "skipped"]
skipped_reasons = sorted({r.get("reason", "unknown") for r in results if r["status"] == "skipped"})
print()
print(f" ⚠ {skipped} skipped: {', '.join(skipped_fixtures)}")
for reason in skipped_reasons:
print(f" reason: {reason}")
print(" Skipped fixtures count as failures for the aggregate.")
print("=" * 50)
sys.exit(0 if failed == 0 and skipped == 0 else 1)
PY
# Render output
remotion-src/out/
hf-src/out/
hf.mp4
diff/
strip/
# Remotion / HF dependencies
node_modules/
package-lock.json
{
"tier": 1,
"name": "title-card-fade",
"composition_id": "TitleCard",
"description": "Solid black background, single 'HELLO' element fades in 0-0.5s, holds 0.5-2.5s, fades out 2.5-3.0s. Tests the most basic Remotion → HyperFrames translation: a single AbsoluteFill, a single useCurrentFrame-driven interpolate, no audio, no media, no custom React components.",
"duration_seconds": 3,
"fps": 30,
"width": 1280,
"height": 720,
"ssim_threshold": 0.95,
"validation": {
"measured_mean_ssim": 0.974,
"measured_min_ssim": 0.972,
"measured_p05_ssim": 0.972,
"measured_p95_ssim": 0.983,
"measured_at": "2026-04-27",
"measured_against": "remotion@4.0 (PNG output, BT.709) vs hyperframes@0.4.15-alpha.1"
},
"translation_notes": [
"Remotion: AbsoluteFill → HF: position:absolute;inset:0 div",
"Remotion: interpolate(frame, [0,15,75,90], [0,1,1,0]) at fps=30 → HF: paused GSAP timeline with three keyframed tweens at 0s, 0.5s, 2.5s with ease:'none' (linear matches Remotion's default linear interpolation)",
"No fonts loaded; both renderers use system Helvetica/Arial fallback. The Linux fallback diverges between Remotion's bundled Chromium and HyperFrames' chrome-headless-shell — same fontWeight:800 renders perceptibly bolder in HF. This costs ~0.025 mean SSIM and is the dominant non-translation noise floor.",
"Remotion config must use setVideoImageFormat('png') + setColorSpace('bt709'); the JPEG default writes yuvj420p (full-range) which costs ~0.05 SSIM vs HF's yuv420p (limited-range)."
],
"rationale": "Threshold 0.95 sits ~0.02 below measured p05. A real translation regression (wrong easing, wrong durations) drops mean SSIM by 0.05+. Encoder/font drift between CI runs is bounded at ~0.01."
}
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>tier-1-title-card</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/3.12.5/gsap.min.js"></script>
<style>
html,
body {
margin: 0;
padding: 0;
width: 1280px;
height: 720px;
overflow: hidden;
background: #0a0a0a;
font-family: Helvetica, Arial, sans-serif;
}
.title {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
color: #ffffff;
font-size: 160px;
font-weight: 800;
letter-spacing: 0.05em;
opacity: 0;
}
</style>
</head>
<body>
<div
id="stage"
data-composition-id="tier-1-title-card"
data-start="0"
data-width="1280"
data-height="720"
data-duration="3"
data-fps="30"
>
<div id="title" class="clip" data-start="0" data-duration="3" data-track-index="0">
<div class="title">HELLO</div>
</div>
<script>
// Translation of Remotion's
// interpolate(frame, [0, 15, 75, 90], [0, 1, 1, 0]) at fps=30
// into a paused GSAP timeline keyed in seconds.
// Frame ranges → time ranges: 0/30=0, 15/30=0.5, 75/30=2.5, 90/30=3.0
const tl = gsap.timeline({ paused: true });
const target = document.querySelector("#title .title");
tl.to(target, { opacity: 1, duration: 0.5, ease: "none" }, 0);
tl.to(target, { opacity: 1, duration: 2.0, ease: "none" }, 0.5);
tl.to(target, { opacity: 0, duration: 0.5, ease: "none" }, 2.5);
window.__timelines = window.__timelines || {};
window.__timelines["tier-1-title-card"] = tl;
</script>
</div>
</body>
</html>
Tier 1 — title-card-fade
What it tests
The simplest non-trivial Remotion → HyperFrames translation. A single text element fades in over the first 0.5 s, holds for 2.0 s, and fades out over the last 0.5 s. No audio, no media, no custom components.
If a translation can't pass T1, it's broken on table-stakes basics: AbsoluteFill, useCurrentFrame, interpolate with multi-segment input, and the timing offset between Remotion's frame-based driver and HF's paused-GSAP driver.
Translation walk-through
| Remotion | HyperFrames |
|---|---|
<AbsoluteFill style={{ backgroundColor: "#0a0a0a" }}> | <body style="background: #0a0a0a"> + a positioned root div |
useCurrentFrame() | dropped — HF seeks the timeline |
interpolate(frame, [0, 15, 75, 90], [0, 1, 1, 0]) at fps=30 | gsap.timeline({ paused: true }) with three .to() calls at offsets 0s/0.5s/2.5s, each ease: "none" |
<div style={{ opacity }}>HELLO</div> | static markup; opacity is animated by the timeline |
The Remotion→HF time conversion is time = frame / fps. So [0, 15, 75, 90] at 30 fps becomes [0, 0.5, 2.5, 3.0] seconds.
How to render and evaluate
# Render Remotion baseline
cd remotion-src && npm install && npm run render
# Renders to remotion-src/out/baseline.mp4
# Render HyperFrames translation
cd ../hf-src && npx hyperframes render --output ../hf.mp4
# Compare with the eval harness (from skill scripts/)
../../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diffexpected.json documents the SSIM threshold (0.95) for this fixture; the calibrated mean against Remotion @ 4.0 with PNG/BT.709 output is 0.974.
{
"name": "tier-1-title-card-remotion",
"version": "0.0.0",
"private": true,
"scripts": {
"render": "remotion render TitleCard out/baseline.mp4"
},
"dependencies": {
"@remotion/cli": "^4.0.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"remotion": "^4.0.0"
}
}
import { Config } from "@remotion/cli/config";
// Match HyperFrames' default render so SSIM diffs measure translation
// fidelity, not encoder differences.
//
// setVideoImageFormat("png") avoids the JPEG limited-range/full-range
// colorspace flag (yuvj420p vs yuv420p) that otherwise costs ~0.05 SSIM.
//
// setColorSpace("bt709") matches HF's BT.709 SDR output.
Config.setVideoImageFormat("png");
Config.setColorSpace("bt709");
Config.setOverwriteOutput(true);
Config.setConcurrency(1);
import { registerRoot } from "remotion";
import { RemotionRoot } from "./Root";
registerRoot(RemotionRoot);
import { Composition } from "remotion";
import { TitleCard } from "./TitleCard";
export const RemotionRoot = () => (
<Composition
id="TitleCard"
component={TitleCard}
durationInFrames={90}
fps={30}
width={1280}
height={720}
/>
);
import { AbsoluteFill, interpolate, useCurrentFrame } from "remotion";
export const TitleCard = () => {
const frame = useCurrentFrame();
// Fade in 0-15, hold 15-75, fade out 75-90.
const opacity = interpolate(frame, [0, 15, 75, 90], [0, 1, 1, 0], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
return (
<AbsoluteFill
style={{
backgroundColor: "#0a0a0a",
justifyContent: "center",
alignItems: "center",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
<div
style={{
fontSize: 160,
fontWeight: 800,
color: "#ffffff",
opacity,
letterSpacing: "0.05em",
}}
>
HELLO
</div>
</AbsoluteFill>
);
};
{
"compilerOptions": {
"target": "ES2018",
"module": "ESNext",
"jsx": "react-jsx",
"strict": true,
"esModuleInterop": true,
"moduleResolution": "node",
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"isolatedModules": true
},
"include": ["src"]
}
# Generated by setup.sh
remotion-src/public/
hf-src/assets/
# Remotion / HF dependencies
remotion-src/node_modules/
remotion-src/package-lock.json
# Render output
remotion-src/out/
hf-src/out/
hf.mp4
diff/
strip/
{
"tier": 2,
"name": "title-image-outro",
"composition_id": "MultiScene",
"description": "Three-scene composition exercising Sequence, spring, interpolate, Audio, Img, and staticFile. Title scene uses Remotion's spring (translated to GSAP back.out as an approximation). Image scene scales an Img (from staticFile) with linear interpolate. Outro scene fades text in linearly. A silent WAV plays throughout at volume 0.5.",
"duration_seconds": 6,
"fps": 30,
"width": 1280,
"height": 720,
"ssim_threshold": 0.95,
"validation": {
"measured_mean_ssim": 0.985,
"measured_min_ssim": 0.963,
"measured_p05_ssim": 0.966,
"measured_p95_ssim": 0.999,
"measured_at": "2026-04-27",
"measured_against": "remotion@4.0 (PNG output, BT.709) vs hyperframes@0.4.15-alpha.1",
"notes": "Spring → back.out(1.4) translation came out cleaner than expected; mean 0.985 leaves substantial headroom over 0.95."
},
"translation_notes": [
"spring({damping:12, stiffness:100, mass:1}) → back.out(1.4) over 0.7s. Spring overshoot+settle and back.out's overshoot+settle have similar shape; budget ~0.02 SSIM for the late-tail curvature mismatch (validated lower than predicted in spec; original notes overestimated drift).",
"<Sequence from durationInFrames> → wrapping div with data-start/data-duration in seconds and explicit gsap.set(opacity, 0/1) at scene boundaries to crossfade in/out cleanly",
"<Audio src volume> → <audio data-start data-duration data-volume>",
"<Img src={staticFile('x')}> → <img src='assets/x'>; setup.sh copies the asset into both fixture trees",
"interpolate with default linear easing → ease:'none' in GSAP",
"Fonts again rely on system Helvetica/Arial; ~0.015 SSIM cost from AA differences"
],
"rationale": "Threshold 0.95 sits ~0.015 below measured p05 (0.966). T2 actually validated cleaner than T1 because the lower title fontWeight (140px vs T1's 160px) shows less of the system-font fallback divergence."
}
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>tier-2-multi-scene</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/3.12.5/gsap.min.js"></script>
<style>
html,
body {
margin: 0;
padding: 0;
width: 1280px;
height: 720px;
overflow: hidden;
background: #0a0a0a;
font-family: Helvetica, Arial, sans-serif;
}
.scene {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
opacity: 0;
}
.title {
font-size: 140px;
font-weight: 800;
color: #ffffff;
letter-spacing: 0.05em;
transform: scale(0);
}
.square {
width: 200px;
height: 200px;
opacity: 0;
transform: scale(0.8);
}
.outro {
font-size: 100px;
font-weight: 600;
color: #ffffff;
}
</style>
</head>
<body>
<div
id="stage"
data-composition-id="tier-2-multi-scene"
data-start="0"
data-width="1280"
data-height="720"
data-duration="6"
data-fps="30"
>
<div id="scene-1" class="scene clip" data-start="0" data-duration="2" data-track-index="0">
<div class="title">Welcome</div>
</div>
<div id="scene-2" class="scene clip" data-start="2" data-duration="2" data-track-index="0">
<img class="square" src="assets/square.png" alt="" />
</div>
<div id="scene-3" class="scene clip" data-start="4" data-duration="2" data-track-index="0">
<div class="outro">Goodbye</div>
</div>
<audio
id="bg-music"
data-start="0"
data-duration="6"
data-track-index="1"
data-volume="0.5"
src="assets/music.wav"
></audio>
<script>
// Translation of Remotion's 3-Sequence MultiScene at fps=30, total 6s.
//
// Scene 1 (0-2s): TitleScene — spring scale on "Welcome".
// Remotion: spring({frame, fps, config: {damping:12, stiffness:100, mass:1}})
// GSAP equivalent: back.out(1.4) over ~0.7s. Spring → ease translation
// is approximate; expect minor differences in the tail of the curve.
//
// Scene 2 (2-4s): ImageScene — opacity 0→1 over 0-0.5s + scale 0.8→1.0 over 0-2.0s.
// Both Remotion interpolates use linear (default), match with ease:"none".
//
// Scene 3 (4-6s): OutroScene — opacity 0→1 over 4-5s, linear.
const tl = gsap.timeline({ paused: true });
// Scene 1 setup: opacity flick on so scene becomes visible at start
const scene1 = document.querySelector("#scene-1");
const title = scene1.querySelector(".title");
tl.set(scene1, { opacity: 1 }, 0);
tl.to(title, { scale: 1, duration: 0.7, ease: "back.out(1.4)" }, 0);
tl.set(scene1, { opacity: 0 }, 2);
// Scene 2
const scene2 = document.querySelector("#scene-2");
const square = scene2.querySelector(".square");
tl.set(scene2, { opacity: 1 }, 2);
tl.to(square, { opacity: 1, duration: 0.5, ease: "none" }, 2);
tl.to(square, { scale: 1.0, duration: 2.0, ease: "none" }, 2);
tl.set(scene2, { opacity: 0 }, 4);
// Scene 3
const scene3 = document.querySelector("#scene-3");
const outro = scene3.querySelector(".outro");
tl.set(scene3, { opacity: 1 }, 4);
tl.fromTo(outro, { opacity: 0 }, { opacity: 1, duration: 1.0, ease: "none" }, 4);
window.__timelines = window.__timelines || {};
window.__timelines["tier-2-multi-scene"] = tl;
</script>
</div>
</body>
</html>
Tier 2 — title-image-outro
What it tests
Three-scene composition. Each scene exercises a different Remotion idiom:
1. Scene 1 (0–2 s) — TitleScene with spring({damping:12, stiffness:100, mass:1}) driving a transform: scale() on text. Tests the lossy spring → GSAP ease translation. 2. Scene 2 (2–4 s) — ImageScene that fades in a staticFile-loaded image and linearly scales it from 0.8 → 1.0. Tests asset paths + linear interpolate. 3. Scene 3 (4–6 s) — OutroScene with a 1-s linear fade-in. Sanity check after the harder scenes.
A silent 6-second WAV plays throughout at volume={0.5}. Tests <Audio> translation.
If a translation passes T2, the skill correctly handles <Sequence> boundaries, <Audio> / <Img> / staticFile, and the Remotion spring → GSAP ease heuristic.
Translation walk-through
| Remotion | HyperFrames |
|---|---|
<Sequence from={0} durationInFrames={60}> | <div data-start="0" data-duration="2" data-track-index="0"> |
spring({frame, fps, config: {damping:12, stiffness:100, mass:1}}) | gsap.to(target, { scale: 1, duration: 0.7, ease: "back.out(1.4)" }) |
<Audio src={staticFile("music.wav")} volume={0.5} /> | <audio src="assets/music.wav" data-start="0" data-duration="6" data-volume="0.5" data-track-index="1"> |
<Img src={staticFile("square.png")} /> | <img src="assets/square.png"> (with setup.sh copying into both trees) |
interpolate(frame, [0, 15], [0, 1]) at 30 fps | gsap.to(target, { opacity: 1, duration: 0.5, ease: "none" }) |
The scene crossfading is a HyperFrames idiom, not a Remotion one: at scene boundaries we gsap.set(scene, { opacity: 0 }) so the previous scene disappears at the right time. Remotion does this implicitly by virtue of <Sequence>'s durationInFrames.
How to render and evaluate
# 1. Generate the binary assets (PNG + WAV) via ffmpeg
./setup.sh
# 2. Render Remotion baseline
cd remotion-src && npm install && npm run render
# 3. Render HyperFrames translation
cd ../hf-src && npx hyperframes render --output ../hf.mp4
# 4. Compare
../../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diffWhy threshold 0.95?
Same threshold as T1 (expected.json codifies it for the orchestrator). Spring → back.out(1.4) came in cleaner than predicted during calibration — the validated mean is 0.985 against the 0.95 gate. If the translation breaks anything else (spring overshoot wrong, stagger off, asset path drift), mean SSIM will fall well below 0.95 — that's the failure signal.
{
"name": "tier-2-multi-scene-remotion",
"version": "0.0.0",
"private": true,
"scripts": {
"render": "remotion render MultiScene out/baseline.mp4"
},
"dependencies": {
"@remotion/cli": "^4.0.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"remotion": "^4.0.0"
}
}
import { Config } from "@remotion/cli/config";
// Match HyperFrames' default render so SSIM diffs measure translation
// fidelity, not encoder differences.
//
// setVideoImageFormat("png") avoids the JPEG limited-range/full-range
// colorspace flag (yuvj420p vs yuv420p) that otherwise costs ~0.05 SSIM.
//
// setColorSpace("bt709") matches HF's BT.709 SDR output.
Config.setVideoImageFormat("png");
Config.setColorSpace("bt709");
Config.setOverwriteOutput(true);
Config.setConcurrency(1);
import { registerRoot } from "remotion";
import { RemotionRoot } from "./Root";
registerRoot(RemotionRoot);
import {
AbsoluteFill,
Audio,
Img,
Sequence,
interpolate,
spring,
staticFile,
useCurrentFrame,
useVideoConfig,
} from "remotion";
const TitleScene = () => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const scale = spring({ frame, fps, config: { damping: 12, stiffness: 100, mass: 1 } });
return (
<AbsoluteFill
style={{
backgroundColor: "#0a0a0a",
justifyContent: "center",
alignItems: "center",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
<div
style={{
fontSize: 140,
fontWeight: 800,
color: "#ffffff",
transform: `scale(${scale})`,
letterSpacing: "0.05em",
}}
>
Welcome
</div>
</AbsoluteFill>
);
};
const ImageScene = () => {
const frame = useCurrentFrame();
const opacity = interpolate(frame, [0, 15], [0, 1], { extrapolateRight: "clamp" });
const scale = interpolate(frame, [0, 60], [0.8, 1.0], { extrapolateRight: "clamp" });
return (
<AbsoluteFill
style={{
backgroundColor: "#0a0a0a",
justifyContent: "center",
alignItems: "center",
}}
>
<Img
src={staticFile("square.png")}
style={{
width: 200,
height: 200,
opacity,
transform: `scale(${scale})`,
}}
/>
</AbsoluteFill>
);
};
const OutroScene = () => {
const frame = useCurrentFrame();
const opacity = interpolate(frame, [0, 30], [0, 1], { extrapolateRight: "clamp" });
return (
<AbsoluteFill
style={{
backgroundColor: "#0a0a0a",
justifyContent: "center",
alignItems: "center",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
<div
style={{
fontSize: 100,
fontWeight: 600,
color: "#ffffff",
opacity,
}}
>
Goodbye
</div>
</AbsoluteFill>
);
};
export const MultiScene = () => (
<AbsoluteFill>
<Sequence from={0} durationInFrames={60}>
<TitleScene />
</Sequence>
<Sequence from={60} durationInFrames={60}>
<ImageScene />
</Sequence>
<Sequence from={120} durationInFrames={60}>
<OutroScene />
</Sequence>
<Audio src={staticFile("music.wav")} volume={0.5} />
</AbsoluteFill>
);
import { Composition } from "remotion";
import { MultiScene } from "./MultiScene";
export const RemotionRoot = () => (
<Composition
id="MultiScene"
component={MultiScene}
durationInFrames={180}
fps={30}
width={1280}
height={720}
/>
);
{
"compilerOptions": {
"target": "ES2018",
"module": "ESNext",
"jsx": "react-jsx",
"strict": true,
"esModuleInterop": true,
"moduleResolution": "node",
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"isolatedModules": true
},
"include": ["src"]
}
#!/usr/bin/env bash
# setup.sh — generate the binary assets this fixture needs.
#
# Both Remotion and HyperFrames variants need a 200x200 blue PNG and a
# 6-second silent WAV. Generating them via ffmpeg keeps binaries out of
# the repo while still letting the fixture render reproducibly.
#
# Run from the fixture root: ./setup.sh
set -euo pipefail
THIS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
if ! command -v ffmpeg >/dev/null 2>&1; then
echo "error: ffmpeg not on PATH" >&2
exit 2
fi
mkdir -p "$THIS_DIR/remotion-src/public" "$THIS_DIR/hf-src/assets"
# 200x200 solid blue PNG, ~200 bytes.
ffmpeg -y -hide_banner -loglevel error \
-f lavfi -i "color=color=#3066be:size=200x200" -frames:v 1 \
"$THIS_DIR/remotion-src/public/square.png"
cp "$THIS_DIR/remotion-src/public/square.png" "$THIS_DIR/hf-src/assets/square.png"
# 6-second silent WAV at 8 kHz mono. ~96 KB if checked in, but it is generated.
ffmpeg -y -hide_banner -loglevel error \
-f lavfi -i "anullsrc=cl=mono:r=8000" -t 6 -acodec pcm_s16le \
"$THIS_DIR/remotion-src/public/music.wav"
cp "$THIS_DIR/remotion-src/public/music.wav" "$THIS_DIR/hf-src/assets/music.wav"
echo "generated:"
ls -la "$THIS_DIR/remotion-src/public/" "$THIS_DIR/hf-src/assets/"
# Render output
remotion-src/out/
hf-src/out/
hf.mp4
diff/
strip/
# Remotion / HF dependencies
node_modules/
package-lock.json
{
"tier": 3,
"name": "stargazed-data-driven",
"composition_id": "Stargazed",
"description": "Data-driven 10-second composition with three scenes, custom React subcomponents reused across scenes, a Zod schema with defaultProps, and a count-up number animation. Translates the realistic shape of a production Remotion composition into HF — without using the runtime adapter from PR #214.",
"duration_seconds": 10,
"fps": 30,
"width": 1280,
"height": 720,
"ssim_threshold": 0.9,
"validation": {
"measured_mean_ssim": 0.953,
"measured_min_ssim": 0.927,
"measured_p05_ssim": 0.938,
"measured_p95_ssim": 0.977,
"measured_at": "2026-04-27",
"measured_against": "remotion@4.0 (PNG output, BT.709) vs hyperframes@0.4.15-alpha.1",
"notes": "Count-up timing in StatsScene shows a few-frame offset between Remotion's manual 1-(1-t)^3 and GSAP's power3.out — both formulas are identical, so the offset comes from sub-frame timing of when the seek + onUpdate fire. Final values converge correctly. Visible only as transient digit mismatches mid-animation; no SSIM impact above the noise floor."
},
"remotion_apis_exercised": [
"Composition with z.object schema and typed defaultProps",
"Sequence (3 nested with computed offsets)",
"AbsoluteFill",
"useCurrentFrame, useVideoConfig",
"interpolate (single-segment, multi-segment, with extrapolation)",
"spring (two configs: damping:12 and damping:14)",
"Custom React subcomponents reused with different props (StatCard ×3)",
"Custom React utility component (AnimatedNumber for count-up)",
"Custom React utility component (UnderlinedText)",
"Per-instance delay via prop (delayInFrames)"
],
"translation_notes": [
"Zod schema + defaultProps → data-* attributes on the root #stage div. The skill emits one data attribute per scalar prop; nested arrays (stats[]) get materialized as repeated HTML markup with per-instance data attributes (data-stat-index, data-stat-value, --card-color).",
"Custom React subcomponents inline as repeated HTML divs. The component prop interface becomes the repeated markup template. This is lossy for components with internal state — fine here because StatCard, AnimatedNumber, UnderlinedText all derive from props alone.",
"AnimatedNumber's frame-driven count-up → a GSAP tween on a { v: 0 } counter object with onUpdate rewriting textContent. GSAP's power3.out is cubic easeOut, matching the Remotion 1-(1-t)^3 manual ease.",
"Two different springs in this composition: damping:12 → back.out(1.4) (snappy), damping:14 → back.out(1.2) (calmer). The 1.4 vs 1.2 overshoot ratio approximates the damping difference.",
"Per-instance stagger via delayInFrames prop translates to a GSAP timeline offset of (i * 0.4)s.",
"Threshold 0.90 reflects: spring → back.out approximation (×2 different configs), the count-up easing curve match (very close but not identical due to sub-frame seek timing), font/AA differences on body text. SSIM well below 0.90 indicates a structural mismatch, not approximation drift."
],
"rationale": "Threshold 0.90 sits ~0.04 below measured p05 (0.938). The wider gap vs T1/T2 reflects T3's bigger approximation budget (2 spring instances + count-up timing + font fallback on multiple text sizes). Mean SSIM below 0.90 = structural mismatch (wrong durations, wrong stagger, missing prop wiring), not approximation drift."
}
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<title>tier-3-data-driven</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/gsap/3.12.5/gsap.min.js"></script>
<style>
html,
body {
margin: 0;
padding: 0;
width: 1280px;
height: 720px;
overflow: hidden;
background: #0a0a0a;
font-family: Helvetica, Arial, sans-serif;
color: #ffffff;
}
.scene {
position: absolute;
inset: 0;
display: flex;
align-items: center;
justify-content: center;
opacity: 0;
}
/* Scene 1 — Title */
.scene-1 {
flex-direction: column;
}
.scene-1 .title {
font-size: 160px;
font-weight: 900;
letter-spacing: 0.05em;
transform: scale(0);
}
.scene-1 .subtitle {
font-size: 36px;
font-weight: 400;
color: #9ca3af;
margin-top: 24px;
opacity: 0;
}
/* Scene 2 — Stats */
.scene-2 {
gap: 48px;
}
.stat-card {
width: 280px;
height: 220px;
background: #1a1a1a;
border-radius: 16px;
border: 2px solid var(--card-color);
display: flex;
flex-direction: column;
justify-content: center;
align-items: center;
opacity: 0;
transform: scale(0);
}
.stat-card .number {
font-size: 72px;
font-weight: 800;
color: var(--card-color);
line-height: 1;
}
.stat-card .label {
font-size: 24px;
font-weight: 500;
color: #9ca3af;
margin-top: 16px;
text-transform: uppercase;
letter-spacing: 0.1em;
}
/* Scene 3 — Outro */
.scene-3 .outro-wrap {
position: relative;
display: inline-block;
opacity: 0;
}
.scene-3 .outro-text {
font-size: 80px;
font-weight: 600;
}
.scene-3 .outro-underline {
position: absolute;
left: 0;
bottom: -8px;
width: 100%;
height: 6px;
background: #fbbf24;
border-radius: 3px;
transform: scaleX(0);
transform-origin: left center;
}
</style>
</head>
<body>
<div
id="stage"
data-composition-id="tier-3-data-driven"
data-start="0"
data-width="1280"
data-height="720"
data-duration="10"
data-fps="30"
data-title="STARGAZED"
data-subtitle="by HeyGen"
data-outro="thanks for watching"
>
<!-- Scene 1: Title -->
<div
id="scene-1"
class="scene scene-1 clip"
data-start="0"
data-duration="3"
data-track-index="0"
>
<div class="title">STARGAZED</div>
<div class="subtitle">by HeyGen</div>
</div>
<!-- Scene 2: Stats — three StatCards as repeated markup with per-instance data attrs -->
<div
id="scene-2"
class="scene scene-2 clip"
data-start="3"
data-duration="4"
data-track-index="0"
>
<div
class="stat-card"
data-stat-index="0"
data-stat-value="1247"
style="--card-color: #fbbf24"
>
<div class="number">0</div>
<div class="label">Stars</div>
</div>
<div
class="stat-card"
data-stat-index="1"
data-stat-value="312"
style="--card-color: #60a5fa"
>
<div class="number">0</div>
<div class="label">Forks</div>
</div>
<div
class="stat-card"
data-stat-index="2"
data-stat-value="48"
style="--card-color: #f87171"
>
<div class="number">0</div>
<div class="label">Issues</div>
</div>
</div>
<!-- Scene 3: Outro -->
<div
id="scene-3"
class="scene scene-3 clip"
data-start="7"
data-duration="3"
data-track-index="0"
>
<div class="outro-wrap">
<div class="outro-text">thanks for watching</div>
<div class="outro-underline"></div>
</div>
</div>
<script>
// ────────────────────────────────────────────────────────────────────
// Translation of Stargazed.tsx
//
// Remotion structure:
// <Composition schema={zod} defaultProps={...} fps=30 dur=300>
// <Sequence 0..90> <TitleScene title subtitle />
// <Sequence 90..210> <StatsScene stats />
// <Sequence 210..300><OutroScene text />
//
// HF structure:
// - root #stage carries data-* with the same props (title, subtitle, outro)
// - 3 scene divs with data-start/data-duration in seconds (frames/fps)
// - Custom React subcomponents inline as repeated HTML; their per-instance
// props become data-* attributes on the markup
// - Animation lives in a single paused GSAP timeline keyed by composition seconds
//
// Frame → time:
// Sequence(0,90) → 0s..3s (title)
// Sequence(90,210) → 3s..7s (stats; 4s window)
// Sequence(210,300) → 7s..10s (outro)
// Stagger(i*12) → i*0.4s (12 frames at 30fps)
// ────────────────────────────────────────────────────────────────────
const tl = gsap.timeline({ paused: true });
// ─── Scene 1: Title ─────────────────────────────────────────────────
const scene1 = document.querySelector("#scene-1");
const title = scene1.querySelector(".title");
const subtitle = scene1.querySelector(".subtitle");
tl.set(scene1, { opacity: 1 }, 0);
// spring({damping:12, stiffness:100, mass:1}) → back.out(1.4) ~0.7s
tl.to(title, { scale: 1, duration: 0.7, ease: "back.out(1.4)" }, 0);
// interpolate(frame, [20,40], [0,1]) at fps=30 → 0.667s..1.333s linear
tl.fromTo(subtitle, { opacity: 0 }, { opacity: 1, duration: 0.667, ease: "none" }, 0.667);
tl.set(scene1, { opacity: 0 }, 3);
// ─── Scene 2: Stats ─────────────────────────────────────────────────
const scene2 = document.querySelector("#scene-2");
const cards = scene2.querySelectorAll(".stat-card");
tl.set(scene2, { opacity: 1 }, 3);
cards.forEach((card, i) => {
const stagger = i * 0.4; // i * 12 frames at 30 fps
const start = 3 + stagger;
const value = Number(card.dataset.statValue);
const numberEl = card.querySelector(".number");
// StatCard entrance:
// spring({damping:14, stiffness:90, mass:1}) → back.out(1.2) ~0.7s
// interpolate(local, [0,12], [0,1]) → 0s..0.4s linear opacity
tl.to(card, { scale: 1, duration: 0.7, ease: "back.out(1.2)" }, start);
tl.to(card, { opacity: 1, duration: 0.4, ease: "none" }, start);
// AnimatedNumber: count from 0 → value with easeOutCubic over 45 frames (1.5s).
// GSAP equivalent: a tween on a counter object with onUpdate rewriting textContent.
// power3.out is GSAP's name for cubic easeOut.
const counter = { v: 0 };
tl.to(
counter,
{
v: value,
duration: 1.5,
ease: "power3.out",
onUpdate: () => {
numberEl.textContent = Math.round(counter.v).toLocaleString();
},
},
start,
);
});
tl.set(scene2, { opacity: 0 }, 7);
// ─── Scene 3: Outro ─────────────────────────────────────────────────
const scene3 = document.querySelector("#scene-3");
const outroWrap = scene3.querySelector(".outro-wrap");
const underline = scene3.querySelector(".outro-underline");
tl.set(scene3, { opacity: 1 }, 7);
// interpolate(frame, [0,12], [0,1]) → 0s..0.4s linear opacity
tl.to(outroWrap, { opacity: 1, duration: 0.4, ease: "none" }, 7);
// interpolate(frame, [10,40], [0,1]) for underline → starts at 7+0.333s, dur 1s
tl.to(underline, { scaleX: 1, duration: 1.0, ease: "none" }, 7.333);
window.__timelines = window.__timelines || {};
window.__timelines["tier-3-data-driven"] = tl;
</script>
</div>
</body>
</html>
Tier 3 — stargazed-data-driven
What it tests
A purpose-built data-driven fixture that exercises the realistic shape of a production Remotion composition without using the runtime adapter from PR #214. If a translation passes T3, the skill correctly handles:
- A
<Composition>with az.objectschema and typeddefaultProps - Custom React subcomponents reused with different props across scenes
- A nested data structure (
stats[]) materialized as repeated HTML with
per-instance attributes
- A frame-driven count-up animation (
AnimatedNumber→ GSAPonUpdate) - Two different
springconfigs translated to two differentback.outovershoots - Per-instance delays via component props (
delayInFrames→ GSAP timeline offsets)
Composition shape
Stargazed (10 s @ 30 fps, 1280×720)
├── Sequence 0–3 s TitleScene
│ ├── title ← spring scale
│ └── subtitle ← linear fade
├── Sequence 3–7 s StatsScene
│ ├── StatCard "Stars" 1247 #fbbf24 (delay 0 frames)
│ ├── StatCard "Forks" 312 #60a5fa (delay 12 frames)
│ └── StatCard "Issues" 48 #f87171 (delay 24 frames)
└── Sequence 7–10 s OutroScene
└── UnderlinedText "thanks for watching" ← scale-in underlineEach StatCard is a custom subcomponent that internally uses AnimatedNumber to count from 0 to the target. AnimatedNumber itself derives the displayed value from useCurrentFrame() + a manual 1 - (1 - t)^3 ease.
The lossy parts (and why threshold = 0.90)
1. `spring → back.out(N)`: two different spring configs in this composition.
{ damping: 12, stiffness: 100, mass: 1 }(title) →back.out(1.4){ damping: 14, stiffness: 90, mass: 1 }(stat card) →back.out(1.2)
Overshoot ratio (1.4 vs 1.2) approximates the damping difference. The late-tail curve of GSAP's back ease and Remotion's spring don't match exactly — costs ~0.03 mean SSIM per spring instance.
2. Count-up easing: AnimatedNumber uses 1 - (1 - t)^3 (cubic ease-out) manually computed in the component. GSAP's power3.out is the same curve shape — should match closely. The displayed integer is rounded each frame in both renderers; minor mismatches occur when the rounded value flips between two numbers on a sub-frame timing difference.
3. Font rendering: same caveat as T1/T2. System Helvetica/Arial fallback produces minor anti-aliasing differences between renderers. Affects the stat card numbers (large weight 800) most.
A mean SSIM below 0.90 in T3 indicates a _structural_ mismatch (wrong scene durations, wrong stagger timing, missing prop wiring), not approximation drift. That's the failure signal we care about. The calibrated mean against Remotion @ 4.0 with PNG/BT.709 output is 0.953.
Translation walk-through (skill cheat sheet)
| Remotion | HyperFrames |
|---|---|
<Composition schema={z.object({...})} defaultProps={...} /> | data-\* attributes on root #stage div |
nested array prop (stats[]) | repeated HTML markup with per-instance data-* attrs |
| custom React subcomponent | inline repeated HTML using the component's prop interface as the template |
<AnimatedNumber from={0} to={value} dur={45} /> (cubic ease-out count-up) | tween on { v: 0 } object with onUpdate rewriting textContent, ease power3.out |
spring({damping:12, stiffness:100}) | back.out(1.4) over ~0.7 s |
spring({damping:14, stiffness:90}) | back.out(1.2) over ~0.7 s |
delayInFrames={i * 12} (per-instance) | GSAP timeline offset (i * 0.4) s |
useVideoConfig() to get fps | dropped — composition fps is in data-fps on #stage |
How to render and evaluate
# Render Remotion baseline (no setup.sh — no binary assets in this fixture)
cd remotion-src && npm install && npm run render
# Render HyperFrames translation
cd ../hf-src && npx hyperframes render --output ../hf.mp4
# Compare
../../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diff{
"name": "tier-3-data-driven-remotion",
"version": "0.0.0",
"private": true,
"scripts": {
"render": "remotion render Stargazed out/baseline.mp4"
},
"dependencies": {
"@remotion/cli": "^4.0.0",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"remotion": "^4.0.0",
"zod": "^3.22.0"
}
}
import { Config } from "@remotion/cli/config";
// Match HyperFrames' default render so SSIM diffs measure translation
// fidelity, not encoder differences.
//
// setVideoImageFormat("png") avoids the JPEG limited-range/full-range
// colorspace flag (yuvj420p vs yuv420p) that otherwise costs ~0.05 SSIM.
//
// setColorSpace("bt709") matches HF's BT.709 SDR output.
Config.setVideoImageFormat("png");
Config.setColorSpace("bt709");
Config.setOverwriteOutput(true);
Config.setConcurrency(1);
import { interpolate, useCurrentFrame } from "remotion";
interface Props {
from: number;
to: number;
durationInFrames: number;
}
/**
* Counts from `from` to `to` over `durationInFrames` with easeOut.
* Driven entirely by useCurrentFrame — deterministic.
*/
export const AnimatedNumber: React.FC<Props> = ({ from, to, durationInFrames }) => {
const frame = useCurrentFrame();
const t = interpolate(frame, [0, durationInFrames], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
// Ease-out cubic — fast start, slow finish, matches the ramp on data dashboards.
const eased = 1 - (1 - t) ** 3;
const value = Math.round(from + (to - from) * eased);
return <>{value.toLocaleString()}</>;
};
import { interpolate, spring, useCurrentFrame, useVideoConfig } from "remotion";
import { AnimatedNumber } from "./AnimatedNumber";
interface Props {
label: string;
value: number;
color: string;
delayInFrames: number;
}
export const StatCard: React.FC<Props> = ({ label, value, color, delayInFrames }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const local = frame - delayInFrames;
const scale = spring({
frame: local,
fps,
config: { damping: 14, stiffness: 90, mass: 1 },
});
const opacity = interpolate(local, [0, 12], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
return (
<div
style={{
width: 280,
height: 220,
background: "#1a1a1a",
borderRadius: 16,
border: `2px solid ${color}`,
display: "flex",
flexDirection: "column",
justifyContent: "center",
alignItems: "center",
opacity,
transform: `scale(${scale})`,
}}
>
<div style={{ fontSize: 72, fontWeight: 800, color, lineHeight: 1 }}>
{local >= 0 ? <AnimatedNumber from={0} to={value} durationInFrames={45} /> : 0}
</div>
<div
style={{
fontSize: 24,
fontWeight: 500,
color: "#9ca3af",
marginTop: 16,
textTransform: "uppercase",
letterSpacing: "0.1em",
}}
>
{label}
</div>
</div>
);
};
import { interpolate, useCurrentFrame } from "remotion";
interface Props {
text: string;
color: string;
}
export const UnderlinedText: React.FC<Props> = ({ text, color }) => {
const frame = useCurrentFrame();
// Underline scales from left over 0-30 frames.
const underlineScaleX = interpolate(frame, [10, 40], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
const opacity = interpolate(frame, [0, 12], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
return (
<div style={{ position: "relative", display: "inline-block", opacity }}>
<div
style={{
fontSize: 80,
fontWeight: 600,
color: "#ffffff",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
{text}
</div>
<div
style={{
position: "absolute",
left: 0,
bottom: -8,
width: "100%",
height: 6,
background: color,
transform: `scaleX(${underlineScaleX})`,
transformOrigin: "left center",
borderRadius: 3,
}}
/>
</div>
);
};
import { registerRoot } from "remotion";
import { RemotionRoot } from "./Root";
registerRoot(RemotionRoot);
import { Composition } from "remotion";
import { z } from "zod";
import { Stargazed, stargazedSchema } from "./Stargazed";
const defaultProps: z.infer<typeof stargazedSchema> = {
title: "STARGAZED",
subtitle: "by HeyGen",
stats: [
{ label: "Stars", value: 1247, color: "#fbbf24" },
{ label: "Forks", value: 312, color: "#60a5fa" },
{ label: "Issues", value: 48, color: "#f87171" },
],
outro: "thanks for watching",
};
export const RemotionRoot = () => (
<Composition
id="Stargazed"
component={Stargazed}
schema={stargazedSchema}
durationInFrames={300}
fps={30}
width={1280}
height={720}
defaultProps={defaultProps}
/>
);
import { AbsoluteFill } from "remotion";
import { UnderlinedText } from "../components/UnderlinedText";
interface Props {
text: string;
}
export const OutroScene: React.FC<Props> = ({ text }) => (
<AbsoluteFill
style={{
justifyContent: "center",
alignItems: "center",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
<UnderlinedText text={text} color="#fbbf24" />
</AbsoluteFill>
);
import { AbsoluteFill } from "remotion";
import { StatCard } from "../components/StatCard";
interface Stat {
label: string;
value: number;
color: string;
}
interface Props {
stats: Stat[];
}
export const StatsScene: React.FC<Props> = ({ stats }) => (
<AbsoluteFill
style={{
justifyContent: "center",
alignItems: "center",
gap: 48,
flexDirection: "row",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
{stats.map((stat, i) => (
<StatCard
key={stat.label}
label={stat.label}
value={stat.value}
color={stat.color}
delayInFrames={i * 12}
/>
))}
</AbsoluteFill>
);
import { AbsoluteFill, interpolate, spring, useCurrentFrame, useVideoConfig } from "remotion";
interface Props {
title: string;
subtitle: string;
}
export const TitleScene: React.FC<Props> = ({ title, subtitle }) => {
const frame = useCurrentFrame();
const { fps } = useVideoConfig();
const titleScale = spring({
frame,
fps,
config: { damping: 12, stiffness: 100, mass: 1 },
});
const subtitleOpacity = interpolate(frame, [20, 40], [0, 1], {
extrapolateLeft: "clamp",
extrapolateRight: "clamp",
});
return (
<AbsoluteFill
style={{
justifyContent: "center",
alignItems: "center",
flexDirection: "column",
fontFamily: "Helvetica, Arial, sans-serif",
}}
>
<div
style={{
fontSize: 160,
fontWeight: 900,
color: "#ffffff",
letterSpacing: "0.05em",
transform: `scale(${titleScale})`,
}}
>
{title}
</div>
<div
style={{
fontSize: 36,
fontWeight: 400,
color: "#9ca3af",
marginTop: 24,
opacity: subtitleOpacity,
}}
>
{subtitle}
</div>
</AbsoluteFill>
);
};
import { AbsoluteFill, Sequence } from "remotion";
import { z } from "zod";
import { TitleScene } from "./scenes/TitleScene";
import { StatsScene } from "./scenes/StatsScene";
import { OutroScene } from "./scenes/OutroScene";
export const stargazedSchema = z.object({
title: z.string(),
subtitle: z.string(),
stats: z.array(
z.object({
label: z.string(),
value: z.number(),
color: z.string(),
}),
),
outro: z.string(),
});
export const Stargazed: React.FC<z.infer<typeof stargazedSchema>> = ({
title,
subtitle,
stats,
outro,
}) => (
<AbsoluteFill style={{ backgroundColor: "#0a0a0a" }}>
<Sequence from={0} durationInFrames={90}>
<TitleScene title={title} subtitle={subtitle} />
</Sequence>
<Sequence from={90} durationInFrames={120}>
<StatsScene stats={stats} />
</Sequence>
<Sequence from={210} durationInFrames={90}>
<OutroScene text={outro} />
</Sequence>
</AbsoluteFill>
);
{
"compilerOptions": {
"target": "ES2018",
"module": "ESNext",
"jsx": "react-jsx",
"strict": true,
"esModuleInterop": true,
"moduleResolution": "node",
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"resolveJsonModule": true,
"isolatedModules": true
},
"include": ["src"]
}
// T4 case 01 — useState drives animation.
//
// Should be detected by lint_source.py as blocker r2hf/use-state.
// The skill should refuse to translate and recommend the runtime interop
// pattern from PR #214.
//
// Why this is a blocker: useState is React's component-local mutable state.
// HF's seek-driven model produces deterministic frames from a single time
// value — there's no per-frame React render cycle to update state on.
import React, { useState } from "react";
import { AbsoluteFill, useCurrentFrame } from "remotion";
export const StateDriven: React.FC = () => {
const frame = useCurrentFrame();
const [hue, setHue] = useState(0);
// Even if this looks innocuous, the setHue call breaks determinism: HF
// can't reproduce React state mutations across seeks.
if (frame % 30 === 0 && hue < 360) {
setHue((h) => h + 30);
}
return (
<AbsoluteFill style={{ background: `hsl(${hue}, 80%, 50%)` }}>
<div>frame {frame}</div>
</AbsoluteFill>
);
};
// T4 case 02 — useEffect with non-empty deps performs side effects per render.
//
// Should be detected by lint_source.py as blocker r2hf/use-effect-deps.
// The skill should refuse to translate.
//
// Why this is a blocker: side effects (network, DOM mutation outside the
// rendered tree, timers) don't translate to a seek-driven model. HF assumes
// the page is fully rendered and pure between seeks.
import React, { useEffect, useRef } from "react";
import { AbsoluteFill, useCurrentFrame } from "remotion";
export const SideEffectDriven: React.FC = () => {
const frame = useCurrentFrame();
const canvasRef = useRef<HTMLCanvasElement>(null);
useEffect(() => {
const canvas = canvasRef.current;
if (!canvas) return;
const ctx = canvas.getContext("2d");
ctx?.fillRect(frame, frame, 10, 10);
}, [frame]);
return (
<AbsoluteFill>
<canvas ref={canvasRef} width={1280} height={720} />
</AbsoluteFill>
);
};
// T4 case 03 — calculateMetadata returns a Promise.
//
// Should be detected by lint_source.py as blocker r2hf/async-metadata.
// The skill should refuse to translate.
//
// Why this is a blocker: HF needs the composition's duration, dimensions,
// and props known up-front to produce HTML and seed the timeline. Async
// metadata fetched from a server at render time has no equivalent in HF —
// the metadata would need to be resolved at build time before the HTML is
// authored.
import React from "react";
import { AbsoluteFill, useCurrentFrame } from "remotion";
interface Props {
text: string;
}
export const AsyncMetadataDriven: React.FC<Props> = ({ text }) => {
const frame = useCurrentFrame();
return (
<AbsoluteFill>
<div>
{text} · frame {frame}
</div>
</AbsoluteFill>
);
};
export const calculateMetadata = async ({ props }: { props: Props }) => {
const response = await fetch(
`https://api.example.com/duration?text=${encodeURIComponent(props.text)}`,
);
const { durationInFrames } = await response.json();
return {
durationInFrames,
fps: 30,
};
};
// T4 case 04 — Imports from a third-party React UI library.
//
// Should be detected by lint_source.py as blocker r2hf/third-party-react-ui.
// The skill should refuse to translate.
//
// Why this is a blocker: a Material-UI Button (or any React UI library
// component) is a React-only abstraction with internal hooks, refs, and
// theme provider context. Translating it to HTML+CSS would require
// re-implementing the design system, which is out of scope for a video
// translation skill. Use the runtime interop pattern from PR #214 to keep
// these components rendering through Remotion's React tree.
import React from "react";
import { Button } from "@mui/material";
import { AbsoluteFill, useCurrentFrame, interpolate } from "remotion";
export const MuiDriven: React.FC = () => {
const frame = useCurrentFrame();
const opacity = interpolate(frame, [0, 30], [0, 1], { extrapolateRight: "clamp" });
return (
<AbsoluteFill style={{ alignItems: "center", justifyContent: "center" }}>
<div style={{ opacity }}>
<Button variant="contained" color="primary">
Click me · frame {frame}
</Button>
</div>
</AbsoluteFill>
);
};
// T4 case 05 — Imports @remotion/lambda for distributed rendering config.
//
// Should be detected by lint_source.py as warning r2hf/lambda-import.
// The skill drops the Lambda code with a note (HF runs single-machine
// today) and translates the rest of the composition.
//
// Why this is a warning, not a blocker: @remotion/lambda config is
// orthogonal to the rendered composition — it's deployment configuration,
// not animation logic. Treating it as a hard blocker would refuse
// translation for compositions that are otherwise clean. The skill drops
// the Lambda calls in step 3 (Generate) and writes a TRANSLATION_NOTES.md
// entry so the user knows to set up HF rendering separately.
import React from "react";
import { renderMediaOnLambda } from "@remotion/lambda";
import { AbsoluteFill, useCurrentFrame, interpolate } from "remotion";
export const LambdaConfigured: React.FC = () => {
const frame = useCurrentFrame();
const opacity = interpolate(frame, [0, 30], [0, 1]);
return (
<AbsoluteFill style={{ opacity }}>
<div>frame {frame}</div>
</AbsoluteFill>
);
};
// Rendered at scale via Lambda — no HF equivalent.
export async function renderViaLambda() {
return renderMediaOnLambda({
region: "us-east-1",
functionName: "remotion-render",
composition: "LambdaConfigured",
serveUrl: "https://example.com/bundle",
inputProps: {},
codec: "h264",
});
}
// T4 case 06 — Patterns that warn but don't block.
//
// Should be detected by lint_source.py with:
// - r2hf/delay-render (warning) — drop the call; HF handles asset readiness
// - r2hf/use-callback (warning) — decorative, drop the wrapper
// - r2hf/use-memo (warning) — decorative, drop the wrapper
//
// 0 blockers expected — the skill should still translate this composition
// after dropping the wrappers. delayRender is paired with continueRender via
// an empty-deps useEffect (mount-once side effect), which doesn't trip the
// use-effect-deps blocker.
import React, { useCallback, useMemo } from "react";
import { AbsoluteFill, delayRender, continueRender, useCurrentFrame, interpolate } from "remotion";
const handle = delayRender();
// Resolve the handle once at module load — no per-frame side effects.
queueMicrotask(() => continueRender(handle));
export const WarningsOnly: React.FC = () => {
const frame = useCurrentFrame();
// useCallback / useMemo — decorative for render-perf in React, no equivalent
// needed in the seek-driven HF model.
const opacity = useMemo(
() => interpolate(frame, [0, 30], [0, 1], { extrapolateRight: "clamp" }),
[frame],
);
const onMount = useCallback(() => {}, []);
return (
<AbsoluteFill style={{ opacity }} onClick={onMount}>
<div>frame {frame}</div>
</AbsoluteFill>
);
};
// T4 case 07 — Locally-defined custom hook.
//
// Should be detected by lint_source.py as warning r2hf/custom-hook.
// 0 blockers expected — the skill can attempt translation if the hook body
// is pure (derives from props/frame alone).
//
// Why this is a warning: custom hooks vary widely in what they do. Some are
// pure derivations of useCurrentFrame (translatable — inline the body); some
// wrap useState/useEffect (blocker — but those will be caught by the other
// rules independently). The warning prompts the agent to inspect the body.
import React from "react";
import { AbsoluteFill, useCurrentFrame, interpolate } from "remotion";
// Custom hook — pure derivation from frame, no state. Translates fine.
function useFadeIn(durationInFrames: number) {
const frame = useCurrentFrame();
return interpolate(frame, [0, durationInFrames], [0, 1], { extrapolateRight: "clamp" });
}
export const CustomHookDriven: React.FC = () => {
const opacity = useFadeIn(30);
return (
<AbsoluteFill style={{ opacity }}>
<div>fading in</div>
</AbsoluteFill>
);
};
// T4 case 08 — Multiple blockers + multiple warnings in one file.
//
// Should report:
// blockers: r2hf/use-state, r2hf/use-effect-deps, r2hf/third-party-react-ui
// warnings: r2hf/use-callback (also r2hf/delay-render via the import chain
// would only fire if delayRender is actually called)
//
// Tests that the linter aggregates findings correctly and does not stop at
// the first blocker.
import React, { useState, useEffect, useCallback } from "react";
import { AbsoluteFill, useCurrentFrame } from "remotion";
import { Card } from "@chakra-ui/react";
interface Item {
id: string;
label: string;
}
export const MixedBlockers: React.FC = () => {
const frame = useCurrentFrame();
const [items, setItems] = useState<Item[]>([]);
useEffect(() => {
fetch("/api/items")
.then((r) => r.json())
.then(setItems);
}, [frame]);
const onClick = useCallback(() => {
setItems((prev) => [...prev, { id: String(prev.length), label: "new" }]);
}, []);
return (
<AbsoluteFill onClick={onClick}>
{items.map((item) => (
<Card key={item.id}>{item.label}</Card>
))}
</AbsoluteFill>
);
};
{
"tier": 4,
"name": "escape-hatch",
"description": "Lint-only fixture set. Each case demonstrates a Remotion pattern the skill cannot or should not translate cleanly. The skill is graded on whether lint_source.py emits the right finding for each case — there are no renders to compare. T4 passes when every case triggers its expected rule and no others.",
"cases": [
{
"file": "01-use-state.tsx",
"expected": {
"blockers": [{ "rule": "r2hf/use-state", "min_count": 1 }],
"warnings": [],
"skill_action": "refuse_translation_recommend_interop"
}
},
{
"file": "02-use-effect-deps.tsx",
"expected": {
"blockers": [{ "rule": "r2hf/use-effect-deps", "min_count": 1 }],
"warnings": [],
"skill_action": "refuse_translation_recommend_interop"
}
},
{
"file": "03-async-metadata.tsx",
"expected": {
"blockers": [{ "rule": "r2hf/async-metadata", "min_count": 1 }],
"warnings": [],
"skill_action": "refuse_translation_recommend_interop"
}
},
{
"file": "04-third-party-react.tsx",
"expected": {
"blockers": [{ "rule": "r2hf/third-party-react-ui", "min_count": 1 }],
"warnings": [],
"skill_action": "refuse_translation_recommend_interop"
}
},
{
"file": "05-lambda-config.tsx",
"expected": {
"blockers": [],
"warnings": [{ "rule": "r2hf/lambda-import", "min_count": 1 }],
"skill_action": "drop_lambda_code_translate_remainder_if_clean"
}
},
{
"file": "06-warnings-only.tsx",
"expected": {
"blockers": [],
"warnings": [
{ "rule": "r2hf/delay-render", "min_count": 1 },
{ "rule": "r2hf/use-callback", "min_count": 1 },
{ "rule": "r2hf/use-memo", "min_count": 1 }
],
"skill_action": "translate_after_dropping_wrappers"
}
},
{
"file": "07-custom-hook.tsx",
"expected": {
"blockers": [],
"warnings": [{ "rule": "r2hf/custom-hook", "min_count": 1 }],
"skill_action": "inline_hook_body_if_pure"
}
},
{
"file": "08-mixed.tsx",
"expected": {
"blockers": [
{ "rule": "r2hf/use-state", "min_count": 1 },
{ "rule": "r2hf/use-effect-deps", "min_count": 1 },
{ "rule": "r2hf/third-party-react-ui", "min_count": 1 }
],
"warnings": [{ "rule": "r2hf/use-callback", "min_count": 1 }],
"skill_action": "refuse_translation_recommend_interop"
}
}
],
"totals": {
"expected_blocker_cases": 5,
"expected_warning_only_cases": 3,
"expected_total_blocker_findings_min": 7,
"expected_total_warning_findings_min": 6
}
}
Tier 4 — escape-hatch
What it tests
T4 is the lint-only tier. There are no renders to diff — the skill is graded on whether it correctly _refuses_ to translate each case (and recommends the runtime interop pattern from PR #214 instead) or, where appropriate, translates after dropping warning-level decorations.
Each cases/*.tsx file is a minimal Remotion composition that demonstrates one specific pattern. The skill should:
1. Run scripts/lint_source.py over the source. 2. Compare the JSON output to expected.json for that case. 3. Take the documented skill_action:
refuse_translation_recommend_interop— print the rationale + link to
the PR #214 interop guide; do not produce HF output.
drop_lambda_code_translate_remainder_if_clean— drop the
@remotion/lambda code with a note; translate the rest only if no other blockers are present.
translate_after_dropping_wrappers— translate normally; drop
useCallback / useMemo / delayRender wrappers.
inline_hook_body_if_pure— inline the custom hook's body if it's a
pure derivation of useCurrentFrame; otherwise bow out.
Cases
| # | File | Expected finding | Notes |
|---|---|---|---|
| 01 | 01-use-state.tsx | blocker r2hf/use-state | useState driving animation |
| 02 | 02-use-effect-deps.tsx | blocker r2hf/use-effect-deps | useEffect/useLayoutEffect with non-empty deps |
| 03 | 03-async-metadata.tsx | blocker r2hf/async-metadata | calculateMetadata returns a Promise |
| 04 | 04-third-party-react.tsx | blocker r2hf/third-party-react-ui | imports @mui/material |
| 05 | 05-lambda-config.tsx | warning r2hf/lambda-import | imports @remotion/lambda — drops, translates |
| 06 | 06-warnings-only.tsx | warnings only | delayRender / useCallback / useMemo |
| 07 | 07-custom-hook.tsx | warning r2hf/custom-hook | locally-defined useFadeIn (export const form) |
| 08 | 08-mixed.tsx | 3 blockers + 1 warning | aggregate-findings test |
Validation
./validate.shThe script runs lint_source.py against each case and asserts:
- Each expected blocker rule fires with severity
blocker. - Each expected warning rule fires with severity
warning(or stronger). lint_source.py's exit code is 1 when blockers are expected, 0 otherwise.
T4 passes when every case matches its expected output. No renders involved.
#!/usr/bin/env bash
# validate.sh — assert lint_source.py output matches expected.json for every T4 case.
#
# T4 has no renders to diff. The skill is graded on whether it correctly
# refuses to translate each case (or drops only the lambda config in case 5,
# or warns appropriately in cases 6 and 7).
#
# Usage:
# ./validate.sh
# Exit 0 on pass.
set -euo pipefail
THIS_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SCRIPTS_DIR="$(cd "$THIS_DIR/../../../scripts" && pwd)"
EXPECTED="$THIS_DIR/expected.json"
if [[ ! -f "$SCRIPTS_DIR/lint_source.py" ]]; then
echo "error: lint_source.py not found at $SCRIPTS_DIR/lint_source.py" >&2
exit 2
fi
if [[ ! -f "$EXPECTED" ]]; then
echo "error: expected.json not found at $EXPECTED" >&2
exit 2
fi
# Drive lint_file() in-process so the per-case overhead is one Python startup,
# not N (8 cases × ~80 ms forking python3 was the dominant cost).
SCRIPTS_DIR="$SCRIPTS_DIR" \
THIS_DIR="$THIS_DIR" \
EXPECTED="$EXPECTED" \
python3 <<'PY'
import json
import os
import sys
from collections import Counter
from pathlib import Path
scripts_dir = Path(os.environ["SCRIPTS_DIR"])
this_dir = Path(os.environ["THIS_DIR"])
expected_path = Path(os.environ["EXPECTED"])
cases_dir = this_dir / "cases"
sys.path.insert(0, str(scripts_dir))
from lint_source import BLOCKER, WARNING, lint_file # noqa: E402
expected = json.loads(expected_path.read_text())
fails: list[str] = []
passes: list[str] = []
for case in expected["cases"]:
file_name = case["file"]
fixture = cases_dir / file_name
if not fixture.exists():
fails.append(f"{file_name}: fixture missing at {fixture}")
continue
findings = lint_file(fixture)
rule_counts: Counter[str] = Counter()
severity_by_rule: dict[str, str] = {}
for f in findings:
rule_counts[f.rule] += 1
severity_by_rule[f.rule] = f.severity
case_failed = False
def assert_rule(expected_entry, expected_severity_floor, kind):
global case_failed
rule = expected_entry["rule"]
min_count = expected_entry["min_count"]
actual = rule_counts[rule]
actual_severity = severity_by_rule.get(rule)
if actual < min_count:
fails.append(f"{file_name}: expected >={min_count} {kind} findings of rule {rule}, got {actual}")
case_failed = True
elif actual_severity not in expected_severity_floor:
fails.append(
f"{file_name}: rule {rule} found but severity={actual_severity!r} (expected {kind})"
)
case_failed = True
for entry in case["expected"]["blockers"]:
assert_rule(entry, {BLOCKER}, "blocker")
for entry in case["expected"]["warnings"]:
assert_rule(entry, {WARNING, BLOCKER}, "warning")
# Implied lint exit code: 1 when blockers are expected, 0 otherwise.
has_blockers = any(f.severity == BLOCKER for f in findings)
expected_has_blockers = bool(case["expected"]["blockers"])
if has_blockers != expected_has_blockers:
fails.append(
f"{file_name}: implied lint exit {1 if has_blockers else 0}, "
f"expected {1 if expected_has_blockers else 0} (blockers expected: {expected_has_blockers})"
)
case_failed = True
if not case_failed:
passes.append(file_name)
print(f"Passed: {len(passes)}")
for name in passes:
print(f" ✓ {name}")
if fails:
print(f"Failed: {len(fails)}")
for msg in fails:
print(f" ✗ {msg}")
sys.exit(1)
sys.exit(0)
PY
Remotion → HyperFrames API Map
Authoritative translation table. Load this reference when starting a translation to know the high-level mapping; load the per-topic references for fragile details (timing, transitions, etc.).
Reading this table
- `drop` = remove from output entirely. The HF runtime handles it.
- `see references/X.md` = the mapping is non-trivial; read the linked file.
- `refuse + interop` = the skill bows out and recommends the runtime adapter
pattern from PR #214.
Composition root
| Remotion | HyperFrames |
|---|---|
<Composition id durationInFrames fps width height> | root <div id="stage" data-composition-id data-start="0" data-duration="<dur/fps>" data-fps data-width data-height> |
defaultProps={...} | data-* attributes on #stage (one per scalar prop). Nested objects/arrays — see parameters.md |
schema={z.object(...)} | not represented in HTML; the schema lives in the agent's translation step only |
calculateMetadata (sync) | resolve at translation time, write concrete values into data-* |
calculateMetadata (async) | refuse + interop — see escape-hatch.md |
registerRoot(RemotionRoot) | drop |
<AbsoluteFill style> | <div style="position:absolute;inset:0;{style}"> |
Sequencing
See sequencing.md for nesting and stagger details.
| Remotion | HyperFrames |
|---|---|
<Sequence from={F} durationInFrames={D}> | <div data-start="<F/fps>" data-duration="<D/fps>" data-track-index="N"> |
<Series> + <Series.Sequence> | siblings with sequential data-start values |
<Loop durationInFrames={D}> | not a primitive — emit a custom GSAP repeat: -1 loop with manual offset math |
<Freeze frame={F}> | drop the wrapper; HF doesn't have running animation outside the seek-driven timeline so freeze is a no-op |
Timing
See timing.md — this is the highest-leverage section.
| Remotion | HyperFrames |
|---|---|
useCurrentFrame() | drop — HF seeks the timeline. The math derived from frame becomes an animatable property of a paused GSAP tween. |
useVideoConfig() for fps / durationInFrames | drop — read from data-fps / data-duration on #stage |
interpolate(frame, [a,b], [x,y]) (linear) | gsap.fromTo(t, {p:x}, {p:y, duration:(b-a)/fps, ease:"none"}) at offset a/fps |
interpolate(frame, [a,b,c,d], [x,y,y,z]) (multi-segment) | three gsap.to calls at offsets a/fps, b/fps, c/fps |
interpolate(..., {easing: Easing.bezier}) | GSAP CustomEase.create("c", "M0,0 C${a},${b} ${c},${d} 1,1") |
spring({frame, fps, config: {damping, stiffness, mass}}) | GSAP back.out(N) — see timing.md for damping → overshoot table |
interpolateColors(frame, range, colors) | gsap.to({...}, { backgroundColor, color, duration, ease }) — GSAP handles color tweens natively |
Easing.in / .out / .inOut(power) | GSAP power<N>.in / power<N>.out / power<N>.inOut |
Media
See media.md for trim, volume ramps, and decoder notes.
| Remotion | HyperFrames |
|---|---|
<Audio src volume> | <audio data-start data-duration data-track-index data-volume src> |
<Audio playbackRate startFrom endAt> | data-playback-rate, data-trim-start, data-trim-end |
<Video src> | <video muted playsinline data-start data-duration data-track-index src> |
<OffthreadVideo> | <video> — HF doesn't need the off-thread variant (uses headless Chrome) |
<Img src> | <img> |
<IFrame src> | <iframe> — HF auto-falls back to screenshot mode for nested iframes |
staticFile("x.png") | "assets/x.png" — copy the file into hf-src/assets/ next to index.html |
delayRender() / continueRender() | drop — HF waits on asset readiness via the Frame Adapter pattern |
Transitions
See transitions.md.
| Remotion | HyperFrames |
|---|---|
<TransitionSeries> + <TransitionSeries.Transition presentation={fade()} /> | manual gsap.to(scene, {opacity: 0/1, duration}) crossfade at the boundary |
slide(), wipe(), clockWipe(), fade() | HF shader-transitions package presets — pick the closest |
linearTiming({durationInFrames}) | duration in seconds (/fps) |
springTiming({config}) | duration in seconds, ease back.out — see timing.md |
Lottie
See lottie.md.
| Remotion | HyperFrames |
|---|---|
<Lottie animationData={data}> | <div id="lottie-N"> + <script>lottie.loadAnimation(...).then(a => window.__hfLottie.push(a))</script> |
loop / playbackRate props | translate to loop / lottie playback options; HF adapter seeks via goToAndStop |
@remotion/lottie runtime | lottie-web from CDN — drop the React wrapper |
Fonts
See fonts.md.
| Remotion | HyperFrames |
|---|---|
loadFont() from @remotion/google-fonts/<Family> | @font-face rule referencing the Google Fonts CSS, OR <link> to Google Fonts in <head> |
Local font via @font-face | same — paste the rule into <style> |
| System font fallback | document the font-fallback divergence cost (see eval.md) |
Parameters
See parameters.md.
| Remotion | HyperFrames |
|---|---|
z.object({foo: z.string()}) | data-foo on #stage (the schema is implicit in HTML structure) |
nested array prop (stats[]) | repeated HTML markup with per-instance data-* attrs |
| Zod default values | bake defaults into the HTML directly |
| Zod runtime validation | not represented; if validation matters, validate in the translation step before emitting HTML |
React patterns
| Remotion | HyperFrames |
|---|---|
| Custom React subcomponent (pure, prop-driven) | inline as repeated HTML using the prop interface as the template |
useState driving animation | refuse + interop |
useReducer driving animation | refuse + interop |
useEffect(fn, [deps]) (non-empty deps) | refuse + interop |
useEffect(fn, []) (mount-once side effect) | drop the effect; use queueMicrotask if startup work is needed |
useCallback, useMemo | drop the wrappers — decorative |
Custom hook (pure derivation of useCurrentFrame) | inline the body |
| Custom hook with state/effects | refuse + interop |
Distributed rendering
@remotion/lambda and @remotion/cloudrun are deployment configuration — orthogonal to the rendered composition itself. The skill emits these as warnings (not blockers) and drops them in step 3 (Generate) with a note in TRANSLATION_NOTES.md. HF is single-machine today; document the gap.
| Remotion | HyperFrames |
|---|---|
@remotion/lambda import | drop the import (warning r2hf/lambda-import) |
renderMediaOnLambda(...) | drop the call; note in TRANSLATION_NOTES.md |
@remotion/cloudrun | drop the import + call; note in TRANSLATION_NOTES.md |
When to bow out entirely
If any blocker pattern is present, recommend the runtime interop pattern from PR #214 instead of attempting translation. See escape-hatch.md.
The blockers are documented in `scripts/lint_source.py` and tested by tier-4-escape-hatch.
When to bow out: the runtime interop pattern
Some Remotion compositions can't be translated cleanly. The skill should recognize them upfront and recommend the runtime interop pattern from PR #214 instead of producing broken HTML.
When to recommend interop
Run scripts/lint_source.py first. If it returns any blocker, recommend interop. The blockers are:
| Rule | What it catches |
|---|---|
r2hf/use-state | useState driving animation |
r2hf/use-reducer | useReducer driving animation |
r2hf/use-effect-deps | useEffect/useLayoutEffect with non-empty deps (side effects) |
r2hf/async-metadata | calculateMetadata returns a Promise |
r2hf/third-party-react-ui | Imports from MUI, Chakra, Mantine, antd, shadcn, Radix, NextUI |
Each of these breaks the seek-driven, deterministic-frame model that HF relies on. Translating them produces silently-wrong output.
What the interop pattern actually does
Per PR #214, the runtime adapter:
1. Bundles the user's Remotion code with React + @remotion/player via esbuild. 2. Mounts a Remotion <Player> inside an HF composition's HTML. 3. Pauses the player on mount. 4. Registers the player on window.__hfRemotion with seekTo(frame), pause(), durationInFrames, fps. 5. HF's render loop seeks the player frame-by-frame via seekTo(frame).
Result: Remotion's React tree renders at HF's deterministic frame ticks. Custom hooks, useState, useEffect, MUI components — all work because Remotion's React reconciler is doing the rendering.
The recommendation message
When the skill detects a blocker, output something like:
The Remotion source uses useState (and others), which can't betranslated to HF's seek-driven HTML model. The recommended path is the
runtime interop pattern: bundle your Remotion code with @remotion/playerand let HF drive it frame-by-frame.
>
See https://github.com/heygen-com/hyperframes/pull/214 for the full
implementation. Quick summary:
>
1. Bundleentry.tsxwith esbuild:npx esbuild entry.tsx --bundle --outfile=dist/bundle.js --format=iife --jsx=automatic
2. Mount the Player and register on window.__hfRemotion:>
```tsx
const playerRef = useRef<PlayerRef>(null);
useEffect(() => {
playerRef.current?.pause();
window.__hfRemotion = window.__hfRemotion || [];
window.__hfRemotion.push({
seekTo: (f) => playerRef.current?.seekTo(f),
pause: () => playerRef.current?.pause(),
durationInFrames,
fps,
});
}, []);
```
>
3. Reference the bundle from your HF index.html and render normally: <script src="dist/bundle.js"></script>The lint output already includes recommendations
lint_source.py emits a recommendation field per finding. Surface those verbatim — they're tuned per blocker rule:
{
"rule": "r2hf/use-state",
"message": "useState detected — Remotion compositions that drive animation via React state are not deterministic frame-capture targets in HyperFrames",
"recommendation": "Use the runtime interop pattern from PR #214 instead of attempting a translation"
}When NOT to bow out: warnings only
Some patterns produce warnings, not blockers — translate after dropping the wrappers:
| Rule | Action |
|---|---|
r2hf/lambda-import | drop the @remotion/lambda config; HF runs single-machine, log gap |
r2hf/delay-render | drop the call; HF handles asset readiness |
r2hf/use-callback | drop the wrapper, inline the function |
r2hf/use-memo | drop the wrapper, compute inline |
r2hf/custom-hook (pure) | inline the hook body if it's a derivation of useCurrentFrame |
r2hf/static-file | replace staticFile("x") with "assets/x" |
r2hf/interpolate-colors | translate to GSAP color tween (see timing.md) |
These are documented in T4 cases 05–07.
r2hf/lambda-import is a warning — not a blocker — because Lambda configuration is orthogonal to the rendered composition. Translating an otherwise-clean Remotion comp shouldn't fail just because the author also configured AWS Lambda for distributed rendering. The skill drops the @remotion/lambda imports and renderMediaOnLambda(...) calls in step 3 (Generate) and writes a TRANSLATION_NOTES.md entry so the user knows to set up HF rendering separately.
When the source has BOTH blockers AND warnings
Bow out. The presence of a single blocker means the skill shouldn't attempt translation — even if the rest of the composition is clean. The user should use interop for the whole thing OR refactor the blocker patterns out of their Remotion source first.
Eval: how to validate a translation end-to-end
Every translation should be measured. The skill ships three scripts and a tiered test corpus that, together, gate translation quality.
The three scripts
| Script | Input | Output |
|---|---|---|
scripts/lint_source.py | Remotion source dir or file | JSON findings + exit code (0 clean, 1 has blockers) |
scripts/render_diff.sh | two MP4 paths | per-frame SSIM + JSON summary (mean, min, p05, p95, pass) |
scripts/frame_strip.sh | two MP4 paths | side-by-side comparison strip PNG for visual debugging |
Run them in this order: lint → render → diff → (if fail) strip.
Per-fixture flow
# 1. Lint the source — blockers mean stop
python3 ../../scripts/lint_source.py ./remotion-src/src/
# 2. Generate any binary assets (T2+T3 only)
[ -f setup.sh ] && ./setup.sh
# 3. Render Remotion baseline
cd remotion-src && npm install && npm run render
# -> remotion-src/out/baseline.mp4
# 4. Render HF translation
cd .. && node ../../../packages/cli/dist/cli.js render hf-src/ --output hf.mp4
# -> hf.mp4
# 5. SSIM diff
../../scripts/render_diff.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./diff
# -> diff/summary.json
# 6. If diff fails, generate frame strip for visual inspection
../../scripts/frame_strip.sh ./remotion-src/out/baseline.mp4 ./hf.mp4 ./strip 8
# -> strip/strip.pngReading diff/summary.json
{
"frame_count": 90,
"mean": 0.974,
"min": 0.972,
"max": 0.999,
"p05": 0.972,
"p95": 0.983,
"threshold": 0.95,
"pass": true
}| Field | What it tells you |
|---|---|
mean | average SSIM across all frames; the headline number |
min | worst frame; below threshold means at least one frame is structurally wrong |
p05 / p95 | 5th / 95th percentile — most frames sit between these |
threshold | from R2HF_SSIM_THRESHOLD env var (default 0.85) |
pass | whether mean >= threshold |
Validated tier thresholds
Calibrated against actual Remotion + HF renders:
| Tier | Composition shape | Mean | Threshold | Margin |
|---|---|---|---|---|
| T1 | single-element fade-in | 0.974 | 0.95 | +0.022 |
| T2 | multi-scene + spring + audio + image | 0.985 | 0.95 | +0.016 |
| T3 | data-driven, custom subcomponents, count-up | 0.953 | 0.90 | +0.038 |
Each fixture's expected.json carries:
ssim_threshold— the gate forpassvalidation— the actual measured numbers from the calibration runtranslation_notes— what's lossy and why
Critical: encoder config
Both Remotion and HF must output the same pixel format for SSIM to be meaningful. Remotion's default JPEG output writes yuvj420p (full-range); HF outputs yuv420p (limited-range). The mismatch costs ~0.05 SSIM.
Every fixture's remotion.config.ts sets:
Config.setVideoImageFormat("png");
Config.setColorSpace("bt709");If the user's source doesn't have these, add them in the translation step — otherwise the diff measures encoder differences, not translation fidelity.
What the noise floor looks like
The dominant non-translation noise is system font fallback divergence. Remotion's bundled Chromium and HF's chrome-headless-shell interpret font-weight: 800 differently when there's no real font installed:
- Remotion HELLO at 160px: medium-weight stroke
- HF HELLO at 160px: heavy-weight stroke
This costs ~0.025 mean SSIM. Visible in T1's frame strip. fonts.md covers how to mitigate (use Inter, load explicit Google Fonts).
Threshold rule of thumb
Set the threshold ~0.02 below measured p05:
- Real translation regressions drop mean by 0.05+ — caught.
- Encoder/font drift between CI runs is bounded at ~0.01 — not caught.
If a calibration run's measured mean is far above your initial threshold guess, _don't_ tighten the threshold to fit. Leave headroom — fixtures re-rendered on different hardware will drift.
When the diff fails
1. Look at `frame_strip.sh` output first. A side-by-side strip at 6–10 evenly-spaced timestamps shows whether the failure is structural (wrong scene durations, missing element) or cosmetic (different font weight, slight timing skew). 2. Check `diff/ssim.log`. Per-frame SSIM tells you _which_ frames failed. Cluster of bad frames in the middle of a scene = animation problem; bad frames at scene boundaries = sequencing problem. 3. Re-read the relevant reference. timing.md for spring/easing issues, sequencing.md for scene boundary issues, media.md for asset loading issues.
CI integration
The fixtures are not yet wired into CI (packages/producer/tests/ runs inside Docker; the skill corpus needs the same). PR 7 of the stack adds the orchestrator that runs all four tiers and emits an aggregated pass report. For now, evaluate by hand per fixture.
Font translation
Fonts are the dominant non-translation noise floor. Same font-weight: 800 renders perceptibly bolder on HF's chrome-headless-shell than on Remotion's bundled Chromium when there's no real font installed. Validation showed this costs ~0.025 mean SSIM at the noise floor.
Pattern: @remotion/google-fonts/<Family>
import { loadFont } from "@remotion/google-fonts/Inter";
loadFont("normal", { weights: ["400", "800"] });Translate to a <link> tag in <head>:
<head>
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
<link
href="https://fonts.googleapis.com/css2?family=Inter:wght@400;800&display=swap"
rel="stylesheet"
/>
<style>
body {
font-family: Inter, sans-serif;
}
</style>
</head>Pull the family name and weights from the import path and loadFont arguments. HF's compiler inlines the Google Fonts CSS at render time, so you don't pay a network round-trip per render.
Pattern: local fonts via @font-face
import { Font } from "remotion";
Font.loadFont("/MyFont.woff2", "MyFont");Translate to a @font-face rule:
<style>
@font-face {
font-family: "MyFont";
src: url("assets/MyFont.woff2") format("woff2");
font-weight: 400;
font-style: normal;
}
</style>Copy the font file into hf-src/assets/ next to the HTML.
Pattern: system font fallback (no font load)
<div style={{ fontFamily: "Helvetica, Arial, sans-serif" }}>...</div>Same string in HF — but be aware: on Linux without a real Helvetica installed (typical CI environment), Remotion and HF fall back to _different_ sans-serif system fonts because they bundle different Chromium versions. This is the noise floor: ~0.025 mean SSIM cost, visible as different stroke widths at large font weights (800+).
If matching the Remotion render exactly matters for a specific fixture, load the same font explicitly — don't rely on system fallback.
When in doubt: use Inter
Inter renders identically across Chromium versions and is free. Translate any "system sans-serif" Remotion comp to Inter when you need to minimize font drift in the validation harness.
Font loading and delayRender
Remotion uses delayRender() to defer the first frame until fonts load. HF's compiler inlines Google Fonts at compile time and waits on @font-face readiness via the Frame Adapter pattern — the delayRender call drops in translation. See media.md.
Multi-weight loading
When Remotion loads multiple weights:
loadFont("normal", { weights: ["400", "500", "700", "800"] });Inline all weights in the Google Fonts URL:
?family=Inter:wght@400;500;700;800&display=swapTranslation rule: enumerate every distinct font-weight value that appears in the composition's CSS (font-weight: 800 → weight 800 must be loaded). If the Remotion source loads weights that aren't actually used, drop them.
Font subsetting
Remotion's loadFont doesn't subset; HF's compiler doesn't either (yet). Don't try to optimize this in translation — it's lossless to keep the same weight set as the Remotion source.
Translation limitations
What the skill explicitly cannot translate, separated from the blocker-list (which is enforced by lint_source.py). These are _known_ gaps — surface them to the user as translation notes when translating the surrounding composition.
React patterns the skill refuses
See escape-hatch.md. Any of these triggers a bow-out:
useState,useReducerdriving animationuseEffect/useLayoutEffectwith non-empty deps (side effects)- async
calculateMetadata - Third-party React UI libraries (MUI, Chakra, Mantine, antd, shadcn, Radix, NextUI)
@remotion/lambda is no longer in this list — it's a warning, not a blocker, because Lambda config is orthogonal to composition rendering. The skill drops the imports and renderMediaOnLambda(...) calls and writes a TRANSLATION_NOTES.md entry. See escape-hatch.md.
Patterns that work with caveats
Volume ramps on <Audio>
Remotion accepts a function for volume:
<Audio src={...} volume={(f) => interpolate(f, [0, 30], [0, 1])} />HF supports static data-volume only. Translation: bake the ramp into the audio file at translation time using ffmpeg afade, OR drop the ramp with a note. The dropped-ramp path produces audibly different output but visually-identical video, so SSIM passes — just flag it.
<Loop> with stateful children
<Loop durationInFrames={30}>
<CounterThatIncrementsViaUseRef />
</Loop>Loop with repeat: -1 works for _visual_ repetition. If the looped child has cross-iteration state (a counter, a randomness seed), HF won't reproduce it identically per iteration. Bow out unless the child is fully deterministic per-iteration.
Remotion's <Img> with crossOrigin
<Img src="https://other-domain.com/x.png" crossOrigin="anonymous" />HF's renderer doesn't enforce CORS the same way Remotion does. Most public images work; private images served with auth headers won't. If the source uses crossOrigin="use-credentials", the asset needs to be downloaded and inlined at translation time.
Custom presentation in <TransitionSeries>
const customPresentation: PresentationComponent = ({ children, presentationProgress }) => {
return <div style={{ filter: `blur(${(1 - presentationProgress) * 20}px)` }}>{children}</div>;
};Pure presentations (transform/filter/opacity computed from progress) translate to GSAP tweens cleanly. Presentations that read useCurrentFrame() internally or have stateful children don't — bow out.
Code-split components (React.lazy)
const HeavyChart = React.lazy(() => import("./HeavyChart"));React.lazy is async and doesn't fit the deterministic-render model. Translate to a regular import; the resulting HF composition will just include all the code upfront.
Patterns that always work
<AbsoluteFill>and<Sequence>(any nesting)useCurrentFrame()derivations:interpolate,spring,Easing,
interpolateColors, manual math
<Audio>,<Video>,<Img>,<IFrame>with simple propsstaticFile()references- Custom React subcomponents that are pure functions of props
- Custom hooks that are pure derivations of
useCurrentFrame @remotion/lottie(translates to HF's Lottie adapter)@remotion/google-fonts/<Family>(translates to<link>or@font-face)- Sync
calculateMetadata(resolved at translation time) <TransitionSeries>with built-in presentations (fade,slide,
wipe, clockWipe, flip, iris)
What the skill never tries to translate
These are out-of-scope by design:
- HDR rendering — HF supports HDR but Remotion doesn't, so there's
nothing to translate from.
- Variable frame rate — both tools assume constant fps.
- Multi-composition `<Composition>` lists — translate one at a
time. The skill prompts the user to choose which composition.
- Remotion Studio props panel — visual prop editing in HF Studio
needs different infrastructure; out of scope.
Reporting gaps to the user
When translation produces _something_ but the something has gaps, write a TRANSLATION_NOTES.md next to the output:
# Translation notes
The following Remotion patterns were translated with caveats:
- `<Audio volume={(f) => ...}>` (line 15): volume ramp dropped — added
static `data-volume="0.5"`. To preserve the ramp, run
`ffmpeg -i music.wav -af "afade=t=in:st=0:d=1" music.faded.wav` and
swap the source file.
- `<HeavyChart>` (line 30): translated as inline HTML. The original
React.lazy boundary was dropped — bundle size unchanged because HF
serves a single HTML file.
If any of these caveats matter, consider the runtime interop pattern
instead.This file is also generated by the skill alongside the HF output, not held in the corpus.
Lottie translation: @remotion/lottie → HF lottie adapter
Lottie animations are a clean translation case — HF has a built-in Lottie adapter that supports both lottie-web and @lottiefiles/dotlottie-web. The adapter auto-discovers animations registered on window.__hfLottie and seeks them per-frame via goToAndStop.
Pattern
import { Lottie } from "@remotion/lottie";
import animationData from "./hello.json";
export const MyComp = () => (
<AbsoluteFill>
<Lottie animationData={animationData} loop={false} />
</AbsoluteFill>
);Translates to:
<div id="stage" ...>
<div id="lottie-anim" style="width:100%;height:100%"></div>
<script src="https://cdnjs.cloudflare.com/ajax/libs/bodymovin/5.12.2/lottie.min.js"></script>
<script>
const anim = lottie.loadAnimation({
container: document.getElementById("lottie-anim"),
renderer: "svg",
loop: false,
autoplay: false,
path: "assets/hello.json",
});
window.__hfLottie = window.__hfLottie || [];
window.__hfLottie.push(anim);
</script>
</div>Key differences from a typical Lottie embed:
autoplay: false— HF drives playback by seekingloop: falsetypically (unless Remotion'sloop={true})window.__hfLottie.push(anim)is what hooks the animation into HF's
per-frame seek
Asset handling
Remotion bundles the animation JSON via webpack import. HF needs the JSON on disk under assets/ and references it via path:
1. Copy hello.json from the Remotion project into hf-src/assets/. 2. Reference as path: "assets/hello.json" in loadAnimation.
For dotlottie (binary) format, swap in @lottiefiles/dotlottie-web:
<script src="https://unpkg.com/@lottiefiles/dotlottie-web"></script>
<canvas id="anim" style="width:100%;height:100%"></canvas>
<script>
const player = new DotLottie({
canvas: document.getElementById("anim"),
src: "assets/hello.lottie",
autoplay: false,
});
window.__hfLottie = window.__hfLottie || [];
window.__hfLottie.push(player);
</script>The HF adapter handles both player APIs (it duck-types goToAndStop vs setCurrentRawFrameValue / seek).
Multiple Lottie animations
Multiple <Lottie> instances in one composition work — push each one onto window.__hfLottie and the adapter will seek all of them in sync:
window.__hfLottie.push(anim1);
window.__hfLottie.push(anim2);
window.__hfLottie.push(anim3);Lottie source isn't actually translation-blocking
Lottie animations encode their own deterministic timeline. They're the _easiest_ part of a Remotion composition to translate because the animation logic is already self-contained — neither Remotion nor HF "animate" them, both just seek them. Translation cost is near-zero.
After Effects → Lottie limitations
Lottie supports a subset of After Effects features. Expressions, most Effects (drop shadow, color overlay), all blend modes beyond Normal/Add/ Multiply, luma mattes, and most 3D parameters are not supported. If the Remotion composition uses a Lottie file that depends on these, the animation will break in BOTH Remotion and HF — this isn't a translation problem, it's a Lottie limitation. See airbnb/lottie/after-effects.md for the full supported feature list.
Loop behavior
Remotion's loop={true} plays the animation continuously. Translate to the lottie-web loop: true setting AND rely on the adapter's natural seek behavior (it'll seek modulo the animation's duration). For non-default playback rates, set playbackRate in loadAnimation and HF will respect it during seek.
Performance note
Per the Lottie adapter docs: lottie-web's goToAndStop(time, isFrame=false) takes time in ms; the adapter passes time * 1000 for precision. This is more accurate than passing frame numbers (especially for animations whose internal fps doesn't match the HF render fps).
Media translation: Audio, Video, Img, IFrame, staticFile
Asset paths
Remotion's staticFile("x.png") resolves to the project's public/ directory. HF uses relative paths from the composition's index.html, conventionally assets/:
<Img src={staticFile("logo.png")} /><img src="assets/logo.png" />When translating, copy the asset from remotion-src/public/x to hf-src/assets/x. Multiple files can be batched with a setup script; see T2's setup.sh for an example pattern.
<Audio>
<Audio src={staticFile("music.wav")} volume={0.5} /><audio
data-start="0"
data-duration="6"
data-track-index="2"
data-volume="0.5"
src="assets/music.wav"
></audio>data-start and data-duration are required — the runtime needs them to schedule the audio. Default to the composition's full duration if Remotion didn't specify trim.
Volume ramps
<Audio src={staticFile("music.wav")} volume={(f) => interpolate(f, [0, 30], [0, 1])} />HF supports static data-volume only for now. Volume ramps need to be applied to the audio file at translation time (with ffmpeg afade) or the ramp is dropped with a translation note.
Trim / playbackRate
<Audio src={staticFile("music.wav")} startFrom={60} endAt={180} playbackRate={1.5} /><audio
data-start="0"
data-duration="<resolved from trim>"
data-trim-start="2"
data-trim-end="6"
data-playback-rate="1.5"
src="assets/music.wav"
></audio>startFrom / endAt are frame indexes; convert to seconds.
<Video> and <OffthreadVideo>
<Video src={staticFile("intro.mp4")} muted playsInline />
<OffthreadVideo src={staticFile("intro.mp4")} muted /><video
muted
playsinline
data-start="0"
data-duration="5"
data-track-index="0"
src="assets/intro.mp4"
></video><OffthreadVideo> is a Remotion-specific optimization for headless rendering. HF runs in headless Chrome already, so the off-thread variant collapses to a regular <video>.
muted and playsinline are required for the runtime to autoplay (browser policy). Always emit them.
<Img>
<Img src={staticFile("logo.png")} style={{ width: 200, height: 200 }} /><img src="assets/logo.png" style="width: 200px; height: 200px;" />Width/height get rounded to integer px. If the original style has animated dimensions, the GSAP tween animates them — see timing.md.
<IFrame>
<IFrame src="https://example.com" /><iframe src="https://example.com"></iframe>When HF detects a nested iframe in a composition, it auto-falls back to screenshot mode rather than the deterministic BeginFrame mode. This costs render performance but produces visibly-correct output. See hyperframes-vs-remotion.mdx for details.
delayRender() / continueRender()
const handle = delayRender();
useEffect(() => {
loadAsset().then(() => continueRender(handle));
}, []);Drop. HF waits on asset readiness via the Frame Adapter pattern — images, videos, fonts, and Lottie animations all signal load completion natively. There's nothing to do at the application level.
When the asset isn't a file
If Remotion's media source is a Buffer, dataURL, or URL.createObjectURL, the asset doesn't exist on disk and can't be copied via setup.sh. Two options:
1. Materialize the asset at translation time — write the buffer to a file in hf-src/assets/. 2. Embed as a data URL directly in the HTML (src="data:image/png;base64,...") for small assets (< 100 KB).
For audio/video Buffers, option 1 is preferred — base64-encoded media bloats the HTML and slows the renderer.
Parameter translation: Zod schemas, defaultProps, calculateMetadata
How a typed Remotion <Composition schema={...} defaultProps={...} /> turns into a parameterized HF composition.
Sync calculateMetadata (translatable)
<Composition
id="MyVideo"
component={MyVideo}
schema={z.object({ title: z.string(), duration: z.number() })}
defaultProps={{ title: "Hello", duration: 90 }}
calculateMetadata={({ props }) => ({
durationInFrames: props.duration,
fps: 30,
})}
/>When calculateMetadata is synchronous and only uses props, resolve it at translation time — call it with defaultProps (or whatever the caller specifies) and write the concrete result into the HTML:
<div
id="stage"
data-composition-id="MyVideo"
data-start="0"
data-duration="3" <!-- 90/30 -->
data-fps="30"
data-title="Hello"
></div>The data-title attribute carries the value through. Code that originally read props.title reads document.getElementById("stage").dataset.title in HF.
Async calculateMetadata (NOT translatable)
<Composition
calculateMetadata={async ({ props }) => {
const res = await fetch(...);
return { durationInFrames: res.duration };
}}
/>Refuse + interop. HF needs composition metadata up-front to seed the HTML. Resolving network calls at translation time defeats the purpose of having dynamic metadata. See escape-hatch.md.
The lint rule r2hf/async-metadata catches this. T4 case 03 tests it.
Default props
defaultProps={{
title: "Hello",
subtitle: "World",
count: 42,
}}Translate to data-* attributes on the root #stage div:
<div id="stage" data-title="Hello" data-subtitle="World" data-count="42">...</div>Convention: propName → data-prop-name (kebab-case). Inside the GSAP script, read via document.getElementById("stage").dataset.propName.
Nested object / array props
defaultProps={{
stats: [
{ label: "Stars", value: 1247, color: "#fbbf24" },
{ label: "Forks", value: 312, color: "#60a5fa" },
],
}}Don't try to encode the array as a JSON data- attribute — HF's runtime doesn't parse those. Materialize the array as repeated HTML markup:
<div id="scene-stats">
<div class="stat-card" data-stat-index="0" data-stat-value="1247" style="--card-color:#fbbf24">
<div class="number">0</div>
<div class="label">Stars</div>
</div>
<div class="stat-card" data-stat-index="1" data-stat-value="312" style="--card-color:#60a5fa">
<div class="number">0</div>
<div class="label">Forks</div>
</div>
</div>The component template (StatCard.tsx) becomes the markup template; each instance gets its scalar props rendered as data-* and CSS custom properties.
Validated in T3 — three StatCards reused with different props, mean SSIM 0.953.
Numeric props that need typed parsing
document.getElementById("stage").dataset.count is a string. Convert at read time:
const count = Number(stage.dataset.count);Or inline values directly into the GSAP script when the data is known at translation time and doesn't need to vary per render.
Boolean props
defaultProps={{ darkMode: true }}Two conventions:
data-dark-mode="true"— read as string, compare=== "true"data-dark-mode(presence/absence) —<div data-dark-mode>for true, omit for false
Pick one and be consistent. The presence/absence form is HTML-idiomatic and pairs well with CSS attribute selectors:
[data-dark-mode] .scene {
background: #000;
}Zod runtime validation
Remotion's schema validates props at composition load. HF doesn't have an equivalent — by the time the HTML is in the renderer, the schema is already gone.
Validate at translation time instead. If the user passes invalid data, fail with a translation error before emitting HTML. This matches Zod's "fail loud" intent without requiring the runtime dependency.
When the composition uses props for computed prop derivation
const Composition: React.FC<Props> = ({ stats }) => {
const total = stats.reduce((acc, s) => acc + s.value, 0);
return <div>{total}</div>;
};Compute the derived value at translation time and bake it into the HTML or a data- attribute. Don't try to express the computation in JS in the HF composition — that adds runtime overhead and makes the HTML stateful in ways that complicate human editing.
If the derivation is non-trivial (involves the array itself, not just scalars), materialize it as static text in the HTML.
import React, { useState, useEffect, useLayoutEffect } from "react";
import { useCurrentFrame, AbsoluteFill, delayRender, continueRender } from "remotion";
import { Button } from "@mui/material";
// Custom hook in `export const useFoo = ...` form — earlier custom-hook
// regex anchored to `^\s*(?:function|const|let)` and missed the `export`
// prefix. This covers the regression.
export const useFadeMixed = (n: number) => {
const f = useCurrentFrame();
return f / n;
};
export const BadComposition: React.FC = () => {
const frame = useCurrentFrame();
const [data, setData] = useState<string | null>(null);
const [handle] = useState(() => delayRender());
// Multi-line useEffect body with commas inside (fillRect args) — regression
// coverage for r2hf/use-effect-deps. An earlier regex `[^,]+` would stop at
// the first comma inside the body and miss the deps array entirely.
useEffect(() => {
fetch("/api/data")
.then((r) => r.json())
.then((d) => {
const ctx = document.createElement("canvas").getContext("2d");
ctx?.fillRect(0, 0, 100, 100);
setData(d.text);
continueRender(handle);
});
}, [handle]);
// Expression-bodied useEffect — the form `useEffect(() => fetch(...), [deps])`
// has no closing `}`, which an earlier regex anchored on. This and the
// useLayoutEffect below cover the false-negative cases Miguel surfaced.
useEffect(() => fetch("/api/heartbeat"), [frame]);
useLayoutEffect(() => (document.title = `frame ${frame}`), [frame]);
return (
<AbsoluteFill>
<Button>{data ?? "loading"}</Button>
<span>{frame}</span>
</AbsoluteFill>
);
};
export const calculateMetadata = async () => {
const res = await fetch("/api/duration");
const { duration } = await res.json();
return { durationInFrames: duration };
};
Related skills
How it compares
Pick remotion-to-hyperframes only for explicit Remotion-to-HyperFrames ports with SSIM validation; use general-video for native HyperFrames authoring from scratch.
FAQ
When should remotion-to-hyperframes be used?
remotion-to-hyperframes applies only when the user explicitly asks to port, convert, migrate, or translate Remotion source to HyperFrames. New HyperFrames authoring, passing Remotion mentions, or matching a Remotion video without migrating source should use general-video instead.
What Remotion patterns block remotion-to-hyperframes translation?
scripts/lint_source.py treats useState, useReducer, useEffect with deps, async calculateMetadata, and third-party React UI libraries like MUI, Chakra, and shadcn as blockers. Those compositions should use the runtime interop escape hatch documented in references/escape-hatch.md.
How does remotion-to-hyperframes verify translation quality?
remotion-to-hyperframes renders Remotion and HyperFrames outputs, then runs scripts/render_diff.sh for SSIM. Thresholds sit about 0.02 below validated tier p05 values; T1 mean SSIM 0.974 and T3 mean 0.953 are documented baselines as of 2026-04-27.
Is Remotion To Hyperframes safe to install?
skills.sh reports 2 of 3 security scanners passed. Review the Security Audits panel on this page before installing in production.