Whether the clip came from Kling, Veo, Seedance, After Effects, or a phone, the web problem is the same: scroll wants random access to frames. An MP4 wants a decoder and keyframes. That is why video.currentTime = progress * duration jumps - especially scrolling back, especially on iOS.

The industry pattern (Apple product pages included) is:

1. Turn the video into numbered stills.
2. Draw the current still on a <canvas> from scroll progress.
3. Serve a lighter set on phones.

You can do that by hand. Or you can generate a production bundle with Scrollsequence’s free local pipeline.

Part 1 - The generic DIY way

1. Extract frames with ffmpeg

# Example: ~24 fps stills, 1600px wide WebP
ffmpeg -i clip.mp4 -vf "fps=24,scale=1600:-1" frame_%04d.webp

Then thin the set. You rarely need every frame. A practical hero is often ~100–150 frames on desktop, fewer on mobile.

If you insist on scrubbing the MP4 anyway, re-encode with frequent keyframes first (-g 1 or -g 2). That helps seeking; it still will not match true frame sequences for reverse scrub and memory control.

2. Scrub on a canvas (vanilla idea)

<section id="pin" style="height: 300vh;">
  <canvas id="seq" style="position: sticky; top: 0; width: 100%; height: 100vh;"></canvas>
</section>

const frames = []; // preload Image() / createImageBitmap for each frame_%04d.webp
const canvas = document.getElementById('seq');
const ctx = canvas.getContext('2d');
let current = -1;

function draw(i) {
  const img = frames[i];
  if (!img) return;
  // size canvas to CSS * devicePixelRatio (cap at 2 on phones)
  ctx.drawImage(img, 0, 0, canvas.width, canvas.height);
}

function onScroll() {
  const el = document.getElementById('pin');
  const rect = el.getBoundingClientRect();
  const total = el.offsetHeight - window.innerHeight;
  const scrolled = Math.min(total, Math.max(0, -rect.top));
  const progress = total > 0 ? scrolled / total : 0;
  const i = Math.round(progress * (frames.length - 1));
  if (i === current) return;
  current = i;
  requestAnimationFrame(() => draw(i));
}

window.addEventListener('scroll', onScroll, { passive: true });

That is the whole “Apple-style” trick in spirit: progress → index → draw, only when the index changes.

3. What DIY usually forgets (and why demos lag)

You can wire GSAP ScrollTrigger instead of the raw scroll math - same idea: scrub a number from 0→1, map to frameIndex.

Part 2 - Free Scrollsequence path: npx scrollsequence vid2ssq

Scrollsequence v2 is built around the same model on every stack: Scene → Layer → Canvas. The asset side is a pipeline that turns raw .mp4 / .webm into multi-resolution folders plus a scrollsequence-manifest.json that React, vanilla JS, and WordPress all consume the same way.

Free vs optional AI (important)

From the Asset Pipeline docs and credits docs:

• 100% free / local: frame extraction, responsive resizing, format/quality, frame step, manifest generation (CLI uses local FFmpeg via Node / Wasm - no credits).
• Optional / API key + credits: --depth (3D depth maps), --track "sports car" (subject tracking), --vision (scene / camera / subject metadata for agents and automation).

So the default “video → scroll-ready bundle” path is free. Depth maps and vision are extras when you want WebGL tilt or agent-readable metadata.

Run the wizard

# Interactive wizard (beta package URL - see docs for current .tgz)
npx https://docs.scrollsequence.com/scrollsequence-2.0.0-beta.52.tgz vid2ssq

# Or if already installed:
npx scrollsequence vid2ssq

Non-interactive example (free local processing):

npx scrollsequence vid2ssq input.mp4 \
  --name hero-sequence \
  --sizes 1280,1920 \
  --format jpg \
  --quality 80

Optional AI (needs SSQ_API):

npx scrollsequence vid2ssq input.mp4 \
  --name car-reveal \
  --sizes 1280,1920 \
  --depth \
  --track "sports car" \
  --vision

What you get that hand-rolled ffmpeg usually skips

1. Multiple size tiers (e.g. 1280 + 1920) so phones are not force-fed desktop frames. The runtime picks a tier from canvas CSS width × devicePixelRatio.
2. scrollsequence-manifest.json - one file describing paths, frame counts, optional files.d depth maps, tracking, and vision.
3. Depth maps (optional) - grayscale per frame for WebGL 2.5D mouse / gyro tilt (shared lower-res depth across color tiers).
4. Tracking + vision (optional) - bounding data and scene/camera/subject JSON so an AI agent (or your own UI) can pin labels to a moving subject or reason about the shot.
5. Same output for WordPress, React, and vanilla - no separate “Elementor-only” asset format.

Inside Next.js / Vite, if a public/ folder exists, the CLI writes straight to public/<name>/ so the dev server can serve frames immediately.

Drop the manifest into the page

Vanilla (free engine pattern):

<div class="scrollsequence-scene" data-ssq-duration="1500" data-ssq-scrub="1">
  <div class="scrollsequence-scene_inner">
    <div class="scrollsequence-layer">
      <div
        class="scrollsequence-canvas2d"
        data-ssq-media-groups='[{
          "type": "manifest",
          "rootDir": "/hero-sequence/",
          "imageSizes": { /* from scrollsequence-manifest.json */ }
        }]'
      >
        <canvas></canvas>
      </div>
    </div>
  </div>
</div>

<script type="module">
  import ScrollsequenceManager from 'scrollsequence';
  import 'scrollsequence/scrollsequence.css';
  ScrollsequenceManager.initialize().preloadAll();
</script>

React / Next.js:

'use client';
import { Scene, Layer, Canvas2d } from 'scrollsequence/react';
import manifest from '@/public/hero-sequence/scrollsequence-manifest.json';

export default function Hero() {
  return (
    <Scene duration={1500}>
      <Layer>
        <Canvas2d
          mediaGroups={[{
            type: 'manifest',
            rootDir: '/hero-sequence/',
            imageSizes: manifest.imageSizes,
          }]}
        />
      </Layer>
    </Scene>
  );
}

WordPress v2 uses the same Canvas 2D + Media Tools vid2ssq flow (basic extraction free in-browser; AI toggles credit-based).

Install note (public beta): package currently installs from the docs .tgz until the npm name settles — see docs.scrollsequence.com/v2.

Which path should you use?

DIY teaches the mechanism. vid2ssq is the same mechanism packaged so you are not reinventing responsive tiers and a manifest every project — and the basic convert stays free.

Docs: Asset Pipeline & CLI · Vanilla · React · WordPress

Originally published on Scrollsequence.

Tutorials like this one from Nicolai Palmkvist that scrub a video with GSAP ScrollTrigger in Elementor look great in the demo. Then you try it on your site and it jumps, freezes when you scroll back up, or completely falls apart on iPhone.

That is not you "doing Elementor wrong." It is how browsers seek video.

What Those Tutorials Actually Do

The usual pattern you see in tutorials:

1. Drop a <video> (or a background video widget) in a tall section.
2. Pin the section with ScrollTrigger.
3. On scroll progress, set:

video.currentTime = progress * duration;

On paper, that cleanly maps scroll distance to video playback time. In practice, the browser does not store every frame as a clean, random-access image.

Modern video codecs store keyframes (I-frames) and delta frames (P-frames and B-frames). Seeking to an arbitrary currentTime forces the browser's video decoder to snap to the nearest keyframe or decode intermediate frames on the fly. As a result, the animation stutters — especially when you scrub backwards.

iOS Safari is the harshest case:

• Stricter seeking throttles.
• Aggressive memory pressure and hardware decoder throttling.
• Mandatory muted + playsinline attributes, or the video will refuse to behave like an inline scrubbable timeline at all.

Quick Fix If You Must Keep the MP4

If you need to stay on video for now, these adjustments will get you the best possible results:

1. Re-encode With Frequent Keyframes

Use FFmpeg to force keyframes at short intervals:

ffmpeg -i input.mp4 -an -c:v libx264 -g 2 -keyint_min 2 -sc_threshold 0 -crf 24 -pix_fmt yuv420p output.mp4

• -g 2 places a keyframe approximately every 2 frames.
• Seeking becomes dramatically smoother because the decoder never has to jump far to find a reference frame.
• Trade-off: file size increases significantly.

2. Keep the Video Short and Light

• Mute it (-an): Audio is completely useless for scroll scrubbing and wastes bandwidth.
• Keep it brief: Prefer 8–12 seconds over a long timeline.
• Cap resolution: Often 1280px wide is more than enough for a hero section.

3. Respect Mobile Video Rules

• Ensure muted and playsinline attributes are set on the <video> element.
• Preload carefully: avoid downloading a massive video file before the section is actually near the viewport.

This can get your MP4 scrubbing to "pretty good." It rarely gets you to Apple-style, frame-perfect control.

Better Approach: Scroll an Image Sequence

Apple product pages (and most high-end scrollytelling websites) do not scrub an MP4. Instead, they render numbered image stills onto an HTML5 <canvas> (or swap images) driven by scroll progress.

1. Extract and Prepare Stills

You can extract frames manually with FFmpeg:

# Extract ~24 fps stills, then thin them to ~100–150 frames total
ffmpeg -i clip.mp4 -vf "fps=24,scale=1600:-1" frame_%04d.jpg

Or run the automated Scrollsequence CLI wizard, which extracts frames, creates responsive resolution folders (e.g., mobile vs. desktop), and builds a manifest:

npx scrollsequence vid2ssq

2. Implement on the Page

If writing custom JavaScript from scratch, you preload frames into an array, place a <canvas> in a sticky container, and draw on scroll:

const frameIndex = Math.round(progress * (frames.length - 1));
if (frameIndex !== currentFrameIndex) {
  currentFrameIndex = frameIndex;
  requestAnimationFrame(() => drawFrame(currentFrameIndex));
}

Or using Scrollsequence Canvas2D (vanilla JS), which handles responsive tiering, sticky pinning, and canvas rendering automatically:

<div
  class="scrollsequence-canvas2d"
  data-ssq-media-groups='[{"type":"manifest","rootDir":"/sequence/","imageSizes":{"1920":{"width":1920,"height":1080,"files":{"i":"1920/%04d.jpg"},"frameCount":150}}}]'
>
  <canvas></canvas>
</div>
<script type="module">
  import ScrollsequenceManager from 'scrollsequence';
  import 'scrollsequence/scrollsequence.css';
  ScrollsequenceManager.initialize().preloadAll();
</script>

Why This Feels Smoother

• Instant random-access: Every frame is a discrete, seekable image — no keyframe guessing or decoder lag.
• Bidirectional symmetry: Scrolling backward is just as instantaneous and fluid as scrolling forward.
• Responsive asset control: Serve a desktop set vs. a lightweight mobile set tailored to device capabilities.

Frame Budget (Where Most Demos Fail)

Lag in image sequence demos is almost never because "GSAP is slow." It is almost always caused by serving too many, overly large frames.

Recommended Starting Budget

Performance Wins (The Scrollsequence Strategy)

• Probability-based preloading (Gauss curve): Don't load frames purely sequentially from 0 to 150. Scrollsequence prioritizes images closest to the user's current scroll position using a Gaussian probability distribution. Frames near the active view load first so scrubbing is instantly responsive, expanding outward as bandwidth allows.
• Multi-size responsive tiers: Generate separate image sizes (e.g., 960px, 1280px, 1920px) matched to screen and container width. Serving an 800KB mobile batch instead of a 20MB 4K sequence prevents crashes and data waste.
• Avoid DOM bloat: Never inject hundreds of <img> tags into the DOM — draw to a single <canvas>.
• Watch iOS memory: Decoded memory is roughly width × height × 4 bytes × frame count. Decoding 150 uncompressed Full HD frames can consume over 1 GB of RAM and crash the mobile Safari tab.

Elementor-Specific Notes

• Pin / sticky height: The height of your container or spacer must match the scroll distance you want (e.g., 300vh). A parent container with an unexpected fixed height or broken overflow is the most common reason the effect appears to do nothing.
• One hero sequence per page at first: Multiple heavy sequences require separate preload arrays and a ScrollTrigger.refresh() after images finish loading so trigger start/end positions recalculate accurately.
• Background video widgets: Elementor's native background video widgets are designed for looping ambient background video — they are not built for interactive scroll scrubbing.

What About CSS Scroll-Driven Animations?

As of Firefox 159, CSS scroll-driven animations (animation-timeline: scroll()) are effectively Baseline for fades, parallax, and progress bars.

However, they do not replace a 100+ frame product sequence. A giant image sprite sheet using steps() works for short micro-animations (10–20 frames), but quickly becomes impractically large for cinematic sequences. For Apple-style scrubbing, <canvas> with image frames remains the industry standard.

Minimal Checklist

• If scrubbing MP4: re-encode with frequent keyframes (-g 2), mute audio (-an), and keep the timeline short.
• If you need frame-perfect scrub: export JPEG frames, render to <canvas>, and map scroll progress → frameIndex.
• Cap frame budgets: keep total frames under control and serve a lighter set to mobile devices.
• Render selectively: only invoke canvas.drawImage() when the calculated frame index actually changes.
• Test on real devices: test on a real iPhone or Android and a mid-range Android phone, not just desktop Chrome DevTools.

Optional: WordPress Without Pasting the Whole Stack

If you are on WordPress / Elementor and would rather not hand-wire FFmpeg, canvas rendering loops, and responsive loaders every time, Scrollsequence is the plugin built specifically for this workflow.

You can upload your frames (or directly convert video in v2) and get an editable, responsive section or block. The checklist above still applies either way — the mechanics of image frames and mobile memory limits remain identical.