Canvas Timeline
Package

Mediabunny Audio & Video Sync Adapter

Optional Mediabunny adapter for timeline media playback and frame access.

Use the Mediabunny adapter when your editor needs decoded canvas frames, local media inputs, or a timeline-aware media monitor instead of one native media element.

Overview

The Mediabunny adapter is for timeline preview monitors that need decoded media, not just a mounted <video> tag. It opens registered sources only when active or explicitly preloaded, paints decoded video frames to a canvas, queues audio through Web Audio, and exposes per-source timing and track metadata.

In React, start with useMediabunnyTimelineMedia from ./react. It creates the adapter and connects it to timeline playback. The package root stays framework-free for imperative integrations. Drop down to useMediabunnyAdapter or createMediabunnyAdapter only when you are building custom sync, sharing an adapter between controls, or owning lifecycle outside React.

Install

pnpm
pnpm add @techsquidtv/canvas-timeline-mediabunny-adapter mediabunny

Usage

Decode media without putting media in timeline state

Timeline clips keep lightweight sourceId values. Your app keeps the actual files, URLs, or Mediabunny inputs outside the timeline and passes matching source descriptors to the adapter.

  1. 1Give each media clip a stable sourceId.
  2. 2Create a sources array where each logical source has a matching sourceId and app-resolved input.
  3. 3Resolve originals or proxies in app state and use fallbacks only for equivalent transport failover.
  4. 4Pass sources and visual/audio layer selectors to useMediabunnyTimelineMedia inside a TimelineProvider.
  5. 5Attach the returned canvasRef to the canvas that receives decoded video frames.
  6. 6Use sourceStateById for lazy loading and source metadata; preload likely next clips and unload resources no longer needed.
  7. 7Use the returned transport helpers for playback and useMediabunnyFrameTime only for UI that needs live decoded-frame timestamps.

Example

Decoded canvas preview
import { useMediabunnyTimelineMedia } from '@techsquidtv/canvas-timeline-mediabunny-adapter/react';


const sourceId = 'clip-source-main';
const sources = [{
  sourceId,
  input: '/media/preview.mp4',
}] as const;
const layers = {
  visuals: { trackKind: 'visual', sourceId },
  audio: { trackKind: 'audio', sourceId },
} as const;


export function DecodedPreview() {
  const media = useMediabunnyTimelineMedia({
    sources,
    layers,
    onError: (error) => console.error(error.reason, error.message),
  });


  return <canvas ref={media.canvasRef} width={1280} height={720} onClick={() => void media.play()} />;
}