DominScript — ffmpeg

12 functionsnamespace ffmpegsource plugins/ffmpeg/FfmpegPlugin.c

Written from a line-by-line source review; every example output is from a real run.

Introduction

The ffmpeg plugin enables video decoding from the script: you can open a video file and read its frames one by one as raw RGBA8 pixels, and optionally play the audio too. The work is done in the background by the system's FFmpeg tool (ffmpeg / ffprobe), which the plugin starts as an external process. The plugin is handle-based: every opened video receives an integer identifier (a decoder handle).

Real and synthetic sources

The source can be of two kinds. With the path of a real video file (for example “film.mp4”) the plugin decodes via the background FFmpeg process. The “synthetic” or “synthetic:WIDTHxHEIGHT” source, by contrast, needs no real file: the plugin produces generated frames — this is ideal for testing and demos, because it works deterministically without an FFmpeg installation or a video file.

Frame format (RGBA8)

The decoded frames are raw RGBA8 pixels: 4 bytes per pixel (red, green, blue, alpha). The blob returned by DecodeNextBlob begins with a 64-byte header (size and format data), followed by the pixels. A WIDTH×HEIGHT frame's blob is therefore 64 + WIDTH×HEIGHT×4 bytes — for example a 32×32 frame is 64 + 4096 = 4160 bytes.

Loading the plugin

plugin "../plugins/print/PrintPlugin";
plugin "../plugins/ffmpeg/FfmpegPlugin";

A typical flow — decoding frames

i32 $d;
blob $frame[8192];
$d = ffmpeg.Open("synthetic:32x32");
$frame = ffmpeg.DecodeNextBlob($d); // one RGBA8 frame
printf("%d bytes\n", $frame.Length);
ffmpeg.Close($d);

What to know about every function (the basics)

Open returns a positive decoder handle; every further operation takes it, and Close releases it at the end. At most 8 decoders can be open at once.
The configuring functions (SetOutputSize / SetRealtime / SetVideoOnly) set a GLOBAL default that affects real sources opened afterwards, and return 1. A synthetic source uses its own size given in the path.
DecodeNext passes the frame blob to a script callback, and returns 1 if there was a frame, 0 at the end of the video. DecodeNextBlob returns the frame blob directly.
Using a non-existent decoder handle raises a runtime error. A wrong argument count or type also causes an error.
FileExists returns 1/0 (always 0 for a “synthetic” path, since that is not a real file).
The frame is always RGBA8 (4 bytes/pixel); the row stride is four times the width.

How to read the signatures

handle is an opened video's decoder identifier, callback is the name of a script-level function. The type after the -> arrow is the return type. For example, ffmpeg.DecodeNextBlob(handle) -> blob takes a decoder handle and returns a frame blob.

Configuration

Setting the global decoding defaults: output size, real-time pacing, video-only. They affect (real) sources opened afterwards.

ffmpeg.SetOutputSize

ffmpeg.SetOutputSize(width, height) -> int

Sets the default size of decoded frames (the output is scaled to this size).

Parameters

Parameter	Type	Description
width	int	The image width in pixels (between 16 and 4096).
height	int	The image height in pixels (between 16 and 4096).

Return value

1 on success. An out-of-range size causes a runtime error.

Example

printf("%d\n", ffmpeg.SetOutputSize(320, 180));

Output after running

When to use

Before opening a real video, if you want the frames scaled to a particular size (for example the size of the display surface). A synthetic source uses its own size given in the path.

ffmpeg.SetRealtime

ffmpeg.SetRealtime(enabled) -> int

Turns real-time pacing on or off (decoding paced to the video's own frame rate).

Parameters

Parameter	Type	Description
enabled	int	1 = real-time pacing on, 0 = off (decode as fast as possible).

Return value

Example

printf("%d\n", ffmpeg.SetRealtime(0));

Output after running

When to use

Turn it on for playback (the frames arrive at the video's speed); turn it off for processing/export, so decoding is as fast as possible.

ffmpeg.SetVideoOnly

ffmpeg.SetVideoOnly(enabled) -> int

Sets whether to process only the video track (ignoring the audio).

Parameters

Parameter	Type	Description
enabled	int	1 = video only, 0 = audio is also available (via StartAudio).

Return value

Example

printf("%d\n", ffmpeg.SetVideoOnly(1));

Output after running

When to use

Turn it on if you only need the frames (faster, simpler); turn it off if you also want to play the audio with StartAudio.

Opening and closing

Opening a video source as a decoder, rewinding it to the start, and closing it.

ffmpeg.Open

ffmpeg.Open(path) -> int

Opens a video source as a decoder and returns a decoder handle.

Parameters

Parameter	Type	Description
path	string	The path of the video file, or “synthetic” / “synthetic:WIDTHxHEIGHT” for the generated source.

Return value

The decoder handle (a positive integer). On error (for example a bad synthetic size or a full decoder table), a runtime error.

Example

i32 $d;
$d = ffmpeg.Open("synthetic:32x32");
printf("%d\n", ($d > 0));
ffmpeg.Close($d);

Output after running

When to use

The first step of decoding. For a real video, give the file's path; for testing, the synthetic source is convenient, since it needs no file or FFmpeg. Always call Close at the end.

ffmpeg.Rewind

ffmpeg.Rewind(handle) -> int

Rewinds the decoder to the start of the video (the next frame will again be the first).

Parameters

Parameter	Type	Description
handle	int	The decoder handle.

Return value

1 on success. A non-existent handle causes a runtime error.

Example

i32 $d;
$d = ffmpeg.Open("synthetic:32x32");
ffmpeg.DecodeNextBlob($d);
printf("%d\n", ffmpeg.Rewind($d));
ffmpeg.Close($d);

Output after running

When to use

To replay or reprocess a video without having to open it again. For a real video it restarts the background worker process from the beginning.

ffmpeg.Close

ffmpeg.Close(handle) -> int

Closes the decoder and frees its resources (the background processes and the buffers).

Parameters

Parameter	Type	Description
handle	int	The decoder handle to close.

Return value

1 on success. A non-existent handle causes a runtime error.

Example

i32 $d;
$d = ffmpeg.Open("synthetic:32x32");
printf("%d\n", ffmpeg.Close($d));

Output after running

When to use

Close every opened decoder when you are done with it — otherwise the background FFmpeg processes and the memory buffers can pile up (at most 8 decoders can be open).

Decoding

Reading the next frame in raw RGBA8 format — as a blob or passed to a callback.

ffmpeg.DecodeNextBlob

ffmpeg.DecodeNextBlob(handle) -> blob

Reads the next frame and returns it as a blob (a 64-byte header + raw RGBA8 pixel data).

Parameters

Parameter	Type	Description
handle	int	The decoder handle.

Return value

The frame as a blob. The blob's size is 64 + width × height × 4. On error (or absence of a frame), a runtime error.

Example

i32 $d;
blob $frame[8192];
$d = ffmpeg.Open("synthetic:32x32");
$frame = ffmpeg.DecodeNextBlob($d);
printf("%d\n", $frame.Length);
ffmpeg.Close($d);

Output after running

When to use

The simplest way to obtain a frame, when you want to process or pass on the pixel data directly in a blob. Repeated calls advance through the frames.

ffmpeg.DecodeNext

ffmpeg.DecodeNext(handle, callback) -> int

Reads the next frame and passes it to a script-level callback (with the frame blob).

Parameters

Parameter	Type	Description
handle	int	The decoder handle.
callback	string	The name of the script-level function to call (it receives a blob argument).

Return value

1 if there was a frame (the callback ran); 0 if there are no more frames (the end of the video). On error, a runtime error.

Example

i32 $d;
$d = ffmpeg.Open("synthetic:32x32");
ffmpeg.DecodeNext($d, "onFrame");
ffmpeg.Close($d);
// ...
i32 onFrame(blob $f)
{
printf("frame: %d bytes\n", $f.Length);
return(0);
}

Output after running

frame: 4160 bytes

When to use

For event-driven processing: the callback runs for each frame. A return of 0 marks the end of the video, so you can conveniently iterate over all frames in a loop.

Audio

Playing the video's audio track separately (when video-only mode is off).

ffmpeg.StartAudio

ffmpeg.StartAudio(handle) -> int

Starts playback of the video's audio track in a separate process.

Parameters

Parameter	Type	Description
handle	int	The decoder handle.

Return value

A success flag (1 if it started). A non-existent handle causes a runtime error.

Example

i32 $d;
$d = ffmpeg.Open("video.mp4");
ffmpeg.StartAudio($d);
// ... audio plays while frames are decoded ...
ffmpeg.StopAudio($d);
ffmpeg.Close($d);

Output after running

(1 if the audio started)

When to use

During playback, when you want to hear the audio alongside the image. For this, video-only mode must be off before opening (SetVideoOnly(0)).

ffmpeg.StopAudio

ffmpeg.StopAudio(handle) -> int

Stops playback of the audio track.

Parameters

Parameter	Type	Description
handle	int	The decoder handle.

Return value

1 on success. A non-existent handle causes a runtime error.

Example

i32 $d;
$d = ffmpeg.Open("video.mp4");
ffmpeg.StartAudio($d);
printf("%d\n", ffmpeg.StopAudio($d));
ffmpeg.Close($d);

Output after running

When to use

To stop the audio (for example when pausing or at the end of playback). Close stops the audio on its own too, but for an explicit pause this is the direct way.

Helper and information

Checking a file's availability and querying the decoder's state.

ffmpeg.FileExists

ffmpeg.FileExists(path) -> int

Tells whether a readable file exists at the given path.

Parameters

Parameter	Type	Description
path	string	The path of the file to check.

Return value

1 if the file exists and is readable; otherwise 0 (always 0 for the “synthetic” alias, since that is not a real file).

Example

printf("%d\n", ffmpeg.FileExists("/no/such.mp4"));

Output after running

When to use

For a check before opening: this avoids trying to open a decoder on a missing file. (The synthetic source need not be checked — it is not a file.)

ffmpeg.Info

ffmpeg.Info(handle) -> string

Returns the decoder's current state as human-readable, key=value text.

Parameters

Parameter	Type	Description
handle	int	The decoder handle.

Return value

The state string (handle, mode, size, format, frame counter, path, etc.).

Example

i32 $d;
$d = ffmpeg.Open("synthetic:32x32");
printf("%s\n", ffmpeg.Info($d));
ffmpeg.Close($d);

Output after running

handle=1 mode=synthetic pacing=none audio=off width=32 height=32
stride=128 format=RGBA8 frame=0 dropped=0 path=synthetic:32x32

When to use

For diagnostics and logging: it shows the decoder's settings and state at a glance (mode, size, format, the number of frames decoded and dropped so far).

Practical notes

What the ffmpeg plugin is good for

Reading a video's frames in raw RGBA8 format for display (for example in an SDL- or OpenGL-based player).
Per-frame processing or analysis (making a thumbnail, saving a frame, motion detection).
Playing a video together with audio (StartAudio), with real-time pacing.
Generated (synthetic) frames for testing and demos, without a real video file or FFmpeg.

Typical flow

The usual flow: set the global options (SetOutputSize/SetRealtime/SetVideoOnly) → Open to open the source → iterate over the frames with repeated DecodeNextBlob (or DecodeNext with a callback) while there are more → optionally StartAudio for sound meanwhile → finally Close. For a replay, Rewind rewinds to the start.

Synthetic source

The “synthetic” or “synthetic:WIDTHxHEIGHT” source needs neither a real video file nor an installed FFmpeg: the plugin generates frames itself. This is ideal for testing, development, and demos, because it is deterministic and runs standalone. That is why this reference's examples use a synthetic source.

Frame format

A DecodeNextBlob frame consists of a 64-byte header (size, format, sequence number) and the raw RGBA8 pixel data that follows, where each pixel is 4 bytes (R, G, B, A). The total blob size is 64 + width × height × 4, and the row stride is width × 4. A 32×32 frame is therefore 4160 bytes.

Error handling

A wrong argument count or type, a non-existent decoder handle, an out-of-range size, a bad synthetic format, and a frame-decoding error are reported by the runtime as a runtime error, and the script stops. The usual states are signaled by a neutral return value: FileExists gives 0/1, DecodeNext gives 1/0 (the latter being the end of the video). The configuring and managing functions return 1 on success.

The fileio file-handling plugin

Reading, writing, checking the existence of, and deleting files — complete function reference