This document provides an overview of the most important classes and structures, and how to use them.
AVResult32-63 represents the result of most FFmpeg functions that return an integer. The return value indicates success or failure:
- Success:
>= 0→IsErrorreturnsfalse - Failure:
< 0→IsErrorreturnstrue
AVResult32-63 can be implicitly converted from and to both int and long.
The structure provides static properties for the most important FFmpeg error codes.
| Code | Name | Description |
|---|---|---|
| -11 | TryAgain | Used during encoding/decoding or filtering when more data needs to be written or read. |
| -22 | InvalidArgument | Indicates missing or incorrect parameters. Check standard output for additional FFmpeg error information. |
| -1 to -32 | … | Other standard GNU error codes |
| Code | Name | Description |
|---|---|---|
"EOF " |
EndOfFile | End of the file has been reached. |
| ... |
Call ThrowIfError() to automatically throw an exception if the value is negative.
AVResult32 res = context.ReceiveFrame(frame);
do while(res == AVResult32.TryAgain)
{
context.SendPacket(packet).ThrowIfError();
res = context.ReceiveFrame(frame);
} The Rational struct represents a rational number, typically used to express frame rates, timebases, or other ratio-based values.
Rational supports implicit conversion from and to double, and TimeSpan.
The Rational struct supports several operators, including:
- Arithmetic operations (
+,-,*,/) - Comparisons (
==,!=,<,>,<=,>=) - Conversions to numeric types (
double,TimeSpan)
The method Rescale can be used to rescale a timestamp (usually stored as a long in TimeBase units) from one TimeBase into another.
// TimeSpan = long * Rational
// GetPresentationTimestamp returns the pts or the best effort pts if pts is not available
TimeSpan pts = frame.GetPresentationTimestamp() * frame.TimeBase;// Conversion from one timebase into another
dstFrame.PresentationTimestamp = dstFrame.TimeBase.Rescale(srcFrame.GetPresentationTimestamp(), srcFrame.TimeBase);Represents a codec in FFmpeg. Provides access to core codec information, supported formats, and allows managed interaction with codec details.
| Property | Type | Description |
|---|---|---|
Name |
string |
Short, symbolic name of the codec (e.g., "h264"). |
LongName |
string |
Human-readable descriptive name of the codec. |
MediaType |
MediaType |
Type of media handled by the codec (video, audio, subtitle). |
CodecID |
CodecID |
FFmpeg codec identifier. |
SupportedFramerates |
ReadOnlySpan<Rational> |
Array of supported frame rates (video codecs). |
SupportedPixelFormats |
ReadOnlySpan<PixelFormat> |
Array of supported pixel formats (video codecs). |
SupportedSampleRates |
ReadOnlySpan<int> |
Array of supported audio sample rates. |
SupportedSampleFormats |
ReadOnlySpan<SampleFormat> |
Array of supported audio sample formats. |
| Method | Description |
|---|---|
GetBestPixelFormat(PixelFormat src, bool alphaUsed, out FFLoss loss) |
Finds the best pixel format for a source format, considering alpha usage. |
GetBestPixelFormat(PixelFormat src) |
Overload ignoring loss and assuming alpha used. |
static ReadOnlyCollection<Codec> GetAllCodecs() |
Retrieves all registered codecs (encoders and decoders). |
static Codec? FindDecoder(CodecID codecID) |
Finds a decoder by codec ID. |
static Codec? FindDecoder(string name) |
Finds a decoder by name. |
static Codec? FindEncoder(CodecID codecID) |
Finds an encoder by codec ID. |
static Codec? FindEncoder(string name) |
Finds an encoder by name. |
// Find a decoder by codec ID
Codec? codec = Codec.FindDecoder(CodecID.H264);
if (codec is not null)
{
Console.WriteLine($"Decoder: {codec?.Name} ({codec?.LongName})");
Console.WriteLine($"MediaType: {codec?.MediaType}, CodecID: {codec?.CodecID}");
// Best pixel format for a given source
FFLoss loss = default;
PixelFormat best = codec?.GetBestPixelFormat(PixelFormat.YUV420P, alphaUsed: false, out loss) ?? PixelFormat.None;
if (best != PixelFormat.None)
Console.WriteLine($"Best PixelFormat: {best}, Loss: {loss}");
}
// Enumerate all codecs
var allCodecs = Codec.GetAllCodecs();The PixelFormat enum defines all supported pixel formats within FFmpeg.
It describes how pixel data is stored and interpreted (e.g., RGB, YUV, planar, packed, endianness, alpha channels, etc.).
The FFmpeg.Images.PixelFormatExtensions static class provides helper methods to simplify working with pixel formats.
| Method | Description | Returns |
|---|---|---|
SwapEndianness() |
Swaps the byte order (endianness) of the pixel format (e.g., RGB24BE ↔ RGB24LE). |
PixelFormat |
PlaneCount() |
Returns the number of image planes for the given pixel format (e.g., 1 for RGB, 3 for YUV). | int |
FindBestPixelFormat(ReadOnlySpan<PixelFormat>) |
Finds the best matching pixel format from a list of candidates. | PixelFormat |
FindBestPixelFormat(bool useAlpha, ReadOnlySpan<PixelFormat>) |
Finds the best matching format considering alpha transparency. | PixelFormat |
FindBestPixelFormat(out FFLoss loss, ReadOnlySpan<PixelFormat>) |
Finds the best matching format and reports conversion loss. | PixelFormat |
FindBestPixelFormat(bool useAlpha, out FFLoss loss, ReadOnlySpan<PixelFormat>) |
Finds the best matching format considering alpha transparency and reports conversion loss. | PixelFormat |
The SampleFormat enum defines all supported audio sample formats within FFmpeg.
It describes how audio samples are stored — integer vs. floating-point, planar vs. packed, bit depth, etc.
The FFmpeg.Audio.SampleExtensions static class provides helper methods for inspecting, converting, and validating audio sample formats.
| Method | Description | Returns |
|---|---|---|
IsPlanar() |
Determines whether the sample format stores audio data in planar layout (one buffer per channel). | bool |
IsPacked() |
Determines whether the sample format stores audio data in packed/interleaved layout. | bool |
AsPlanar() |
Converts the current sample format to its planar equivalent (e.g., Int16 → Int16Planar). |
SampleFormat |
AsPacked() |
Converts the current sample format to its packed equivalent (e.g., Float32Planar → Float32). |
SampleFormat |
GetBytesPerSample() |
Returns the number of bytes per individual sample. | int |
GetBitsPerSample() |
Returns the number of bits per individual sample. | int |
GetBitsPerSample(AutoGen._AVCodecID) |
Returns the number of bits per sample for a given FFmpeg codec ID. | int |
GetName() |
Returns the FFmpeg string name for the sample format (e.g., "s16p", "fltp"). |
string |
ValidateType<T>() |
Ensures that a given unmanaged .NET type matches the sample format (throws if not). | void |
GetSampleFormatType() |
Returns the .NET type (byte, short, float, etc.) corresponding to the sample format. |
Type |
The DiscardFlags enum specifies which packets of an AVStream should be discarded during decoding or processing.
var mediaSource = MediaSource.Open(file);
int videoIndex = mediaSource.FindBestStream(MediaType.Video);
// Discard all packets from every stream
foreach (var stream in mediaSource.Streams)
stream.Discard = DiscardFlags.All;
// Keep only the main video stream active
mediaSource.Streams[videoIndex].Discard = DiscardFlags.Default;The DeviceType enum specifies the hardware device types available for hardware‑accelerated decoding in FFmpeg.
| DeviceType | Description |
|---|---|
None |
No hardware acceleration device. |
VDPAU |
Video Decode and Presentation API for Unix — Linux hardware decoding. |
CUDA |
NVIDIA GPU hardware-accelerated decoding. |
VAAPI |
Video Acceleration API — Linux/Unix. |
DXVA2 |
Microsoft DirectX Video Acceleration 2 — Windows. |
QSV |
Intel Quick Sync Video — Intel platforms. |
VideoToolbox |
Apple hardware-accelerated video API on macOS/iOS. |
D3D11VA |
Direct3D 11 Video Acceleration — Windows. |
DRM |
Direct Rendering Manager — Linux GPU acceleration. |
OpenCL |
Open Computing Language — cross-platform GPU/CPU. |
MediaCodec |
Android hardware-accelerated video API. |
Vulkan |
Cross-platform graphics & compute API. |
D3D12VA |
Direct3D 12 Video Acceleration — Windows. |
AMF |
AMD Advanced Media Framework — AMD hardware acceleration. |
OHCODEC |
OpenHarmony hardware acceleration. |
- Intended for decoding: choose a
DeviceTypewhen creating a decoder context for hardware acceleration. - Availability depends on OS, GPU, drivers, and FFmpeg build.
- Unsupported devices may fall back to software decoding or throw an error.
- See FFmpeg Wiki: HWAccelIntro
MediaSource video = MediaSource.Open(file, deviceType: DeviceType.Vulkan);Most classes contain unmanaged data and implement IDisposable.
Represents a decoded audio or video frame in memory.
| Method | Description |
|---|---|
static AVFrame.Allocate() |
Allocates a new empty frame. |
static AVFrame.Allocate(PixelFormat format, int width, int height) |
Allocates a video frame with initialized image buffer. |
static AVFrame.Allocate(SampleFormat format, ChannelLayout layout, int samples, int? sampleRate = null) |
Allocates an audio frame with initialized sample buffers. |
static AVFrame.Allocate(SampleFormat format, int channels, int samples, int? sampleRate = null) |
Allocates an audio frame using a default channel layout. |
| Property | Type | Description |
|---|---|---|
Width |
int |
Coded width of the video frame. |
Height |
int |
Coded height of the video frame. |
PixelFormat |
PixelFormat |
Pixel format for video frames. |
SampleFormat |
SampleFormat |
Sample format for audio frames. |
SampleRate |
int |
Audio sample rate in Hz. |
SampleCount |
int |
Number of audio samples per channel. |
ChannelLayout |
ChannelLayout_ref |
Channel layout of the audio frame. |
Data |
ReadOnlySpan<IntPtr> |
Pointers to the frame’s data buffers. |
LineSize |
ref int_array8 |
Stride/line sizes for each buffer. |
PresentationTimestamp |
long |
Presentation timestamp (PTS) in stream time base units. |
TimeBase |
Rational |
Time base for frame timestamps. |
Duration |
long |
Duration of the frame in the same units as PTS. |
| Method | Description |
|---|---|
GetPresentationTimestamp() |
Returns the PTS if available, otherwise best-effort timestamp. |
CreateBuffer(int align = 1) |
Allocates new buffers for the frame. Must set Pixel/Sample format, dimensions or channels first. |
GetData(int index) |
Returns a span of bytes for the specified plane/channel. |
GetBufferSpan(int bufferIndex) |
Returns a span of bytes for the specified buffer. |
Reference(AVFrame src) |
Makes the current frame reference another frame (shallow copy). |
Unreference() |
Releases all buffers and resets the frame for reuse. |
Dispose() |
Disposes the AVFrame. |
// Allocate a video frame
using var frame = AVFrame.Allocate(PixelFormat.RGB24, 1920, 1080);
// Access pixel data
using Span<byte> plane0 = frame.GetData(0);
// Allocate an audio frame
var audioFrame = AVFrame.Allocate(SampleFormat.Float32, 2, 1024, 44100);
// Reference another frame
var clone = AVFrame.Allocate();
clone.Reference(frame);
// Reset frame for reuse
frame.Unreference();Represents a compressed audio or video packet in memory.
| Method | Description |
|---|---|
static AVPacket.Allocate() |
Allocates a new empty packet. |
static AVPacket.Allocate(int size) |
Allocates a packet with a pre-allocated payload. |
| Property | Type | Description |
|---|---|---|
PresentationTimestamp |
long |
Packet PTS in stream time base units. |
PresentationTime |
TimeSpan |
PTS expressed as a TimeSpan using the packet's TimeBase. |
DecompressionTimestamp |
long |
Packet DTS in stream time base units. |
DecompressionTime |
TimeSpan |
DTS expressed as a TimeSpan using the packet's TimeBase. |
Data |
Span<byte> |
Raw packet data. |
Size |
int |
Size of the packet data in bytes. |
StreamIndex |
int |
Index of the stream this packet belongs to. |
Duration |
long |
Duration in stream time base units. |
Position |
long |
Byte position of the packet in the stream. |
TimeBase |
Rational |
Time base for the packet's timestamps. |
| Method | Description |
|---|---|
Unreference() |
Releases the packet and any associated data. |
Clone() |
Creates a new packet referencing the same data (shallow copy). |
Dispose() |
Disposes the AVPacket. |
// Allocate a new packet
using var packet = AVPacket.Allocate();
// Receive a packet from the demuxer
formatContext.ReadFrame(packet);
// Send the packet to the decoder
codecContext.SendPacket(packet);
// Access raw packet data
Span<byte> buffer = packet.Data;
// Clone the packet (shallow copy)
using var clone = packet.Clone();
// Release packet resources
packet.Unreference();Represents a managed wrapper around FFmpeg’s native _AVFormatContext, providing container-level functionality shared by both demuxing and muxing operations.
This class is typically used indirectly through DemuxerContext (input) and MuxerContext (output).
It implements IDisposable because it owns unmanaged FFmpeg resources.
- Wraps the native
_AVFormatContext* - Manages container metadata, stream descriptors, chapters, and format options
- Synchronizes managed stream objects with the native stream array
- Supports binding to a custom
IOContext - Provides format-level option querying via
OptionQueryBase - Handles cleanup of the underlying native context and any associated I/O resources
| Property | Type | Description |
|---|---|---|
Flags |
FormatContextFlags |
Flags applied to the format context. |
StreamCount |
int |
Number of streams present in the container. |
Streams |
IReadOnlyList<AVStream> |
Managed list of stream descriptors synchronized with the native context. |
Url |
string? |
Media URL or file path associated with the context, if available. |
Metadata |
AVDictionary_ref |
Container-level metadata. |
StartTimeRealTime |
DateTime? |
Real-time clock value of the first packet, when provided by the container. |
Chapters |
ChapterList |
List of chapters defined in the container. |
| Method | Description |
|---|---|
FindBestStream(MediaType type) |
Returns the index of the best-matching stream for the specified media type. |
SetContext(IOContext context, IOOptions options, int bufferSize) |
Associates a custom I/O context and configures it for reading or writing. |
GetOutputTimestamp(int streamIndex, out TimeSpan timestamp, out TimeSpan wallClock) |
Retrieves FFmpeg’s computed output timestamps for muxing operations. |
Dispose() |
Releases the underlying _AVFormatContext* and any associated resources. |
- This class is not intended for direct allocation or opening of media files.
UseDemuxerContextfor reading andMuxerContextfor writing. - It functions as the core container representation used internally by both workflows.
Represents an output-oriented wrapper around FFmpeg’s _AVFormatContext, used for creating and writing container formats such as MP4, MKV, MP3, TS, and others.
MuxerContext is responsible for allocating output contexts, configuring streams, writing headers, writing packets, and finalizing the output file.
This class inherits from FormatContext and extends it with muxing-specific functionality.
- Allocates and configures output format contexts
- Manages output streams and associated codec parameters
- Handles file or custom I/O binding for writing
- Writes headers, packets, trailers, and flushes buffers
- Ensures proper disposal of unmanaged FFmpeg structures
| Property | Type | Description |
|---|---|---|
OutputFormat |
OutputFormat? |
The format used for muxing, derived from the underlying _AVFormatContext.oformat. |
| Method | Description |
|---|---|
Open(string? filename, OutputFormat? format) |
Creates a muxer context using a filesystem output target. May also allocate an FFmpeg-managed file handle. |
Open(Stream stream, OutputFormat format, bool closeOnDispose = true) |
Opens a muxer context using a managed .NET stream. |
Open(IOContext context, OutputFormat format) |
Opens a muxer context using a custom I/O implementation. |
| Method | Description |
|---|---|
AddStream(Codec codec) |
Creates a new stream using the given codec. Initializes codec parameters (ID, type). |
AddStream(Codec codec, ICodecParameters parameters) |
Creates a new stream and copies codec parameters into the stream. |
AddStream(CodecContext codec) |
Creates a stream from an existing initialized codec context. Copies codec parameters and uses its time base. |
| Method | Description |
|---|---|
WriteHeader() |
Writes the container header using default options. |
WriteHeader(AVDictionary dict) |
Writes the header using a single dictionary of muxing options. |
WriteHeader(AVMultiDictionary dict) |
Writes the header using multiple option dictionaries. |
WriteHeader(IDictionary<string,string> dict) |
Writes the header and returns updated options after FFmpeg processing. |
| Method | Description |
|---|---|
WriteFrame(IPacket? packet) |
Writes a packet directly without interleaving. |
WriteFrameInterleaved(IPacket? packet) |
Writes a packet with automatic interleaving. |
WritePacket(AVPacket? packet) |
Writes a packet directly without interleaving. |
WritePacketInterleaved(AVPacket? packet) |
Writes a packet with automatic interleaving. |
| Method | Description |
|---|---|
WriteTrailer() |
Writes the container trailer and final metadata. |
Flush() |
Flushes the underlying I/O context if present. |
| Member | Description |
|---|---|
ToString() |
Returns the long name of the output format or "Unknown" when unavailable. |
using MuxerContext context = MuxerContext.Open("output.mp4",null!)!;
using CodecContext ctx = CodecContext.Open(Codec.FindEncoder(CodecID.H264)!.Value);
// ... Set codec parameters (ctx);
// adds a stream using the codec context and copies the codec parameters
// its advised to set the codec parameters manually instead of copieng during remuxing,
// since they might not be valid between Decoder and Encoder
var videoStream = context.AddStream(ctx);
videoStream.TimeBase = ctx.TimeBase = new Rational(1, 30); // 30 fps
var audioStream = context.AddStream(Codec.FindEncoder(CodecID.AAC)!.Value);
audioStream.TimeBase = new Rational(1, 48000); // 48 kHz
var codecParameters = audioStream.CodecParameters; // its a struct, VS, does not like changing Properties that are structs.
codecParameters.SampleRate = 48000;
codecParameters.ChannelLayout = ChannelLayout.CreateStereo();
codecParameters.SampleFormat = SampleFormat.Float32Planar;
audioStream.SetOption("key", "value"); // example to set stream option
context.WriteHeader();
// ... Encode and write frames
while (...)
{
AVPacket packet = GetSomePacketFormSomeEncoder();
packet.StreamIndex = videoStream.Index; // or audioStream.Index
context.WriteFrameInterleaved(packet);
}
context.WriteTrailer();
context.Dispose(); // or use 'using' statementRepresents an input-oriented wrapper around FFmpeg’s _AVFormatContext used for reading container formats such as MP4, MKV, MP3, TS, and others.
This class inherits from FormatContext and extends it with demuxing-specific functionality.
| Constructor | Description |
|---|---|
protected DemuxerContext(AutoGen._AVFormatContext* context) |
Initializes a new instance of DemuxerContext with an existing native _AVFormatContext. Ownership of the context depends on internal management. |
| Property | Type | Description |
|---|---|---|
InputFormat |
InputFormat? |
Gets the input format associated with this context, if available. |
StartTime |
long |
Start time of the media file in AV_TIME_BASE units. |
Duration |
long |
Duration of the media file in AV_TIME_BASE units. |
BitRate |
long |
Overall bit rate of the media file in bits per second. |
Chapters |
ChapterList |
List of chapters in the media file. Constructed from the current context. |
Opens an input media source and initializes a DemuxerContext. Overloads exist depending on the type of source:
| Source Type | Input Format (InputFormat?) |
Options (IDictionary<string,string> or derived) |
Find Stream Info (bool) |
Notes |
|---|---|---|---|---|
File path (string) |
Optional | Optional | Optional | Opens input from a file or URL. |
Stream (Stream) |
Optional | Optional | Optional | Stream must support reading and seeking. |
IOContext (IOContext) |
Optional | Optional | Optional | IOContext handles low-level read/write operations; may support seeking. |
Parameter notes:
- Input Format: Required for non-file-based sources or to override automatic detection.
- Options: General key/value dictionary controlling input behavior; single or multi-dictionary supported internally.
- Find Stream Info: If
true, callsavformat_find_stream_infoafter opening to populate stream metadata. FFmpeg might read a few frames to set the AVStream Properties to the correct values.
All overloads internally handle dictionary conversions and FFmpeg context initialization.
| Method | Description |
|---|---|
AVResult32 FindStreamInfo(...) |
Finds stream information from the media file. |
Rational GuessFrameRate(...) |
Guesses the frame rate of a stream using an optional frame. |
AVResult32 Seek(Rational time, int streamIndex) |
Seeks to a specific time in a given stream. Throws if index out of range. |
AVResult32 Seek(long timestamp, int streamIndex) |
Seeks to a specific timestamp in a given stream. |
AVResult32 ReadFrame(AVPacket packet) |
Reads a frame from the input media. Sets packet time base if not already set. |
AVResult32 ReadPacket(AVPacket packet) |
Reads a frame from the input media. Sets packet time base if not already set. |
Dispose() |
Disposes the context and underlying resources. |
TimeSpan seek = TimeSpan.FromSeconds(10);
using var demuxerContext = DemuxerContext.Open(videoFile, true); // searchStreamInfo = true
int videoIndex = context.FindBestStream(MediaType.Video); // find Best Video Stream
// Discard all other streams
foreach (var stream in context.Streams)
stream.Discard = DiscardFlags.All;
context.Streams[index].Discard = DiscardFlags.Default;
// Open Decoder, with the correct parameters
Codec c = Codec.FindDecoder(demuxerContext.Streams[videoIndex].CodecParameters.CodecId)!.Value;
using CodecContext decoder = CodecContext.Open(c, demuxerContext.Streams[videoIndex].CodecParameters);
decoder.TimeBase = input.Streams[videoIndex].TimeBase; // set codec TimeBase just to be sure
using AVFrame frame = AVFrame.Allocate();
using var packet = AVPacket.Allocate();
var result = input.Seek(seek, videoIndex);
if (result.IsError)
return result;
// decoder.FlushBuffers(); // we seeked so flush the codecs internal buffers if something was written to it
do
{
do
{
result = input.ReadFrame(packet);
if (result.IsError)
return result;
if(packet.Flags.HasFlag(PacketFlags.Discard) || packet.StreamIndex != videoIndex)
{
result = AVResult32.TryAgain;
continue;
}
result = decoder.SendPacket(packet);
if (result.IsError)
return result;
result = decoder.ReceiveFrame(frame);
} while (result.IsTryAgain);
} while (!result.IsError && (frame.GetPresentationTimestamp()+frame.Duration) * frame.TimeBase < seek);
return frame;