Framefilters

Framefilters are the building blocks of pipelines. The base class Limef::ff::FrameFilter defines the go() / cc() / pass() interface. Use Limef::ff::SimpleFrameFilter for one-to-one filters and Limef::ff::SplitFrameFilter for one-to-many forks.

namespace ff

Framefilters.

Typedefs

using json = nlohmann::json

Enums

enum class ChannelOrder

Output channel order for colour conversions.

Values:

enumerator RGB

Red-Green-Blue (PyTorch / PIL convention)

enumerator BGR

Blue-Green-Red (OpenCV convention)

Functions

static size_t serialize_frame_wrapper(uint8_t *buffer, size_t buffer_size, Frame *frame)

class AnnexBFrameFilter : public Limef::ff::SimpleFrameFilter

#include <annexb.h>

FrameFilter that converts H.264/H.265 video from AVCC to Annex B format.

Activates the h264_mp4toannexb / hevc_mp4toannexb BSF when the incoming stream is in AVCC (length-prefixed) format, i.e. extradata[0] != 0x00. Typical sources: demuxed MP4 files, RTSP streams from hardware cameras.

NVENC always outputs Annex B natively, regardless of AV_CODEC_FLAG_GLOBAL_HEADER, so this filter is a no-op for NVENC-encoded streams.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

struct AudioTranscodeContext

#include <audiotranscode.h>

A Context variable that defines the audio codecs and their specific requirements.

NOTE: Only supports planar audio channel formats.

Public Functions

inline AudioTranscodeContext(AVCodecID codec, int bitrate, AVSampleFormat sample_format, int sample_size, const std::vector<int> &flags, const std::map<std::string, std::string> &opts = {})

Constructor with options.

Parameters:

• codec – Audio codec ID

• bitrate – Target bitrate

• sample_format – Target sample format (AVSampleFormat). NOTE: must be a planar format.

• calc – Sample calculation method

• flags – Codec flags

• opts – Dictionary options for encoder

Public Members

AVCodecID codec_id = {AV_CODEC_ID_AAC}

AV_CODEC_ID of the audio codec. Use to define the target codec.

int bit_rate = {44100}

Default / good bitrate for the codec. Feel free to change.

AVSampleFormat sample_format = {AV_SAMPLE_FMT_FLTP}

The default planar format for AAC.

int sample_size = {1024}

Sample size for the encoder.

const std::vector<int> codec_flags

Just store raw FFmpeg flags.

std::map<std::string, std::string> options

New field: dictionary options for the encoder.

Public Static Functions

static inline AudioTranscodeContext AAC()

Advanced Audio Codec (AAC)

static inline AudioTranscodeContext AAC(const std::map<std::string, std::string> &opts)

AAC with custom options.

Parameters:

opts – Dictionary options to override defaults

Returns:

AudioTranscodeContext

class AudioTranscodeFrameFilter : public Limef::ff::SimpleFrameFilter

#include <audiotranscode.h>

FrameFilter that transcodes audio to a different codec.

Takes a CodecFrame first to get stream information, then accepts audio PacketFrames, transcodes them and outputs new PacketFrames with the desired audio codec (target codec)

If the incoming codec matches the target codec, then passthrough is enabled and audio packets are passed downstream as-is

Param name:

Name for this framefilter

Param ctx:

A Context that defines the audio codec and it’s requirements

Param next:

(optional) next framefilter in the chain

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class CodecAssertFrameFilter : public Limef::ff::SimpleFrameFilter

#include <codecassert.h>

FrameFilter that asserts the incoming stream matches an allowed codec pair.

On the first CodecFrame it inspects the video and audio codec IDs and checks them against a caller-supplied list of CodecPair values. If no pair matches the filter logs a fatal error and inutilizes itself: all subsequent frames (including the non-matching CodecFrame) are silently dropped.

When a pair matches, the CodecFrame and all following frames are forwarded downstream normally.

Public Functions

explicit CodecAssertFrameFilter(std::string name, std::vector<CodecPair> allowed_pairs)

Parameters:

• name – Label for logging.

• allowed_pairs – List of codec pairs that are considered valid. AV_CODEC_ID_NONE in either slot means “don’t check that track type”.

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class CodecCacheFrameFilter : public Limef::ff::SimpleFrameFilter

#include <codec_cache.h>

FrameFilter that caches and can resend CodecFrames.

Acts as a passthrough filter for all frames, but when it encounters a CodecFrame, it makes an internal copy that can be resent later. This is useful when a new client/component needs to receive codec parameters mid-stream without waiting for a new CodecFrame to arrive naturally.

Thread-safe: can be called from multiple threads.

Example usage:

// In setup code
CodecCacheFrameFilter codec_cache("codec_cache");
source.getOutput().cc(codec_cache).cc(next_filter);

// ... later when a new component joins
codec_cache.reSend(); // Resends cached CodecFrame to next_filter

Public Functions

inline explicit CodecCacheFrameFilter(std::string name = "CodecCacheFrameFilter")

Construct a new Codec Cache Frame Filter.

Parameters:

• name – Unique name for this filter

• next – Optional pointer to the next filter in chain

~CodecCacheFrameFilter() override = default

Destroy the Codec Cache Frame Filter.

Cleans up any cached codec frame.

inline virtual void go(const Limef::frame::Frame *frame) override

Process incoming frame.

Passes all frames downstream. For CodecFrames, also maintains an internal copy.

Parameters:

frame – The frame to process

inline bool reSend()

Resend the cached CodecFrame downstream.

If a CodecFrame has been cached, sends a copy downstream. Thread-safe: can be called from any thread.

Returns:

true if a CodecFrame was sent

Returns:

false if no CodecFrame is available to send

inline void reset()

Reset the filter state.

Clears any cached CodecFrame. After calling this method, hasCodecFrame() will return false and reSend() will not send anything until a new CodecFrame is received.

Thread-safe: can be called from any thread.

inline bool hasCodecFrame() const

Check if a CodecFrame has been cached.

Thread-safe method to check if a CodecFrame is available.

Returns:

true if a CodecFrame has been cached

Returns:

false if no CodecFrame is cached

class CodecMessageFrameFilter : public Limef::ff::SimpleFrameFilter

#include <codec_message.h>

FrameFilter that sends a JSON message through a pipe when it encounters a CodecFrame.

Passes all frames through to the next filter unchanged, but when it encounters a CodecFrame, it sends a JSON message through the provided pipe for external monitoring.

Thread-safe: can be used from multiple threads.

Example usage:

// Create a pipe for receiving messages
auto [sender_pipe, receiver_pipe] = Limef::comm::Pipe::create();

// Create filter
CodecMessageFrameFilter message_filter("codec_monitor", &sender_pipe);

// Connect to pipeline
source.getOutput().cc(message_filter).cc(next_filter);

// In another thread or context, monitor for codec messages
std::string json_str;
if (receiver_pipe.recv(json_str)) {
    auto msg = json::parse(json_str);
    std::cout << "New codec: video=" << msg["has_video"]
              << ", audio=" << msg["has_audio"] << std::endl;
}

Public Functions

inline explicit CodecMessageFrameFilter(std::string name = "CodecMessageFrameFilter", Limef::comm::Pipe *pipe = nullptr)

Construct a new Codec Message Frame Filter.

Parameters:

• name – Unique name for this filter

• pipe – Pipe to send codec messages through

• next – Optional pointer to the next filter in chain

inline void setPipe(Limef::comm::Pipe *pipe)

Set the pipe to send messages through.

Parameters:

pipe – Pipe for sending messages

inline virtual void go(const Limef::frame::Frame *frame) override

Process incoming frame.

Passes all frames downstream. For CodecFrames, sends a JSON message through the pipe.

Parameters:

frame – The frame to process

struct CodecPair

#include <codecassert.h>

A (video, audio) codec pair used by CodecAssertFrameFilter.

Use AV_CODEC_ID_NONE for either field to mean “don’t check this track type”. If the source has no stream of that type the field is treated as AV_CODEC_ID_NONE.

Examples:

CodecPair{AV_CODEC_ID_H264, AV_CODEC_ID_AAC}   // H264 video + AAC audio
CodecPair{AV_CODEC_ID_H264, AV_CODEC_ID_NONE}  // H264 video, any/no audio
CodecPair{AV_CODEC_ID_NONE, AV_CODEC_ID_OPUS}  // any/no video, Opus audio

class ComposeFrameFilter : public Limef::ff::FrameFilter

#include <compose.h>

Base class for composite framechain.

Allows creating complex framefilters by composing simpler ones. Subclasses should create and chain their specific filters in their constructor, for example:

class FMP4FrameFilter : public ComposeFrameFilter {
public:
    FMP4FrameFilter(const AudioParams& params) : ComposeFrameFilter("FMP4") {
        // Chain them together
        this.chain = {
             new AudioDecoder(params),
             new Muxer()
         };
    }
};

NOTE: ComposeFrameFilter will delete the framefilters you have created in-place in the vector initialization at dtor time, so no need to worry about them.

Subclassed by Limef::ff::FMP4FrameFilter, Limef::ff::RTSPMuxerFrameFilter, Limef::ff::WebMFrameFilter, Limef::ff::WebRTCMuxerFrameFilter

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Go is called by the upstream framefilter.

In this case, simply forward to first internal framefilter

inline virtual FrameFilter &cc(FrameFilter &next_) override

In this case, chain internal last filter to the next_

inline virtual void clear() override

Call clear() of the internal last framefilter

class CountDecodedFrameFilter : public Limef::ff::SimpleFrameFilter

#include <count.h>

Count DecodedFrames and report count + fps on demand.

All frames are passed downstream unchanged. Only DecodedFrames (FrameClass::Decoded) are counted. Timing starts on the first DecodedFrame seen.

report() and getCount() are safe to call from any thread.

Example:

CountDecodedFrameFilter counter("counter");
camera.getOutput().cc(counter);
// ... run for a while ...
counter.report();   // logs: "counter: 150 frames in 10.0s → 15.0 fps"

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

inline void report() const

Log frame count and mean fps since the first DecodedFrame.

inline long getCount() const

Return number of DecodedFrames received so far.

inline long sinceLastFrame() const

Milliseconds since the last DecodedFrame; -1 if none has arrived yet.

class CountTensorFrameFilter : public Limef::ff::SimpleFrameFilter

#include <count.h>

Count TensorFrames and report count + fps on demand.

All frames are passed downstream unchanged. Only TensorFrames (FrameClass::Tensor) are counted. Timing starts on the first TensorFrame seen.

report() and getCount() are safe to call from any thread.

Example:

CountTensorFrameFilter counter("counter");
camera.getOutput().cc(counter);
// ... run for a while ...
counter.report();   // logs: "counter: 150 frames in 10.0s → 15.0 fps"

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

inline void report() const

Log frame count and mean fps since the first TensorFrame.

inline long getCount() const

Return number of TensorFrames received so far.

inline long sinceLastFrame() const

Milliseconds since the last TensorFrame; -1 if none has arrived yet.

class CpuSwScaleConverter : public Limef::ff::DecodedToTensorConverter

#include <decoded_to_tensor_cpu.h>

Converts any CPU DecodedFrame → TensorFrame (c, h, w) UInt8 via libswscale.

Colour input → GBRP intermediate → CHW(3, H, W) via permuted dst[] pointers (zero extra copy). Greyscale input → GRAY8 → CHW(1, H, W) directly.

ChannelOrder::RGB → channels [R, G, B] in slots [0, 1, 2]. ChannelOrder::BGR → channels [B, G, R] in slots [0, 1, 2].

Public Functions

inline virtual bool accepts(const frame::DecodedFrame &df) const override

Return true if this converter handles df (may inspect format and location).

virtual bool convert(const frame::DecodedFrame &src, frame::TensorFrame &dst, ChannelOrder order) override

Convert src into dst.

dst is the filter’s internal TensorFrame (grow-only, reused across calls). After a successful call, dst.planes[0] has shape (c, h, w), dtype UInt8.

Parameters:

• src – Incoming video DecodedFrame (isVideo() guaranteed by caller).

• dst – Output TensorFrame to fill (grow-only).

• order – Desired output channel order.

Returns:

true on success; false to signal a conversion error (frame is dropped).

inline virtual const char *converterName() const override

Short name used in log messages.

class CUDAScaleFrameFilter : public Limef::ff::SimpleFrameFilter

#include <cuda_scale_frame.h>

Scales NV12 CUDA DecodedFrames to a target resolution on the GPU.

Uses bilinear interpolation. Y and UV planes are scaled independently, preserving the NV12 semi-planar layout (interleaved UV).

Output is always BufferLocation::CUDA (plain cudaMalloc, no hw_frames_ctx), regardless of whether the input was CUDA or CUDA_FFMPEG.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

struct CUDAScaleParams

#include <cuda_scale_frame.h>

Configuration for CUDAScaleFrameFilter.

Public Members

int dst_width = {0}

Target width in pixels; 0 = keep source width.

int dst_height = {0}

Target height in pixels; 0 = keep source height

AVPixelFormat output_format = {AV_PIX_FMT_NV12}

Output pixel format. AV_PIX_FMT_NV12 (default) preserves NV12 layout. AV_PIX_FMT_YUV420P scales AND de-interleaves the UV plane in one GPU pass, producing a CUDA frame with planar U/V ready for DecodedDownloadFrameFilter + VP8.

class CudaToFFmpegFrameFilter : public Limef::ff::SimpleFrameFilter

#include <cuda_to_ffmpeg.h>

Converts BufferLocation::CUDA frames to BufferLocation::CUDA_FFMPEG.

Allocates an AVHWFramesContext (AV_HWDEVICE_TYPE_CUDA, NV12, pooled) on the first incoming CUDA frame and copies each frame into the pool via D2D cudaMemcpy2D. The output frame has hw_frames_ctx set and can be consumed directly by h264_nvenc / hevc_nvenc / av1_nvenc without an upload step.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class DecodedDownloadFrameFilter : public Limef::ff::SimpleFrameFilter

#include <decoded_transfer.h>

Downloads CUDA/CUDA_FFMPEG DecodedFrames to CPU memory (BufferLocation::CPU, NV12).

Grow-only output buffer: reallocated only when a larger frame arrives. Non-CUDA video frames and audio frames are passed through unchanged.

For TensorFrame transfers see tensor_transfer.cpp.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class DecodedToTensorConverter

#include <decoded_to_tensor.h>

Abstract base for pixel-format converters used by DecodedToTensorFrameFilter.

Each subclass handles one (or a small group of closely related) AVPixelFormats. Converters are owned by the filter and never exposed to users.

The convert() call must be reentrant-safe for the same object (called once per frame, always from the filter’s driving thread).

Subclassed by Limef::ff::CpuSwScaleConverter

Public Functions

virtual bool accepts(const frame::DecodedFrame &df) const = 0

Return true if this converter handles df (may inspect format and location).

virtual bool convert(const frame::DecodedFrame &src, frame::TensorFrame &dst, ChannelOrder order) = 0

Convert src into dst.

dst is the filter’s internal TensorFrame (grow-only, reused across calls). After a successful call, dst.planes[0] has shape (c, h, w), dtype UInt8.

Parameters:

• src – Incoming video DecodedFrame (isVideo() guaranteed by caller).

• dst – Output TensorFrame to fill (grow-only).

• order – Desired output channel order.

Returns:

true on success; false to signal a conversion error (frame is dropped).

virtual const char *converterName() const = 0

Short name used in log messages.

class DecodedToTensorFrameFilter : public Limef::ff::SimpleFrameFilter

#include <decoded_to_tensor.h>

FrameFilter that converts DecodedFrame → TensorFrame (c, h, w).

Usage:

DumpFrameFilter dump("dump");
DecodedToTensorFrameFilter d2t("d2t", ChannelOrder::RGB);
d2t.cc(dump);
// … drive with decoded frames …

Public Functions

explicit DecodedToTensorFrameFilter(std::string name = "DecodedToTensorFrameFilter", ChannelOrder out_order = ChannelOrder::RGB)

Parameters:

• name – Identifier used in log messages.

• out_order – Desired output channel order (default: RGB).

virtual void go(const frame::Frame *f) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class DecodedUploadFrameFilter : public Limef::ff::SimpleFrameFilter

#include <decoded_transfer.h>

Uploads CPU DecodedFrames to plain CUDA memory (BufferLocation::CUDA).

Allocates cudaMalloc’d NV12 output buffers lazily on the first video frame. Grow-only: the buffer is reallocated only when a larger frame arrives. Non-video frames and frames already on GPU are passed through unchanged.

For TensorFrame transfers see tensor_transfer.cpp.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

struct DecodedUploadParams

#include <decoded_transfer.h>

Configuration for CPU→CUDA upload.

Only BufferLocation::CUDA is supported as target.

class DecodingFrameFilter : public Limef::ff::SimpleFrameFilter

#include <decoding.h>

A FrameFilter that decodes frames.

• Receives Limef::frame::CodecFrames and Limef::frame::PacketFrames

• Produces Limef::frame::DecodedFrames

• Uses internally the Limef::decode::Decoder class

This framefilter uses both Limef::frame::CodecFrames and Limef::frame::PacketFrames that pass through it: the former is used to set up decoders and the latter ones are passed on to the set’ted up decoders as-is.

Output from this framefilter is a Limef::frame::DecodedFrame.

Always when this framefilter receives a new PacketFrame, it feeds it to the decoder and after that drains the decoder, i.e. calls the decoder until all decoded frames are fetched from the decoder and sent downstream.

Supports multiple decoder backends via the factory pattern:

// Default: FFmpeg software decoding
DecodingFrameFilter decode;

// Custom FFmpeg params (more threads, etc.)
FFmpegDecoderParams params;
params.thread_count = 4;
DecodingFrameFilter decode("decode", params);

// V4L2 hardware decoding (when available)
V4L2NVDecoderParams v4l2_params;
v4l2_params.device = "/dev/video0";
DecodingFrameFilter decode("decode", v4l2_params);

Note

For hardware frame lifetime details (when using VAAPI/CUDA decoding), see the “Hardware Frame Lifecycle” section in Limef::decode::Decoder.

Public Functions

explicit DecodingFrameFilter(std::string name = "DecodingFrameFilter", const Limef::decode::VideoDecoderParams &params = Limef::decode::FFmpegDecoderParams{})

Construct a decoding framefilter.

Parameters:

• name – Identifier for logging

• params – Decoder parameters (FFmpegDecoderParams, V4L2NVDecoderParams, etc.)

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class DumpFrameFilter : public Limef::ff::SimpleFrameFilter

#include <dump.h>

Dump info about the Frame

Uses Frame’s dump method to print information about the frame to stdout

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class DumpPacketFrameFilter : public Limef::ff::SimpleFrameFilter

#include <dump_packet.h>

A FrameFilter that dumps information about PacketFrames.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class EmptyFrameFilter : public Limef::ff::SimpleFrameFilter

#include <empty.h>

Dummy FrameFilter

A FrameFilter

that does nothing.

A use case: some object’s internal framefilter that start a framefilter chain

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class EncodeCheckFrameFilter : public Limef::ff::SimpleFrameFilter

#include <encode_check.h>

Validates stream codec types and controls downstream frame flow.

Waits for initial CodecFrame and checks if its codecs match allowed lists. If match, passes all frames downstream. If no match, blocks further frames and sends warning InfoFrame.

Public Functions

EncodeCheckFrameFilter(std::string name = "EncodeCheckFilter", const std::vector<AVCodecID> &allowed_video_codecs = {}, const std::vector<AVCodecID> &allowed_audio_codecs = {}, std::string warning_message = "Unsupported codec detected")

Construct filter with allowed codec lists.

Parameters:

• name – Filter name

• allowed_video_codecs – List of allowed video codec IDs

• allowed_audio_codecs – List of allowed audio codec IDs

• warning_message – Message for log and InfoFrame on codec mismatch

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class EncodingFrameFilter : public Limef::ff::SimpleFrameFilter

#include <encoding.h>

A FrameFilter that encodes frames.

• Receives Limef::frame::StreamFrames and Limef::frame::DecodedFrames

• Produces Limef::frame::CodecFrame (once, after encoder init) and Limef::frame::PacketFrames

• Uses internally the Limef::encode::Encoder class

This framefilter uses both Limef::frame::StreamFrames and Limef::frame::DecodedFrames that pass through it: the former is used to configure encoder timing (fps, timebase) and the latter ones are encoded into packets.

Output sequence:

1. When encoder initializes (after first DecodedFrame with dimensions):

   • Sends CodecFrame downstream with encoder’s codec parameters

   • This allows downstream decoders to initialize

2. For each encoded packet:

   • Sends PacketFrame downstream

Has separate encoders for video and audio - routes DecodedFrames to the correct encoder based on their MediaType.

Always when this framefilter receives a new DecodedFrame, it feeds it to the appropriate encoder and after that drains the encoder, i.e. calls the encoder until all encoded packets are fetched and sent downstream.

Type checking:

• If a CodecFrame (encoded stream info) is received instead of StreamFrame, a warning is logged - this likely indicates misrouted frames.

Public Functions

explicit EncodingFrameFilter(std::string name = "EncodingFrameFilter", const Limef::encode::VideoEncoderParams &video_params = Limef::encode::FFmpegEncoderParams{}, const Limef::encode::VideoEncoderParams *audio_params = nullptr)

Construct an encoding framefilter.

Example:

// Using FFmpeg encoder (default)
FFmpegEncoderParams ffmpeg_params;
ffmpeg_params.codec_id = AV_CODEC_ID_H264;
EncodingFrameFilter enc("encoder", ffmpeg_params);

// Using V4L2 encoder (when available)
V4L2EncoderParams v4l2_params;
v4l2_params.device = "/dev/video2";
EncodingFrameFilter enc("encoder", v4l2_params);

Parameters:

• name – Filter name for logging

• video_params – Encoding parameters for video (variant of FFmpegEncoderParams, V4L2EncoderParams)

• audio_params – Encoding parameters for audio (optional, nullptr to disable)

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

void requestKeyFrame()

Request a keyframe from the video encoder on its next output.

Rare exception to the “no public setters on framefilters” rule (see CLAUDE.md, “Thread Safety: FrameFilter Chains”): safe to call from any thread (e.g. a WebRTC PLI callback) because it’s mutex-protected against the driving thread’s own use of video_encoder_ in go()/ setupEncoders() (which may delete/recreate the encoder concurrently).

No-op if there is no video encoder yet (e.g. before the first StreamFrame/DecodedFrame arrives).

class FMP4FrameFilter : public Limef::ff::ComposeFrameFilter

#include <fmp4.h>

Composite filter that muxes to fragmented MP4 for browser MediaSource Extensions.

Creates a chain of:

1. CodecAssertFrameFilter — rejects unsupported codec pairs (see table below)

2. GateFrameFilter — toggles the stream on/off; call open() to start

3. MuxerFrameFilter — muxes packets into fMP4 boxes

4. MP4PartitionFrameFilter — splits the muxed stream into ftyp/moov/moof/mdat RawFrames

Audio transcoding is not performed here — the upstream pipeline is responsible for providing the correct codec pair.

Public Functions

inline void open()

Call internal Limef:ff::SwithFrameFilter’s open method.

inline void close()

Call internal Limef:ff::SwithFrameFilter’s close method.

template<typename T>
class FrameFifoFrameFilter : public Limef::ff::SimpleFrameFilter

#include <framefifo_.h>

Feeding FrameFifo

A FrameFilter that feeds a FrameFifo

Param FrameFifo:

: the FrameFifo this framefilter is feeding

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class FrameFilter

#include <framefilter.h>

Subclassed by Limef::ff::ComposeFrameFilter, Limef::ff::SimpleFrameFilter, Limef::ff::SplitFrameFilter, Limef::ff::SwitchFrameFilter

Public Functions

virtual void clear() = 0

Removes framefilter downstream connection.

virtual FrameFilter &cc(FrameFilter &next) = 0

Register next framefilter in the chain.

This method does any necessary initialization. Framefilter is not fully functional until this method has been called. Should call init.

inline const std::string &getName() const

Initialize framefilter.

Get the name of this framefilter

virtual void go(const Limef::frame::Frame *frame) = 0

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class GateFrameFilter : public Limef::ff::SimpleFrameFilter

#include <gate.h>

Frame filter that can Gate packet stream on/off with timestamp correction.

Handles CodecFrames and PacketFrames with special logic, passes all other frame types through:

• Caches received CodecFrame(s) for later use whether the gate is open or closed (always listening for CodecFrames)

• Doesn’t pass PacketFrames by default (closed state). All other packets are passed through.

• When opened:

  • If has a cached CodecFrame: sends it downstream and starts passing PacketFrames if conditions are met (see below)

  • If no CodecFrame yet: waits for it, then sends it and starts passing packets if conditions are met (see below)

• When video stream present:

  • Waits for keyframe before passing any packets

  • After keyframe, passes all packets

• When only audio stream:

  • Passes packets immediately after opening (no keyframe concept)

• When closed:

  • Stops passing packets

  • Maintains CodecFrame cache .. keeps listening/caching CodecFrame(s) and passing all other packets than PacketFrames downstream

Thread-safe: can be opened/closed from any thread.

Public Functions

explicit GateFrameFilter(std::string name = "GateFrameFilter")

Construct a new GateFrameFilter.

Parameters:

• name – Identifier for this filter

• next – Next filter in chain

void cleanup()

Resets state.

virtual void go(const Limef::frame::Frame *frame) override

Process incoming frames.

For CodecFrame:

• Cache for later use

For PacketFrame when open:

• Adjust timestamps to start from zero

• For video, wait for keyframe before passing

• For audio, pass immediately

All other frame types pass through.

Parameters:

frame – Input frame

void open()

Open the Gate and start passing packets.

If we have CodecFrame:

• Send it downstream

• Start passing PacketFrame(s) (after keyframe for video)

If no CodecFrame:

• Wait for CodecFframe to appear before taking any other action

• Meanwhile pass downstream all other than PacketFrame(s)

void close()

Close the Gate and stop passing packets.

Resets internal state:

• Stops passing packets

• Maintains CodecFrame cache

class H264StripParamSetsFrameFilter : public Limef::ff::SimpleFrameFilter

#include <h264_strip_param_sets.h>

Strips SPS (NAL type 7) and PPS (NAL type 8) from H.264 Annex B packets after the first occurrence, preventing NVDEC from firing repeated V4L2_EVENT_SOURCE_CHANGE events on every GOP boundary.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class InfoFrameFilter : public Limef::ff::SimpleFrameFilter

#include <info.h>

Captures InfoFrames and stores their messages.

Passes all other frames downstream but captures InfoFrames. Messages can be retrieved later using popMessage().

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

bool hasMessage() const

Check if there are messages available.

Returns:

true if messages in queue

std::string popMessage()

Get oldest message from queue.

Returns:

Message string or empty string if none available

class MP4PartitionFrameFilter : public Limef::ff::SimpleFrameFilter

#include <mp4part.h>

Repartitions fragmented MP4 stream into proper box-aligned chunks.

Input: Raw byte blobs from FFmpeg muxer that may contain:

• Partial MP4 boxes

• Multiple MP4 boxes

• Parts of multiple MP4 boxes

Output: Clean-cut MP4 boxes as individual frames

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class MP4ProbeFrameFilter : public Limef::ff::SimpleFrameFilter

#include <mp4_probe.h>

Analyzes and logs fragmented MP4 boxes as they pass through.

Outputs box information in format: <BOX_TYPE/size=N> for incomplete boxes <BOX_TYPE/size=N/complete> for complete boxes

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class MuteAudioFrameFilter : public Limef::ff::SimpleFrameFilter

#include <muteaudio.h>

Drops all audio from the stream.

CodecFrame: copies the frame but removes all audio streams before passing downstream. PacketFrame with audio media type: silently dropped. All other frame types: passed through unchanged.

Use this upstream of WebRTCMuxerFrameFilter when the file/RTSP source carries an audio codec that WebRTC does not accept (e.g. PCM S16LE).

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class MuxerFrameFilter : public Limef::ff::SimpleFrameFilter

#include <muxer.h>

FrameFilter that muxes elementary streams into a container format.

This is a very general muxing framefilter that produces RawFrame(s) that are processed by downstream framefilters and threads as just muxed media stream: just raw bytestream

For more specifically muxed stream, feel free to implement more complementary framefilters, see for example RTPMuxerFrameFilter.

Takes a CodecFrame first to get stream information, then accepts elementary PacketFrames (H.264, AAC, etc.) as input and outputs RawFrames containing muxed data in specified container format.

The first output will contain format headers required for initializing downstream demuxers.

Internals:

AVFormatContext refers to AVIOContext AVIOContext refers to a custom writeCallback

walkthrough: go -> processPacket -> av_interleaved_write_frame -> returns immediately on some instances, when av_interleaved_write_frame is called, writeCallback is activated internally by ffmpeg -> pass

Public Functions

void close()

flush the muxer

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

void setOption(const char *key, const char *value)

Set muxer options using FFmpeg AVDictionary key/value pairs

class MuxingContext

#include <muxer.h>

Parameters for muxing streams into containers.

Public Functions

inline MuxingContext(const std::string &format, const std::map<std::string, std::string> &custom_options)

Create context with custom options

Public Members

const std::string format_name

FFmpeg format name (e.g. “mp4”, “flv”, etc.)

std::map<std::string, std::string> options

Custom muxer options (key-value pairs for AVDictionary)

Public Static Functions

static inline MuxingContext MKV()

Ready-made context for Matroska

static inline MuxingContext FMP4()

Ready-made context for fragmented MP4 A separate fragment for each frame: more balanced fragmentation

static inline MuxingContext FMP4_KEYFRAG()

Ready-made context for fragmented MP4 One fragment per each keyframe - all other frames are included in that same macrofragment

static inline MuxingContext WebM()

Ready-made context for streaming WebM (VP8 / VP9).

cluster_time_limit forces a new Cluster every N milliseconds, giving the browser regular segment boundaries without needing the muxer to seek back and fill in a Cluster size (which is impossible with custom IO). Clusters written this way carry unknown size (0x01FFFFFFFFFFFFFF), which WebMPartitionFrameFilter handles by detecting the next Cluster start.

Audio note: WebM uses Vorbis or Opus, not AAC. If the upstream source carries AAC audio, the muxer will reject it. For video-only VP8/VP9 sources (e.g. USBCameraThread with no audio) this is fine.

class PutInfoFrameFilter : public Limef::ff::SimpleFrameFilter

#include <putinfo.h>

Injects InfoFrames into the pipeline from an external thread.

All frames are passed downstream unchanged. Calling put() from any thread queues a message; the next go() call (on the driving thread) drains the queue and emits an InfoFrame per message before forwarding the regular frame.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

void put(const std::string &msg)

Queue a message for injection as an InfoFrame. Thread-safe — may be called from any thread.

class RTPMuxerFrameFilter : public Limef::ff::SimpleFrameFilter

#include <rtpmuxer.h>

FrameFilter that muxes elementary streams into RTP packets.

Takes a CodecFrame first to get stream information and generate SDP, then accepts PacketFrames and outputs RTPFrames.

Flow:

CodecFrame → RTPMuxerFrameFilter
                ├── SDPFrame (once, downstream)
                └── (waits for packets)

PacketFrame → RTPMuxerFrameFilter
                └── RTPFrame (per RTP/RTCP packet)

Usage:

RTPMuxerFrameFilter rtp_muxer("rtp");
rtp_muxer.cc(downstream);

// Send CodecFrame first - SDPFrame will be passed downstream
rtp_muxer.go(&codec_frame);

// Then send packets - RTPFrames will be passed downstream
rtp_muxer.go(&packet_frame);

The RTP muxer uses FFmpeg’s RTP format. Each track gets its own AVFormatContext because RTP only supports one stream per context. SDP is generated from all tracks using av_sdp_create().

Incoming PacketFrames must already be in the correct bitstream format (e.g. Annex B for H.264/H.265). Use AnnexBFrameFilter upstream if needed.

Public Functions

explicit RTPMuxerFrameFilter(std::string name = "RTPMuxerFrameFilter", int packet_size = 1400)

Parameters:

• name – label for logging

• packet_size – max size in bytes of each muxed RTP packet (default 1400, i.e. a plain-Ethernet path MTU of ~1500 minus IP/UDP headers). Lower this when the packets must additionally survive a tunnel/VPN hop (e.g. Tailscale, MTU 1280) between this muxer and the client, since nothing downstream fragments an oversized RTP/SRTP packet - see WebRTCMuxerFrameFilter.

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

struct RTPTrackContext

#include <rtpmuxer.h>

Context for a single RTP track (video or audio)

Each track has its own FFmpeg muxer context because RTP format supports only one stream per context.

Public Members

AVFormatContext *fmt_ctx = {nullptr}

RTP muxer for this track.

int track_index = {0}

Track index (0=video, 1=audio, etc.)

AVRational input_time_base = {1, 1000}

Original input timebase for rescaling.

uint32_t ssrc = {0}

SSRC assigned by FFmpeg’s RTP muxer.

int64_t last_input_dts = {AV_NOPTS_VALUE}

Last input DTS before offset (for discontinuity detection)

int64_t max_output_dts = {AV_NOPTS_VALUE}

Maximum output DTS ever sent (for offset calculation)

int64_t dts_offset = {0}

Cumulative offset for loops.

class RTSPMuxerFrameFilter : public Limef::ff::ComposeFrameFilter

#include <rtspmuxer.h>

Composite filter that canonizes a stream for RTSP/RTP delivery.

Chains: CodecAssertFrameFilter → AnnexBFrameFilter → RTPMuxerFrameFilter

The codec assert rejects unsupported codec combinations with a fatal log and drops all subsequent frames. The Annex B filter converts H.264/H.265 from AVCC (length-prefixed) to Annex B (start-code) format as required by the RTP muxer. Non-H.264/H.265 streams pass through AnnexBFrameFilter unchanged.

class ShmemServerFrameFilter : public Limef::ff::SimpleFrameFilter

#include <shmemserver.h>

Server-side framefilter that serializes frames to shared memory.

Accepts DecodedFrames and RawFrames and serializes them directly to shared memory ring buffer. Other frame types are ignored.

This is a terminal framefilter: frame flow stops here, cannot be connected to downstream filters.

The shared memory ring buffer is initialized in the constructor and cleaned up in the destructor automatically.

Usage:

EventFd event_fd;
Limef::shmem::ShmemRingbufferContext ctx("decoder_ring", 5, 8*1024*1024, 100, event_fd);
ShmemServerFrameFilter server("shmem-server", ctx);

// Chain in framefilter pipeline (terminal filter)
decoder.cc(server);

Subclassed by ShmemServerFrameFilterPy

Public Functions

ShmemServerFrameFilter(const std::string &name, const Limef::shmem::ShmemRingbufferContext &ctx)

Construct server framefilter.

Automatically initializes the shared memory ring buffer.

Parameters:

• name – Name for logging and identification

• ctx – Context defining shared memory parameters

• next – Should be nullptr - this is a terminal filter

Throws:

std::runtime_error – if shared memory setup fails

~ShmemServerFrameFilter() override

Destructor - automatically cleans up shared memory.

virtual void go(const Limef::frame::Frame *frame) override

Process incoming frames.

DecodedFrames and RawFrames are serialized to shared memory. All other frame types are ignored (logged and dropped).

After each successful write the ring buffer increments the eventfd counter (via setEventFd()), waking any client blocked in select/poll.

Parameters:

frame – Incoming frame to process

inline const Limef::shmem::ShmemRingbufferContext &getContext() const

Get the context used by this server.

Returns:

const Limef::shmem::ShmemRingbufferContext& Context reference

size_t getOverflowCount() const

Get number of overflow events detected.

Overflows occur when frames are too large for the ring buffer cells or when the ring buffer is full.

Returns:

size_t Overflow count since creation or last reset

void resetOverflowCount()

Reset the overflow counter to zero.

bool isActive() const

Check if the server is currently active.

Returns:

true if ring buffer is available for writing

virtual void setLogger(std::shared_ptr<spdlog::logger> logger)

Set the logger for this framefilter and its ring buffer.

This hides the base class implementation to also set the ring buffer’s logger.

Parameters:

logger – Logger instance to use

class SimpleFrameFilter : public Limef::ff::FrameFilter

#include <simple.h>

A base class for simple one-to-one framefilters.

A FrameFilter of this class can be connected to only one framefilter in the framefilter chain, i.e.

FrameFilter ff1();
FrameFilter ff2();
FrameFilter ff3();
FrameFilter ff4();
ff1.cc(ff2).cc(ff3).cc(ff4)

For subclasses, you should only implement the virtual method Limef::ff::SimpleFrameFilter::go

Subclassed by Limef::ff::FrameFifoFrameFilter< Limef::frame::DecodedFrame >, Limef::ff::FrameFifoFrameFilter< Limef::frame::TensorFrame >, Limef::ff::AnnexBFrameFilter, Limef::ff::AudioTranscodeFrameFilter, Limef::ff::CUDAScaleFrameFilter, Limef::ff::CodecAssertFrameFilter, Limef::ff::CodecCacheFrameFilter, Limef::ff::CodecMessageFrameFilter, Limef::ff::CountDecodedFrameFilter, Limef::ff::CountTensorFrameFilter, Limef::ff::CudaToFFmpegFrameFilter, Limef::ff::DecodedDownloadFrameFilter, Limef::ff::DecodedToTensorFrameFilter, Limef::ff::DecodedUploadFrameFilter, Limef::ff::DecodingFrameFilter, Limef::ff::DumpFrameFilter, Limef::ff::DumpPacketFrameFilter, Limef::ff::EmptyFrameFilter, Limef::ff::EncodeCheckFrameFilter, Limef::ff::EncodingFrameFilter, Limef::ff::FrameFifoFrameFilter< T >, Limef::ff::GateFrameFilter, Limef::ff::H264StripParamSetsFrameFilter, Limef::ff::InfoFrameFilter, Limef::ff::MP4PartitionFrameFilter, Limef::ff::MP4ProbeFrameFilter, Limef::ff::MuteAudioFrameFilter, Limef::ff::MuxerFrameFilter, Limef::ff::PutInfoFrameFilter, Limef::ff::RTPMuxerFrameFilter, Limef::ff::ShmemServerFrameFilter, Limef::ff::StreamGateFrameFilter, Limef::ff::SwScaleFrameFilter, Limef::ff::TensorToDecodedFrameFilter, Limef::ff::ThrottleFrameFilter, Limef::ff::WebMPartitionFrameFilter, Limef::ff::WritePNGFrameFilter

Public Functions

inline virtual void pass(const Limef::frame::Frame *frame) final

Define how frames are passed downstream.

Used by method Limef::ff:FrameFilter::go Calls Limef::ff:FrameFilter::go method(s) of the downstream filter(s)

inline virtual FrameFilter &cc(FrameFilter &next_) override

Register next framefilter in the chain.

This method does any necessary initialization. Framefilter is not fully functional until this method has been called. Should call init.

inline virtual void clear() override

Removes framefilter downstream connection.

class SplitFrameFilter : public Limef::ff::FrameFilter

#include <split.h>

A base class for one-to-many framefilters.

A FrameFilter of this class can be connected to several downstream framefilters:

FrameFilter ff1();
FrameFilter ff2();
SplitFrameFilter ff3();
// connect many downstream framefilters:
ff1.cc(ff2).cc(ff3).cm(ff41,ff42,ff43)
// connect many downstream framefilters one-by-one
ff3.cc(ff41)
ff3.cc(ff42)
// disconnect:
ff3.dc(ff41);
ff3.dc(ff42)

Maintains an internal registry of connected framefilters

NOTE: this framefilter is thread safe, i.e. you can use methods cc, cm and dc after threads have been started on the filterchain

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

inline virtual void pass(const Limef::frame::Frame *frame) final

Define how frames are passed downstream.

Used by method Limef::ff:FrameFilter::go Calls Limef::ff:FrameFilter::go method(s) of the downstream filter(s)

inline virtual FrameFilter &cc(FrameFilter &next) override

Register next framefilter in the chain.

This method does any necessary initialization. Framefilter is not fully functional until this method has been called. Should call init.

inline virtual void clear() override

Removes framefilter downstream connection.

class StreamGateFrameFilter : public Limef::ff::SimpleFrameFilter

#include <streamgate.h>

Passes only source-originated StreamFrames; drops relay replays.

Acts as a gate on the StreamFrame::from_source flag:

• from_source == true → frame passes through unchanged.

• from_source == false → frame is silently dropped.

All other frame types (PacketFrame, DecodedFrame, TensorFrame, SignalFrame, CodecFrame, …) pass through unconditionally.

Public Functions

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class SwitchFrameFilter : public Limef::ff::FrameFilter

#include <switch.h>

Routes incoming frames to exactly one of N output terminals.

Only the currently active terminal receives frames. Calling toggle() at runtime changes which terminal is active. When a terminal is first activated, any cached CodecFrame and StreamFrame metadata are replayed automatically before the next live frame is forwarded, so the new downstream has the codec/stream context it needs even if those one-time metadata frames have already passed.

Caching rules:

• CodecFrame and StreamFrame are cached keyed by their slot number.

• A newer frame for the same slot replaces the cached one.

• All other frame types are never cached.

Public Functions

inline explicit SwitchFrameFilter(std::string name = "switchframefilter", int n = 1)

Parameters:

• name – Filter name (for logging)

• n – Number of output terminals (must be >= 1)

inline FrameFilter &cc(int index, FrameFilter &next)

Connect output terminal index to next.

Parameters:

• index – Terminal index in [0, n)

• next – Downstream framefilter

Returns:

Reference to next (for chaining further downstream)

inline virtual FrameFilter &cc(FrameFilter &next) override

Connect terminal 0 to next.

Convenience override compatible with the standard cc() chain API. Equivalent to cc(0, next).

inline void toggle(int index)

Activate output terminal index.

Pass -1 to deactivate all terminals (frames are silently dropped). When activating a terminal, a lazy replay of cached metadata is scheduled: the next call to go() will forward all cached CodecFrame / StreamFrame entries before the live frame.

Parameters:

index – Terminal to activate, or -1 to deactivate all

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

inline virtual void clear() override

Removes framefilter downstream connection.

class SwScaleFrameFilter : public Limef::ff::SimpleFrameFilter

#include <swscale.h>

Converts bitmap frames to specified pixel format.

For DecodedVideoFrames:

• If already in target format, passes downstream as-is

• If different format, converts using swscale

For StreamFrames: makes a deep copy, updates the video stream’s pixel format to target_format, then passes the copy downstream. This keeps downstream consumers (encoders, shmem servers) informed of the actual output format.

All other frame types pass through unchanged.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class TensorToDecodedFrameFilter : public Limef::ff::SimpleFrameFilter

#include <tensor_to_decoded.h>

Converts TensorFrame (c, h, w) UInt8 → DecodedFrame.

CPU planes → GBRP or GRAY8 DecodedFrame (via memcpy). GPU planes → AV_PIX_FMT_CUDA / NV12 DecodedFrame (via CUDA kernel, LIMEF_CUDA only). Also emits a StreamFrame when the stream starts, dimensions change, or CPU/GPU mode changes.

Public Functions

explicit TensorToDecodedFrameFilter(std::string name = "TensorToDecodedFrameFilter", ChannelOrder in_order = ChannelOrder::RGB)

Parameters:

• name – Identifier used in log messages.

• in_order – Channel order of the input tensor (default: RGB).

virtual void go(const frame::Frame *f) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class ThrottleFrameFilter : public Limef::ff::SimpleFrameFilter

#include <throttle.h>

Rate-limit payload frames by minimum inter-frame interval.

Forwards a payload frame (Packet, Decoded, Tensor) downstream only when at least interval microseconds have elapsed since the last forwarded payload frame, as measured by the frame’s absolute timestamp.

• Non-payload frames (Signal, Codec, Stream, SDP, RTP, Info, …) pass through unconditionally.

• Payload frames with an invalid absolute timestamp pass through unconditionally.

• The very first payload frame always passes through (no prior reference point).

Two orthogonal controls:

• Rate-limiter (setState / setInterval): drops frames that arrive too soon after the previous forwarded frame.

• Gate (open / close): drops all payload frames when closed, regardless of timestamps or the rate-limiter state.

All control methods are mutex-protected and safe to call from any thread.

Public Functions

inline void setState(bool enabled)

Enable or disable the rate-limiter.

When enabled (default), payload frames are forwarded only when the inter-frame interval condition is met. When disabled, all payload frames pass through immediately with no timestamp check — equivalent to removing the filter from the chain. Non-payload frames are unaffected by this flag. Thread-safe.

inline bool getState() const

Return true if the rate-limiter is currently enabled. Thread-safe.

inline void setInterval(int interval_ms)

Change the minimum inter-frame interval.

Sets the minimum time (in milliseconds) that must elapse between two consecutively forwarded payload frames. Setting 0 ms passes every frame (since any delta ≥ 0), but still incurs the timestamp check overhead; prefer setState(false) for a true pass-through. Thread-safe; takes effect on the next go() call.

inline void close()

Close the gate — drop all payload frames until open() is called.

Acts independently of the rate-limiter (setState). When the gate is closed, no payload frames reach downstream regardless of timestamps or the enabled flag. Non-payload frames (Signal, Codec, Stream, Info, …) still pass through. Thread-safe.

inline void open()

Open the gate — resume forwarding payload frames.

Also resets the internal timestamp reference so the very next payload frame passes immediately (no waiting for the interval to elapse from the last frame before close() was called). Thread-safe.

inline bool isOpen() const

Return true if the gate is currently open. Thread-safe.

inline virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class WebMFrameFilter : public Limef::ff::ComposeFrameFilter

#include <webm.h>

Composite filter that muxes a VP8/VP9/AV1 stream to WebM for browser MSE.

Creates the chain: CodecAssertFrameFilter → GateFrameFilter → MuxerFrameFilter(WebM) → WebMPartitionFrameFilter

Output: one RawFrame per MSE segment:

• Init segment = EBML header + Segment header + Info + Tracks

• Media segment = one WebM Cluster (contains video blocks, ~1 s each)

Intended downstream: WebSocketServerThread (same as FMP4FrameFilter).

Public Functions

inline void open()

Open the gate and let frames flow.

inline void close()

Close the gate; frames are dropped.

class WebMPartitionFrameFilter : public Limef::ff::SimpleFrameFilter

#include <webmpart.h>

Repartitions a WebM/EBML byte stream into MSE-friendly segments.

Input: raw byte blobs from ffmpeg’s WebM muxer — arbitrary chunk sizes, not aligned to EBML element boundaries.

Output: one RawFrame per logical MSE segment:

• Init segment = EBML header + Segment header + Info + Tracks (everything before the first Cluster element)

• Media segment = one complete Cluster element

The WebM Cluster element is the MSE media-segment boundary, analogous to moof+mdat in fragmented MP4. The browser expects to receive the init segment first, then a stream of Clusters (each containing one or more video/audio blocks).

The parser walks the EBML tree at two depths:

• DOC / SEGMENT level: 4-byte IDs (first byte 0x10–0x1F)

• CLUSTER level: 1-byte IDs (first byte 0x80–0xFF)

Clusters from ffmpeg with custom IO have unknown size (streaming mode); the end of a Cluster is detected by the appearance of the next EBML element with a 4-byte ID (first byte 0x10–0x1F).

Assumption: Cluster children produced by ffmpeg’s webm muxer always carry known sizes (SimpleBlock, BlockGroup, Timecode). Unknown-size children are flagged as warnings and parsing stops until the next blob.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.

class WebRTCMuxerFrameFilter : public Limef::ff::ComposeFrameFilter

#include <webrtcmuxer.h>

Composite filter that canonizes a stream for WebRTC delivery.

Chains: CodecAssertFrameFilter → RTPMuxerFrameFilter

Public Functions

inline explicit WebRTCMuxerFrameFilter(std::string name = "WebRTCMuxerFrameFilter", int mtu = 1200)

Parameters:

• name – label for logging

• mtu – path MTU to the client in bytes (default 1200, matching WebRTCServerThread’s default). Nothing downstream fragments an oversized RTP/SRTP packet - libdatachannel’s own mtu config only bounds its own SCTP/DTLS traffic, not media tracks - so the RTP muxer’s packet size must itself be kept at or below mtu

  . Pass the same value given to WebRTCServerThread; lower both when routing over a VPN/tunnel such as Tailscale (MTU 1280) to avoid EMSGSIZE (“datagram is

  too large”) errors.

class WritePNGFrameFilter : public Limef::ff::SimpleFrameFilter

#include <write_png.h>

Writes each decoded video frame to a PNG file in a directory.

Pass-through: every frame is forwarded downstream unchanged after being written to disk.

Filenames are frame000001.png, frame000002.png, … (6-digit counter). The directory is created on construction.

Only CPU-resident video frames are written. CUDA/GPU frames are passed through with a one-time warning — insert DecodedDownloadFrameFilter upstream to materialise GPU frames before reaching this filter.

The source pixel format is automatically converted to RGB24 via libswscale before PNG encoding, so any AVPixelFormat is accepted.

Public Functions

virtual void go(const Limef::frame::Frame *frame) override

Implementation of the framefilter action.

Uses Limef::ff:FrameFilter::pass to pass frames downstream.

The framefilter ensures that the frame is not modified. If it needs a modified frame it has to take an internal copy of the frame first.