SkiaCamera: video effects and subtitles in .NET MAUI

Real-Time Video Processing and AI Subtitles in .NET MAUI with SkiaCamera

SkiaCamera lets you capture photos and videos from mobile device cameras while applying effects, overlays, and AI-generated subtitles in real time. Every frame passes through a Skia canvas before encoding, seamlessly integrating processed data into the final MP4 without any post-production. To get started, install the DrawnUi.Maui.Camera NuGet package and initialize DrawnUI in MauiProgram.cs with builder.UseDrawnUi().

Place the component inside a draw:Canvas with RenderingMode="Accelerated":

<Grid VerticalOptions="Fill" HorizontalOptions="Fill">
    <draw:Canvas RenderingMode="Accelerated" Gestures="Lock">
        <camera:SkiaCamera x:Name="Camera" CaptureMode="Video" />
    </draw:Canvas>
</Grid>

Pin the container to Fill to avoid rendering artifacts. For proper video orientation, lock the app to portrait mode: on Android via ScreenOrientation.SensorPortrait in MainActivity.cs, and on iOS through UISupportedInterfaceOrientations in Info.plist.

Set the required permissions:

Camera.NeedPermissionsSet = NeedPermissions.Camera | NeedPermissions.Gallery | NeedPermissions.Microphone;

Start the camera after the canvas's first render or on navigation events.

Camera Control and Frame Processing

The IsOn property handles start/stop. The camera pauses automatically when the app goes to the background. The preview renders via SkiaImage in the Display property, perfect for adding blur or other effects.

Enable real-time processing:

UseRealtimeVideoProcessing = true;
ProcessFrame = OnFrameProcessing;
ProcessPreview = OnFrameProcessing;

Handlers receive a DrawableFrame with SKCanvas, SKImage, Scale, and an IsPreview flag. During recording, the preview shows a scaled-down version of the processed frame—what you see is what you get (WYSIWYG).

Here's an example custom controller AppCamera:

public partial class AppCamera : SkiaCamera
{
    public AppCamera()
    {
        NeedPermissionsSet = NeedPermissions.Camera | NeedPermissions.Gallery | NeedPermissions.Microphone;
        InjectGpsLocation = true;
        UseRealtimeVideoProcessing = true;
        VideoQuality = VideoQuality.Standard;
        EnableAudioRecording = true;
        ProcessFrame = OnFrameProcessing;
        ProcessPreview = OnFrameProcessing;
    }
}

Video Recording with Pre-Recording

Start recording:

if (CameraControl.IsRecording)
    await CameraControl.StopVideoRecording();
else
    await CameraControl.StartVideoRecording();

Move to the gallery on completion:

private async void OnVideoRecordingSuccess(object sender, CapturedVideo capturedVideo)
{
    var publicPath = await CameraControl.MoveVideoToGalleryAsync(capturedVideo, MauiProgram.Album);
}

Pre-recording uses an in-memory ring buffer to capture 3–10 seconds before hitting REC:

CameraControl.EnablePreRecording = true;
CameraControl.PreRecordDuration = TimeSpan.FromSeconds(5);

IsPreRecording and IsRecording properties are bindable for UI updates. Canceling clears the buffer: StopVideoRecording(true).

Ideal for unpredictable action like sports, wildlife, or motion-triggered security cams.

SKSL Shaders for Video Effects

Apply filters to previews, photos, and videos using SKSL shaders. Override RenderPreviewForProcessing and RenderFrameForRecording:

CameraControl.VideoEffect = ShaderEffect.Movie;

Shaders render on SKPaint before encoding, baking the effect into the MP4. The UI demo includes a filter switcher: Noir, Movie, and more.

AI Subtitles and Audio Visualization

OpenAI-powered subtitle generation integrates in real time and embeds directly into videos. An audio visualizer (EQ) overlays as a responsive element, reacting to device rotation via RotationChanged.

Settings sections:

• Input: camera selection, format, mode.
• Processing: audio monitoring, subtitles, shaders.
• Output: audio/video, pre-recording, quality.

The UI rotates button icons independently of screen lock.

Key Takeaways

• SkiaCamera processes frames on GPU via Skia canvas for real-time effects baked into the final video.
• Pre-recording with a ring buffer captures events before the trigger without constant disk writes.
• SKSL shaders apply to previews and recordings without post-processing; ProcessFrame/Preview callbacks distinguish modes.
• OpenAI AI subtitles and EQ overlays embed in MP4; UI adapts to orientation.
• Auto-pauses in background, bindable properties for recording/pre-recording.

— Editorial Team