Skip to content

Optimize camera preview with native macOS GPU path #1909

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
richiemcilroy merged 24 commits into from
Jun 10, 2026
Merged

Optimize camera preview with native macOS GPU path #1909

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
51c31e4
feat(rendering): add create_bgra_texture to IOSurface cache
richiemcilroy Jun 10, 2026
44c2952
refactor(camera-ffmpeg): extract sample_buf_as_ffmpeg conversion
richiemcilroy Jun 10, 2026
6535b57
feat(camera-effects): add Linux ONNX Runtime dynamic loading
richiemcilroy Jun 10, 2026
5c89dc5
perf(recording): optimize camera feed sender and ffmpeg path
richiemcilroy Jun 10, 2026
d9856b7
feat(frame-ws): track websocket frame subscriber count
richiemcilroy Jun 10, 2026
0c43092
refactor(frame-ws): pass subscriber count in editor websockets
richiemcilroy Jun 10, 2026
fe0fa4a
feat(camera-native): add GPU-native IOSurface frame converter
richiemcilroy Jun 10, 2026
d7bc4bb
build(desktop): add cap-camera-ffmpeg macOS dependency
richiemcilroy Jun 10, 2026
7205dcd
build(desktop): enable Win32 threading for camera preview QoS
richiemcilroy Jun 10, 2026
4d4129f
feat(general-settings): roll out native camera preview default on macOS
richiemcilroy Jun 10, 2026
b2d83fc
feat(camera): add native preview renderer with IOSurface GPU path
richiemcilroy Jun 10, 2026
89c3de7
perf(camera-legacy): optimize websocket camera preview pipeline
richiemcilroy Jun 10, 2026
55707bf
refactor(lib): wire native camera preview sender and state channel
richiemcilroy Jun 10, 2026
6849624
refactor(windows): use native camera preview settings helper
richiemcilroy Jun 10, 2026
c59f5c7
feat(recording): prefer 60fps 16:9 camera formats up to 720p
richiemcilroy Jun 10, 2026
fd68542
feat(webgpu-renderer): support configurable adapter power preference
richiemcilroy Jun 10, 2026
44d19bd
perf(socket): optimize camera preview websocket frame rendering
richiemcilroy Jun 10, 2026
a5bf8b9
refactor(camera): use shared websocket renderer in legacy preview
richiemcilroy Jun 10, 2026
bf549b2
refactor(overlay): use shared websocket renderer in inline preview
richiemcilroy Jun 10, 2026
c91f82c
docs(settings): update native camera preview experimental description
richiemcilroy Jun 10, 2026
e6e7caa
Update Cargo.lock
richiemcilroy Jun 10, 2026
ef3a82a
fix: address camera preview PR feedback
richiemcilroy Jun 10, 2026
2e6868e
fix: address remaining camera feedback
richiemcilroy Jun 10, 2026
02a6aa9
fix: validate nv12 frame metadata
richiemcilroy Jun 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions apps/desktop/src-tauri/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,7 @@ objc = "0.2.7"
swift-rs = "1.0.6"
tauri-nspanel = { git = "https://github.com/ahkohd/tauri-nspanel", branch = "v2" }
cidre = { workspace = true }
cap-camera-ffmpeg = { path = "../../../crates/camera-ffmpeg" }

[target.'cfg(not(all(target_os = "macos", target_arch = "x86_64")))'.dependencies]
parakeet-rs = "0.3.4"
Expand All @@ -160,6 +161,7 @@ windows = { workspace = true, features = [
"Win32_Foundation",
"Win32_System",
"Win32_System_Power",
"Win32_System_Threading",
"Win32_System_WinRT",
"Win32_UI_WindowsAndMessaging",
"Win32_Graphics_Gdi",
Expand Down
7 changes: 3 additions & 4 deletions apps/desktop/src-tauri/examples/desktop-display-transport-benchmark.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -220,7 +220,8 @@ async fn main() {
};

let (frame_watch_tx, frame_watch_rx) = watch::channel(None);
let (ws_port, ws_shutdown_token) = create_watch_frame_ws(frame_watch_rx).await;
let (ws_port, ws_shutdown_token) =
create_watch_frame_ws(frame_watch_rx, Default::default()).await;

println!("DISPLAY_WS_URL=ws://127.0.0.1:{ws_port}");
println!(
Expand Down Expand Up @@ -248,7 +249,7 @@ async fn main() {
let ws_frame = match output {
EditorFrameOutput::Nv12(frame) => {
let ws_format = match frame.format {
GpuOutputFormat::Nv12 => WSFrameFormat::Nv12,
GpuOutputFormat::Nv12 => WSFrameFormat::Nv12 { full_range: false },
GpuOutputFormat::Rgba => WSFrameFormat::Rgba,
};
let metadata_bytes = match frame.format {
Expand Down Expand Up @@ -292,8 +293,6 @@ async fn main() {
let renderer = match Renderer::spawn_with_telemetry(
render_constants.clone(),
frame_cb,
&recording_meta,
meta.as_ref(),
layers_rx,
Some(telemetry.clone()),
) {
Expand Down
Loading
Loading