[Addin]

Summary

Windows capture currently copies every frame back to the CPU with a blocking Map(READ), then copies it again into an FFmpeg frame. This removes the second copy: the mapped staging texture is wrapped directly as an FFmpeg-backed frame, and instant/fragmented M4S recordings can encode straight from the captured D3D11 texture through Media Foundation. The existing FFmpeg encoder remains the fallback.

Changes

• scap-direct3d: staging-texture ring with in-use tracking; bounded D3D11_MAP_FLAG_DO_NOT_WAIT polling by default (spin for up to 4 ms, then sleep in 100 µs steps to an 8 ms deadline); CAP_READBACK_BLOCKING=1 for an unbounded Map; D3D11_CREATE_DEVICE_VIDEO_SUPPORT; WARP fallback uses BGRA_SUPPORT only; one zeroed padding row.
• scap-ffmpeg: the mapped staging texture is an AVBufferRef-backed AVFrame; the per-frame pool and memcpy are gone.
• enc-mediafoundation / mediafoundation-ffmpeg: explicit-bitrate constructor, sequence_header(), 2 s keyframe spacing, sample_to_packet, bounded encoder-tail drain; iterate MFTs, negotiate types inside the candidate loop, and read the sequence header from GetOutputCurrentType.
• enc-ffmpeg: SegmentedPacketMuxer writes externally encoded H.264 into the existing DASH init/m4s/manifest and declares the encoded size, not the capture size.
• win_fragmented_m4s: MF texture input is preferred on non-AMD hardware; setup failure falls back to FFmpeg; a runtime MF failure finalizes written segments and emits MuxerCrashed.
• scap-ffmpeg/examples/cli.rs: --bench A/B, --path validation, MinUpdateInterval, and failure counters that only count exercised paths.
• Removed the unused win_segmented* muxers and DiskSpaceCallback plumbing.

Test machine

• HP Pavilion Gaming Notebook
• Intel Core i7-6700HQ, 4C/8T, 2.6 GHz
• 16 GB DDR3L-1600 (2×8 GB)
• NVIDIA GeForce GTX 950M, 4 GB, driver 581.94
• Intel HD Graphics 530, driver 31.0.101.2115
• Windows 11 24H2, build 26100.9711
• 1920×1080 @ 60 Hz
• Rust 1.88.0; benchmark harness x86_64-pc-windows-gnu, workspace validation x86_64-pc-windows-msvc

Performance

Results summary

Resolution	Metric	Before	After	Improvement	Speedup
1280x720	p50	3.10 ms	2.64 ms	14.8%	1.17x
1600x900	p50	2.92 ms	2.03 ms	30.5%	1.44x
1920x1080	p50	4.12 ms	2.82 ms	31.6%	1.46x
1920x1080	p99	13.62 ms	8.21 ms	39.7%	1.66x

The 1080p after path trades 11 missed readbacks out of 881 frames for the lower latency; the bounded wait remains the default and CAP_READBACK_BLOCKING=1 is still available.

Instant mode, compared with the FFmpeg fallbacks:

Metric	MF texture input	NVENC	libx264	vs NVENC	vs libx264
Total CPU	8.1%	11.2%	52.3%	27.7% lower (1.38x)	84.5% lower (6.46x)
Stop latency	70.6 ms	140.7 ms	89.0 ms	49.8% lower (1.99x)	20.7% lower (1.26x)

Readback A/B (--bench --seconds 30; 60 s runs were consistent)

resolution	path	p50	p95	p99	max	frames / misses
1280×720	old	3.10 ms	3.66 ms	3.93 ms	4.21 ms	891 / 0
1280×720	new	2.64 ms	3.08 ms	3.16 ms	3.38 ms	891 / 0
1600×900	old	2.92 ms	3.11 ms	3.25 ms	6.28 ms	891 / 0
1600×900	new	2.03 ms	2.21 ms	2.26 ms	2.69 ms	891 / 0
1920×1080	old	4.12 ms	4.58 ms	13.62 ms	15.18 ms	892 / 0
1920×1080	new	2.82 ms	3.02 ms	8.21 ms	8.92 ms	881 / 11

The bounded default wins at every resolution: p50 3.10→2.64 ms (720p), 2.92→2.03 ms (900p), 4.12→2.82 ms (1080p); 1080p p99 drops 13.62→8.21 ms. CAP_READBACK_BLOCKING=1 is slightly faster at 1080p (2.71/2.12/2.94 ms p50 at 720p/900p/1080p with zero misses) but can stall on slow hardware.

The remaining cost is GPU readback latency, not the memcpy: CopySubresourceRegion + Flush submits in ~60 µs, while Map waits ~2–3.5 ms for the copy. Encoding from the texture avoids that map.

Instant mode

metric	MF texture input	FFmpeg fallback (h264_nvenc)	FFmpeg software (libx264)
CPU, avg total / per core	8.1% / 1.0%	11.2% / 1.4%	52.3% / 6.5%
decoded frames / duration	758 / 30.19 s	750 / 30.58 s	766 / 30.23 s
bitrate	7.84 Mbps	7.80 Mbps	7.93 Mbps
segments	13	13	13
stop latency	70.6 ms	140.7 ms	89.0 ms

MF uses 8.1% total CPU versus 11.2% for NVENC and 52.3% for software x264, with the lowest stop latency; bitrate is comparable at 7.8–7.9 Mbps.

Validation

• cargo fmt --all --check: pass.
• cargo clippy --workspace --exclude cap-desktop --all-features --locked -- -D warnings (Windows MSVC): pass.
• cargo build --workspace --exclude cap-desktop --locked (Windows MSVC): pass.
• Affected tests: scap-direct3d 5/5, cap-mediafoundation-ffmpeg 1/1.
• Full workspace tests: scap-direct3d and cap-mediafoundation-ffmpeg pass. Remaining failures are hardware/environment-specific (NVENC segment timing, unavailable HEVC MF, pre-existing chaos/editor tests); the camera-pts-late MP4 MF export case is outside this path and needs a follow-up.

Scope

• AMD remains excluded from the MF path, matching the existing MP4 hardware path.
• The Windows MP4 hardware muxer (H264StreamMuxer) is unchanged; this PR targets instant/fragmented M4S.
• 4K and WARP performance were not measured (1080p panel; WARP accepts BGRA only).
• On Optimus, the recording executable needs the Windows high-performance GPU preference (GpuPreference=2) for NVIDIA MFT activation.

Confidence Score: 5/5

The PR appears safe to merge, with no new actionable issues found.

Summary

This PR reduces Windows frame copies and adds Media Foundation texture encoding for fragmented recordings, with FFmpeg as the setup fallback.

• Windows capture frames reach FFmpeg without a second pixel copy.
• Windows fragmented recordings can encode captured textures through Media Foundation.
• Unused Windows segmented muxers and their exports are removed.
• The capture example can compare old and new readback timing.

_{Reviews (3) · Last reviewed commit: "fix(scap-ffmpeg): mark mapped readback f..." · Reviewed by Greptile}

[vercel]

@Addin is attempting to deploy a commit to the Cap Software Inc Team on Vercel.

A member of the Team first needs to authorize it.

[Addin]

@greptileai review

[Addin]

@greptileai review

[richiemcilroy]

Thanks @Addin, this is a lot of careful work and the measurements on the Pavilion are really useful. We can't merge it as is. These are the things we found:

1. Default readback now drops frames. With the 8 ms READBACK_BUDGET (crates/scap-ffmpeg/src/direct3d.rs:10, map_readback in crates/scap-direct3d/src/lib.rs), every FFmpeg path (all AMD users, the OOP muxer, and any MF fallback) duplicates frames under GPU load, spins the CPU for up to 4 ms per frame, and can drop tail frames at stop. Your own 1080p run shows 11 misses out of 881. Please make the waiting Map the default and the deadline opt-in.
2. Studio mode changes too, not only Instant. Studio recordings also go through start_media_foundation_encoder (win_fragmented_m4s.rs:631). If MF fails mid-recording, the rest of the video is discarded (:813) with no FFmpeg fallback, and the user is never told because desktop-gpui doesn't consume MuxerCrashed. We'd like the texture path limited to Instant, or behind a setting, until it's validated on more hardware, and a runtime failure should fall back rather than end the recording.
3. Colour. The MF video processor's colour-space setup (crates/enc-mediafoundation/src/video_processor.rs:99) leaves the output on BT.601, and SegmentedPacketMuxer writes no colour tags. The FFmpeg path produces tagged BT.709 limited range, so shares may play back with shifted colours. Please output BT.709 and tag the stream.
4. Stop errors. DrainIncomplete after 250 ms (crates/enc-mediafoundation/src/video/h264.rs:788-845) now fails WindowsMuxer and the camera MP4 stop with "Hardware encoder error", where main returned Ok. DRAIN is also sent before END_OF_STREAM. A slow drain shouldn't fail an otherwise good recording.
5. Encoder selection. The MP4 and camera MF encoders now try every hardware encoder, which changes the pick on hybrid laptops, and the 2 s keyframe interval may be ignored by some hardware encoders. Please keep the current selection for those paths unless that change is the goal, and say so in the description if it is.

Smaller things: the 285-line scap-ffmpeg/examples/cli.rs benchmark rewrite would be easier to review as its own PR, several new /// comments narrate the code (AGENTS.md asks for none), and CAP_READBACK_BLOCKING is read on every frame instead of once.

Before merging we'll also need CI to run on Windows and a real-device pass: AMD under GPU load, Intel iGPU, NVIDIA, a hybrid laptop, ARM64, pause/resume, window resize, crash recovery, and 4K60. Removing the unused win_segmented* muxers and DiskSpaceCallback is fine.