Refine media capture interop source placement and startup fallback#57
Merged
Conversation
- Add MarshalMedia.h with marshalling for MediaDeviceKind enum and media state enums - Implement EnumerateDevices() to query Windows audio devices using WASAPI - Add MediaDeviceInfo.Create() factory method for interop to create device info records - Support both audio input and audio output device enumeration - Add placeholder for video device enumeration (future work) - All compilation warnings only, no errors Related to issue #49 BugzId:48
Add minimal GetUserMedia implementation that validates input arguments. Full implementation will require: - Native WebRTC VideoTrackSourceInterface/AudioSourceInterface implementations - Audio/video device source creation using Windows APIs (WASAPI for audio, DirectShow/WinRT for video) - MediaStream construction with tracks from sources - PeerConnectionFactory::CreateVideoTrack and CreateAudioTrack integration Builds successfully with no errors. BugzId:48
Implement MVP GetUserMedia that creates audio and video tracks using the native WebRTC PeerConnectionFactory API. New files: - WebRtcInterop/Media/SimpleVideoSource.h/cpp: Minimal VideoTrackSourceInterface implementation that satisfies the WebRTC API contract without capturing video. Can be upgraded to real capture (DirectShow/WinRT) later. Changes: - MediaDevices::GetUserMedia() now: * Validates audio/video constraints (at least one must be true) * Gets the native PeerConnectionFactory instance * Creates a native MediaStream * Creates audio track with null source (allowed by WebRTC API) * Creates video track with SimpleVideoSource * Returns managed MediaStream wrapper Benefits: - Unblocks BasicVideoChat rendering and track management - MediaStream tracks can be enumerated, muted, and events fire - Real audio/video capture can be integrated later as an upgrade - No dependency on DirectShow or WASAPI yet Tests: All 107 managed API tests pass BugzId:48
Implement MVP constrainable behavior for MediaStreamTrack by removing NotImplementedException paths in constraint APIs. - Add per-track applied_constraints_ state in WebRtcInterop::Media::MediaStreamTrack - Initialize to empty MediaTrackConstraints in constructors - Implement GetConstraints() to return current applied constraints - Implement ApplyConstraints(null) to clear constraints to an empty set - Implement ApplyConstraints(non-null) to persist supplied constraints This provides a stable, non-throwing baseline for apps using the constraint APIs while full native capability/settings marshalling is completed later. BugzId:48
Use Google WebRTC's own VideoCaptureFactory::CreateDeviceInfo path to enumerate Windows video input devices instead of the local stub. - Implement EnumerateVideoDevices() via VideoCaptureModule::DeviceInfo NumberOfDevices/GetDeviceName and map to MediaDeviceInfo - Populate video device id/label/groupId from WebRTC-provided unique/product ids - Fix managed->native string conversion in GetUserMedia track/stream ids using marshal_as<std::string> instead of wide-char reinterpret casts - Add strmiids.lib to interop link dependencies (required by DirectShow IIDs referenced by webrtc.lib device_info_ds/video_capture_ds objects) BugzId:48
Implement the next post-MVP MediaDevices hardening items. - Add MediaDevices lifetime management with timer-based device polling - Track known devices and raise OnDeviceChange when device snapshots differ - Populate DeviceChangeEventArgs with full current list and inserted-device subset - Enforce requested device availability in GetUserMedia (audio/video) - Convert track creation/add failures to MediaStreamException instead of returning partial/empty streams This keeps MediaDevices behavior predictable for app callers while preserving MVP capture architecture. BugzId:48
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Moves
CameraVideoSourcefromWebRtcInterop\MediatoWebRtcInterop\to reflect that it is a WebRTC capture-source primitive, and keepsMediaDevicesas the consumer. This also includes the recent capture capability fallback startup flow (tiered capabilities + best-match probing) so camera startup is more resilient.Change Area (required)
WebRtcNet.Api/ managed-only behavior)Required Evidence (for behavioral/API changes)
1) W3C reference (required when applicable)
getUserMedia()resolves with aMediaStreamwhose tracks satisfy the requested kinds/constraints, or rejects when constraints/device access cannot be satisfied.enumerateDevices()returns snapshots of available media devices.2) Google source reference (required when applicable)
Provide pinned references: path + revision context (commit SHA or branch/tag).
modules/video_capture/video_capture_factory.*modules/video_capture/video_capture_defines.hapi/media_stream_interface.hbranch-heads/7778(repo pipeline target used by this project)VideoCaptureFactory::Create(...),CreateDeviceInfo(), and capability matching/start semantics as the primary implementation reference.3) Intended observable behavior (required when applicable)
Describe externally observable behavior (state transitions, event ordering, timing, errors).
GetUserMedia(audio/video)creates a local stream with requested tracks; video uses camera-backed capture source and now attempts fallback capture capabilities when preferred mode cannot start.OnDeviceChangeon detected device set changes.4) Divergence from Google reference (required if diverging)
Test Evidence (required)
Tests added/updated
Commands run + results
dotnet test WebRtcNet.Api.UnitTests\WebRtcNet.Api.UnitTests.csproj→ Passed (214/214)MSBuild.exe WebRtcInterop\WebRtcInterop.Framework.vcxproj -p:Configuration=Debug -p:Platform=x64→ SucceededChange-area test gate confirmation
If environment unavailable (required when checked above)
WebRtcInterop.UnitTests) is not currently available in this environment.Documentation
Spec/Crosswalk Review (required for W3C-aligned behavior changes)
docs\standards\crosswalk\webrtcnet-api-to-spec.mddocs\standards\specs\index\spec-map.md.\scripts\update-spec-docs.ps1) when neededMedia capture API status (not implemented / partially implemented)
Current known gaps in the Media Capture surface:
MediaStreamConstraints.AudioConstraints/VideoConstraintsare accepted but not fully enforced in interop; current selection is primarily kind-based (Audio/Video) with capability fallback for video startup.MediaDevices.GetSupportedConstraints()currently returns defaultMediaTrackSupportedConstraintswithout mapping real supported constraint flags.InputDeviceInfo.GetCapabilities()is not implemented and throwsNotImplementedException(WebRtcNet.Api\Media\InputDeviceInfo.cs).MediaStreamTrackcapability/settings/constraint application is partial:GetCapabilities()returns an empty default object.GetSettings()returns an empty default object.ApplyConstraints(...)stores constraints locally but does not apply them to native tracks.