Get video capture (getUserMedia) pixel format with no chroma subsampling (YUV444 or RGB24)
Baptiste Parsy
Despite conducting a good number of experiments and thorough research, I could not find a way to obtain a full color video/webcam stream with no chroma subsampling in chromium. The goal is to be able to display a video stream from a capture card and displaying it directly onto a canvas and drawing some overlay elements on top, so network throughput is not an issue, but the stream quality is. My company is making analyses and overlays over video streams of endoscopes, thus the need to output a good quality stream back to the clinicians.
I have been testing by feeding a still image into v4l2 with ffmpeg with RGB24 pixel format (YUV444 doesn't seem to be supported by v4l2). However, whatever I do, I cannot change the resulting stream to be something other than vp8 I420 but I cannot find how to change the vp8 codec used with another one supporting a full color profile.
I attempted using the experimental MediaStreamTrackProcessor since it said it would allow me to access raw stream bytes, however these are already I420 unless I missed a way to change it. But this makes me wonder if there could be some other experimental APIs or ongoing w3c proposals I could have missed and which could achieve what I need. I came to realize that I could have use MediaRecorder too which might have been simpler.
Here is the test code I have been using:
navigator.mediaDevices.enumerateDevices().then(async devices => {
console.log(devices);
const videoDevice = devices.find(device => device.kind === "videoinput" && device.label === "Virtual Webcam");
const stream = await navigator.mediaDevices.getUserMedia({
video: {
deviceId: videoDevice.deviceId,
width: {ideal: 1920},
height: {ideal: 1080},
frameRate: {ideal: 25},
}
});
const [track] = stream.getVideoTracks();
const {readable} = new MediaStreamTrackProcessor({track});
const reader = readable.getReader();
const result = await reader.read();
console.log(result.value.format); // Outputs I420 no matter which ffmpeg pix_fmt
});
If there is no simpler solution, my company is willing to spend some people and some time to contribute to chromium to make it happen if it must come to it, but it is quite difficult to know if this is realistic as I fear we would need to write a serious and backed proposal first if I understand correctly. I hope to obtain some advice on the feasibility of this; and it would be truly amazing and incredibly helpful if you knew a very approximate number of developers and amount of time you think that such an endeavour would require.
Thanks a lot for your time!
Joe Mason
--
--
Chromium Discussion mailing list: chromium...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-discuss