Gabriel Brito would like Greg Thompson, Zequan Wu, Chromium LUCI CQ, Fredrik Hernqvist, Olga Sharonova and Henrik Andreasson to review this change.
Reland "[gDM] Window audio capture WASAPI implementation"
This is a reland of commit 42f725b8b8d20758e49c0b9cbcaded7bc6fdfe95
Original change's description:
> [gDM] Window audio capture WASAPI implementation
>
> Adds the process-based audio capture logic that will power
> getDisplayMedia window capture on Windows devices. This CL also adds
> tests and testing mocks for the Windows WASAPI interfaces. This CL
> builds on the work in
> https://chromium-review.googlesource.com/c/chromium/src/+/5092148.
>
> Bug: 40947205
> Change-Id: I971b5b6a9c715b45ddcdbcc40466b5da8f2f816e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6442207
> Reviewed-by: Henrik Andreasson <hen...@chromium.org>
> Reviewed-by: Olga Sharonova <ol...@chromium.org>
> Reviewed-by: Zequan Wu <zequ...@google.com>
> Commit-Queue: Gabriel Brito <gabrie...@microsoft.com>
> Reviewed-by: Greg Thompson <g...@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#1461128}
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Commit-Queue | +1 |
Hi! PTAL when convenient.
@g...@chromium.org I am removing you from the reviewer set just because your approval is not strictly required for this change and I don't want to take your time unnecessarily. But, please, feel free to review it if you'd like.
Thanks!
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
I created an ASan build and verified that all tests now work as intended.
To be clear: the changes here are on the fake client only: No other functional changes are made afaik.
Correct?
Code-Review | +1 |
I created an ASan build and verified that all tests now work as intended.
To be clear: the changes here are on the fake client only: No other functional changes are made afaik.
Correct?
I could see the diff in https://chromium-review.googlesource.com/c/chromium/src/+/6557667/1..2 and the change is on the fake clients only. Hence Resolved.
*buffer_size = kBufferSizeFrames;
Just FYI, my experience from real devices is that this size is often a multiple of 10ms and should depend on the sample rate. It should work with any size I but given that we also have access to the sample rate it might be more clean to do the same for the fake class. Today, the FIFO will have to adapt more in the fake client than in a real client.
Not needed now but just wanted to leave a comment.
Example:
```
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1022] CreateFifoIfNeeded
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1052] endpoint_buffer_size_frames=4800
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1053] frame_size_bytes_=4
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1054] packet_size_bytes_=1920
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1055] buffers_required=20
```
In the example above, the fake client will use 256 and a real client will use 4800.
Henrik AndreassonI created an ASan build and verified that all tests now work as intended.
To be clear: the changes here are on the fake client only: No other functional changes are made afaik.
Correct?
I could see the diff in https://chromium-review.googlesource.com/c/chromium/src/+/6557667/1..2 and the change is on the fake clients only. Hence Resolved.
Yep. thanks for double checking!
Just FYI, my experience from real devices is that this size is often a multiple of 10ms and should depend on the sample rate. It should work with any size I but given that we also have access to the sample rate it might be more clean to do the same for the fake class. Today, the FIFO will have to adapt more in the fake client than in a real client.
Not needed now but just wanted to leave a comment.
Example:
```
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1022] CreateFifoIfNeeded
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1052] endpoint_buffer_size_frames=4800
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1053] frame_size_bytes_=4
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1054] packet_size_bytes_=1920
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1055] buffers_required=20
```In the example above, the fake client will use 256 and a real client will use 4800.
Fix applied.
It seems that I don't have access issue 323486298. Is it ok for me to be able to see it?
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Code-Review | +1 |
*buffer_size = kBufferSizeFrames;
Gabriel BritoJust FYI, my experience from real devices is that this size is often a multiple of 10ms and should depend on the sample rate. It should work with any size I but given that we also have access to the sample rate it might be more clean to do the same for the fake class. Today, the FIFO will have to adapt more in the fake client than in a real client.
Not needed now but just wanted to leave a comment.
Example:
```
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1022] CreateFifoIfNeeded
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1052] endpoint_buffer_size_frames=4800
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1053] frame_size_bytes_=4
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1054] packet_size_bytes_=1920
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1055] buffers_required=20
```In the example above, the fake client will use 256 and a real client will use 4800.
Fix applied.
It seems that I don't have access issue 323486298. Is it ok for me to be able to see it?
b/323486298 is unrelated to your work and I can't find it as crbug.com/323486298 either. Note sure where you got 323486298 from. Can you create your own/new issue and us it instead.
*buffer_size = kBufferSizeFrames;
Gabriel BritoJust FYI, my experience from real devices is that this size is often a multiple of 10ms and should depend on the sample rate. It should work with any size I but given that we also have access to the sample rate it might be more clean to do the same for the fake class. Today, the FIFO will have to adapt more in the fake client than in a real client.
Not needed now but just wanted to leave a comment.
Example:
```
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1022] CreateFifoIfNeeded
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1052] endpoint_buffer_size_frames=4800
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1053] frame_size_bytes_=4
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1054] packet_size_bytes_=1920
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1055] buffers_required=20
```In the example above, the fake client will use 256 and a real client will use 4800.
Henrik AndreassonFix applied.
It seems that I don't have access issue 323486298. Is it ok for me to be able to see it?
b/323486298 is unrelated to your work and I can't find it as crbug.com/323486298 either. Note sure where you got 323486298 from. Can you create your own/new issue and us it instead.
I was just about to write more-or-less the same thing.
should_stop_streaming_ = true;
there's no synchronization around this (accessed by multiple threads), nor do i see anything that ensures that the tasks posted below (which use `Unretained`) will not run after this instance is destroyed. although this is test code, it would be good to fix this so that we don't end up with flaky tests.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
next_packet_size_(buffer_size_frames) {}
CHECK(frame_size_bytes_);
(since we are dividing by it later)
static_cast<UINT32>(buffer_data_.size() / frame_size_bytes_);
This is a known constant value - precalculate and use here and in GetNextPacketSize()?
should_stop_streaming_ = true;
there's no synchronization around this (accessed by multiple threads), nor do i see anything that ensures that the tasks posted below (which use `Unretained`) will not run after this instance is destroyed. although this is test code, it would be good to fix this so that we don't end up with flaky tests.
+1
if (buffer_ready_event_handle_) {
SetEvent(buffer_ready_event_handle_);
}
Do we expect IAudioClient to stop synchronously? In that case, shouldn't this be called only if `should_stop_streaming_` is true?
An typical alternative to should_stop_streaming would be: posting from Stop() to base::ThreadPool a task to signal an event, and waiting in Stop() for it to be signaled. (This would mean all the earlier posted tasks have cycled through.)
Code-Review | +1 |
I took the liberty of fixing the remaining comments to ensure that we can land this essential CL again as soon as possible.
I have pending work which need the new structure.
CHECK(frame_size_bytes_);
(since we are dividing by it later)
Done
static_cast<UINT32>(buffer_data_.size() / frame_size_bytes_);
This is a known constant value - precalculate and use here and in GetNextPacketSize()?
Done
*buffer_size = kBufferSizeFrames;
Gabriel BritoJust FYI, my experience from real devices is that this size is often a multiple of 10ms and should depend on the sample rate. It should work with any size I but given that we also have access to the sample rate it might be more clean to do the same for the fake class. Today, the FIFO will have to adapt more in the fake client than in a real client.
Not needed now but just wanted to leave a comment.
Example:
```
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1022] CreateFifoIfNeeded
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1052] endpoint_buffer_size_frames=4800
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1053] frame_size_bytes_=4
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1054] packet_size_bytes_=1920
[20028:24184:VERBOSE1:media\audio\win\audio_low_latency_input_win.cc:1055] buffers_required=20
```In the example above, the fake client will use 256 and a real client will use 4800.
Henrik AndreassonFix applied.
It seems that I don't have access issue 323486298. Is it ok for me to be able to see it?
Greg Thompsonb/323486298 is unrelated to your work and I can't find it as crbug.com/323486298 either. Note sure where you got 323486298 from. Can you create your own/new issue and us it instead.
I was just about to write more-or-less the same thing.
Done
Olga Sharonovathere's no synchronization around this (accessed by multiple threads), nor do i see anything that ensures that the tasks posted below (which use `Unretained`) will not run after this instance is destroyed. although this is test code, it would be good to fix this so that we don't end up with flaky tests.
+1
Done
if (buffer_ready_event_handle_) {
SetEvent(buffer_ready_event_handle_);
}
Do we expect IAudioClient to stop synchronously? In that case, shouldn't this be called only if `should_stop_streaming_` is true?
An typical alternative to should_stop_streaming would be: posting from Stop() to base::ThreadPool a task to signal an event, and waiting in Stop() for it to be signaled. (This would mean all the earlier posted tasks have cycled through.)
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
// Post a task to signal the event after all tasks are flushed.
if the intent here is to have this task run after any existing call to `StreamData`, then all tasks need to be posted to the same `SequencedTaskRunner`. also note that this won't prevent a previously-posted delayed task from running. you could introduce a `WeakPtrFactory` and post tasks w/ a weak ptr to prevent them from running after destruction.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Code-Review | +1 |
// Post a task to signal the event after all tasks are flushed.
if the intent here is to have this task run after any existing call to `StreamData`, then all tasks need to be posted to the same `SequencedTaskRunner`. also note that this won't prevent a previously-posted delayed task from running. you could introduce a `WeakPtrFactory` and post tasks w/ a weak ptr to prevent them from running after destruction.
Thanks Greg. I hope my last version addresses your comments. PTAL.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
// Post a task to signal the event after all tasks are flushed.
Henrik Andreassonif the intent here is to have this task run after any existing call to `StreamData`, then all tasks need to be posted to the same `SequencedTaskRunner`. also note that this won't prevent a previously-posted delayed task from running. you could introduce a `WeakPtrFactory` and post tasks w/ a weak ptr to prevent them from running after destruction.
Thanks Greg. I hope my last version addresses your comments. PTAL.
on second thought, i think i gave you some bad advice with that WeakPtr suggestion. i think it's better to use a SequenceBound to run a timer. something close to this should do the trick:
```
class FakeIAudioClient {
public:
...
private:
class DataStreamer {
public:
explicit DataStreamer(HANDLE buffer_ready_event_handle);
DataStreamer(const DataStreamer&) = delete;
DataStreamer& operator=(const DataStreamer&) = delete;
~DataStreamer();
void Stop(base::WaitableEvent &stop_event);
private:
void StreamData();
const HANDLE buffer_ready_event_handle_;
base::RepeatingTimer timer_;
};
base::SequenceBound<DataStreamer> streamer_;
};
FakeIAudioClient::DataStreamer::DataStreamer(HANDLE buffer_ready_event_handle)
: buffer_ready_event_handle_(buffer_ready_event_handle) {
// Stream once immediately.
StreamData();
// And again every kSamplingPeriodMs milliseconds.
timer_.Start(FROM_HERE, base::Milliseconds(kSamplingPeriodMs),
base::BindRepeating(&FakeIAudioClient::DataStreamer::StreamData,
base::Unretained(this)));
}
FakeIAudioClient::DataStreamer::~DataStreamer() = default;
void FakeIAudioClient::DataStreamer::Stop(base::WaitableEvent &stop_event) {
timer_.Stop();
stop_event.Signal();
}
void FakeIAudioClient::DataStreamer::StreamData() {
if (buffer_ready_event_handle_) {
::SetEvent(buffer_ready_event_handle_);
}
}
void FakeIAudioClient::Start() {
if (!streamer_.is_null()) {
return; // Already running.
}
// Start the streamer, giving it the client's event to be signaled.
streamer_.emplace(base::ThreadPool::CreateSequencedTaskRunner({}),
buffer_ready_event_handle_);
}
void FakeIAudioClient::Stop() {
if (streamer_.is_null()) {
return; // Not running.
}
// Stop the streamer and wait for it to acknowledge.
base::WaitableEvent stop_event;
streamer_.AsyncCall(&DataStreamer::Stop).WithArgs(std::ref(stop_event));
stop_event.Wait();
// Destroy the streamer.
streamer_.Reset();
}
```
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Code-Review | +1 |
// Post a task to signal the event after all tasks are flushed.
Now that was a fancy pattern :-)
Thanks for the detailed feedback Greg.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Code-Review | +1 |
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Code-Review | +1 |
Yes. Also verified ASan. Only required some minor/trivial changes in your proposal. Very impressive!
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
zequ...@google.com: we need your OK again for `build/config/win/BUILD.gn` in this attempt to reland a previously reverted CL.
It is my assumption that you will still be the author in git blame for this CL. Hope I did not mess this up here by uploading some final changes.
Looks like we still need an OK from zequ...@google.com.
Code-Review | +1 |
Commit-Queue | +1 |
I did try `git commit --amend --author="Gabriel Brito <gabrie...@microsoft.com>" --no-edit` and then `git cl upload` but I am not sure that we are OK yet.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
I took the liberty of fixing the remaining comments to ensure that we can land this essential CL again as soon as possible.
I have pending work which need the new structure.
No problem on my side. Thanks for doing it! It would have taken me days to fix all of it given out time zone differences.
It is my assumption that you will still be the author in git blame for this CL. Hope I did not mess this up here by uploading some final changes.
Looks like we still need an OK from zequ...@google.com.
I added a simple comment to upload a new patchset. I guess now it looks like how it was before.
@hen...@chromium.org thanks for working through the additional comments.
Also, thanks everyone for the additional feedback. While I did not implement it, I read through everything and understood the idea behind `DataStreamer`. The previous implementation was lacking for using `PostDelayedTask` and `base::Unretained` indiscriminately.
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Reland "[gDM] Window audio capture WASAPI implementation"
This is a reland of commit 42f725b8b8d20758e49c0b9cbcaded7bc6fdfe95
Whenever the *Size fake functions (GetNextPacketSize, GetBufferSize,
etc) were called, we were returning the size in "number of bytes"
instead of "number of frames". For example, a dual channel frame in
Windows has 4 bytes, which results in the fifo always trying to read off
bounds in the test data buffer.
The WAVEFORMATEXT struct has enough information that allows us to
calculate this information at runtime and simulate it properly. The
latest patchset adds this. Tested on a local Windows ASAN build and the
tests executed successfully.
Original change's description:
> [gDM] Window audio capture WASAPI implementation
>
> Adds the process-based audio capture logic that will power
> getDisplayMedia window capture on Windows devices. This CL also adds
> tests and testing mocks for the Windows WASAPI interfaces. This CL
> builds on the work in
> https://chromium-review.googlesource.com/c/chromium/src/+/5092148.
>
> Bug: 40947205
> Change-Id: I971b5b6a9c715b45ddcdbcc40466b5da8f2f816e
> Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6442207
> Reviewed-by: Henrik Andreasson <hen...@chromium.org>
> Reviewed-by: Olga Sharonova <ol...@chromium.org>
> Reviewed-by: Zequan Wu <zequ...@google.com>
> Commit-Queue: Gabriel Brito <gabrie...@microsoft.com>
> Reviewed-by: Greg Thompson <g...@chromium.org>
> Cr-Commit-Position: refs/heads/main@{#1461128}
Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |