How to modify the Captured Sound
sanju
I'm new to DirectX application development, and i seek some help from
this group.
Problem Statement:
I've a USB microphone, that sends the sound data via USB at 44.1khz. I
need to capture the sound and analyze the data and process it.
I've gone through the source code of DirectX c++ sample of
'capturesound' and there they easily capture the data in the capture
buffer and write the captured data to a wav file.
What kind of data is the raw data, which is coming from the USB port
to the Capture buffer?
The data which is in capture buffer can be written to a wav file, When
i want the data which is actually in the capture buffer, the read
operation from the capture buffer the same as what we get from the USB
port?
When i read the data from the capture buffer, is it possible to
analyze and process each and every byte and store it in another buffer
instead of saving it into a file of .wav format?
Please answer. Thank you.
Regards,
sanjay
sanju
'capturesound' application, the voice is not getting recorded, however
when i use some other application like gtalk to do voice chat,the
other person is able to hear me, what may be the reason for this
failure?
Chris P.
Make sure the recording mixer is set to the correct source (XP), or make
sure the appropriate device is the default (Vista).
--
http://www.chrisnet.net/code.htm
[MS MVP for DirectShow / MediaFoundation]
Chris P.
The data is captured as PCM (Pulse Code Modulation) data. The most common
format is 16-bit PCM where the signal is amplitude sampled 44100 times a
second with the values ranging from -32768 to +32767. Google PCM audio
format for the details.
It is easy to process the audio in real time on a PC without saving to
disk. On a fast PC I can process hundreds of channels.
sanju
i'm also showing the data that is captured as in the following printf
(pbCaptureData is of type BYTE* ).
printf("\n\nThe data is %hu",*pbCaptureData);
It shows the data recieved as numbers between 0-255, like
0
231
12
56
89
23
112
....and so on.. But when there is no sound(for example, i've stopped
the playback sound which was getting recorded) it still shows some
values like
13
44
56
36
78
236
,..and so on...when there is no sound generated at all(from microphone
or from a media file) how can it show these values?..is it noise or it
is printing junk data getting printed in both these cases?
sanju
When some device sends the data,say for example, a probe, which is
sending ome data at 44.1khz, which may actually not come under the
audible sund..if i use this direct x application, then it will
automatically convert the data into the pcm wave format by modifying
the actual data and hence the actual data which is sent by the device
is lost. So is there any way where i can make use of the direct x just
to get the data what is coming from the device?..Correct me if my
question itself is wrong.
Chris P.
Capture data is not modified except for sample rate conversion. If you do
not wish to have sample rate conversion then you have to record at the
native sample rate of the device.
The data is not of type BYTE unless you are capturing at 8-bit resolution
(which would be poor quality). If you are capturing at 16-bit 1-channel
the data is of type SHORT and you will need to add a cast to a pointer of
type short*.
There may be some small fluctuations in the data values even if there is
not audio present.
Select your recording source using the recording mixer control panel, in
XP/2K use "sndvol32 /r" or select it from inside the "Sounds & Audio
Devices" control panel in the recording section.
sanju
> not wish to have sample rate conversion then you have to record at the
> native sample rate of the device.
Thanks Chris.
How to achieve what you have said? Because when i specify it as
waveformat of pcm type then automatically the sample rate conversion
takes place, how to record it at native sample rate?
Chris P.
> How to achieve what you have said? Because when i specify it as
> waveformat of pcm type then automatically the sample rate conversion
> takes place, how to record it at native sample rate?
In XP there is no way to know for sure, for most devices it is going to be
44.1 or 48kHz. In Vista you can query what the current sampling rate is
using WASAPI.
sanju
sanju
> ok. thanks a lot chris.
Chris,
Even when i gave it as type short*, i got data like below, when there
was no sound to record.
without sound-> 23, 9, 7 ,11, 6, 17, 9, 11, 13, 7, 12, 8, 7, -1, -3...
with sound-> -4, -9, 4, -7, -5, 3, -2, 14 6, 12, 14...
the code is as below.
RecordCapturedData()
{
SHORT* pbCaptureData = NULL;
DWORD dwCaptureLength;
SHORT* pbCaptureData2 = NULL;
DWORD dwCaptureLength2;
SHORT* buf;
if( FAILED( hr = g_pDSBCapture->GetCurrentPosition( &dwCapturePos,
&dwReadPos ) ) )
{
printf("\nGetCurrentPosition Failed");
return S_FALSE;
}
lLockSize = dwReadPos - g_dwNextCaptureOffset;
if( lLockSize < 0 )
lLockSize += g_dwCaptureBufferSize;
// Block align lock size so that we are always write on a boundary
lLockSize -= (lLockSize % g_dwNotifySize);
if( lLockSize == 0 )
return S_FALSE;
// Lock the capture buffer down
if( FAILED( hr = g_pDSBCapture->Lock( g_dwNextCaptureOffset,
lLockSize,
(LPVOID*)&pbCaptureData,
&dwCaptureLength,
(LPVOID*)&pbCaptureData2,
&dwCaptureLength2, 0L ) ) )
printf("\nLock Failed");
if( FAILED( hr = g_pWaveFile->Write( dwCaptureLength,
(BYTE*)pbCaptureData,
&dwDataWrote ) ) )
printf("\nWrite Failed");
buf=(SHORT*)malloc(dwCaptureLength);
memcpy(buf,pbCaptureData,dwCaptureLength);
printf("\n\nThe data is %hd %hd",*pbCaptureData,*buf);
}
What could be wrong?. Please help.
Regards,
Sanjay
Chris P.
> Even when i gave it as type short*, i got data like below, when there
> was no sound to record.
>
> without sound-> 23, 9, 7 ,11, 6, 17, 9, 11, 13, 7, 12, 8, 7, -1, -3...
> with sound-> -4, -9, 4, -7, -5, 3, -2, 14 6, 12, 14...
How are you determining these values?
sanju
> How are you determining these values?
I'm printing the values what is coming into the Capture buffer, i.e
pbCaptureData.'Without sound' values of pbCaptureData are the values
that were obtained when the system is muted, and 'with sound' values
were obtained when there was a playback song/sound. Am I making myself
clear on this?
Chris P.
> I'm printing the values what is coming into the Capture buffer, i.e
> pbCaptureData.'Without sound' values of pbCaptureData are the values
> that were obtained when the system is muted, and 'with sound' values
> were obtained when there was a playback song/sound. Am I making myself
> clear on this?
You are capturing audio input, not output. What you are playing on the
system is completely irrelevant to the input capture unless you set the
input source to 'Stereo Mix', 'What U Hear', or equivalent.
sanju
My doubts could sound obvious and stupid. Thanks for responding.
> You are capturing audio input, not output. What you are playing on the
> system is completely irrelevant to the input capture
But, before i write to the wave file it has to be the output data
captured in the capture buffer right?
i.e when i call the wavefile write as below, it has to write the data
captured in the capture buffer to a wav file.correct?
g_pWaveFile->Write( dwCaptureLength, (BYTE*)
pbCaptureData,&dwDataWrote )
If i'm wrong then how to capture the output data and display it?
Regds,
Sanjay
Chris P.
> But, before i write to the wave file it has to be the output data
> captured in the capture buffer right?
> i.e when i call the wavefile write as below, it has to write the data
> captured in the capture buffer to a wav file.correct?
> g_pWaveFile->Write( dwCaptureLength, (BYTE*)
> pbCaptureData,&dwDataWrote )
>
> If i'm wrong then how to capture the output data and display it?
You are always capturing the input, you cannot directly capture the output.
Setting the input to a loopback mode so that the output is sent to the
input is the easiest way to capture the output audio. Open the recording
mixer from the control panel and set the input source to 'Stereo Mix'.
If you need to also capture microphone input at the same time then it
becomes much more complicated, at which time a virtual audio device becomes
necessary.
sanju
>
> Setting the input to a loopback mode so that the output is sent to the
> input is the easiest way to capture the output audio. Open the recording
> mixer from the control panel and set the input source to 'Stereo Mix'.
Oh!..Okay..How is the output different from the input?..what are the
changes the driver(sound driver that is enumerated) is making to the
input to get the different output? And using the loopback, the output
is sent as input to the capture buffer or as an input to the driver
again? Is'nt this very dependent on the driver the system uses?
because, if the driver does not support 'what you hear'/'stereo mix'
then we cannot get the actual output data at all!
> If you need to also capture microphone input at the same time then it
> becomes much more complicated, at which time a virtual audio device becomes
> necessary.
Is the above loopback mode applicable to the sound playing in the
system and not to the sound coming from the microphone?, if i dont
have to capture the playback sound playing in the system and capture
the sound only from the microphone, then can the loopback mode be
avoided?, or do i have to make use of the same loopback to send the
output as an input and then capture input?
---
Thanks,
Sanjay
Chris P.
> Oh!..Okay..How is the output different from the input?..what are the
> changes the driver(sound driver that is enumerated) is making to the
> input to get the different output? And using the loopback, the output
> is sent as input to the capture buffer or as an input to the driver
> again? Is'nt this very dependent on the driver the system uses?
> because, if the driver does not support 'what you hear'/'stereo mix'
> then we cannot get the actual output data at all!
Correct, if the system doesn't support 'what you hear'/'stereo mix' then
you can not capture the output audio easily. Windows does not provide a
method to capture output audio easily, although there is a method starting
in Vista that makes this possible if the software supports it.
> Is the above loopback mode applicable to the sound playing in the
> system and not to the sound coming from the microphone?, if i dont
> have to capture the playback sound playing in the system and capture
> the sound only from the microphone, then can the loopback mode be
> avoided?, or do i have to make use of the same loopback to send the
> output as an input and then capture input?
Yes. If you are just capturing the microphone then that is regular capture
and does not need the loopback. You can not capture the loopback and the
microphone at the same time.
sanju
> Yes. If you are just capturing the microphone then that is regular capture
> and does not need the loopback. You can not capture the loopback and the
> microphone at the same time.
then from the way i'm capturing data(as in previous posts), is the
data i'm outputting and the data coming from the microphone same?
Chris P.
> then from the way i'm capturing data(as in previous posts), is the
> data i'm outputting and the data coming from the microphone same?
If your source is set to microphone, then yes.
sanju
> If your source is set to microphone, then yes.
we enumerate for the sound driver in the system, and capture the data
coming to that driver, if there is an mp3 file playing and if i
connect microphone and record it, then both these sounds will be
recorded, correct? then when i output the data coming from the driver,
it will be only microphone's sound that will be displayed. Am i right?
Chris P.
> we enumerate for the sound driver in the system, and capture the data
> coming to that driver, if there is an mp3 file playing and if i
> connect microphone and record it, then both these sounds will be
> recorded, correct? then when i output the data coming from the driver,
> it will be only microphone's sound that will be displayed. Am i right?
You can capture either the microphone data or the loopback data, not both
at the same time (with the exception of some soundblaster cards).
Obviously if you are playing back through speakers then the microphone is
going to pick up the sound from the speakers and hence you would capture
the audio from the mp3 indirectly with significantly reduced quality.
Sanjay
is the data in my buffer same as what is there in the capture buffer
if i capture in this way?..
HRESULT RecordCapturedData()
{
HRESULT hr;
VOID* pbCaptureData = NULL;
DWORD dwCaptureLength=0;
VOID* pbCaptureData2 = NULL;
DWORD dwCaptureLength2=0;
//UINT dwDataWrote;
DWORD dwReadPos;
DWORD dwCapturePos;
LONG lLockSize;
static int i;
/*if( NULL == g_pWaveFile )
return S_FALSE;*/
if( NULL == g_pDSBCapture )
return S_FALSE;
if( FAILED( hr = g_pDSBCapture->GetCurrentPosition
( &dwCapturePos, &dwReadPos ) ) )
{
printf("\nGetCurrentPosition Failed");
return S_FALSE;
}
lLockSize = dwReadPos - g_dwNextCaptureOffset;
if( lLockSize < 0 )
lLockSize += g_dwCaptureBufferSize;
// Block align lock size so that we are always write on a boundary
lLockSize -= (lLockSize % g_dwNotifySize);
if( lLockSize == 0 )
return S_FALSE;
// Lock the capture buffer down
if( FAILED( hr = g_pDSBCapture->Lock( g_dwNextCaptureOffset,
lLockSize,
&pbCaptureData,
&dwCaptureLength,
&pbCaptureData2,
&dwCaptureLength2, 0L ) ) )
printf("\nLock Failed");
buf=(byte*)malloc(100000000);
memcpy(buf,pbCaptureData,dwCaptureLength);
// Move the capture offset along
g_dwNextCaptureOffset += dwCaptureLength;
g_dwNextCaptureOffset %= g_dwCaptureBufferSize; // Circular buffer
if( pbCaptureData2 != NULL )
{
g_dwNextCaptureOffset += dwCaptureLength2;
g_dwNextCaptureOffset %= g_dwCaptureBufferSize; // Circular
buffer*/
}
// Unlock the capture buffer
g_pDSBCapture->Unlock( (LPVOID*)pbCaptureData, dwCaptureLength,
(LPVOID*)pbCaptureData2,
dwCaptureLength2 );
return S_OK;
}
if not, how can i copy the data in the capture buffer to my buffer?
Sanjay
I'm capturing it from the microphone.
In simple words, how can i write the data of capture buffer into an
open memory buffer instead of a wave file?
I need to do it because i'm processing the signal data received from
the microphone, and writing and reading from a file will make it slow,
so i want to write it to a memory/buffer and pass the buffer pointer
to my signal processing matlab application. So how can i write the
data from the capture buffer to another buffer?
Chris P.
> I need to do it because i'm processing the signal data received from
> the microphone, and writing and reading from a file will make it slow,
> so i want to write it to a memory/buffer and pass the buffer pointer
> to my signal processing matlab application. So how can i write the
> data from the capture buffer to another buffer?
If you are passing it to something that is running in the same process
space, memcpy. If it is in another process then you have to use memory
mapped space.
Sanjay
> If you are passing it to something that is running in the same process
> space, memcpy. If it is in another process then you have to use memory
> mapped space.
Thanks chris. Finally I got something right. :)
To copy the data from capture buffer to my buffer, the following tasks
are needed->
1.lock the capture buffer.
2.memcpy to my buffer(if in the same process).
3.unlock the capture buffer.
So, when i create a wave file and write the values that are stored in
the buffer, and play the file, it should play the recorded data from
the microphone. correct?
One more to add, the last parameter for memcpy is the dwCaptureLength
(or could be even locksize of the buffer) and the data of the source
will be always pbCaptureData?
So when i say memcpy(mybuffer,pbCaptureData,dwCaptureLength);
recording at 44.1khz,16bit,2channels, the pbCaptureBuffer
(capturebuffer) has data of 22048 bytes that has to be copied to
destination buffer(my buffer) for processing. right?
When its 2 channels, is it ok to have a single dimensional array as
buffer or i must have 2 dimensional array as buffer?
please reply
Chris P.
> Thanks chris. Finally I got something right. :)
> To copy the data from capture buffer to my buffer, the following tasks
> are needed->
> 1.lock the capture buffer.
> 2.memcpy to my buffer(if in the same process).
> 3.unlock the capture buffer.
> So, when i create a wave file and write the values that are stored in
> the buffer, and play the file, it should play the recorded data from
> the microphone. correct?
Correct.
> One more to add, the last parameter for memcpy is the dwCaptureLength
> (or could be even locksize of the buffer) and the data of the source
> will be always pbCaptureData?
If you are doing one shot recordings then the buffer will start from the
beginning. If you are recording continuously then the buffer is circular,
and you should remember your last read position and measure the distance to
the write cursor to determine the length of data available.
> So when i say memcpy(mybuffer,pbCaptureData,dwCaptureLength);
> recording at 44.1khz,16bit,2channels, the pbCaptureBuffer
> (capturebuffer) has data of 22048 bytes that has to be copied to
> destination buffer(my buffer) for processing. right?
I'm not quite sure what you mean by this question. If you had a 1 second
recording then you would have 44100 stereo 16-bit samples, this would equal
44100 * 2 * 2 = 176400 bytes.
> When its 2 channels, is it ok to have a single dimensional array as
> buffer or i must have 2 dimensional array as buffer?
1 dimensional array is correct. You must deinterleave the data as you
process it if you need to process the channels separately.
Sanjay
when i give the buffer data that i'm copying to as byte, i get values
like 128,127 which is correct for processing.
Now if i give buffer format as double, for processing, i'm getting
data something like this
-1.065814e^-301
-1.65814e^-302
1.35650e^-330...and so on..where as i can expect it to be between -1
and +1. what could be wrong?
and also, does the wave file write function do any sort of data
conversion before writing to a file?..just got this thought. i'm not
sure if the question is valid. :)
Chris P.
The format of the buffer is the format at which you opened the device. If
you opened it as 16-bit stereo then the buffer is an array of signed words
with the left channel in the even index and the right channel in the odd.
If you want to have floating point values then you must do the conversion
yourself. To convert the data to a range of -1.0 to +1.0 you divide by the
max value, so to convert 16-bit to floating point you divide by 32768.
You can also request to open the device as floating point by specifying the
format type as WAVE_FORMAT_IEEE_FLOAT, which is 32-bit floating point
equivalent to the C 'float' type.
Sanjay
> If you want to have floating point values then you must do the conversion
> yourself. To convert the data to a range of -1.0 to +1.0 you divide by the
> max value, so to convert 16-bit to floating point you divide by 32768.
I took the data type of buffer as short*, and then captured the data
and I write the short* type to wav file by typecasting to byte*, but
when i play the wav file, i get jerks in sound. I know the short* is
16 bit and byte* is 8 bit, but i'm recording at 16bit, stereo format
so using short* is justified but wav file takes only byte* as
parameter, so i must type cast shrt* to byte*. Is there any alternate
solution to it?
i'm writing to a wave file like this->
if( FAILED( g_pWaveFile->Write( k, (BYTE*)MyBuf,&dwDataWrote ) ) )
where MyBuf is of type short*. When I DSP the floating point values
and play those floating point values, do i get the proper recorded
audio without any jerks?
Chris P.
> i'm writing to a wave file like this->
> if( FAILED( g_pWaveFile->Write( k, (BYTE*)MyBuf,&dwDataWrote ) ) )
>
> where MyBuf is of type short*. When I DSP the floating point values
> and play those floating point values, do i get the proper recorded
> audio without any jerks?
You may be telling it the wrong size to write.
Sanjay
> You may be telling it the wrong size to write.
oh okay. When it was byte*, i was writing 8000 bytes, now when it is
short* i will have to write double that value. Spot on.
but When I DSP the floating point values and play those floating point
Chris P.
> but When I DSP the floating point values and play those floating point
> values, do i get the proper recorded audio without any jerks?
Sure, as long as you tell it the correct number of samples.
Sanjay
> On Tue, 28 Jul 2009 08:33:28 -0700 (PDT), Sanjay wrote:
> > but When I DSP the floating point values and play those floating point
> > values, do i get the proper recorded audio without any jerks?
>
> Sure, as long as you tell it the correct number of samples.
>
> [MS MVP for DirectShow / MediaFoundation]
thanks chris.