[gentoo-user] [OT] ffmpeg video+audio capture question

[Walter Dnes]

I'm trying to do a video+audio capture of my desktop, e.g. a Youtube
video playing, or whatever. I've got video capture working, but not
audio. I've compiled ffmpeg with both "alsa" and "oss" USE flags and
tried the commands...

ffmpeg -f alsa -ac 2 -i hw:0 -video_size 1920x1080 -framerate 25 -f x11grab -i :0.0 output.mp4


ffmpeg -f oss -i /dev/dsp -video_size 1920x1080 -framerate 25 -f x11grab -i :0.0 output.mp4

In both case the video capture works, and ffmpeg claims to be
capturing audio, but I don't get any audio capture at all. What am I
doing wrong? Any more USE flags to set?

--
Walter Dnes <walt...@waltdnes.org>
I don't run "desktop environments"; I run useful applications

[Mick]

On Wednesday, 18 September 2019 10:10:16 BST Walter Dnes wrote:
> I'm trying to do a video+audio capture of my desktop, e.g. a Youtube
> video playing, or whatever. I've got video capture working, but not
> audio. I've compiled ffmpeg with both "alsa" and "oss" USE flags and
> tried the commands...
>
> ffmpeg -f alsa -ac 2 -i hw:0 -video_size 1920x1080 -framerate 25 -f x11grab
> -i :0.0 output.mp4

The above should work, but only if your active audio card has Card ID 0. If
your PC uses HDMI as the first card you'll need to adjust /etc/asound.conf
accordingly to switch to your analogue audio device, or perhaps try hw:1 in
the above incantation instead.

Also, check your Mic is not muted with alsamixer, or your desktop GUI
equivalent.

> ffmpeg -f oss -i /dev/dsp -video_size 1920x1080 -framerate 25 -f x11grab -i
> :0.0 output.mp4

I don't have oss experience and as I understand it pre-OSS4 is mono only(?),
but again the above ought to work, as long as there is no clash with alsa.
You may need jackd to plug in/out the requisite input devices and sinks - too
complicated for just a simple audio recording.

> In both case the video capture works, and ffmpeg claims to be
> capturing audio, but I don't get any audio capture at all. What am I
> doing wrong? Any more USE flags to set?

Did you try USE="alsa, -speex"?

--
Regards,

Mick

[Walter Dnes]

On Wed, Sep 18, 2019 at 11:16:41AM +0100, Mick wrote

> On Wednesday, 18 September 2019 10:10:16 BST Walter Dnes wrote:
> >
> > ffmpeg -f alsa -ac 2 -i hw:0 -video_size 1920x1080 -framerate 25 -f x11grab
> > -i :0.0 output.mp4
>
> The above should work, but only if your active audio card has Card
> ID 0. If your PC uses HDMI as the first card you'll need to adjust
> /etc/asound.conf accordingly to switch to your analogue audio device,
> or perhaps try hw:1 in the above incantation instead.

"aplay -l" outputs

**** List of PLAYBACK Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: CX20641 Analog [CX20641 Analog]
Subdevices: 0/1
Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 3: Generic Digital [Generic Digital]
Subdevices: 1/1
Subdevice #0: subdevice #0

If I try "hw:1" it dies immediately (No such file or directory)

No /etc/asound.conf or $HOME/.asoundrc on my system. But there is a
large (9596 bytes) /usr/share/alsa/alsa.conf

> Also, check your Mic is not muted with alsamixer, or your desktop GUI
> equivalent.

Enabled them, but no difference.

> Did you try USE="alsa, -speex"?

"emerge -pv mlayer" shows that speex is disabled.

The console I launch ffmpeg from shows a lot of stuff like...

[alsa @ 0x559c480c94c0] ALSA buffer xrun.
frame= 2337 fps= 25 q=28.0 size= 11008kB time=00:01:33.05 bitrate= 969.1kbits/frame= 2350 fps= 25 q=28.0 size= 11008kB time=00:01:33.05 bitrate= 969.1kbits/frame= 2362 fps= 25 q=28.0 size= 11008kB time=00:01:33.05 bitrate= 969.1kbits/frame= 2375 fps= 25 q=28.0 size= 11008kB time=00:01:33.05 bitrate= 969.1kbits/[alsa @ 0x559c480c94c0] ALSA buffer xrun.
frame= 2387 fps= 25 q=28.0 size= 11264kB time=00:01:35.21 bitrate= 969.2kbits/frame= 2400 fps= 25 q=28.0 size= 11264kB time=00:01:35.21 bitrate= 969.2kbits/frame= 2412 fps= 25 q=28.0 size= 11264kB time=00:01:35.21 bitrate= 969.2kbits/frame= 2425 fps= 25 q=28.0 size= 11264kB time=00:01:35.21 bitrate= 969.2kbits/[alsa @ 0x559c480c94c0] ALSA buffer xrun.

When I playback with mplayer and exit, there's a bunch of diagnostics.
Towards the end it says...

==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 48000 Hz, 2 ch, floatle, 0.1 kbit/0.00% (ratio: 17->384000)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
AO: [alsa] 48000Hz 2ch floatle (4 bytes per sample)
Starting playback...
Could not find matching colorspace - retrying with -vf scale...
Opening video filter: [scale]
Movie-Aspect is undefined - no prescaling applied.
[swscaler @ 0x7f28e3f20100]bicubic scaler, from yuv444p to yuv420p using MMXEXT
VO: [xv] 1920x1080 => 1920x1080 Planar YV12
A: 38.9 V: 35.9 A-V: 3.003 ct: 3.588 0/ 0 17% 12% 0.0% 0 0
Timing looks severely broken, resetting
A: 38.9 V: 39.6 A-V: -0.717 ct: 3.805 0/ 0 17% 12% 0.0% 0 0

The message about timing may be a clue.

[Mick]

On Wednesday, 18 September 2019 19:03:02 BST Walter Dnes wrote:
> On Wed, Sep 18, 2019 at 11:16:41AM +0100, Mick wrote
>
> > On Wednesday, 18 September 2019 10:10:16 BST Walter Dnes wrote:
> > > ffmpeg -f alsa -ac 2 -i hw:0 -video_size 1920x1080 -framerate 25 -f
> > > x11grab
> > > -i :0.0 output.mp4
> >
> > The above should work, but only if your active audio card has Card
> > ID 0. If your PC uses HDMI as the first card you'll need to adjust
> > /etc/asound.conf accordingly to switch to your analogue audio device,
> > or perhaps try hw:1 in the above incantation instead.
>
> "aplay -l" outputs
>
> **** List of PLAYBACK Hardware Devices ****
> card 0: PCH [HDA Intel PCH], device 0: CX20641 Analog [CX20641 Analog]
> Subdevices: 0/1
> Subdevice #0: subdevice #0
> card 0: PCH [HDA Intel PCH], device 3: Generic Digital [Generic Digital]
> Subdevices: 1/1
> Subdevice #0: subdevice #0
>
> If I try "hw:1" it dies immediately (No such file or directory)

Yes, you only have one card 0. The first device (default) is the analogue.

What does 'arecord -l' show?

> No /etc/asound.conf or $HOME/.asoundrc on my system. But there is a
> large (9596 bytes) /usr/share/alsa/alsa.conf

You would need to create a asound.conf card to switch from the HDMI to
analogue as the default card, if your hardware was like mine - but it is not.


Let's try something else, specifying a sub-device. What happens with:

-i hw:0,1

or

-i hw:0,0,1


Adjust the above according to what arecord shows.

I think timing issues should not affect the audio (it could be out of sync
with the video). The ffmpeg output seems noisy and you're getting alsa buffer
over/under runs. Try this:

-f alsa -thread_queue_size 2048

or use 1024, if 2048 takes too long to allow the recording to start. This
should do away with the message 'ALSA buffer xrun'.

The mplayer output shows you have an audio stream alright and I bet ffprobe
will show the same. Unless you're trying to record frequencies not audible by
the human ear, I can think of one more thing.

Maximise the Mic's capture volume and any boost settings available. Once I
was trying to perform a voice recording using the built in microphone on a
lesser quality laptop, not understanding why I couldn't hear any audio on
playback. Well, I had to push the capture volume all the way up to 100%
before I heard audio being played back. What a relief! I thought I was going
deaf. LOL!!

HTH.
--
Regards,

Mick

[Walter Dnes]

On Wed, Sep 18, 2019 at 09:14:44PM +0100, Mick wrote

> Yes, you only have one card 0. The first device (default) is the analogue.
>
> What does 'arecord -l' show?

**** List of CAPTURE Hardware Devices ****

card 0: PCH [HDA Intel PCH], device 0: CX20641 Analog [CX20641 Analog]

Subdevices: 1/1
Subdevice #0: subdevice #0

card 0: PCH [HDA Intel PCH], device 2: CX20641 Alt Analog [CX20641 Alt Analog]

Subdevices: 1/1
Subdevice #0: subdevice #0

> Let's try something else, specifying a sub-device. What happens with:
>
> -i hw:0,1
>
> or
>
> -i hw:0,0,1

Both fail with No such file or directory.

> The ffmpeg output seems noisy and you're getting alsa buffer
> over/under runs. Try this:
>
> -f alsa -thread_queue_size 2048

That gets rid of the warnings and Alsa xrun messages. But still no
help. I did some further digging. It appears that the instructions I
followed are for capture devices (USB microphones, etc). According to
https://trac.ffmpeg.org/wiki/Capture/ALSA if I want to record from a
running application, I need to load the "snd_aloop" module and mention
it in a local .asoundrc. According to the linux kernel config help...

CONFIG_SND_ALOOP:

| Say 'Y' or 'M' to include support for the PCM loopback device. |
| This module returns played samples back to the user space using |
| the standard ALSA PCM device |

Oh well, there is a new kernel, so I'll build it with the module, and
get back later to see how things work.

[Adam Carter]

On Thu, Sep 19, 2019 at 9:58 AM Walter Dnes <walt...@waltdnes.org> wrote:

On Wed, Sep 18, 2019 at 09:14:44PM +0100, Mick wrote

> Yes, you only have one card 0.  The first device (default) is the analogue.
>
> What does 'arecord -l' show?

**** List of CAPTURE Hardware Devices ****
card 0: PCH [HDA Intel PCH], device 0: CX20641 Analog [CX20641 Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: PCH [HDA Intel PCH], device 2: CX20641 Alt Analog [CX20641 Alt Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

I don't run "desktop environments"; I run useful applications

I think for what you are trying to do, the capture devices/mic are not relevant - there will be no signal on them. The playback device have the signal, and ffmpeg needs to capture of those signals.

> -i hw:0,1
>
> or
>
> -i hw:0,0,1

  Both fail with No such file or directory.

FWIW audacity has a drop down list of the output devices, and on my system they're all shown as hw:X,Y. There's no hw:X or hw:X,Y,Z. I also notice that the same hw:X,Y is used for both an playback and capture device, so i guess that means that '-i hw:0.1' is using the hw:0,1 capture device, which for you has no signal, and therefore ffmpeg doesnt capture anything.

Hopefully there's some way to capture the output digitally, but if not, a cable connecting the analogue out to the analogue in should work.

[Mick]

I'm sorry - I had misunderstood the original requirement! I thought Walter
wanted to capture the whole desktop (not a single application window within
it) and record voice along with it - not the sound of the application. The
command shown does that - records a screencast of the desktop and voice from
the microphone.

Having read the first email to the list once more, I can confirm the ffmpeg
command will work, but only with the audio loopback configured.

It should also work nicely with jackd - which along with qjackctl would be my
preference.

https://qjackctl.sourceforge.io/qjackctl-index.html

--
Regards,

Mick

[Walter Dnes]

On Wed, Sep 18, 2019 at 07:57:51PM -0400, Walter Dnes wrote

>
> I did some further digging. It appears that the instructions I
> followed are for capture devices (USB microphones, etc). According to
> https://trac.ffmpeg.org/wiki/Capture/ALSA if I want to record from a
> running application, I need to load the "snd_aloop" module and mention
> it in a local .asoundrc. According to the linux kernel config help...
>
> CONFIG_SND_ALOOP:
>
> | Say 'Y' or 'M' to include support for the PCM loopback device. |
> | This module returns played samples back to the user space using |
> | the standard ALSA PCM device |
>
> Oh well, there is a new kernel, so I'll build it with the module, and
> get back later to see how things work.

That failed differently. With the local .asoundrc present, I loaded
the snd_aloop module. It showed up as a capture device in alsamixer.
But I had no sound on the speakers when playing Youtube. Even worse, I
was unable to unload the snd_aloop module. When trying to do so, I got

FATAL ERROR: Module in use.

The normal way around that is to unload any other modules using it.
But I have everything, except snd_aloop, built into the kernel. So
there was nothing to unload to release snd_aloop. I ended up renaming
.asoundrc, and rebooting.

[Mick]

I suggest you give jackd a spin. It should what you want, but may I ask why
you need to record off your browser whatever youtube shows? Can't you use
youtube-dl to download the youtube video/audio?

--
Regards,

Mick

[Dale]

I use a add-on to download videos from sites youtube-dl doesn't work
with.  I think it is called Video Download Helper.  It works pretty
well.  It also allows you to pick what resolution you want if there is
more than one available.  My DSL isn't fast enough for HD videos but I
can use the download tool to select a HD stream when I want to. 

Just another option.

Dale

:-)  :-) 

[Walter Dnes]

On Fri, Sep 20, 2019 at 02:09:58PM +0100, Mick wrote

>
> I suggest you give jackd a spin.

There appear to be "media-sound/jack" and "media-sound/jack2" ebuilds.
Both of them are tagged "~amd64". There's also
media-sound/jack-audio-connection-kit Which one(s) do I use?

> It should what you want, but may I ask why you need to record off
> your browser whatever youtube shows? Can't you use youtube-dl to
> download the youtube video/audio?

Youtube is a bad example to use. There are various other HTML5
streaming websites, which don't work with youtube-dl. Plus I also want
to be able to record video games and any other desktop app in general.
What I want is to "tee" the audio output so that I can record while
monitoring it.

[Ian Zimmerman]

On 2019-09-23 16:22, Walter Dnes wrote:

> There appear to be "media-sound/jack" and "media-sound/jack2" ebuilds.
> Both of them are tagged "~amd64". There's also
> media-sound/jack-audio-connection-kit Which one(s) do I use?

It's the last one, I'm pretty sure.

> Youtube is a bad example to use. There are various other HTML5
> streaming websites, which don't work with youtube-dl. Plus I also want
> to be able to record video games and any other desktop app in general.
> What I want is to "tee" the audio output so that I can record while
> monitoring it.

Long ago, I digitized my vinyl and audio tape collection this way, with
the input coming over good old analog audio cable. I did it with pure
ALSA, no jack was needed. I _think_ there was a knob either in
alsamixer or in arecord to do the "tee", but I don't really remember.

--
Please don't Cc: me privately on mailing lists and Usenet,
if you also post the followup to the list or newsgroup.
To reply privately _only_ on Usenet and on broken lists
which rewrite From, fetch the TXT record for no-use.mooo.com.

[Adam Carter]

> Youtube is a bad example to use.  There are various other HTML5
> streaming websites, which don't work with youtube-dl.  Plus I also want
> to be able to record video games and any other desktop app in general.
> What I want is to "tee" the audio output so that I can record while
> monitoring it.

Long ago, I digitized my vinyl and audio tape collection this way, with
the input coming over good old analog audio cable.  I did it with pure
ALSA, no jack was needed.  I _think_ there was a knob either in
alsamixer or in arecord to do the "tee", but I don't really remember.

When you're digitising audio you can just connect the tape/LP output to a line in input. AFAICT the "tee" is only needed for capturing digital output that's coming from the PC itself.

Also, for LPs, if you don't run the LP output into an amplifier phono input, it will miss out on the EQ reversal that occurs there. The physical characteristics of vinyl media mean that some frequencies are harder to encode in a vinyl groove than others (cant remember if its high or low freqs) so as part of the manufacturing process they put the signal through a filter before its written to vinyl, then the phono input on the amp reverses that filter to restore the signal. I'd imagine you could use a digital EQ to restore the signal if you don't have an amp with phono input handy.

[Mick]

On Tue, 24 Sep 2019 at 04:56, Ian Zimmerman <i...@very.loosely.org> wrote:
>
> On 2019-09-23 16:22, Walter Dnes wrote:
>
> > There appear to be "media-sound/jack" and "media-sound/jack2" ebuilds.
> > Both of them are tagged "~amd64". There's also
> > media-sound/jack-audio-connection-kit Which one(s) do I use?
>
> It's the last one, I'm pretty sure.

media-sound/jack is an old CD ripping application, so this is not what you want.

media-sound/jack2 is the C++ coded jack daemon for multi-processor PCs.

media-sound/jack-audio-connection-kit is (I think) the jack1 daemon coded in C.

For Jack1 Vs Jack2 features have a look here:

https://github.com/jackaudio/jackaudio.github.com/wiki/Q_difference_jack1_jack2

> > Youtube is a bad example to use. There are various other HTML5
> > streaming websites, which don't work with youtube-dl. Plus I also want
> > to be able to record video games and any other desktop app in general.
> > What I want is to "tee" the audio output so that I can record while
> > monitoring it.
>
> Long ago, I digitized my vinyl and audio tape collection this way, with
> the input coming over good old analog audio cable. I did it with pure
> ALSA, no jack was needed. I _think_ there was a knob either in
> alsamixer or in arecord to do the "tee", but I don't really remember.

For streaming websites which do not offer a download button, you can
launch the browser developer tab, select Network/Media and keep an eye
on the traffic as you click on the website to start streaming the
video. The developer tab will list the actual path of the video file.
Open another tab in the browser and paste the full URL to download it,
or you could try curl/wget et al.

tee-ing the audio output of desktop applications/windows may be doable
with dmix and asoundrc syntax, but I wouldn't know what this looks
like today with any certainty - last time I used dmix was in the mid
00s. Anyway, you tried this and it didn't work.

Jackd was created for the very purpose of plumbing I/O audio into
various configurations, so tweaking it to do the recording you require
should be eminently doable and using one of the various Gtk/Qt front
ends makes it even easier/quicker.
--
Regards,
Mick