using ffmpeg command line with python's subprocess module

[iMath]

I have few wav files that I can use either of the following command line mentioned here
https://trac.ffmpeg.org/wiki/How%20to%20concatenate%20%28join,%20merge%29%20media%20files
to concatenate


ffmpeg -f concat -i <(for f in ./*.wav; do echo "file '$f'"; done) -c copy output.wav
ffmpeg -f concat -i <(printf "file '%s'\n" ./*.wav) -c copy output.wav
ffmpeg -f concat -i <(find . -name '*.wav' -printf "file '%p'\n") -c copy output.wav


anyone know how to convert either of them to work with python's subprocess module, it would be better if your solution is platform independent .

[Chris Angelico]

On Mon, Dec 2, 2013 at 10:34 PM, iMath <redsto...@163.com> wrote:
> I have few wav files that I can use either of the following command line mentioned here
> https://trac.ffmpeg.org/wiki/How%20to%20concatenate%20%28join,%20merge%29%20media%20files
> to concatenate
>
>
> ffmpeg -f concat -i <(for f in ./*.wav; do echo "file '$f'"; done) -c copy output.wav
> ffmpeg -f concat -i <(printf "file '%s'\n" ./*.wav) -c copy output.wav
> ffmpeg -f concat -i <(find . -name '*.wav' -printf "file '%p'\n") -c copy output.wav

In bash, the <(...) notation is like piping: it executes the command
inside the parentheses and uses that as standard input to ffmpeg. So
if you work out what the commands are doing (it looks like they emit a
line saying "file '...'" for each .wav file in the current directory,
possibly including subdirectories) and replicate that in Python, you
should be able to make it cross-platform.

ChrisA

[Ben Finney]

Chris Angelico <ros...@gmail.com> writes:

> On Mon, Dec 2, 2013 at 10:34 PM, iMath <redsto...@163.com> wrote:

> > ffmpeg -f concat -i <(for f in ./*.wav; do echo "file '$f'"; done) -c copy output.wav
> > ffmpeg -f concat -i <(printf "file '%s'\n" ./*.wav) -c copy output.wav
> > ffmpeg -f concat -i <(find . -name '*.wav' -printf "file '%p'\n") -c copy output.wav
>

> In bash, the <(...) notation is like piping: it executes the command
> inside the parentheses and uses that as standard input to ffmpeg.

Not standard input, no. What it does is create a temporary file to
contain the result, and inserts that file name on the command line. This
is good for programs that require an actual file, not standard input.

So the above usage seems right to me: the ‘ffmpeg -i FOO’ option is
provided with a filename dynamically created by Bash, referring to a
temporary file that contains the output of the subshell.

--
\ “Welchen Teil von ‘Gestalt’ verstehen Sie nicht? [What part of |
`\ ‘gestalt’ don't you understand?]” —Karsten M. Self |
_o__) |
Ben Finney

[Chris Angelico]

On Tue, Dec 3, 2013 at 8:19 AM, Ben Finney <ben+p...@benfinney.id.au> wrote:
> Chris Angelico <ros...@gmail.com> writes:
>
>> On Mon, Dec 2, 2013 at 10:34 PM, iMath <redsto...@163.com> wrote:

>> > ffmpeg -f concat -i <(for f in ./*.wav; do echo "file '$f'"; done) -c copy output.wav
>> > ffmpeg -f concat -i <(printf "file '%s'\n" ./*.wav) -c copy output.wav
>> > ffmpeg -f concat -i <(find . -name '*.wav' -printf "file '%p'\n") -c copy output.wav
>>

>> In bash, the <(...) notation is like piping: it executes the command
>> inside the parentheses and uses that as standard input to ffmpeg.
>
> Not standard input, no. What it does is create a temporary file to
> contain the result, and inserts that file name on the command line. This
> is good for programs that require an actual file, not standard input.

Ah, sorry, my bad - bit rusty on my arcane bashisms. In any case, the
general point of what I was saying is: Figure out what's happening,
and replicate that. In this case, that means creating a file, then,
rather than putting it on stdin - that's probably actually easier
anyway.

ChrisA

[iMath]

在 2013年12月3日星期二UTC+8上午5时19分21秒，Ben Finney写道：

so is there any way to create a temporary file by Python here ?

[rusi]

On Tuesday, December 3, 2013 6:45:42 AM UTC+5:30, iMath wrote:
> so is there any way to create a temporary file by Python here ?

http://docs.python.org/2/library/tempfile.html

[Alain Ketterlin]

Ben Finney <ben+p...@benfinney.id.au> writes:

> Chris Angelico <ros...@gmail.com> writes:
>
>> On Mon, Dec 2, 2013 at 10:34 PM, iMath <redsto...@163.com> wrote:
>> > ffmpeg -f concat -i <(for f in ./*.wav; do echo "file '$f'"; done) -c copy output.wav
>> > ffmpeg -f concat -i <(printf "file '%s'\n" ./*.wav) -c copy output.wav
>> > ffmpeg -f concat -i <(find . -name '*.wav' -printf "file '%p'\n") -c copy output.wav
>>
>> In bash, the <(...) notation is like piping: it executes the command
>> inside the parentheses and uses that as standard input to ffmpeg.
>
> Not standard input, no. What it does is create a temporary file to
> contain the result, and inserts that file name on the command line. This
> is good for programs that require an actual file, not standard input.

Just in case (it may not be relevant to the current discussion): it may
not be a file, it will more probably be a FIFO (i.e., not seekable).
Here is the relevant part of the manual page:

| Process Substitution
| Process substitution is supported on systems that support named
| pipes (FIFOs) or the /dev/fd method of naming open files. It
| takes the form of <(list) or >(list). The process list is run
| with its input or output connected to a FIFO or some file in
| /dev/fd. The name of this file is passed as an argument to the
| current command as the result of the expansion.

-- Alain.

[iMath]

在 2013年12月3日星期二UTC+8下午5时33分09秒，Alain Ketterlin写道：

thanks for your reply, but is there any method that we can convert this to python ?

[iMath]

在 2013年12月3日星期二UTC+8上午9时42分11秒，rusi写道：

> On Tuesday, December 3, 2013 6:45:42 AM UTC+5:30, iMath wrote:
>
> > so is there any way to create a temporary file by Python here ?
>
>
>
> http://docs.python.org/2/library/tempfile.html

I use the following code to do the test ,but error occurred ,it prompts system cannot find specified files ,but the files are indeed exists there ,any help ?

with tempfile.TemporaryFile() as fp:
fp.write(("file '"+'a1.mp3'+"'\n").encode('utf-8'))
fp.write(("file '"+'a2.mp3'+"'\n").encode('utf-8'))

subprocess.call(['ffmpeg/ffmpeg', '-f', 'concat','-i',fp, '-c', 'copy', 'a3.mp3'])

[iMath]

在 2013年12月3日星期二UTC+8上午9时42分11秒，rusi写道：

> On Tuesday, December 3, 2013 6:45:42 AM UTC+5:30, iMath wrote:
>
> > so is there any way to create a temporary file by Python here ?
>
>
>
> http://docs.python.org/2/library/tempfile.html

[Andreas Perstinger]

iMath <redsto...@163.com> wrote:
>I use the following code to do the test ,but error occurred ,it
>prompts system cannot find specified files ,but the files are indeed
>exists there ,any help ?
>
>with tempfile.TemporaryFile() as fp:
> fp.write(("file '"+'a1.mp3'+"'\n").encode('utf-8'))
> fp.write(("file '"+'a2.mp3'+"'\n").encode('utf-8'))
>
> subprocess.call(['ffmpeg/ffmpeg', '-f', 'concat','-i',fp, '-c',
> 'copy', 'a3.mp3'])

Basic rule: Always copy'n'paste the exact traceback you get.

I don't think you are running the code you have posted because your
subprocess call doesn't work:

>>> import tempfile, subprocess
>>> with tempfile.TemporaryFile() as fp:
... subprocess.call(["foo", "bar", fp, "baz"])
...
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/usr/lib/python3.3/subprocess.py", line 520, in call
with Popen(*popenargs, **kwargs) as p:
File "/usr/lib/python3.3/subprocess.py", line 820, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.3/subprocess.py", line 1380, in _execute_child
restore_signals, start_new_session, preexec_fn)
TypeError: Can't convert '_io.BufferedRandom' object to str implicitly

"fp" is a file object, but subprocess expects a list of strings as
its first argument.

Bye, Andreas

[Chris Angelico]

On Wed, Dec 4, 2013 at 8:38 PM, Andreas Perstinger <andip...@gmail.com> wrote:
> "fp" is a file object, but subprocess expects a list of strings as
> its first argument.

More fundamentally: The subprocess's arguments must include the *name*
of the file. This means you can't use TemporaryFile at all, as it's
not guaranteed to return an object that actually has a file name.

There's another problem, too, and that's that you're not closing the
file before expecting the subprocess to open it. And once you do that,
you'll find that the file no longer exists once it's been closed. In
fact, you'll need to research the tempfile module a bit to be able to
do what you want here; rather than spoon-feed you an exact solution,
I'll just say that there is one, and it can be found here:

http://docs.python.org/3.3/library/tempfile.html

ChrisA

[iMath]

在 2013年12月4日星期三UTC+8下午5时38分27秒，Andreas Perstinger写道：

sorry for my fuss.
it prompts
StdErr: -c: No such file or directory

[Mark Lawrence]

On 06/12/2013 06:23, iMath wrote:

Dearest iMath, wouldst thou be kind enough to partake of obtaining some
type of email client that dost not sendeth double spaced data into this
most illustrious of mailing lists/newsgroups. Thanking thee for thine
participation in my most humble of requests. I do remain your most
obedient servant.

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[Ned Batchelder]

On 12/6/13 4:23 AM, Mark Lawrence wrote:
> On 06/12/2013 06:23, iMath wrote:
>
> Dearest iMath, wouldst thou be kind enough to partake of obtaining some
> type of email client that dost not sendeth double spaced data into this
> most illustrious of mailing lists/newsgroups. Thanking thee for thine
> participation in my most humble of requests. I do remain your most
> obedient servant.
>

iMath seems to be a native Chinese speaker. I think this message,
though amusing, will be baffling and won't have any effect...

--Ned.

[iMath]

在 2013年12月6日星期五UTC+8下午5时23分59秒，Mark Lawrence写道：

yes ,I am a native Chinese speaker.I always post question by Google Group not through email ,is there something wrong with it ?
your english is a little strange to me .

[iMath]

在 2013年12月4日星期三UTC+8下午6时51分49秒，Chris Angelico写道：

I think you mean I should create a temporary file by NamedTemporaryFile(). After tried it many times, I found there is nearly no convenience in creating a temporary file or a persistent one here ,because we couldn't use the temporary file while it has not been closed ,so we couldn't depend on the convenience of letting the temporary file automatically delete itself when closing, we have to delete it later by os.remove() after it has been used in that command line.

code without the with statement is here ,but it is wrong ,it shows this line

c:\docume~1\admini~1\locals~1\temp\tmp0d8959: Invalid data found when processing input


fp=tempfile.NamedTemporaryFile(delete=False)
fp.write(("file '"+fileName1+"'\n").encode('utf-8'))
fp.write(("file '"+fileName2+"'\n").encode('utf-8'))


subprocess.call(['ffmpeg', '-f', 'concat','-i',fp.name, '-c', 'copy', fileName])
fp.close()

[Chris Angelico]

On Sat, Dec 7, 2013 at 1:54 AM, iMath <redsto...@163.com> wrote:
> fp=tempfile.NamedTemporaryFile(delete=False)
> fp.write(("file '"+fileName1+"'\n").encode('utf-8'))
> fp.write(("file '"+fileName2+"'\n").encode('utf-8'))
>
>
> subprocess.call(['ffmpeg', '-f', 'concat','-i',fp.name, '-c', 'copy', fileName])
> fp.close()

You need to close the file before getting the other process to use it.
Otherwise, it may not be able to open the file at all, and even if it
can, you might find that not all the data has been written.

But congrats! You have successfully found the points I was directing
you to. Yes, I was hinting that you need NamedTemporaryFile, the .name
attribute, and delete=False. Good job!

ChrisA

[Mark Lawrence]

You can see the extra lines inserted by google groups above. It's not
too bad in one and only one message, but when a message has been
backwards and forwards several times it's extremely irritating, or worse
still effectively unreadable. Work arounds have been posted on this
list, but I'd recommend using any decent email client.

The English I used was archaic, please ignore it :)

[rusi]

On Friday, December 6, 2013 8:22:48 PM UTC+5:30, iMath wrote:
> 在 2013年12月6日星期五UTC+8下午5时23分59秒，Mark Lawrence写道：
> > On 06/12/2013 06:23, iMath wrote:
> > Dearest iMath, wouldst thou be kind enough to partake of obtaining some
> > type of email client that dost not sendeth double spaced data into this
> > most illustrious of mailing lists/newsgroups. Thanking thee for thine
> > participation in my most humble of requests. I do remain your most
> > obedient servant.

> yes ,I am a native Chinese speaker.I always post question by Google Group not through email ,is there something wrong with it ?

Yes but its easily correctable

I recently answered this question to another poster here

https://groups.google.com/forum/#!searchin/comp.lang.python/rusi$20google$20groups|sort:date/comp.lang.python/C51hEvi-KbY/KSeaMFoHtcIJ

[rusi]

On Friday, December 6, 2013 8:42:02 PM UTC+5:30, Mark Lawrence wrote:
> The English I used was archaic, please ignore it :)

"Archaic" is almost archaic
"Old" is ever-young

:D

[Steven D'Aprano]

On Fri, 06 Dec 2013 06:52:48 -0800, iMath wrote:

> yes ,I am a native Chinese speaker.I always post question by Google
> Group not through email ,is there something wrong with it ? your
> english is a little strange to me .

Mark is writing in fake old-English style, the way people think English
was spoken a thousand years ago. I don't know why he did that. Perhaps he
thought it was amusing.

There are many problems with Google Groups. If you pay attention to this
forum, you will see dozens of posts about "Managing Google Groups
headaches" and other complaints:

- Google Groups double-spaces replies, so text which should appear like:

line one
line two
line three
line four

turns into:

line one
blank line
line two
blank line
line three
blank line
line four

- Google Groups often starts sending HTML code instead of plain text

- it often mangles indentation, which is terrible for Python code

- sometimes it automatically sets the reply address for posts to go
to Google Groups, instead of the mailing list it should go to

- almost all of the spam on his forum comes from Google Groups, so many
people automatically filter everything from Google Groups straight to
the trash.

There are alternatives to Google Groups:

- the mailing list, pytho...@python.org

- Usenet, comp.lang.python

- the Gmane mirror:

http://gmane.org/find.php?list=python-list%40python.org


and possibly others. You will maximise the number of people reading your
posts if you avoid Google Groups. If for some reason you cannot use any
of the alternatives, please take the time to fix some of the problems
with Google Groups. If you search the archives, you should find some
posts by Rusi defending Google Groups and explaining what he does to make
it more presentable, and (if I remember correctly) I think Mark also
sometimes posts a link to managing Google Groups.



--
Steven

[Mark Lawrence]

On 06/12/2013 15:34, Steven D'Aprano wrote:
(if I remember correctly) I think Mark also
> sometimes posts a link to managing Google Groups.
>

You do, and here it is https://wiki.python.org/moin/GoogleGroupsPython

[rusi]

On Friday, December 6, 2013 9:23:47 PM UTC+5:30, Mark Lawrence wrote:
> On 06/12/2013 15:34, Steven D'Aprano wrote:
>
> (if I remember correctly) I think Mark also
>
> > sometimes posts a link to managing Google Groups.
>
> >
>
> You do, and here it is https://wiki.python.org/moin/GoogleGroupsPython

That link needs updating.

Even if my almost-automatic correction methods are not considered
kosher for some reason or other, the thing that needs to go in there
is that GG has TWO problems

1. Blank lines
2. Long lines

That link only describes 1.

Roy's yesterday's post in "Packaging a proprietary python library"
says:

> I, and Rusi, know enough, and take the effort, to overcome its
> shortcomings doesn't change that.

But in fact his post takes care of 1 not 2.

In all fairness I did not know that 2 is a problem until rurpy pointed
it out recently and was not correcting it. In fact, I'd take the
trouble to make the lines long assuming that clients were intelligent
enough to fit it properly into whatever was the current window!!!

So someone please update that page!

[Mark Lawrence]

This is a community so why don't you?

[MRAB]

On 06/12/2013 15:34, Steven D'Aprano wrote:

> On Fri, 06 Dec 2013 06:52:48 -0800, iMath wrote:
>
>> yes ,I am a native Chinese speaker.I always post question by Google
>> Group not through email ,is there something wrong with it ? your
>> english is a little strange to me .
>
> Mark is writing in fake old-English style, the way people think English
> was spoken a thousand years ago. I don't know why he did that. Perhaps he
> thought it was amusing.
>

[snip]
You're exaggerating. It's more like 500 years ago. :-)

[rusi]

On Friday, December 6, 2013 9:55:54 PM UTC+5:30, Mark Lawrence wrote:

> On 06/12/2013 16:19, rusi wrote:

> > So someone please update that page!

> This is a community so why don't you?

Ok done (at least a first draft)
I was under the impression that anyone could not edit

[rusi]

I was going to say the same until I noticed the "the way people think English
was spoken..."

That makes it unarguable -- surely there are some people who (wrongly) think so?

[Gregory Ewing]

Probably. They're surprisingly far off, though. Here's
a sample of actual 1000-year-old English:

http://answers.yahoo.com/question/index?qid=20100314001840AAygUaq

--
Greg

[iMath]

在 2013年12月6日星期五UTC+8下午10时59分43秒，Chris Angelico写道：

we don't have permission to use the temporary file while it has not been closed,but when the file is closed , it will be destroyed by default(delete=True),but once we set delete=False,then we couldn't depend on the convenience of letting the temporary file automatically delete itself after it has been used(then we have to delete it later by os.remove()) ,thus there is nearly no convenience in creating a temporary file or a persistent one when using NamedTemporaryFile.

I think this is a design flaw and should be improved like this :when delete=True,the file should be removed upon destruction of the object or on function return or on garbage collected,only when delete=False,the file can remain on the disk .

[Andreas Perstinger]

iMath <redsto...@163.com> wrote:
>we don't have permission to use the temporary file while it has not
>been closed,but when the file is closed , it will be destroyed by
>default(delete=True),but once we set delete=False,then we couldn't
>depend on the convenience of letting the temporary file automatically
>delete itself after it has been used(then we have to delete it later
>by os.remove()) ,thus there is nearly no convenience in creating a
>temporary file or a persistent one when using NamedTemporaryFile.

In your OP you asked for a platform independent solution thus I don't
think it's possible to do what you want if the OS doesn't support
opening a file another time while it's still open.

Otherwise, on my Linux system this works:

>>> import tempfile, subprocess
>>> with tempfile.NamedTemporaryFile() as fp:
... fp.write(b"foo\nbar\nbaz\n")
... fp.flush()
... subprocess.call(["cat", fp.name])
...
12
foo
bar
baz
0

Bye, Andreas