Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Difference between Popen and open() for reading a file

1,523 views
Skip to first unread message

J

unread,
Apr 22, 2010, 2:28:01 PM4/22/10
to Python List
I was reading something from a code review a little while ago and saw
something that's got my curiosity up...

Say I had a file, foo.txt that I wanted to read from, only one time
and only read.

So what's the difference between this:

mylist = Popen(["cat","foo.txt"], stdout=PIPE).communicate()[0].splitlines()

and this:

f = open('foo.txt')
mylist = f.readlines()
f.close

Is there a reason why you would not use subprocess.Popen for this,
other than just not relying on external programs to perfrom the task?

what if that file only has one line in it, and that's all you're
interested in, and the file is guaranteed to only have that one line
it it?

For example:

say foo.txt contained only the number 123456789

what's wrong with doing this:

my_int = int(commands.getoutput('cat foo.txt')

or via the subprocess.Popen method mentioned above?

Chris Rebert

unread,
Apr 22, 2010, 2:47:44 PM4/22/10
to J, Python List

- Unnecessary complexity; I don't think anyone can credibly argue the
versions that use cat are simpler than the just-read-the-file-directly
version
- Unnecessary dependency (Does Windows have cat? I honestly don't know.)
- Pointless waste of CPU time; the versions invoking cat are slower
and there is no corresponding reward for enduring the decrease in
performance
- Bad error reporting; if foo.txt is nonexistent, the cat-free version
will give you a nice juicy IOError with full traceback, whereas the
cat versions will just silently give you an empty string

In other words, why *would* you want to use cat for this? The versions
using it have absolutely no benefits that I can find over the cat-free
code.

Cheers,
Chris
--
http://blog.rebertia.com

MRAB

unread,
Apr 22, 2010, 3:04:33 PM4/22/10
to Python List
J wrote:
> I was reading something from a code review a little while ago and saw
> something that's got my curiosity up...
>
> Say I had a file, foo.txt that I wanted to read from, only one time
> and only read.
>
> So what's the difference between this:
>
> mylist = Popen(["cat","foo.txt"], stdout=PIPE).communicate()[0].splitlines()
>
> and this:
>
> f = open('foo.txt')
> mylist = f.readlines()
> f.close
>
That last line should be:

f.close()

> Is there a reason why you would not use subprocess.Popen for this,
> other than just not relying on external programs to perfrom the task?
>
> what if that file only has one line in it, and that's all you're
> interested in, and the file is guaranteed to only have that one line
> it it?
>
> For example:
>
> say foo.txt contained only the number 123456789
>
> what's wrong with doing this:
>
> my_int = int(commands.getoutput('cat foo.txt')
>
> or via the subprocess.Popen method mentioned above?

Why would you want to call an external program to do something when you
could do it directly? It seems like a strange thing to do, like driving
to your kitchen! :-)

Dave Angel

unread,
Apr 22, 2010, 3:18:15 PM4/22/10
to J, Python List

J wrote:
> I was reading something from a code review a little while ago and saw
> something that's got my curiosity up...
>
> Say I had a file, foo.txt that I wanted to read from, only one time
> and only read.
>
> So what's the difference between this:
>
> mylist = Popen(["cat","foo.txt"], stdout=PIPE).communicate()[0].splitlines()
>
> and this:
>
> f = open('foo.txt')
> mylist = f.readlines()
> f.close
>

> Is there a reason why you would not use subprocess.Popen for this,
> other than just not relying on external programs to perfrom the task?
>
> what if that file only has one line in it, and that's all you're
> interested in, and the file is guaranteed to only have that one line
> it it?
>
> For example:
>
> say foo.txt contained only the number 123456789
>
> what's wrong with doing this:
>
> my_int = int(commands.getoutput('cat foo.txt')
>
> or via the subprocess.Popen method mentioned above?
>
>

The same difference as between handing the paper boy three bucks, versus
flying to London to open an account, making a deposit, going to a branch
in Sydney and asking for a bank check, then flying back home and taking
the paper boy with you to the bank to cash it.

When you use subprocess, you're creating a whole new process in order
the copy the file into another one, which you then read. Takes much
more resources, time, and is much more likely to break. For example,
what if this program runs on a system without "cat" ?

DaveA

J

unread,
Apr 22, 2010, 3:54:47 PM4/22/10
to Python List
On Thu, Apr 22, 2010 at 15:18, Dave Angel <da...@ieee.org> wrote:

> The same difference as between handing the paper boy three bucks, versus
> flying to London to open an account, making a deposit, going to a branch in
> Sydney and asking for a bank check, then flying back home and taking the
> paper boy with you to the bank to cash it.
>
> When you use subprocess, you're creating a whole new process in order the
> copy the file into another one, which you then read.  Takes much more
> resources, time, and is much more likely to break.  For example, what if
> this program runs on a system without "cat" ?
>
> DaveA

Thanks for this (and the other responses everyone)... I'm still
really in the "learning" process and the reasons to NOT do it that way
make perfect sense to me (and I love DaveA's explanation)...

Learning something every day...

Though, if I could afford it, I wouldn't mind a trip to London and
then Sydney...

Aahz

unread,
Apr 30, 2010, 5:17:40 PM4/30/10
to
In article <mailman.2142.1271960...@python.org>,

J <dreadpi...@gmail.com> wrote:
>
>Say I had a file, foo.txt that I wanted to read from, only one time
>and only read.
>
>So what's the difference between this:
>
>mylist = Popen(["cat","foo.txt"], stdout=PIPE).communicate()[0].splitlines()
>
>Is there a reason why you would not use subprocess.Popen for this,
>other than just not relying on external programs to perfrom the task?

http://uuoc.com/
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

0 new messages