[erlang-questions] file:read_file won't read from /proc

[Martin DeMello]

Eshell V5.7.2 (abort with ^G)
1> {ok, F} = file:open("/proc/cpuinfo", [read]).
{ok,<0.37.0>}
2> file:read(F, 100).
{ok,"processor\t: 0\nvendor_id\t: GenuineIntel\ncpu family\t:
6\nmodel\t\t: 15\nmodel name\t: Intel(R) Core(TM)2 Du"}
3> file:close(F).
ok
4> {ok, B} = file:read_file("/proc/cpuinfo").
{ok,<<>>}

Why is B empty?

martin

________________________________________________________________
erlang-questions (at) erlang.org mailing list.
See http://www.erlang.org/faq.html
To unsubscribe; mailto:erlang-questio...@erlang.org

[Attila Rajmund Nohl]

2010/3/4, Jachym Holecek <fr...@netbsd.org>:
[...]
> The read_file/1 thing check file size and then attempts to read exactly
> that many bytes. The size of /proc/cpuinfo is zero[*], thus read_file/1
> correctly reads zero bytes and represents the result with empty binary.
>
> Gotta love UN*X for its sense of consistency, right? ;-)

Even the classic UNIX utilities can't get it right occasionally:

ethanl@mwlx405: ~>wc /proc/sys/kernel/panic
1 1 2 /proc/sys/kernel/panic
ethanl@mwlx405: ~>wc -c /proc/sys/kernel/panic
0 /proc/sys/kernel/panic
ethanl@mwlx405: ~>cat /proc/sys/kernel/panic | wc -c
2

[Jachym Holecek]

Hello,

# Martin DeMello 2010-03-04:

> 4> {ok, B} = file:read_file("/proc/cpuinfo").
> {ok,<<>>}
>
> Why is B empty?

The read_file/1 thing check file size and then attempts to read exactly

that many bytes. The size of /proc/cpuinfo is zero[*], thus read_file/1
correctly reads zero bytes and represents the result with empty binary.

Gotta love UN*X for its sense of consistency, right? ;-)

Regards,
-- Jachym

[*] Checked on NetBSD and Linux.

[Mikael Pettersson]

Jachym Holecek writes:
> Hello,
>
> # Martin DeMello 2010-03-04:
> > 4> {ok, B} = file:read_file("/proc/cpuinfo").
> > {ok,<<>>}
> >
> > Why is B empty?
>
> The read_file/1 thing check file size and then attempts to read exactly
> that many bytes. The size of /proc/cpuinfo is zero[*], thus read_file/1
> correctly reads zero bytes and represents the result with empty binary.
>
> Gotta love UN*X for its sense of consistency, right? ;-)

/proc files behave more like pipes than regular files,
because their contents are not stored but generated on the fly.

You really have to 'cat' them and not rely on stat to tell
you their sizes in advance.

[Richard O'Keefe]

On Mar 5, 2010, at 5:17 AM, Attila Rajmund Nohl wrote:

> 2010/3/4, Jachym Holecek <fr...@netbsd.org>:
> [...]
>> The read_file/1 thing check file size and then attempts to read
>> exactly
>> that many bytes. The size of /proc/cpuinfo is zero[*], thus
>> read_file/1
>> correctly reads zero bytes and represents the result with empty
>> binary.
>>
>> Gotta love UN*X for its sense of consistency, right? ;-)
>
> Even the classic UNIX utilities can't get it right occasionally:
>
> ethanl@mwlx405: ~>wc /proc/sys/kernel/panic
> 1 1 2 /proc/sys/kernel/panic
> ethanl@mwlx405: ~>wc -c /proc/sys/kernel/panic
> 0 /proc/sys/kernel/panic
> ethanl@mwlx405: ~>cat /proc/sys/kernel/panic | wc -c

Here's what the Single Unix Specification, 4th edition says:
[st_size is set for "shared memory object"s and "typed memory
object"s]

For symbolic links, the ... value of the st_size member shall be
set
to the length of the pathname contained in the symbolic link not
including any terminating null byte.

For all other file types defined in this volume of POSIX.1-2008,
the structure members st_mode, st_ino, st_dev, st_uid, st_gid,
st_atim, st_ctim, and st_mtim shall have meaningful values and
the value of the member st_nlink shall be set to the number of
links to the file.

What's missing is *any* statement at all about what st_size should
be for anything other than a symbolic link, a shared memory object,
or a typed memory object.

If instead of looking at 'stat' you look at '<sys/stat.h>', you find

off_t st_size
For regular files, the file size in bytes.
For symbolic links, the length in bytes of the pathname
contained in the symbolic link.
[SHM]

[Martin DeMello]

On Thu, Mar 4, 2010 at 9:37 PM, Jachym Holecek <fr...@netbsd.org> wrote:

> The read_file/1 thing check file size and then attempts to read exactly
> that many bytes. The size of /proc/cpuinfo is zero[*], thus read_file/1
> correctly reads zero bytes and represents the result with empty binary.

Okay, that makes sense. So what would the best way to read "files" in
/proc be? Is there any standard erlang way to read a stream till you
get an EOF?

martin

[Jachym Holecek]

# Martin DeMello 2010-03-05:

> On Thu, Mar 4, 2010 at 9:37 PM, Jachym Holecek <fr...@netbsd.org> wrote:
>
> > The read_file/1 thing check file size and then attempts to read exactly
> > that many bytes. The size of /proc/cpuinfo is zero[*], thus read_file/1
> > correctly reads zero bytes and represents the result with empty binary.
>
> Okay, that makes sense. So what would the best way to read "files" in
> /proc be? Is there any standard erlang way to read a stream till you
> get an EOF?

Not sure if there's such function in standard libraries, you can always
go with a utility function much in the spirit of your original example:

raw_read_file(Path) ->
{ok, File} = file:open(Path, [read, binary]),
raw_read_loop(File, []).

raw_read_loop(File, Acc) ->
case file:read(File, 1024) of
{ok, Bytes} ->
raw_read_loop(File, [Acc | Bytes]);
eof ->
file:close(File),
iolist_to_binary(Acc);
{error, Reason} ->
file:close(File),
erlang:error(Reason)
end.

Regards,
-- Jachym

[Dmitry Belayev]

of course
iolist_to_binary(lists:reverse(Acc));

[Dmitry Belayev]

Sorry, I was wrong.
It isn't general practice to [Acc | Bytes] but surely it works.

[Vlad Dumitrescu]

>> Okay, that makes sense. So what would the best way to read "files" in
>> /proc be? Is there any standard erlang way to read a stream till you
>> get an EOF?

Maybe os:cmd("cat /proc/cpuinfo") could work?

regards,
Vlad

[Robert Virding]

In your original question you showed a way of doing this using
file:open, file:read and file:close. By doing file:read until it
returns eof you should be able to read the whole file. It won't be
very efficient for large files but for small files there should be no
problems.

Robert