Reading last (or last few) lines from a very large file

[Bratish Goswami]

Hi,

How can I read last line or last few lines from a large file in
node.js?

Basically I need to have a lookup over a log file which is increasing
by size. I am trying to use watchFile to get the change and then read
the last appended line.

Any help is appreciated.

-Bratish

[Oleg "Sannis" Efimov]

"Unix way" is to use specialized tools such as tail[1]. I think -f
option is useful fior you. You can execute tail in child process.

[1]: http://en.wikipedia.org/wiki/Tail_(Unix)

[Bratish Goswami]

'tail -f' wont work for me. Because eventually it will get bigger and
the problem will be the same...
Is there any way to do it programmatically?

-Bratish



On Oct 4, 6:02 pm, "Oleg \"Sannis\" Efimov" <efimo...@gmail.com>
wrote:

> "Unix way" is to use specialized tools such as tail[1]. I think -f
> option is useful fior you. You can execute tail in child process.
>
> [1]:http://en.wikipedia.org/wiki/Tail_(Unix)
>

[mscdex]

On Oct 5, 1:12 am, Bratish Goswami <bratishgosw...@gmail.com> wrote:
> 'tail -f' wont work for me. Because eventually it will get bigger and
> the problem will be the same...

I'm not sure I understand how 'tail -f' executed in a child process in
node wouldn't work for you.

[Kami]

If you are interested in doing "tail -F" programmatically in node,
that is how I did it using watchFile -
http://github.com/cloudkick/cast/blob/master/lib/util/tail.js#L38

[Marco Rogers]

If you're saying you don't want it to update with -f, you can just run
tail and specify the number of lines.

> tail -5 /your/file/here.txt

[Vitali Lovich]

Another way would be to seek to the end of the file, mark that offset, wait for change, seek to the end of the file, print out the contents between the two markers - that will represent the added content (not necessarily split on lines though).

--
You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.

[mscdex]

On Oct 5, 7:45 pm, Vitali Lovich <vlov...@gmail.com> wrote:
> Another way would be to seek to the end of the file, mark that offset, wait
> for change, seek to the end of the file, print out the contents between the
> two markers - that will represent the added content (not necessarily split
> on lines though).

I was going to suggest that earlier, but I was under the impression
there was an issue with addressing very large files due to the limits
of the 'number' data type? There was a thread about it somewhere but I
wasn't sure if a fix or workaround had been made available.

[Bratish Goswami]

Sorry to inform you all lately. I've solved this issue in a similar
line as suggested by Vitali Lovich.

Have a look in my implementation at http://gist.github.com/612849
and let me know if any optimization possible.

Thanks
Bratish