Handling big files with sys.io.FileInput

[Renaud Bardet]

Hi list,

I'm writing a neko script to migrate a big xml file (~13Gb) to a db and I think I'm hitting the stream length limit,

I figured since FileInput.tell returns an Int that the reading head used is an Int and therefore a FileInput can only handle files as big as a Int (so 2^30-1 on neko ?)

is that right ?

Should there be a way to handle those big streams with a BigInt ?

also FileSystem.FileStat.size is an Int

Cheers,

Renaud

[Nicolas Cannasse]

Le 20/05/2013 19:45, Renaud Bardet a �crit :

Yes, but it's not yet done. This requires using specific C API which
usually uses int64 values.

Best,
Nicolas

[Renaud Bardet]

I had to write an equivalent script in python to meet the deadline (performing right now),

but I'm interested in having a solution to that, is it possible to open the file as a pipe ?

I guess I could read stdin and call my script with neko ingestor.n < my_catalog.xml

Le lundi 20 mai 2013 15:57:17 UTC-4, Nicolas Cannasse a écrit :

Le 20/05/2013 19:45, Renaud Bardet a �crit :

[Cauê Waneck]

If you're using Haxe 3.0 RC, you can use the Java target and access Java's STD library for file handling. I think you can even access it by accessing sys.io.FileInput's private variables to find the 64-bit version of tell()

2013/5/20 Renaud Bardet <renaud...@gmail.com>

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---
You received this message because you are subscribed to the Google Groups "Haxe" group.
For more options, visit https://groups.google.com/groups/opt_out.