What is the best way to turn a file into a lazy linear stream?

[d4v3y_5c0n3s]

What the title says.  Basically, I want to use a solution that:

 - does not require GC

 - uses the type system to prevent the user from freeing the file before they are done with the lazy evaluation

 - is not too inconvenient for the user to use

Let me know if you need an example of what I'm talking about.

[d4v3y_5c0n3s]

    Okay, so looking back I should have provided more details in the last post, as it does not explain the issues I was encountering etc.  I posted it right before I went to bed, and that's why I rushed it.

    Anyways, I wanted to explain how I solved my problem so that others could learn.  Basically, I was using ATS' built-in libc binding to open and read a file.  The pointer for the file was a linear type, I wasn't using the prelude's non-linear file type.  What my code did is read characters to a 'stream_vt' with the 'getc' function until it reached the end of the file.  I also planned to have other functions that would discard parts of the stream I didn't need so that I could turn the values I needed into other values (AKA, typical file-parsing stuff.)  The problem came when I needed to close the file.  Despite the use of linear types, freeing the file pointer would not prevent the stream from being access due to its laziness.  My solution was to essentially combine the view for the file pointer and the stream into a single type, and thus requiring both to be freed at once.

    Moral of the story?  Linear types are amazing, one of the reasons I love ATS, but they are not a silver bullet.  They too are limited by what can be statically inferred by the compiler, and effects such as lazy values cannot be predicted by the compiler.  I hope someone was able to learn something from my post, and let me know if you have any questions.

[gmhwxi]

Sorry, I didn't get a chance to read this post until now.

If you use a (linear) FILEptr to create a linear stream (of chars), then the FILEptr is "buried" inside

the stream and thus it can no longer be accessed.

>>My solution was to essentially combine the view for the file pointer and the stream into a single type, and thus requiring both to be freed at once.

To me, this seems to be the only type-safe solution (as the alternative you mentioned above must make use of some unsafe features).

--Hongwei

[d4v3y_5c0n3s]

No worries, I understand sometimes people get busy. :)

  I eventually decided against using linear streams to process files in my code, at least in the way I was using it.  My code would result in being relatively slow, due to only retrieving one character from the file at a time instead of getting a large chunk of text (which is faster because we are making fewer requests to the system.)  Also, I found myself essentially needing to either rewrite regular expressions to using character streams, or to turn my streams into strings, which struck me as being too roundabout to be practical.  The only positive of using streams in the way that I originally planned to was that the memory footprint would be reduced, a tradeoff I didn't see the value of in my case.

  A better way to use streams in my case is to have a stream of strings rather than a stream of characters.  This way, my code will run fast while still benefiting from laziness by only reading the file when needed.