Reading a file via Enum of lines

[TR NS]

It seems to me there must be a better way to read in a file iterating over each line. The only documented pattern I have found thus far requires multiple methods and recurses for every line of the file --which has to be a lot of overhead for a good size file. E.g.

    def load_letters do
      filename = "data/letters.dat"
      if File.exists?(filename) do
        input_file = File.open!(filename, [:read, :utf8])
        HashDict.new process_file(input_file, [])
      end
    end

    def process_file(input_file, listing) do
      line = IO.read(input_file, :line)
      if (line != :eof) do
        process_file(input_file, [process_line(line) | listing])   # recurse
      else
        listing
      end
    end

    def process_line(line) do
      row = String.strip(line)
      [rank, entry] = String.split(row, %r/\s+/)
      {rank, _} = String.to_float(rank)
      {entry, rank}
    end

What I would like to able to do, is something like (warning, pseudo-code ahead):

    def load_letters do
      filename = "data/letters.dat"
      if File.exists?(filename) do
        stream = File.stream!(filename, [:read, :utf8], :line)  # how to get this?
        data = Enum.reduce stream, [], fn(line, listing) ->
          [process_line(line) | listing]
        end
        HashDict.new data
      end
    end

[José Valim]

What I would like to able to do, is something like (warning, pseudo-code ahead):

    def load_letters do
      filename = "data/letters.dat"
      if File.exists?(filename) do
        stream = File.stream!(filename, [:read, :utf8], :line)  # how to get this?
        data = Enum.reduce stream, [], fn(line, listing) ->
          [process_line(line) | listing]
        end
        HashDict.new data
      end
    end

Your pseudo code is meant to work just fine. :) Except you call File.stream!(filename, [:read, :utf8]), which defaults to :line. The third parameter is only supported on Elixir master.

[TR NS]

On Wednesday, August 14, 2013 5:01:43 AM UTC-4, José Valim wrote:

Your pseudo code is meant to work just fine. :) Except you call File.stream!(filename, [:read, :utf8]), which defaults to :line. The third parameter is only supported on Elixir master.

LOL. When one's pseudo-code actually turns out to be real code, well I'd say that bodes well for the language! :)