Stream head / tail

[Jonatan Männchen]

I would like to propose two new functions for the `Stream` module:

Head Function

@spec head(Enumerable.t) :: Stream.element

This function is the equivalent of the following code:

[element] = stream
|> Stream.take(1)
|> Enum.to_list

Tail Function

@spec tail(Enumerable.t) :: Enumerable.t # %Stream{...}

This function returns a new stream without the head element.

Syntax Sugar

It would further be awesome if the following would work:

[element1, elementN... | rest] = stream # I'm aware that this would not work as a function head / case etc.
%Stream{} = rest

Real Wold Example

This capability would for example enable the following (using https://hexdocs.pm/crontab):

def setup do
  ~e[*/10] # Every 10 Minutes
  |> Crontab.Scheduler.get_next_run_dates() # Creating a Stream with dates matching schedule
  |> run(fn date ->
    # Do something useful
  end)
end


def run(schedules, callback) do
  [next_date | rest] = schedules # With Syntax Sugar

  next_date = Stream.head(schedules) # Without Syntax Sugar
  rest = Stream.tail(schedules) # Without Syntax Sugar

  next_date
  |> DateTime.from_naive!("Etc/UTC")
  |> DateTime.diff(DateTime.utc_now(), :millisecond)
  |> min(0)
  |> Process.sleep()

  callback.(next_date)

  run(schedules, callback)
end

Problems

There's some problems with this exact approach since pattern matching doesn't work with streams. Therefore all of the above will need some work. It should however illustrate the intention of this proposal.

[José Valim]

The reason why such operations doesn't exist on stream is because streams may encapsulate resources. For example, Repo.stream is a lazy database query. Having operations such as Stream.head(...) | Stream.tail(...) means you would have to execute the query twice, one to get the head, and another for the tail, which can be very wasteful in those cases.

You probably could be tackled without head and tail, by first applying all transforms to streams and then running it:

~e[*/10]
|> Crontab.Scheduler.get_next_run_dates()
|> Stream.each(fn date ->

  date
  |> DateTime.from_naive!("Etc/UTC")
  |> DateTime.diff(DateTime.utc_now(), :millisecond)
  |> min(0)
  |> Process.sleep()

  callback.(date)
end)
|> Stream.run()

The stream programming model requires you to think about working with the collection as a whole, instead of item by item.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/31cc6e90-2235-4f31-aa76-9fc045b8aa88o%40googlegroups.com.

[Jonatan Männchen]

I'm aware that this does not make sense for every type of stream. A very large part of streams depends on either the last item (like this example or for example a Fibonacci implementation) or something like a pointer (for example File.stream). In those cases a solution like that should work without problems.

The example I posted was supposed to be very simple and could easily be re-arranged to work with the current stream implementation. It was supposed to illustrate that you can handle the rest of a stream separately. There's a lot of cases where this is not as simple.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-l...@googlegroups.com.