fastest way to remove nils
Glen Rubin
I usually use the filter function, but I see there are other functions
like remove that should also do the trick.
lprefo...@softaddicts.ca
"Elapsed time: 0.053219 msecs"
(1 2 4)
user=> (time (remove nil? [ nil 1 2 nil 4]))
"Elapsed time: 0.065092 msecs"
(1 2 4)
Choose the flavor your prefer...
Luc P.
Glen Rubin <rubi...@gmail.com> wrote ..
> What is the fastest way to remove nils from a sequence?
>
> I usually use the filter function, but I see there are other functions
> like remove that should also do the trick.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first
> post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
Luc P.
================
The rabid Muppet
Miki
"Elapsed time: 0.079312 msecs"
()
user=> (time (filter identity (repeat 1000000 nil)))
"Elapsed time: 0.070249 msecs"
()
Seems like filter is a bit faster, however YMMV
Stuart Campbell
user=> (time (filter identity [ nil 1 2 nil 4]))
"Elapsed time: 0.053219 msecs"
(1 2 4)
user=> (time (remove nil? [ nil 1 2 nil 4]))
"Elapsed time: 0.065092 msecs"
(1 2 4)
Choose the flavor your prefer...
Luc P.
Those aren't exactly equivalent:
user> (filter identity [1 2 false 3 nil 4])
(1 2 3 4)
user> (remove nil? [1 2 false 3 nil 4])
(1 2 false 3 4)
Regards,
Stuart
Mark Engelberg
lprefo...@softaddicts.ca
lprefo...@softaddicts.ca
Mark Engelberg <mark.en...@gmail.com> wrote ..
> (keep identity l) is preferable to (filter identity l)
>
Steve Purcell
> user=> (time (remove nil? (repeat 1000000 nil)))
> "Elapsed time: 0.079312 msecs"
> ()
> user=> (time (filter identity (repeat 1000000 nil)))
> "Elapsed time: 0.070249 msecs"
> ()
>
> Seems like filter is a bit faster, however YMMV
You're not timing the execution, just the construction of a lazy seq:
user> (time (remove nil? (repeat 1000000 nil)))
"Elapsed time: 0.044 msecs"
()
user> (time (doall (remove nil? (repeat 1000000 nil))))
"Elapsed time: 772.469 msecs"
()
-Steve
Robert McIntyre
user> (def nil-seq (doall (interleave (repeat 1e5 nil) (repeat 1e5
"whatever"))) )
#'user/nil-seq
user> (time (doall (keep identity nil-seq)))
"Elapsed time: 122.485848 msecs"
user> (time (doall (remove nil? nil-seq)))
"Elapsed time: 149.71484 msecs"
--Robert McIntyre
Ken Wesson
> So, just to be clear,
>
> user> (def nil-seq (doall (interleave (repeat 1e5 nil) (repeat 1e5
> "whatever"))) )
> #'user/nil-seq
>
> user> (time (doall (keep identity nil-seq)))
> "Elapsed time: 122.485848 msecs"
>
> user> (time (doall (remove nil? nil-seq)))
> "Elapsed time: 149.71484 msecs"
I have to run these all a few times before the times quit shrinking
(JITting being done). Then:
user=> (time (do (doall (keep identity nil-seq)) nil))
"Elapsed time: 70.24324 msecs"
nil
user=> (time (do (doall (remove nil? nil-seq)) nil))
"Elapsed time: 40.47016 msecs"
nil
user=> (time (do (doall (filter identity nil-seq)) nil))
"Elapsed time: 36.70256 msecs"
nil
It seems on my system keep identity is actually almost twice as SLOW
as remove nil? and filter identity is slightly faster; filter identity
is almost exactly twice as fast as keep identity. Filter identity
does, of course, discard falses as well as nils, but when that's
acceptable it's worth knowing it's fastest.
I wonder if the difference is that I'm using the -server vm?
It's especially interesting, in light of the likelihood that identity
is probably compiling away to nothing when the JIT is done inlining
everything, that both the slowest and the fastest use identity. It
suggests that keep is inherently somewhat slow, in particular.
Glen Rubin
best way. still nice to learn about the keep fn. thx!
Ken Wesson
> So, it seems the way I was removing nils using filter was already the
> best way. still nice to learn about the keep fn. thx!
I don't know if we can conclude that yet. It seems to be fastest on
one system with one combination of Java version, VM settings, and
clojure version and non-fastest on another system with another
combination of Java version, VM settings, and clojure version.
Moritz Ulrich
->
(remove nil? [...])
On Fri, Oct 26, 2012 at 12:53 PM, kevin roth
<kevin.c....@gmail.com> wrote:
> Be careful with the (filter identity ...) which will also remove "falses"
> from seqs.
> (filter identity [nil 2 3 nil false true 4])
> => (2 3 true 4)
> Since (identity false) and (identity nil) returns respectively false and nil
> they are BOTH rejected by filter.
>
> This could do the trick:
> (filter (comp not nit?) [nil 2 3 nil false true 4])
> => (2 3 false true 4)
Softaddicts
In Clojure a falsy value is either nil or false.
In interop I ensure that null and false when returned in the upper layers
are made consistent (including Boolean objects set to false).
Too much potential trouble in my opinion to let this slip everywhere in your
code, it's not semantically coherent with Clojure.
Luc P.
> Be careful with the (filter identity ...) which will also remove "falses"
> from seqs.
> (filter identity [nil 2 3 nil false true 4])
> => (2 3 true 4)
> Since (identity false) and (identity nil) returns respectively false and
> nil they are BOTH rejected by filter.
>
> This could do the trick:
> (filter (comp not nit?) [nil 2 3 nil false true 4])
> => (2 3 false true 4)
>
> On Wednesday, November 17, 2010 2:37:16 AM UTC+1, Luc wrote:
>
> > "Elapsed time: 0.053219 msecs"
> > (1 2 4)
> >
> > user=> (time (remove nil? [ nil 1 2 nil 4]))
> > "Elapsed time: 0.065092 msecs"
> > (1 2 4)
> >
> > Choose the flavor your prefer...
> >
> > Luc P.
> >
> > >
> > > I usually use the filter function, but I see there are other functions
> > > like remove that should also do the trick.
> > >
> > > --
> > > You received this message because you are subscribed to the Google
> > > Groups "Clojure" group.
> > your first
> > > post.
> > > To unsubscribe from this group, send email to
> > > http://groups.google.com/group/clojure?hl=en
> > Luc P.
> >
> > ================
> > The rabid Muppet
> >
> >
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
Softaddicts<lprefo...@softaddicts.ca> sent by ibisMail from my ipad!
kevin roth
Softaddicts
If you pass false and nil values to a library and expect them to be processed
differently, you may end up with a big surprise.
Luc
> > Softaddicts<lprefo...@softaddicts.ca <javascript:>> sent by ibisMail from