Sorting maps when inspecting

[Eric Meadows-Jönsson]

Currently struct and map keys are in indeterminate order when they are inspected. Due to implementation details in the VM maps that have <32 keys are in term order but for larger maps and structs the order is random.

I propose always sorting the map keys when they are inspected so that it's easier to read and scan the results.

--

Eric Meadows-Jönsson

[Paul Schoenfelder]

+1 from me!

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAM_eapis0_a16Z6HqptYtc2ekCQ46-eh_uzHZtQYkfQ1yHE%2B2g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[jisaa...@gmail.com]

I'd be a little nervous that new devs might get the impression that maps are always sorted....

(I'm not necessarily opposed to the idea though)

On Monday, February 13, 2017 at 9:31:31 PM UTC-8, Paul Schoenfelder wrote:

+1 from me!

On Mon, Feb 13, 2017 at 4:49 PM, Eric Meadows-Jönsson <eric.meado...@gmail.com> wrote:

Currently struct and map keys are in indeterminate order when they are inspected. Due to implementation details in the VM maps that have <32 keys are in term order but for larger maps and structs the order is random.

I propose always sorting the map keys when they are inspected so that it's easier to read and scan the results.

--

Eric Meadows-Jönsson

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

[OvermindDL1]

I honestly would not, the map documentation very clearly states at https://hexdocs.pm/elixir/Map.html that:

> Key-value pairs in a map do not follow any order

:-)

[Norbert Melzer]

And who does read any documentation when he sees the same output on the cli every time? Documenting the behaviour hasn't helped for charlists either...

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/49866d3d-07fb-4218-b537-860d0bbb546c%40googlegroups.com.

[Pedro Medeiros]

Just during the inspection, it's fine for me.

I believe it's gonna help a lot during the development cicle

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/49866d3d-07fb-4218-b537-860d0bbb546c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Pedro Henrique de Souza Medeiros
----------------------------------
Cel: +55 (61) 9197-0993
Email: pedr...@gmail.com

Beautiful is better than ugly,
Explicit is better than implicit,
Simple is better than complex,
Complex is better than complicated.

The Zen of Python, by Tim Peters

[Eric Meadows-Jönsson]

Maps with less keys than 32 are printed in sorted order today so new devs would already be confused. This change would make it more consistent at least.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/2d64f24b-df25-4644-acd6-f5d5633ea720%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Eric Meadows-Jönsson

[Onorio Catenacci]

+1 Eric

[Ben Wilson]

This has my vote as well. It would make visually inspecting maps a lot easier.

On Tuesday, February 14, 2017 at 7:36:05 PM UTC-5, Onorio Catenacci wrote:

+1 Eric

[Amos King]

Before jumping on this should we be asking why the decision was made to not sort if there are more than 32 keys? Maybe there is a performance concern, and if we want sorted keys we should be sending a different message?

Amos King

Owner

Binary Noggin

http://binarynoggin.com #business

http://thisagilelife.com #podcast

=======================================================
I welcome VSRE emails. Learn more at http://vsre.info/
=======================================================

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/934ffbee-44a6-40fb-9ecf-e24d48624f3f%40googlegroups.com.

[Onorio Catenacci]

Hi Amos,

This is solely discussing the _display_ of the keys--not the storage.

--
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/Lc2fWthKQSM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAJr6D4S81ZqQ%2BaZQrw_rohpPXxwBpUWz1WsAEUZ82fjT02U7Pg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Onorio Catenacci
 
http://onor.io

http://www.google.com/+OnorioCatenacci

[Louis Pilfold]

Hey Amos

As I understand it the ordering of maps is an Erlang implementation detail that is not to be relied upon. If you want an ordered dictionary a map probably is not the correct data structure for you.

Cheers,

Louis

On Wed, 15 Feb 2017 at 14:35 Amos King <am...@binarynoggin.com> wrote:

Before jumping on this should we be asking why the decision was made to not sort if there are more than 32 keys? Maybe there is a performance concern, and if we want sorted keys we should be sending a different message?

Amos King

Owner

Binary Noggin

http://binarynoggin.com #business

http://thisagilelife.com #podcast

=======================================================
I welcome VSRE emails. Learn more at http://vsre.info/
=======================================================

On Tue, Feb 14, 2017 at 7:41 PM, Ben Wilson <benwil...@gmail.com> wrote:

This has my vote as well. It would make visually inspecting maps a lot easier.

On Tuesday, February 14, 2017 at 7:36:05 PM UTC-5, Onorio Catenacci wrote:

+1 Eric

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/934ffbee-44a6-40fb-9ecf-e24d48624f3f%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAJr6D4S81ZqQ%2BaZQrw_rohpPXxwBpUWz1WsAEUZ82fjT02U7Pg%40mail.gmail.com.

[Onorio Catenacci]

Again, the question is the _display_ of the keys; absolutely nothing would change about how the keys are stored.  

"I propose always sorting the map keys when they are inspected" (emphasis mine). 

Please read back to the start of the thread if you care to comment.

[Amos King]

I did read all of it before I commented. I'm not sure if I'm going to output a really large map that I want to wait for it to sort. I guess I might want to wait. It would depend on why I want to sort. I guess I need to look at how I usually use inspect. I normally know the keys I'm looking for and don't find myself outputting a large map. When I do I copy it to another tool to look through and inspect keys and values. 

I'm asking if there is a reason we might not want to sort. Sometimes the question to ask is about a reason not to do it. I'm not against this I just want to explore ideas.

Thanks for your proposal and starting the conversation.

Amos

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/968dfad6-6889-4975-a413-91bbc48968ed%40googlegroups.com.

[Tallak Tveide]

I believe it would be useful to not sort if there are very many keys as well. A huge map would presumably be inspected in a short while, skipping keys after the first few. Inspecting a huge map could become cpu/memory intensive, which I would find surprising... anyways +1 from me

[Bruce Tate]

Good points,  but isn't inspecting a map generally a development time activity? Especially big maps?

Sometimes, optimizing people is the right thing to do. When you're looking at big lists of ecto objects and working to pick out an ID, having things in order is a big deal. 

-bt

On Wed, Feb 15, 2017 at 2:17 PM, Tallak Tveide <tal...@gmail.com> wrote:

I believe it would be useful to not sort if there are very many keys as well. A huge map would presumably be inspected in a short while, skipping keys after the first few. Inspecting a huge map could become cpu/memory intensive, which I would find surprising... anyways +1 from me

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/8a1f314b-efdb-468d-8dae-ba6492aa0f95%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Bruce Tate
President, RapidRed, LLC
Phone: 512.772.4312
Fax: 512 857-0415

Author of Seven Languages in Seven Weeks, Deploying Rails Applications, From Java to Ruby, Rails: Up and Running, Beyond Java, 6 others.

[Dmitry Belyaev]

If I dump a map to logs I don't care about its order, I only want it to be as fast as possible. Logging already introduces a serious performance hit, and sorting would make logging even more expensive performance wise.

As was mentioned previously if a map is small it's already sorted. So it would be a waste of time although insignificant. But if it's huge it would spend both cpu and memory to vuild sorted structure, and later would spend more cpu on its garbage collection. I don't like the idea of losing production performance only for development convenience.

[Robert Virding]

How do you mean "when they are inspected"? If you do it then all functions which step through the keys would have to be fixed and some like fold and map could be very difficult to implement in an efficient manner. If you don't do it for all then it is pointless.

Robert

[Onorio Catenacci]

I'm assuming he means IO.inspect but I may be wrong.

--
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/Lc2fWthKQSM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/5e3f4e37-bb6c-4688-bb86-5d430e095726%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

[Eric Meadows-Jönsson]

My guess is that the sorting time will be insignificant compared to sending the data down your logging infrastructure regardless if it's just printing to a terminal or if it's sending data over a socket. Both require syscalls and IO which is usually the more time consuming part.

I think it's wrong to prioritize performance instead of usability for Inspect since the functionality is made for humans. If performance actually is more important than readability then there are lots of optimizations we can do to Inspect, for example dropping algebra documents and pretty printing.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/2649817D-21D7-4C50-B477-7038486C9849%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Eric Meadows-Jönsson

[Jean Parpaillon]

One of the most important reason when choosing a data structure over another is the cost of operations, if not, there is no interest in choosing, for instance, a Map over a Proplist.
Of course there is a cost in sorting a map, and it is not insignificant.

Why not just adding a Map.sort/1 function ?

Jean

Le jeudi 16 février 2017 12:54:43 UTC+1, Eric Meadows-Jönsson a écrit :

My guess is that the sorting time will be insignificant compared to sending the data down your logging infrastructure regardless if it's just printing to a terminal or if it's sending data over a socket. Both require syscalls and IO which is usually the more time consuming part.

I think it's wrong to prioritize performance instead of usability for Inspect since the functionality is made for humans. If performance actually is more important than readability then there are lots of optimizations we can do to Inspect, for example dropping algebra documents and pretty printing.

On Wed, Feb 15, 2017 at 10:04 PM, Dmitry Belyaev <be.d...@gmail.com> wrote:

If I dump a map to logs I don't care about its order, I only want it to be as fast as possible. Logging already introduces a serious performance hit, and sorting would make logging even more expensive performance wise.

As was mentioned previously if a map is small it's already sorted. So it would be a waste of time although insignificant. But if it's huge it would spend both cpu and memory to vuild sorted structure, and later would spend more cpu on its garbage collection. I don't like the idea of losing production performance only for development convenience.

On 16 February 2017 07:25:51 GMT+11:00, Bruce Tate <br...@rapidred.com> wrote:

Good points,  but isn't inspecting a map generally a development time activity? Especially big maps?

Sometimes, optimizing people is the right thing to do. When you're looking at big lists of ecto objects and working to pick out an ID, having things in order is a big deal. 

-bt

On Wed, Feb 15, 2017 at 2:17 PM, Tallak Tveide <tal...@gmail.com> wrote:

I believe it would be useful to not sort if there are very many keys as well. A huge map would presumably be inspected in a short while, skipping keys after the first few. Inspecting a huge map could become cpu/memory intensive, which I would find surprising... anyways +1 from me

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/8a1f314b-efdb-468d-8dae-ba6492aa0f95%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Bruce Tate
President, RapidRed, LLC
Phone: 512.772.4312
Fax: 512 857-0415

Author of Seven Languages in Seven Weeks, Deploying Rails Applications, From Java to Ruby, Rails: Up and Running, Beyond Java, 6 others.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/2649817D-21D7-4C50-B477-7038486C9849%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Eric Meadows-Jönsson

[Michał Muskała]

Inspect is a way for providing debug information. Performance should be considered only after satisfying usability is provided. 

I'm +1 for the change.

Michał.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/3dbae292-3978-41d3-bb54-3b006d694260%40googlegroups.com.

[Jean Parpaillon]

Before usability and performance, there is control, because we are speaking about a language, not a system.

With a Map.sort/1 function, you can control the performance, with automatic sorting you can not.
And with a Map.sort/1 function, you have usability.

If you want out of control languages, just pick one over popular existing ones: javascript, Pythom, Ruby... But please keep elixir clean ;)

Jean

[Parker Selbert]

IO.inspect already performs formatting and transformations while printing data. That is its purpose, to help present data for debugging during development.

Leaving inspect calls in production is already a performance problem, let alone inspecting maps with more than 32 entries.

This would fix an unexpected inconsistency, I'm in favor of it. +1 from me. 

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/84ad4498-7e76-4306-a925-83609bbaf128%40googlegroups.com.

[Allen Madsen]

+1

Allen Madsen
http://www.allenmadsen.com

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/84ad4498-7e76-4306-a925-83609bbaf128%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/E3BFEF00-7475-4716-8525-8B8BE1A4419B%40sorentwo.com.

[Onorio Catenacci]

Again for the 3rd time, we're not talking about sorting the maps to store them; we're talking about sorting the maps when they're displayed for inspection.  The main use case for inspection is debugging. 

We are also talking about the _default_ way of displaying large (more than 32 entries) maps for inspection.  Solely for inspection.  

Besides that, we already have a Map.sort/1.  It's called Enum.sort/1.

--
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/Lc2fWthKQSM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/84ad4498-7e76-4306-a925-83609bbaf128%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

[Tallak Tveide]

I had this discussion with myself and decided not to post. The reason is that you actually have good control: if you want just bypass the inspect function and roll your own printing functions that are optimized.

My proposition earlier was just saying that, when sorting, if you discover many elements (say n > 1000), just don't sort and select the first few elements to inspect. I believe Elixir inspect will anyways not show all elements for large maps (i could be wrong here). My concern was more that you could for example connect to a production node to inspect the state, and if you had a huge map, it could affect runtime performance for the rest of the system unexpectedly. It's a corner case, as you probably wouldn't use a map for such large data collections but rather a database.

So I'm still +1 with or without the optimization :-)

--
You received this message because you are subscribed to a topic in the Google Groups "elixir-lang-core" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/elixir-lang-core/Lc2fWthKQSM/unsubscribe.
To unsubscribe from this group and all its topics, send an email to elixir-lang-core+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/84ad4498-7e76-4306-a925-83609bbaf128%40googlegroups.com.

[Peter Hamilton]

I'm concerned about the logging case for large maps. I propose just increasing the threshold to something practically high, but low enough that performance is predictable.

To unsubscribe from this group and all its topics, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/84ad4498-7e76-4306-a925-83609bbaf128%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAHfREDXUsLtBSyuw0OunChCCY3bFPdufzqwbSgBF%2BD9pqaoo6w%40mail.gmail.com.

[Eric Meadows-Jönsson]

I'm not really concerned about the performance of sorting a map compared to the already big performance hit of dumping the resulting binary into your logging system, be it the terminal or an UDP socket, both which requires syscalls and IO, and which should be slower than sorting a map.

Remember that we already traverse the entire map three times when inspecting. 1. Converting the map to a list 2. Checking if the list is a keyword list and then to generate the algebra document. If performance is proven to actually be an issue we can use the inspect :limit option to only take the first N elements of the map and only sorting them.

To unsubscribe from this group and all its topics, send an email to elixir-lang-core+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/84ad4498-7e76-4306-a925-83609bbaf128%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAHfREDXUsLtBSyuw0OunChCCY3bFPdufzqwbSgBF%2BD9pqaoo6w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "elixir-lang-core" group.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-core+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/CAOMhEnxQajyNp9-XX4P5%3D7Eu5VA_CGSWZ1SHzGVSH9gH6vin5g%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Eric Meadows-Jönsson