These match error messages are killing me

[Edward Stembler]

I'm trying to read an Excel xlsx spreadsheet and save it as a tab-separated file.  I'm using the excellent and csv packages.  My code looks like this:

def xlsx_to_tsv(xlsx_file_name) when is_binary(xlsx_file_name) do
  tsv_file = File.open!(tsv_file_name(xlsx_file_name), [:write])

  Excellent.parse(xlsx_file_name, 0)
  |> CSV.encode(separator: ?\t)
  |> Enum.each(&IO.write(tsv_file, &1))
end


def tsv_file_name(file_name) when is_binary(file_name) do
  base = Path.basename(file_name, ".xlsx")
  dir = Path.dirname(file_name)
  Path.join(dir, "#{base}.tsv")
end

My unit test is failing with this delightful message:

1) test xlsx_to_tsv (MyImporterTest)
   test/my_importer_test.exs:14
   ** (MatchError) no match of right hand side value: {:error, {:EXIT, {{:badmatch, "c:/Users/myuser/Projects/my_importer/test/fixtures/myexcelfile.xlsx"}, [{:zip, :binary_io, 2, [file: 'zip.erl', line: 1724]}, {:zip, :get_end_of_central_dir, 3, [file: 'zip.erl', line: 1313]}, {:zip, :get_central_dir, 3, [file: 'zip.erl', line: 1269]}, {:zip, :do_unzip, 2, [file: 'zip.erl', line: 380]}, {:zip, :unzip, 2, [file: 'zip.erl', line: 370]}, {Excellent, :file_content, 2, [file: 'lib/excellent.ex', line: 57]}, {Excellent, :parse, 2, [file: 'lib/excellent.ex', line: 12]}, {MyImporter, :xlsx_to_tsv, 1, [file: 'lib/my_importer.ex', line: 8]}, {MyImporterTest, :"test xlsx_to_tsv", 1, [file: 'test/my_importer_test.exs', line: 18]}, {ExUnit.Runner, :exec_test, 2, [file: 'lib/ex_unit/runner.ex', line: 294]}, {:timer, :tc, 1, [file: 'timer.erl', line: 166]}, {ExUnit.Runner, :"-spawn_test/3-fun-1-", 3, [file: 'lib/ex_unit/runner.ex', line: 242]}]}}}
   stacktrace:
     lib/excellent.ex:57: Excellent.file_content/2
     lib/excellent.ex:12: Excellent.parse/2
     (my_importer) lib/my_importer.ex:8: MyImporter.xlsx_to_tsv/1
     test/my_importer_test.exs:18

How am I supposed to figure out what this means? If I look at the excellent repo README.md and spec test, it doesn't look like it requires any lhs pattern to match.

[Booker Bense]

Excellent is trying to use the erlang zip libraries to unzip this file

"c:/Users/myuser/Projects/my_importer/test/fixtures/myexcelfile.xlsx"

Unfortunately, the Excellent library expects an Erlang char_list for 

the file path name, since it passes it down to Erlang functions like :zip.unzip

Try this

Excellent.parse(String.to_char_list(xlsx_file_name), 0) 

IMHO, an Elixir library should do this for you. 

- Booker C. Bense

[Booker Bense]

I meant that if you are writing an Elixir library that interfaces with erlang libs, your library

should do the String.to_char_lists that are required. Not that the elixir runtime should 

do this for you. 

- Booker C. Bense 

[John W Higgins]

Instead of complaining about a perfectly valid, complete, and easily helpful error message - have you tried anything at all in terms of working through what the error message says? Have you looked at the stack trace and checked anything to see if there is something that the error message fully explains to you?

It really is all there if you take the time to think it through.

And to help - the error message would come from something like this

{:ok, ... } = { :error, ... }

Meaning that you are expecting the right hand side to return a tuple starting with {:ok} and you got back one that started with {:error}. In other words "no match for right hand side valud of ....."

Go have a look at the Excellent library (specifically where the stack trace points you to) - looking for something that has a left side that looks like { :ok, .... } and that's what has the right side hand returning { :error, ... }. You then need to see why you are getting an error returned from the right hand side. Probably a bad spreadsheet or you are passing in a bad value.

Sometimes Elixir blows up pretty hard - but the error message here is actually fairly comprehensive.

John

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/c6f0e5e4-6ae6-42b2-bf8c-d60f0cef08b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Peter Hamilton]

John - It's been said many times here that a confusing error message is a bug. I don't think it is appropriate to call expressing confusion "complaining". I appreciate that you took the time to teach him how to decipher the message, but we shouldn't discourage people from expressing that initial confusion. That feedback, especially from people new to Elixir, is how we make things even better than they already are.

Even after working through the error message contents, this is sufficiently deep in the call stack as to be confusing. I agree with Booker that Excellence should handle this case and would encourage Edward to file a bug with that library.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAPhAwGz-cj%3Dqa5wR9aS%2BKDo3%2BbtdbF9wsE1vD%2BBf%2Bo-4RZ%2BmCg%40mail.gmail.com.

[Paulo Almeida]

To complement the previous response I'll just add that I ended up using Apache POI to read xlsx files (both 2007 and 2010 versions). I'm using jinterface to interop with the JVM. Here's the guide I used: http://inaka.net/blog/2013/09/05/from-erlang-to-java-and-back-again-1/

I'll push the code to github during the weekend and add the link to this thread.

Regards,

Paulo

[Edward Stembler]

Thank you Booker, Peter, and John to some extent.

I admit that I posted this message out of frustration.  It's also true that I could've spent more time investigating this.  On the other hand, I believe that an error message like that could be greatly improved. Software is written and read by people, so the error messages should be unequivocally clear and informative. We should strive to reduce the amount of time required to understand error messages.

Some languages are making progress in this area.  Ruby is adding did_you_mean, Python 3 bubbles up the masked expection, and look at the improved error messages Elm has.  Elixir should also continue in this vein.

Earlier today, I was helping a collegue debug his R script which was producing a particularly confounding error message: unexpected '}' in "}". It had nothing to do with the number of brackets he had, but instead with a string with double forward slashes...  We can do better.

[John W Higgins]

Give a man a fish - and he eats for a day - teach a man how to fish and he eats forever.

My apologies for being the only person that actually tried to teach you how to fish today. Take my answer out of the thread and you still have no idea what "no match of right hand side" means.

I'll make sure to never try again. Lesson learned.

John

--

You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/92ade85e-b877-491c-9523-17682df27c6a%40googlegroups.com.

[Ben Wilson]

The elixir community views new users as an integral part of building a robust and developing user base and we want to encourage people to ask questions, particularly as Elixir is for many a first introduction to functional programming and its related idiosyncrasies.

Edward, you originally asked how you're supposed to figure out what the issue is. One of the best options early on is to simply go look at the pattern in question. That would take you here: https://github.com/leifg/excellent/blob/0.0.1/lib/excellent.ex#L57 where you'll see that it's expecting the call to :zip to return a nice :ok related tuple, but instead from your pasted error you see you got an error tuple. This is a great step forward, because we now know that the real question we're trying to answer is "why is :zlip returning an error".

So what's our next step? Well, let's just consult the :zip.extract documentation now that we know that that's the error. If we take a look here: http://www.erlang.org/doc/man/zip.html#extract-1 we find exactly what Booker suggested early on. The first argument to extract expects a file:name() type if it's supposed to be a path, and that resolves down to an erlang string, which we in elixir use with single and not double quotes.

Now, hopefully it's clear why this particular example happened the way it did. However, your concern about the vagueness of bad match errors still stands. If you look carefully you'll notice that the first line of the MatchError has this `{:error, {:EXIT, {{:badmatch` bit. Believe it or not that IS the error returned from the erlang library. Not very helpful I agree.

However, the nice thing is that particularly when dealing with elixir dependencies there's a great deal of care taken to provide a high quality developer experience, such that errors returned are more descriptive than the one returned by :zip.

Hopefully you feel a little less lost about how to deal with those errors, let me know if you have any further questions.

[Ben Wilson]

I should add that from a language perspective, I don't know of much that can be done. It's worth noting that the error you experienced is very very different than the problem set the ruby gem you linked or the R example you mentioned. The latter two have to do with supplying information on invalid code, and providing suggestions about how to create valid code. In your case you had perfectly valid code, you simply passed data to a library that didn't like the data.

The solution here is to work as a community to make sure that when our libraries error they return useful errors. I know Jose and others are hard at work improving Elixir's compile time analysis to help catch errors and ambiguous cases that trip up users (such as |> and () related changes). However, this particular question I think rests more on library writers than language developers.

[Edward Stembler]

John, I do appreciate the fishing lesson!  Though I felt your preamble could have been worded differently.

Don't give up!

On Friday, November 20, 2015 at 11:05:29 PM UTC-5, John Higgins wrote:

[José Valim]

John,

I have been teaching folks how to fish on these particular waters almost every day, for almost 5 years. How the content is delivered is as important as the content itself. We see this lesson even in Elixir itself: would Elixir be gaining traction as today if we didn't have a good way to get started? Elixir is the content, Mix is helping it to be delivered.

We appreciate your contribution to this and other threads. We hope there will be more to come, we are all learning here.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/CAPhAwGwizuyMjf2jmUiduF_a5pj6SZ%2B4ne%3DoZ7V72L9fzk%3Dtcw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Paulo Almeida]

Edward,

Not a direct answer to your original post, but after trying Excellent I ended up using Apache POI. Here's the code, in case you want to try this approach: https://github.com/pma/xlsx_reader.

To me it was also a first exercise on how to do Elixir-Java interop.

Regards,

Paulo

[Heri Sim]

Thanks for forming an inclusive community Jose. I think efforts such as Credo is a reflection on our shared values.

[Edward Stembler]

Just a follow-up... After converting my string to a char list, I was able to encounter the real error:

     ** (throw) {:EXIT, {:badarg, [{:re, :run, [nil, {:re_pattern, 1, 0, 0, <<69, 82, 67, 80, 131, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 255, 255, 255, 255, 255, 255, 255, 255, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 64, 0, 0, 0, ...>>}, [{:capture, :all, :index}, :global]], [file: 're.erl', line: 729]}, {Regex, :do_replace, 4, [file: 'lib/regex.ex', line: 485]}, {Excellent, :calculate_type, 2, [file: 'lib/excellent.ex', line: 136]}, {Excellent, :event, 3, [file: 'lib/excellent.ex', line: 68]}, {:xmerl_sax_parser_utf8, :event_callback, 2, [file: 'xmerl_sax_parser_utf8.erl', line: 3308]}, {:xmerl_sax_parser_utf8, :parse_attributes, 3, [file: 'xmerl_sax_parser_utf8.erl', line: 797]}, {:xmerl_sax_parser_utf8, :parse_document, 2, [file: 'xmerl_sax_parser_utf8.erl', line: 185]}, {:xmerl_sax_parser_utf8, :parse, 2, [file: 'xmerl_sax_parser_utf8.erl', line: 117]}, {Excellent, :parse, 2, [file: 'lib/excellent.ex', line: 14]}

Someone already created an issue for this: https://github.com/leifg/excellent/issues/6#issuecomment-154066588

[Avdi Grimm]

On Fri, Nov 20, 2015 at 5:56 PM Peter Hamilton <petergh...@gmail.com> wrote:

John - It's been said many times here that a confusing error message is a bug. I don't think it is appropriate to call expressing confusion "complaining". I appreciate that you took the time to teach him how to decipher the message, but we shouldn't discourage people from expressing that initial confusion. That feedback, especially from people new to Elixir, is how we make things even better than they already are.

Indeed. I find the the Elm team's perspective on compiler errors to be ideal: http://elm-lang.org/blog/compilers-as-assistants 

[Onorio Catenacci]

Hi Edward,

Maybe it's just me but I don't consider that error dramatically better than the stack trace you first showed.  Both of them seem to contain a lot of noise in addition to the true problem.  And in both cases, I'd be looking at the top of the stack and working my way down to the true issue. 

By the way, not to indulge myself in hair-splitting details I don't think that is the same error as the previous two that were associated with that issue. The line numbers are not the same, hence I doubt they're the same problem.

--

Onorio

[José Valim]

Maybe it's just me but I don't consider that error dramatically better than the stack trace you first showed.  Both of them seem to contain a lot of noise in addition to the true problem.  And in both cases, I'd be looking at the top of the stack and working my way down to the true issue. 

Yup. I have improved the guards in the Regex module so the error happens earlier in the stack, pointing to a better location.