First of all, thanks for pointing out the bug with characters not in the latin1 range. That's something I need to test and fix soon.
Actually, when I said non-ascii characters last time, I should have said characters from the extended ascii set. I was having problems using binary to list because it was giving me the wrong key length. Maybe this shell output can explain it better:
iex(2)> String.length "£"
1
iex(3)> byte_size "£"
2
iex(4)> cl = String.to_char_list "£"
[163]
iex(5)> length cl
1
iex(6)> bl = :binary.bin_to_list "£"
[194, 163]
iex(7)> length bl
2
When it gets to the NIF, this character will be interpreted as one character, and if I send it a key_len of 3 (including the NULL character), it will not work.
My point about the erlang version is just that it was useful to clear things up in my own mind on how to approach this. I am well aware that an Erlang string is equivalent to an Elixir char list.
I've started
a new branch, which I think is a lot cleaner (the string is converted to a charlist as soon as it enters the module) and maybe explains what I'm trying to do better.