What is the proper way to use Net::LDAP when sending UTF-8 strings

[danabr]

Hi,

When  I upgraded from Ruby 1.8.7 to 1.9.2 (and the latest version of net-ldap) I faced a lot of encoding issues. First, I made sure all my strings used the same encoding (UTF-8). However, sending UTF-8 encoded strings to net-ldap caused crashes (about not being able to concatenate UTF-8 and ASCII-BIT strings). I solved the problems by using  force_encoding("ASCII-8BIT") on all input to net-ldap. As far as I have been able to tell, the data seems to be correctly formated when I look in the LDAP directory.

Now, is this hack the proper way to deal with non-ASCII data in net-ldap? I thought LDAP v3 defaulted to UTF-8 strings, and that net-ldap would follow that default. If this is the proper way to deal with this issue, why not add the conversion to net-ldap itself, so users don't have to worry?

//Daniel Abrahamsson

[Austin Ziegler]

First: my apologies for not getting your message and membership approved faster.

I don't really have an answer for you about this, as I thought that
we'd gone through to make sure that the current version of net-ldap
was UTF-8 safe.

Now that your message is here, hopefully someone will be able to walk
you through the right questions to get an answer.

-a
--
Austin Ziegler • halos...@gmail.com • aus...@halostatue.ca
http://www.halostatue.ca/ • http://twitter.com/halostatue

[danabr]

To elaborate, the problem only appears if you use special characters. For example, the string "whatisthis" encoded in UTF-8 works just fine (if my memory does not fail me, this is because the representation of this particular string is the same in UTF-8 and ASCII), but the string "denna sträng fungerar ej" will cause a failure because of the "ä".

A test with a string with some special characters should not be too hard to add to the test suite, right?

//Daniel

[R. O'Connell]

Aha this definitely sounds serious. When we made net-ldap work in 1.9 we essentially just modified the String#to_ber to not do byte level string scanning which broke in 1.9, but I don't recall we ever went through and made it UTF safe yet. For now forcing ASCII-8 would be the workaround until we can get into it.

This should absolutely be added to the test suite, which we're converting to all rspec and modifying to attach to a real LDAP server. Thanks for catching this.

[danabr]

Ok, thanks for looking into it. Can I help out in any way?

//Daniel

[danabr]

I've recently been working on adding support for UTF-8 (or in fact, any encoding) to net-ldap. When it comes to sending data, it turns out we can just treat it as raw byte strings internally, as long as the client and server agree on the encoding. This seems reasonable to me. One has to do one fix to String#to_ber, though, and make it use "ASCII-8BIT" encoded strings. You can see the change here: https://github.com/danabr/ruby-net-ldap/commit/2bc334d2904c68ad2ca0cb8e95661a87cbae2632

This has worked great for me in the project I am working on, which uses UTF-8 and an OpenLDAP server. The code must be tested on more setups, though.

Now, so far I have only been dealing with the easy part, namely sending data to the server. It becomes a little bit more complicated when receiving data. Ideally, I would like it to work something like this:

  Net::LDAP.open(..., :encoding => 'utf-8') do |ldap|

      ldap.search(...) do |entry|

        entry[:dn] # utf-8

         entry[:cn][0] # utf-8

      end

  end

The question is where the proper place to place the decoding is. I've been thinking about storing the encoding as a property of the connection, and then pass on the encoding to Net::LDAP::PDU to do the work. But I don't really see a "clean" way of doing it.

Do you have any suggestions?

[krogebry]

I might have a fix for this here: https://github.com/ruby-ldap/ruby-net-ldap/pull/44

As far as I can tell it might help you out.