HELP I broke my app after integrating utf characters

[braun...@gmail.com]

I had a perfectly running webapp in Nitrogen.

2 days ago I replaced all strings to valid spanish characters.

For example, I had strings like "buenos dias", where "i" must have
acute accent: "buenos días".

I changed many many strings that are
emited like strings embedded as Nitrogen elements ["buenos días"...].
Some other strings are embedded as js code, like "window.alert('buenos días')".

Now Im getting errors in places where everything was running fine.

In this particular trace the string is now  "Debe indicar una descripción breve", look how the trace data breaks just at the "ó" character.

What is the origin of these bugs?

Should I emit those strings as binary strings like <<"buenos días"/utf8>>. Thats the syntax, right?

  crasher:
    initial call: cowboy_clear:connection_process/5
    pid: <0.23012.38>
    registered_name: []
    exception error: bad argument
      in function  iolist_to_binary/1
         called as iolist_to_binary([<<"nitrogen_system_event:">>,
                                     {error,
                                      <<"Nitrogen.$set_param('pageContext', 'QW9Qm03U28vyCbAGHZGGElIQmuAf3XGqEVhVTT9zyyJKJjmtSCAFxiwmPr_DzFbW0zEEM6DiwX1vEB6_0xcm2Wf9NSFXOXn7Dn1vwEYzSb3-18cYoj0k1M2Nbf5XD-xpuknqo6eXatLcPynhSc6ncEwSZisLFbjJQNfIRgqwqkEIroZn71uK4YeMtgVNc-T1wlge8LzY5BftIWjfXisardS22xC4bmwk-fdBwD6ReoCCpQoYewad8spw5AyxfriJDyu6h6ZH-V3lcejcIFEJ2sHeHR9cnfl2vM9cI-_sacM1X76vv0GGjEd5eMVnCZmtNrsl2WgvnTYXUmZEOavrKUakqCqkdgEAcQgyjXu34B9Wj1NopI1RQW9hbdYJUYjVUPenlbn1aXbPJZWgerR-Pl1Mm_MnxrsuFXT12xdLzfp-4uXL');\n\nNitrogen.$anchor('.wfid_temp3967810', 'page');\nNitrogen.$anchor('.wfid_temp3967810', '.ezy_error');Nitrogen.$remove(\".wfid_temp3967810\", \".ezy_error\", \"\");\nNitrogen.$anchor('.wfid_temp3967810', '.wfid_temp3967810');var elem = obj('slogan'); if (elem.parentElement.classList.contains('input')) var target = elem.parentElement; else if (elem.dataset.role=='spinner') var target = elem.parentElement; else var target = elem; $('<div class=\"ezy_error\" style=\"color:red\">Debe indicar un slogan breve</div>').insertAfter(target);\nNitrogen.$anchor('.wfid_temp3967810', '.wfid_temp3967810');var elem = obj('descripcion'); if (elem.parentElement.classList.contains('input')) var target = elem.parentElement; else if (elem.dataset.role=='spinner') var target = elem.parentElement; else var target = elem; $('<div class=\"ezy_error\" style=\"color:red\">Debe indicar una descripci">>,
                                      [[[[[[[<<"ón breve (entre 2 y 80 palabras)</div>').insertAfter(target);\nNitrogen.$anchor('.wfid_temp3967810', 'page');Nitrogen.$cancel_system_reconnection_event('temp3967906');">>]]]]]]]}])
      in call from cowboy_simple_bridge_anchor:massage_reply/2 (src/cowboy_bridge_modules/cowboy_simple_bridge_anchor.erl, line 90)
      in call from cowboy_websocket:handler_call/6 (src/cowboy_websocket.erl, line 454)
      in call from cowboy_http:loop/2 (src/cowboy_http.erl, line 201)
    ancestors: [<0.130.0>,<0.129.0>,ranch_sup,<0.121.0>]

[Jesse Gumm]

Hey Franklin,

The {error, Binary1, Binary2} tuple indicates that the unicode:characters_to_binary/1 call is failing, then passing that busted tuple to simple_bridge.  There's something about that that breaks Erlang's unicode handler:

Specifically, in nitrogen_core, there are these calls, both of which end with wf:to_unicode_binary/1.

https://github.com/nitrogen/nitrogen_core/blob/master/src/wf_core.erl#L73-L82

If you were to wrap a try-catch around those wf:to_unicode_binary functions, you could try to get to why the unicode handler is crashing on those lines.  It may be enough to try to wrap your unicode binaries with <<"abc"/utf8>> as you've suggested.

Specifically, I did a quick test:

> wf:to_unicode_binary("Debe indicar una descripción breve").
<<"Debe indicar una descripción breve"/utf8>>

> wf:to_unicode_binary(<<"Debe indicar una descripción breve">>).
{error,<<"Debe indicar una descripci">>,<<"ón breve">>}

So if I pass just the string "Debe indicar una descripción breve" to wf:to_unicode_binary/1, it works, but if I pass the naive binary <<"Debe indicar una descripción breve">> (without the /utf8), it crashes.

So I suspect if, in your code, you're doing binaries, make sure you have the /utf8 flag in the binary.

-Jesse

--
You received this message because you are subscribed to the Google Groups "Nitrogen Project / The Nitrogen Web Framework for Erlang" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nitrogenweb...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nitrogenweb/2be55803-7447-47cf-8c2f-0906c45ff6b1n%40googlegroups.com.

--

Jesse Gumm
Owner, Sigma Star Systems
414.940.4866 || sigma-star.com || @jessegumm

[braun...@gmail.com]

wf:to_unicode_binary worked for me. Just read the 'unicode' module documentation and it all makes sense now. Thanks.