Edifact in UTF-8 problem

[Jakub]

Hello guys,

I have created a few fully working channels so far, but this week I started receiving files I cannot handle..
It seems very simple, the only problem is that it's encoded in UTF-8.

When I open the file with Notepad and change "UTF-8" to "ANSI" the translation works like a charm.
If the character set is untouched I get this error:
 "InMessageError: [A57]: Edi file does not start with a valid interchange."

What I tried, naturally, is set the proper character set in edifact.py file, but it's completely ignored.
The route processing crashes in the second phase already (Infile error) before even edifact.py is compiled.

I don't believe the files are impossible to be read. What am I doing wrong here?

Thanks for any answers

[henk-jan ebbers]

hi Jacub,

bots should derives the character set from the edifact message.
what character-set does the edifact UNB segment say it is (UNOA, UNOB, etc).
does it crash or do you get an error? (what error do you get?)

can you send me the edifact file for examination (if wished you can send it directly to hjeb...@gmail.com )

kind regards,
henk-jan

> --
> You received this message because you are subscribed to the Google Groups "Bots Open Source EDI Translator" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to botsmail+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Jakub]

hi Henk-jan

I'm getting an error as written before:

 "InMessageError: [A57]: Edi file does not start with a valid interchange."

The UNB segment says UNOA but again, this is ignored.. If I change it to anything else, nothing happens.
I already sent you the files.

Best regards,
 Jakub

[henk-jan ebbers]

the files are uft-8 with BOM. (http://en.wikipedia.org/wiki/Byte_order_mark)
edifact does have uft-8 as UNOY-character set.

edifact supports no BOM
In fact they even explicitly state that (when using utf-8) the start characters of an edifact file should all be parseble as ASCII (in order to find out the character set).

so there files have 2 problems:
- BOM
- not indicating the right character set in the UNB

kind regards,
henk-jan

[Jakub]

Thanks for the answer. After your message I made some more research and in deed the file wasn't sent from our partner as UTF-8 with BOM.
The problem was that these files go via a third-party operator and instead of just being passed through completely are passed through and set as UTF-8 (default charset) on the output.

Cheers:)

[henk-jan ebbers]

adding a BOM is not 'added value';-))

henk-jan

On 03/05/2013 03:37 PM, Jakub wrote:
> Thanks for the answer. After your message I made some more research and in deed the file wasn't sent from our partner as UTF-8 with BOM.
> The problem was that these files go via a third-party operator and instead of just being passed through completely are passed through and set as UTF-8 (default charset) on the output.
>
> Cheers:)
>
> W dniu wtorek, 5 marca 2013 11:37:46 UTC+1 użytkownik eppye napisał:
>

> the files are uft-8 with BOM. (http://en.wikipedia.org/wiki/Byte_order_mark <http://en.wikipedia.org/wiki/Byte_order_mark>)

> edifact does have uft-8 as UNOY-character set.
>
> edifact supports no BOM
> In fact they even explicitly state that (when using utf-8) the start characters of an edifact file should all be parseble as ASCII (in order to find out the character set).
>
> so there files have 2 problems:
> - BOM
> - not indicating the right character set in the UNB
>
> kind regards,
> henk-jan
>
>
> On 03/05/2013 11:27 AM, Jakub wrote:
> > hi Henk-jan
> >
> > I'm getting an error as written before:
> > "InMessageError: [A57]: Edi file does not start with a valid interchange."
> >
> > The UNB segment says UNOA but again, this is ignored.. If I change it to anything else, nothing happens.
> > I already sent you the files.
> >
> > Best regards,
> > Jakub
> > --
> > You received this message because you are subscribed to the Google Groups "Bots Open Source EDI Translator" group.

> > To unsubscribe from this group and stop receiving emails from it, send an email to botsmail+u...@googlegroups.com <javascript:>.
> > For more options, visit https://groups.google.com/groups/opt_out <https://groups.google.com/groups/opt_out>.