Issue when importing CSV with BOM

[Lukas Heidbreder]

Hi everyone.

We have stumbled into an issue regarding CSV imports, namely if the CSV file is UTF-8 with BOM.

An otherwise correct CSV file has a BOM will show

> org.adempiere.exceptions.AdempiereException:

> org.adempiere.exceptions.AdempiereException:

> Field <field name> not found in tab <current tab>

Removing the BOM (e.g. using a hex editor like PSPad) fixes the issue.

Has anyone else encountered and solved the issue?

Thanks in advance!
Lukas

[Carlos Antonio Ruiz Gomez]

Yes, I think the BOM is not accepted.

[Martin Schönbeck]

Hi,

while it's of course recommended to never use excel but to take calc, many people don't obey this recommendation. And because excel is to dumb to correctly open csv-files with utf-8 without a BOM many programs exporting csv-files now enter a BOM. And excel does so, too, when writing a csv-file. So I think it's necessary to implement this, especially because it's not easy to detect the reason of the resulting error.

Regards,

Martin

[Carlos Antonio Ruiz Gomez]

Agree, would be good to accept that case, which is somehow common.

[Christophe CapBarthodia]

Hello,
Have you implemented a solution to handle this case?
Or do we need to create a plugin?

Regards

Christophe

[Carlos Antonio Ruiz Gomez]

I think is not implemented.  Indeed I think it doesn't have a JIRA ticket.

Instead of doing it in a plugin I would suggest to open a JIRA ticket and contribute a pull request for core.

Regards,

Carlos Ruiz

[Christophe CapBarthodia]

Hi Carlos,

Actually, we'd like to manage this for both CSV import and export.
The version we need this for is Idempiere 10.
When I tried to open a JIRA, I saw that it only works from version 11 and higher.
Since this is the first JIRA we're opening, I don't want to make a mistake.
Do I need to open it in IDP 11, and will we be responsible for modifying the CORE?
Thanks for your feedback.

regards 

Christophe

[Carlos Antonio Ruiz Gomez]

Hi Christophe,

You can open the JIRA ticket here:

https://idempiere.atlassian.net/secure/CreateIssue.jspa

The flow in iDempiere is that we always make the fixes, improvements, etc directly in the branch master.

So, master (at this moment release-13) get always the commits first.

After a few days usually bug fixes and non-disruptive improvements are integrated into the maintained release (at this moment release-12).

So, a pull request must be always created against the master branch.  If that becomes too complicated for you andyou have a patch that works for release-10, then you can attach it to the ticket and ask for help to create a pull request based on it.

Regards,

Carlos Ruiz