Thoughts on adding UTF-8 BOM to DROID export CSV

[ross-spencer]

Hi All,

Wrestling with Unicode here. We need the handling of the DROID export with UTF-8 characters encoded to be a bit simpler. 

UTF-8 support has recently been added to DROID which has helped our work with macrons in Maori language no end.

We then manipulate the DROID output for various purposes. Using tools to do this isn't a problem. The issue only arises when we have to add a human stage, e.g. the addition of an extra column to the DROID sheet to annotate information or some such. At the moment this means opening it in Excel 2010, and having to follow the steps listed here: http://stackoverflow.com/a/6488070

It's not a bad approach but the more people that have to do this, and the consistency required is difficult to achieve. 

It seems that adding the UTF-8 BOM to the file helps Excel to recognize the file immediately as UTF-8 which is the most desirable behavior. 

I guess my questions are: 

• Does anyone know if this is at odds with any CSV 'specification' 
• If not, is there an issue adding this support to DROID? - Maybe a UTF-8 with BOM option? 

Thanks,

Ross

[Brian O'Reilly]

Thanks for raising this isue Ross, we'll take this on boadr when reviewing the next round of development isues for DROID early in 2015.
Regards, Brian