Modifying encoding to UTF8 no BOM

[David Viard]

Hello BBEdit 

I have an issue with an export from Excel as TXT or CSV.

Excel adds a BOM to TXT / CSV files on export which kills my Shell Script.

I have to :

1. open the file in BBEdit change encoding and CTL to UNIX and Save (with the right encoding) or in OpenOffice and save again…
   
2. Rename the File from .csv to .txt 
   

• It's tedious
  
• I have multiple files to do this way
  
• Coworkers might forget this part of the process and generate an error (File not Found)
  

So, I am inquiring of a scripting solution to handle poInts 1 & 2

Was thinking of a Shell Script and a BBEdit but not sure how to do it and include this in my Script

Thanks for any input 

[Patrick Woolsey]

This sounds to me like a job for a *text factory* with the _Change Text Encoding_ action. :-)

For details, please see the section titled "Text Factories" starting on page 155 of the PDF manual (Help -> User Manual) and I reckon that should suffice to get you started.


Regards,

Patrick Woolsey
==
Bare Bones Software, Inc. <https://www.barebones.com/>

[David Viard]

Hello Patrick,
Thanks for your answer.

I'll try to be more precise.

It seems the tendency is to use UTF8 no BOM (default) and UTF-16 with BOM. But MICROSOFT Excel (on My Mac, Saves as UTF8+BOM(default) with Windows CRLF (!) 😖.

Context : I have a Shell Script that :

1. reads a CSV file with Paths to files (in a folder hierarchy of various folders/subfolders), 
2. Applies optional extension filters,
3. downloads files to a folder as a flat structure (all on one level),
   
4. The Script has also process & error logging,… 
   
5. Moves the processed CSV Files with the logs (With client Name/timestamp…)

I did not use AppleScript.

So either my Shell Script misses something or I need to launch BBEDIT just for this conversion, prior to the Script execution. 

Do BBEDIT cmd line Tools handle this? I am not sure from my quick read of the manual.

Would I need some AppleScript to trigger BBEdit & handle the save Dialog options?

Note, on my last test & log, I get a "xargs: unterminated quote" error, and no Files processed.

I can share the Script if that helps

Thanks a lot