Is CsvFix Check wrong or am I missing a fundamental here?

[William Finke]

CsvFix Check appears to declare valid *.csv files containing double-quotes as invalid with the error "Embedded newline in xxx at line 1".

Couldn't get what I thought were valid files to pass the 1.6 CsvFix Check. So I tried the uninstall/reinstall and the files in .\data. Same error on every one that had double-quotes.

Went to a brand spanking new Windows 8.1 machine and installed 1.5 - same error (on birthdays.csv for example).

Uninstalled 1.5 and installed 1.2 - same thing.


Are the data\*.csv files truly invalid or is there some piece of info I am missing?

[Neil Butterworth]

It's a bug.

[William Finke]

Should I go enter an issue to track this?

You would not believe how much time I spent trying to figure out what was going wrong on my side. I guess it shows how much I am impressed by this body of work.

It looks like the fault occurs in csved_check (my recollection), not in a common routine. If anyone can confirm this, my confidence is restored.

[Neil Butterworth]

No need to post an issue - I will fix it in the next release. Also, the check command  uses a different CSV parser from the other commands, so they are not affected.

[marcde...@gmail.com]

Hi Neil,

I have just encountred this bug with my
test file "csvin3comma.csv", content:
Field1,Field2,Field3,Field4
One,,Two,Three
What,,"Is,",It
What,,Is,It
What,,"""Yes,No""",It
What,,Is,It
"""What""",,Is,It
"""What""","""yes, no",Is,It
20,,aaa20,No
20,"""",aaa20,Yes
21,"""Hello""",abc21,"""Sir"""
23,,xyz00,","
24,"""Yes!""",pdq24,9
28,,89,3
25,"""Yes,No""","26,pd",q24
25,,"26,pd","q""Sir""24"

csvfix check csvin3comma.csv =>
Embedded newline in csvin3comma.csv at line 3
What,,"Is,",It

Have you already fixed this bug? It would be great if this bug is fixed.
Maybe you can create a minor Update version only with this bug fix?

Thank you very much.

[Neil Butterworth]

It is fixed in the BitBucket Mercurial repository: https://bitbucket.org/neilb/csvfix/commits/93804d497be6e41a0b71864d15d2a8346cc63364

But I'm afraid that if you want the fix, you will need to build CSVfix yourself.

[marcde...@gmail.com]

Thank you Neil.
I have very few knowledge in C and no knowledge with C++ but I have a
Code::Blocks IDE installed so I tried to open the csvfix.cbp with it and build:

||=== Build: Debug in csvfix (compiler: GNU GCC Compiler) ===|
 ..\alib\lib\alib.a||No such file or directory|
||=== Build failed: 1 error(s), 0 warning(s) (1 minute(s), 51 second(s)) ===|

Unfortunately I can't succeed to get a Windows exe file.

[Neil Butterworth]

The C::B build process is a bit complicated - you need to build alib before csvfix, using its own .cbp file. Simplest, if you have installed the GCC tools that come with C::B, and if you have them on your PATH, is to open a command-line shell in the project root  directory and simply do make win.

[marcde...@gmail.com]

Thank you Neil.
I have now created in the project root directory a file "build.bat" with the content:
path=c:\Program Files\CodeBlocks\MinGW\bin\;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem
mingw32-make.exe win

After executing build.bat, csvfix.exe was created successfully in the bin directory.
With my test file, there is now no output for:
csvfix check csvin3comma.csv

So I think everything is correct now? BTW, were there other changes since Version 1.6?
Thank you again.

[Neil Butterworth]

Yes, if there is no output, it is working. Regarding the changes since release 1.6. there are actually quite  a few (more than I remembered) though nothing that would invalidate usage of previous releases. You can see them in the Mercurial repository, or in the Help&Doc database csvfix-manual.hnd  (for which you will need the free HelpNDoc program). As the documentation is more up-to-date than I remembered, I may consider doing a 1.7 release, but don't hold your breath.

[marcde...@gmail.com]

Hi Neil, great thank you! An official 1.7 release would be of course even better.