File-split

76 views
Skip to first unread message

Pete

unread,
Nov 13, 2014, 7:59:09 PM11/13/14
to csv...@googlegroups.com
I have a large csv file that I wish to split by state codes. File-split will do what I want,I'm think, but does the file need to be sorted by state to work?

The files has close to 200M records and I do not want to loop through it multiple times, but rather export a row to the appropriate file, if the file exists, else create the file.

Can this be done with csv fix without trying to sort the data?

Thanks

Neil Butterworth

unread,
Nov 14, 2014, 5:32:20 PM11/14/14
to csv...@googlegroups.com
To use file_split, the input file does not need to be sorted, and CSVfix will only need to make a single pass over it.

Regards,

Neil Butterworth

ly...@southerlyengineering.com

unread,
May 19, 2016, 5:29:22 PM5/19/16
to csvfix
I just tried to spilt by the 13th column a record with about 334,000 rows. It ran for awhile but no files were created and no errors.
How do I know what is wrong?

Neil Butterworth

unread,
May 19, 2016, 8:14:32 PM5/19/16
to csvfix, ly...@southerlyengineering.com
CSVfix  should  have no problems processing this file, and if it  didn't produce an error message, it probably has done so. are you sure you are looking in the right directory for the output files?
Reply all
Reply to author
Forward
0 new messages