Mongoimport - How to specify types of fields efficently?

[jayatdolp...@gmail.com]

Hi,

I am quite new at mongodb and I try to use a public dataset csv file into mongo db.

When I use a simple mongoimport, it works well but the types for fields are mixed. For instance I can have string and int32 for the same field.

This is causing a problem when I try to visualise the data in visual data discovery software.

What I would like to do is to specify the type of each field. Like this I hope to get one consistent type per field.

I m using MongoDB 3.4 community server with mmpavp1 storage.

I use this command:

.\mongoimport --port 28018 --db Crimes --collection Chicago3 --type csv --columnsHaveTypes --headerline  --file F:\Demos\30-Swarm\SimplifiedCrimes_-_2001_to_present.csv

and I get " Failed: could not parse type from header ID"

When I try to specifically give a type to columns:

.\mongoimport --port 28018 --db Crimes --collection Chicago2 --type csv --columnsHaveTypes --fields col1.int32,col2.string,col3.date,col4.string,col5.string,col6.string,col7.string,col8.string,col9.string,col10.string,col11.string,col12.string,col13.int32,col14.int32,col15.string,col16.int32,col17.int32,col18.int32,col19.date,col20.double,col21.double,col22.string  --file F:\Demos\30-Swarm\SimplifiedCrimes_-_2001_to_present.csv

What i get is col1,2,3 ... as names of fields (which is unfortunately expected but if I use the real names it does not import) and the types are most of them as "document"

Can someone help me to achieve something that seems ... easy to do.

Any help is much appreciated.

Jerome

[Kevin Adistambha]

Hi,

When I try to specifically give a type to columns:

.\mongoimport —port 28018 —db Crimes —collection Chicago2 —type csv —columnsHaveTypes —fields col1.int32,col2.string,col3.date,col4.string,col5.string,col6.string,col7.string,col8.string,col9.string,col10.string,col11.string,col12.string,col13.int32,col14.int32,col15.string,col16.int32,col17.int32,col18.int32,col19.date,col20.double,col21.double,col22.string —file F:\Demos\30-Swarm\SimplifiedCrimes_-_2001_to_present.csv

The example in the —columnsHaveTypes page specified that you could specify the field type in the header (i.e. first line) of your CSV file. However, the example also mentions that:

Field names must be in the form of <colName>.<type>(<arg>)

I think you are missing the brackets in your type definitions. For example, I created this example CSV:

col_one.string(),col_two.int64(),col_three.boolean()
"string1",123,true
"string2",456,false

Importing it into MongoDB using mongoimport:

mongoimport --type csv --columnsHaveTypes --headerline test.csv

Which results in:

> db.test.find()
{ "_id" : ObjectId("5948caf656490a758778a17f"), "col_one" : "string1", "col_two" : NumberLong(123), "col_three" : true }
{ "_id" : ObjectId("5948caf656490a758778a180"), "col_one" : "string2", "col_two" : NumberLong(456), "col_three" : false }

If you’re still having issues with mongoimport, could you post the full error message, your CSV file, and your full MongoDB version (i.e. 3.4.4)?

Best regards,
Kevin

​