How do I make a Django model from a tab-delimited data file?

[Tom Tanner]

I have a tab-delimited data file that looks something like this:

NAME S1903_C02_001E state county tract State-County-Tract-ID
Census Tract 201, Autauga County, Alabama 66000 01 001 020100 01001020100
Census Tract 202, Autauga County, Alabama 41107 01 001 020200 01001020200
Census Tract 203, Autauga County, Alabama 51250 01 001 020300 01001020300

I want to make a Django model named `MyModel` with three columns: "name", "data", and "geoid", which correspond to the file's columns "NAME", "S1903_C02_001E", and "State-County-Tract-ID." Can I do this via command line, or with a custom Python script? I'm running my Django project locally on a computer running Debian 9.3. 

[Andréas Kühne]

Hi,

You will have to parse the CSV file manually with a custom management command (at least that is what I would do). All you need to do is open the file, split each row with a "," and then  import the correct columns to the model.

You can also use something like pandas to convert the CSV file into something that you can create the models from. But that may be overkill in your case.

Regards,

Andréas

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/389d8f38-6dbc-43a0-8a69-d20aa13ca844%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kasper Laudrup]

Hi,

On 2018-01-09 08:09, Andréas Kühne wrote:
> You will have to parse the CSV file manually with a custom management
> command (at least that is what I would do). All you need to do is open
> the file, split each row with a "," and then  import the correct columns
> to the model.

Unfortunately, CSV is a really bad, non-standardized format and simply
splitting each row with a "," will cause you all kinds of issues with
espacing values with a "," and other things.

I would suggest using the python CSV library for this:

https://docs.python.org/3/library/csv.html

Just though that was worth mentioning.

Kind regards,

Kasper Laudrup

[Andréas Kühne]

Kasper,

You are correct :-), I just assumed (probably incorrectly) that the data was just like his example - in that case it would have been easiest :-)

Regards,

Andréas

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscribe@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/83268fb2-22f6-2d6e-c14c-0015d77c48f0%40stacktrace.dk.

[Tom Tanner]

Thanks you two. I'll check out that parser.

[Scot Hacker]

Another approach is to use postgres' COPY command:

https://www.postgresql.org/docs/9.6/static/sql-copy.html

which knows how to parse and import tab-delimited files (and is crazy fast). Then, once the table exists, use Django's `inspectdb` management command to generate a Model corresponding to the table. It may need a little manual massaging afterwards, but I've done this on several projects and it works very well for most purposes.

./s