Encoding problem loading world data in tutorial

[Carlos de Infierno]

Hello, this is my first post here ... I'm following the geodjango tutorial and during the LayerMapping section, while running "load.run()" something internally is crashing on non-ascii characters within the dataset.  I tried changing the encoding to "UTF-8" but same error.  Here is the output:

Saved: Monaco
Saved: Palestine
Saved: Montenegro
Saved: Mayotte
Failed to save the feature (id: 141) into the model with the keyword arguments:
{'iso2': u'AX', 'pop2005': 0, 'area': 0, 'region': 150, 'lon': 19.952, 'iso3': u'ALA', 'subregion': 154, 'fips': u'', 'lat': 60.198, 'un': 248, 'mpoly': u'MULTIPOLYGON (((20.210831 59.979996000000142,20.203327 59.976944,20.137215 59.991661,20.129162 59.994995,20.004166000000112 60.
....
00108 60.317497,19.862221 60.312492,19.838942 60.319824,19.834274 60.322159,19.779999 60.349159,19.782776 60.354439,19.808331 60.377495000000124,19.824997 60.39222,19.85722 60.403610000000128,19.868610000000103 60.403610000000128,19.90583 60.399994,20.084721 60.350273)))', 'name': u'\xc3\x85land Islands'}
'ascii' codec can't encode characters in position 7-8: ordinal not in range(128)
>>>

I searched for this error but don't find anything specifically regarding geodjango.  Can anyone help me get past this step?  Thank you!
Carlos

[Alex Mandel]

It's a shapefile so it's probably latin1 encoded (aka 1252)

If you're on a non english version of windows that could be an issue.
Linux tends not to care.

When you setup Postgis and the template or django db did you specify an
encoding?

Then there's the oddball handling of encoding in Python, are you using
Python 2 or Python 3?

FYI, I usually load my data directly to POSTGIS once django has made the
table using ogr2ogr, shp2pgsql or QGIS.

-Alex

On 04/23/2015 08:02 AM, Carlos de Infierno wrote:
> Hello, this is my first post here ... I'm following the geodjango

> <https://docs.djangoproject.com/en/1.6/ref/contrib/gis/tutorial/> tutorial

[Charles Cosse]

Alex, thank you for the clues!

On Friday, April 24, 2015 at 12:24:40 AM UTC-6, Alex wrote:

It's a shapefile so it's probably latin1 encoded (aka 1252)  

If you're on a non english version of windows that could be an issue.
Linux tends not to care.

When you setup Postgis and the template or django db did you specify an
encoding?

I think you have identified the problem here.  I followed the postgis installation page linked to from the geodjango tutorial.  And in the postgis installation it does indeed specify UTF-8 in the createdb command.  So the geodjango tutorial page should have a note about that ...
 

FYI, I usually load my data directly to POSTGIS once django has made the
table using ogr2ogr, shp2pgsql or QGIS.

 Without changing anything I was able to import the shapefile into the database via these commands:

shp2pgsql -W LATIN1 world/data/TM_WORLD_BORDERS-0.3.shp>world_borders.sql

psql -d geodjango geo < world_borders.sql

I guess I'm back in business now.  Thank you again!
Charles

[eduard...@gmail.com]

Hi,

I've managed to fix the problem. Here are the steps to acomplish it:

Open the file on QGIS using "Add Layer", "Add Vector Layer"
Right-click on the shapefile and choose "Open attribute table"
Select the row with "ISO3" equals to "ALA" and click on the pencil icon
Rename the "Name" of the record to "Aland Islands"
Click on the pencil again and save the changes
Right-click on the shapefile again and save it in another folder, using the desired encoding, in my case, "UTF-8"

I hope it helps.

Cheers,
Eduardo