eve-db problem importing eve data

[Streamweaver]

I expect this is a known problem but just wanted to post it.

When running the import of the CCP dump via manage.py the data import
throws a sqlite3 Table Does Not Exist error when it gets to the
planetSchematics table.

Checking the sqlite evedata dump these tables indeed are not contained
in that export and the remaining planet tables are not either.

I expect I can just comment those tables out of the IMPORT_LIST list
but wanted to check if there was something I was missing first.

Thanks for any feedback.

[Gregory Taylor]

Can you post a link to the URL you got that data dump from?

> --
> You received this message because you are subscribed to the Google Groups "Django and EVE Online" group.
> To post to this group, send email to djang...@googlegroups.com.
> To unsubscribe from this group, send email to django-eve+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/django-eve?hl=en.
>
>

[Streamweaver]

I picked it up from the URL in the readme file http://eve.no-ip.de/dom111/

I downloaded the file dom111-sqlite3-v1.db.bz2 and not the eam version
as directed.

The import itself seems to run fine until I hit the planetSchematics
import step.

On Nov 2, 10:47 am, Gregory Taylor <gtay...@gc-taylor.com> wrote:
> Can you post a link to the URL you got that data dump from?
>

[Streamweaver]

So looking at the latest Eve Online Toolkit page the latest DB dump
version appears to be Tyrannis_1.0.4_33864_db.7z and using the URL
convention I tried typing it into my browser and found the following
http://eve.no-ip.de/tyr104/

Am downloading try104-sqlite3-v1.db.bz2 to give that a try and see if
the import works. Download and subsequent import will take awhile
though so will update everyone.

If this new location is indeed accurate I"ll update the documentation
and submit a patch.

On Nov 2, 10:47 am, Gregory Taylor <gtay...@gc-taylor.com> wrote:

> Can you post a link to the URL you got that data dump from?
>

[Streamweaver]

Tried the latest version of the data dump with Tyrannis 1.0.4

No Joy there, threw the following Error. Anyone know what version of
the dump to use?

- crpActivities: 100% [========================================]
Time: 00:00:00
Traceback (most recent call last): ]
ETA: --:--:--
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/
__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/local/lib/python2.6/dist-packages/django/core/management/
__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/
base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.6/dist-packages/django/core/management/
base.py", line 218, in execute
output = self.handle(*args, **options)
File "/home/sturnbu/NetBeansProjects/evetools/src/evetools/apps/
eve_db/management/commands/eve_import_ccp_dump.py", line 124, in
handle
util.run_importers(util.IMPORT_LIST)
File "/home/sturnbu/NetBeansProjects/evetools/src/evetools/apps/
eve_db/ccp_importer/util.py", line 140, in run_importers
importer.prep_and_run_importer(conn)
File "/home/sturnbu/NetBeansProjects/evetools/src/evetools/apps/
eve_db/ccp_importer/importers/importer_classes.py", line 29, in
prep_and_run_importer
self.import_row(row)
File "/home/sturnbu/NetBeansProjects/evetools/src/evetools/apps/
eve_db/ccp_importer/importers/system.py", line 30, in import_row
graphic.name = row['urlWeb']
IndexError: No item with that key


On Nov 2, 1:29 pm, Streamweaver <streamwea...@gmail.com> wrote:
> I picked it up from the URL in the readme filehttp://eve.no-ip.de/dom111/

[Matt]

I'm getting that exact error message with that data dump as well.

[Matt]

Yeah, just verified that the schema in the sqlite dump is wonky:

sqlite> .schema eveGraphics
CREATE TABLE "eveGraphics" (
"graphicID" smallint(6) NOT NULL default '0',
"graphicFile" varchar(500) NOT NULL,
"description" varchar(16000) NOT NULL,
"obsolete" tinyint(1) NOT NULL,
"graphicType" varchar(100) default NULL,
"collidable" tinyint(1) default NULL,
"explosionID" smallint(6) default NULL,
"directoryID" int(11) default NULL,
"graphicName" varchar(64) NOT NULL,
PRIMARY KEY ("graphicID")
);
CREATE INDEX "eveGraphics_IX_explosionID" ON
"eveGraphics" ("explosionID");

vs the mysql dump's:

mysql> describe eveGraphics;
+-------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| graphicID | smallint(6) | NO | PRI | NULL | |
| url3D | varchar(100) | YES | | NULL | |
| urlWeb | varchar(100) | YES | | NULL | |
| description | varchar(1000) | YES | | NULL | |
| published | tinyint(1) | YES | | NULL | |
| obsolete | tinyint(1) | YES | | NULL | |
| icon | varchar(100) | YES | | NULL | |
| urlSound | varchar(100) | YES | | NULL | |
| explosionID | smallint(6) | YES | MUL | NULL | |
+-------------+---------------+------+-----+---------+-------+

On Nov 2, 1:40 pm, Streamweaver <streamwea...@gmail.com> wrote:

[Streamweaver]

Overall it seems the current supported schema is prior to Tyr1.04.
They changed the way they deal with graphics, moving Icons off to
their own table. The code is going to have to be migrated to support
the new schema.

That said it appears an earlier version of Tyr is supported by the
current eve-db codbase and that it's no longer compatable with the Dom
schema.

[Matt]

So can someone link a static dump that works?

[Streamweaver]

I never did get a dump to work on import. Instead of casting about I
just went ahead and modified Eve-DB to work with Tyr1.04.

Essentially all you need to do is add an EVEIcon model and modify the
existing models and imports as per the schema change notes in the
Tyr1.04 datadump.

I was going to submit a patch for this but could never find way to
attach it or who to send it to. It's likely this change would
significantly impact any website currently in production though since
it changes the entire way graphics are displayed.

[Gregory Taylor]

Patches can be attached to an issue on the django-eve-db issue tracker:

http://code.google.com/p/django-eve-db/issues/list

If you get it posted there, I'll look it over and get it committed.

Greg

[Matt]

The Tyranis v1.0.1 import worked fine without modifications.

[Greg Taylor]

As a follow-up on this thread, we've moved to github now, so please
feel free to fork the django-eve-db project and send pull requests my
way with fixes.

django-eve-db on github: http://code.google.com/p/django-eve-db/

[Greg Taylor]

And as a further follow-up, I've done a whole lot of changes to all of
the django-eve projects. None of these are earth-shattering, just lots
of cleanup. One of the things I did was fix the importers to work with
the latest tyr1.0.4 DB. Ironically, just in time for them to release
Incursion :)

So get the new github versions of the django-eve projects, try those,
and let me know if you still have trouble.