how to import from wordpress

[elliot]

hi!

thanks for byteflow, i have been looking through the code and running
on my ubuntu server and it is really nice!

i am confused about how to import from wordpress though...
i have set ENABLE_IMPORT to True, and run syncdb, but I don't see how
to get the data out of my wordpress tables into the byteflow tables.

if you can give me some hints, i'd be happy to write this up on the
wiki or on my blog, to help out the next person who comes along.

cheers,
-elliot

[Alexander Solovyov]

On Fri, May 16, 2008 at 8:29 AM, elliot <elliot...@gmail.com> wrote:
> if you can give me some hints, i'd be happy to write this up on the
> wiki or on my blog, to help out the next person who comes along.

Saying honestly I'd like to rewrite importer to use Wordpress'
exported XML, but now it just uses database, and you need to point
Django to database with Wordpress tables.

Though there was some reports that Wordpress has changed database
schema since wpimport was written and it may not work. :-( I haven't
got time to look at that, but we have plans to write something like
generic importing framework for simplification of new import
facilities creation.

P.S. If you have skills and passion to fix WP importer, I'll be really
glad to include your fixes. :D

--
Alexander

[Salubrium]

> i am confused about how to import fromwordpressthough...
> i have set ENABLE_IMPORT to True, and run syncdb, but I don't see how
> to get the data out of mywordpresstables into the byteflow tables.

As Alex said, you need to point the setup of Byteflow to your existing
Wordpress database. Theres a couple of downsides to this method:

1. Byteflow is no longer up to date with the current Wordpress DB
schema (2.5.1 in my case - and I don't know what version it was
current with)
2. You are tied to using Mysql unless you convert your Wordpress DB
from mysql into whatever else.. I wanted to use sqlite, personally and
it's not trivial to move from mysql to sqlite and then I found out it
was expecting an old schema.. so I gave up :).

I think importing via XML would be a much better approach, though I
don't have the time to do this at the moment.

[Alexander Solovyov]

On Tue, Jun 3, 2008 at 8:10 AM, Salubrium
<bil...@serverspacesolutions.com> wrote:
> 1. Byteflow is no longer up to date with the current Wordpress DB
> schema (2.5.1 in my case - and I don't know what version it was
> current with)

2.2.something :-)

> 2. You are tied to using Mysql unless you convert your Wordpress DB
> from mysql into whatever else.. I wanted to use sqlite, personally and
> it's not trivial to move from mysql to sqlite and then I found out it
> was expecting an old schema.. so I gave up :).

You're done this with dumpdata/loaddata, right? Hmmm... maybe I'll
write some tutorial for those who want to change their DBMS.

> I think importing via XML would be a much better approach, though I
> don't have the time to do this at the moment.

Hehe. There is already working XML-importer from Greg Hearsfield,
which I probably will merge tomorrow into main repository. ;-)

--
Alexander

[Greg Heartsfield]

For the impatient, see:
http://gregheartsfield.com/~scsibug/WXRImporter.py

Place in byteflow root directory, and run to see usage information. I
haven't yet cleaned it up to run in a directory like apps/import, but
that should happen by this weekend. I've only tried it on a couple
WXR exports so far, so please let me know your findings if you use it.

-Greg

> --~--~---------~--~----~------------~-------~--~----~
> You received this message because you are subscribed to the Google Groups "byteflow-users" group.
> To post to this group, send email to byteflo...@googlegroups.com
> To unsubscribe from this group, send email to byteflow-user...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/byteflow-users?hl=en
> -~----------~----~----~----~------~----~------~--~---
>

[Corbin]

Greg,

I just used your WXRImporter to migrate my data from WordPress into
byteflow.
I only ran into one issue (see full trace below). I didn't realize
that I had some comments
in WordPress from people using the same name with different email
addresses.
This caused the import to fail with "username is not unique" errors.
I wasn't quite sure how to best handle this... but I adjusted the
importer code to fallback to
looking up by "email address only" prior to creating a new user.
This allowed me to get a clean
import. Looking at my imported data via the admin interface, it
looks like everything worked out
OK.

Great code (WXRImporter and byteflow) -- I plan to switch my blog from
WordPress to byteflow in the next few days.

Thanks,

Brian Corbin
http://rockthefatknot.com

The mods I made to WXRImporter.py to get a clean import of my data:

-----------------------------------
BEGIN-----------------------------------
--- OriginalWXRImporter.py 2008-06-19 00:47:52.000000000 -0400
+++ WXRImporter.py 2008-06-19 00:25:42.000000000 -0400
@@ -165,20 +165,33 @@
# unapproved, and "-a" option was used.
if ((not approved_comments_only) or commentNode.approved):
try:
+ logging.debug("Looking up user: %s [email: %s]..." %
(author, author_email))
commentuser = User.objects.get(username=author,
email=author_email)
logging.debug("Found user %s, email: %s" %
(commentuser.username,

commentuser.email))
except ObjectDoesNotExist:
- commentuser = User()
- commentuser.username = author
- commentuser.site = author_url
- commentuser.name = author
- commentuser.email = author_email
- commentuser.set_password(generate_password(10))
- commentuser.save();
- globals()["loaded_users"]+=1
- logging.info("Created user: %s, email: %s" % (author,
-
author_email))
+
+ matchingusers = User.objects.filter(email=author_email)
+ if len(matchingusers) == 1:
+ #use existing user based on email match
+ commentuser = matchingusers[0]
+ else:
+ #Try to avoid integrity errors (username is not
unique)
+ matchingusers = User.objects.filter(username=author)
+ if len(matchingusers) > 0:
+ commentuser = matchingusers[0]
+ else:
+ logging.debug("User [%s] not found. Creating..."
% author)
+ commentuser = User()
+ commentuser.username = author
+ commentuser.site = author_url
+ commentuser.name = author
+ commentuser.email = author_email
+ commentuser.set_password(generate_password(10))
+ commentuser.save()
+ globals()["loaded_users"]+=1
+ logging.info("Created user: %s, email: %s" %
(author,
+
author_email))
commentNode.user = commentuser
commentNode.save()
globals()["loaded_comments"]+=1

-----------------------------------
END--------------------------------------

----------------------------------Original Error
Information------------

Traceback (most recent call last):
File "WXRImporter.py", line 241, in <module>
main()
File "WXRImporter.py", line 65, in main
process_item(i)
File "WXRImporter.py", line 135, in process_item
process_comment(post,c)
File "WXRImporter.py", line 186, in process_comment
commentuser.save();
File "/Library/Python/2.5/site-packages/django/db/models/base.py",
line 272, in save
self.save_base()
File "/Library/Python/2.5/site-packages/django/db/models/base.py",
line 334, in save_base
result = manager._insert(values, return_id=update_pk)
File "/Library/Python/2.5/site-packages/django/db/models/
manager.py", line 127, in _insert
return insert_query(self.model, values, **kwargs)
File "/Library/Python/2.5/site-packages/django/db/models/query.py",
line 735, in insert_query
return query.execute_sql(return_id)
File "/Library/Python/2.5/site-packages/django/db/models/sql/
subqueries.py", line 300, in execute_sql
cursor = super(InsertQuery, self).execute_sql(None)
File "/Library/Python/2.5/site-packages/django/db/models/sql/
query.py", line 1474, in execute_sql
cursor.execute(sql, params)
File "/Library/Python/2.5/site-packages/django/db/backends/util.py",
line 18, in execute
return self.cursor.execute(sql, params)
File "/Library/Python/2.5/site-packages/django/db/backends/sqlite3/
base.py", line 136, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.IntegrityError: column username is not unique

----------------------------------------------------------------------------------

>  application_pgp-signature_part
> 1KDownload

[Corbin]

>  application_pgp-signature_part
> 1KDownload

[Alexander Solovyov]

On Thu, Jun 19, 2008 at 8:11 AM, Corbin <corb...@gmail.com> wrote:
> import. Looking at my imported data via the admin interface, it
> looks like everything worked out
> OK.

Nice to hear that. :-) If I'll get time in next few days, I'll
probably try to integrate it...

By the way, maybe it'll be better to integrate it as manage.py command
and not web interface? Thought we'll lost ability to choose what we
want to import, but do we really need it?

--
Alexander

[Greg Heartsfield]

I tried to start on making it into a proper django app (instead of a
script) last week, but really didn't get anywhere. Integrating it
into manage.py sounds like a good idea to me.

Corbin, thanks for testing, and thanks for the patches!

-Greg Heartsfield