MySQLDataSource stream and MongoDBDataTarget example

[Kenny Meyer]

Can you give me a simple example of getting MySQL records from a table, and storing them into MongoDB collection?

What I did so far is this:

import brewery

import brewery.ds as ds

import brewery.metadata as metadata

from sqlalchemy import Table, Column, Integer, String, Text, DateTime, ForeignKey

from sqlalchemy import create_engine, MetaData

engine = create_engine(

  "postgresql://postgres@localhost/abc"

)

metadata = MetaData(engine)

# fields = metadata.FieldList(["articulo", "titulo"])

stream = ds.SQLDataSource(connection=engine, table="articulos", schema="abc")

print stream.fields

target = ds.MongoDBDataTarget(collection="articulos",

                              database="test_database")

for row in stream.records():

  print "Appending row " + str(row)

  target.append(row)

This raises an exception: AttributeError: 'SQLDataSource' object has no attribute 'dataset' which is a bug, because that instance variable is not initialized.

I have noticed that examples are very sparse and that documentation is grammatically wrong and missing. Good opportunity for patches, I guess. :)

[Stefan Urbanek]

Hi Kenny,

On 30.4.2012, at 13:36, Kenny Meyer wrote:

Can you give me a simple example of getting MySQL records from a table, and storing them into MongoDB collection?

What I did so far is this:

<code snipped>

This raises an exception: AttributeError: 'SQLDataSource' object has no attribute 'dataset' which is a bug, because that instance variable is not initialized.

Thanks for reporting, fixed (and pushed on github).

I have noticed that examples are very sparse and that documentation is grammatically wrong and missing. Good opportunity for patches, I guess. :)

You are more than welcome :-)

Regards,

Stefan Urbanek

Twitter: @Stiivi

Home: http://stiivi.com

Brewery: http://databrewery.org

Github: https://github.com/Stiivi