migrating substanced database to postgres

[Jo G]

Hi, so I have been tasked with migrating our data.fs to a postgres database. If I open it from within my substanceD application then I can see all the objects and I can see all the references using objectmap.

I would however like to have a separate python application to open the data.fs, read out the properties of the objects and save them to postgres. I have my django app (department standard now:)) and I have zodb and substanced installed in the virtual environment.

 I get objects with __Broken_state and that's all fine as I know the properties of the objects and I can just iterate through the folders and read them off. However - I am not easily able to sort out the references. I can get an instance of objectmap but none of the methods work as they are all expecting full information about the objects rather than broken ones.

So, I probably need to install more of my original substanceD project - exactly how much do I need so the objects aren't actually broken?  

Yours hopefully 

[Tres Seaver]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

If this was my task: rather than trying to open the ZODB from your
Django virtualenv, I would add views to the SubstanceD app to spit out
data which could easily be consumed by whatever RDBMS tools you use (CSV,
JSON, whatever). Assuming the Django app is sitting in front of a
database whose schema you know well, I think the "dump to a file and then
load" strategy would be a lot faster to implement.


Tres.
- --
===================================================================
Tres Seaver +1 540-429-0999 tse...@palladion.com
Palladion Software "Excellence by Design" http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlp00OEACgkQFXKVXuSL+CPjtgCdEXuvzcTGR1yAvFnG5Ij7wlvR
ZssAnivDh5yaW70OjFi4P55GbGqeDeWG
=cvL8
-----END PGP SIGNATURE-----

[Leonardo Rochael Almeida]

As with any ZODB app, your Python process that is accessing ZODB needs to have all the Python modules that define the models that were saved in the ZODB.

That is: all modules that contain models (classes) which instances are present in the ZODB need to be **importable** (that's the key word) by the Python process that is doing the migration.

And they need to be importable in the exact same "package path" as when their instances were added to ZODB.

For example. If you had an instance in ZODB of a model whose "path" was "mypackage.mymodule.MyModel", then from the Python process that is doing the export, you need to be able to do without error:

    from mypackage.mymodule import MyModel

Then your objects will no longer be broken.

--
You received this message because you are subscribed to the Google Groups "substanced-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to substanced-users+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Carlos de la Guardia]

Hi,

like Tres said, it's probably better to do it from the substanced side. You can create views like he suggested, or you could write an export script. Here is one that I used before, which could serve as an example:

https://gist.github.com/cguardia/9b36e90f5d03d91220ad266f8ddc092a

Carlos de la Guardia

---

From: Jo G <joannegil...@gmail.com>
To: substanced-users <substanc...@googlegroups.com>
Sent: Friday, February 2, 2018 2:51 PM
Subject: migrating substanced database to postgres

--
You received this message because you are subscribed to the Google Groups "substanced-users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to substanced-use...@googlegroups.com.

[Jo G]

Leonardo, so if I have a 'book' object then I may just need to have the class describing the book and the associated schema for that book available for import? 

To unsubscribe from this group and stop receiving emails from it, send an email to substanced-use...@googlegroups.com.

[Jo G]

hanks. you may be correct. i thought sdump might do it but my file is too big.

[Jo G]

that is very useful.

[Leonardo Rochael Almeida]

Hi Jo,

On 2 February 2018 at 19:27, Jo G <joannegil...@gmail.com> wrote:

Leonardo, so if I have a 'book' object then I may just need to have the class describing the book and the associated schema for that book available for import? 

Exactly.

Cheers,

Leo

To unsubscribe from this group and stop receiving emails from it, send an email to substanced-users+unsubscribe@googlegroups.com.