importing 1.0.90 SQLite database in 3.1.7 Postgres

[Corrado Topi]

Dear all,

I am trying to move from 1.0.90, wich is not working anymore on my ubuntu 20.04.1, to 3.1.7. I used SQLite on 1.0.90, and I am using PostgreSQL 13 on 3.1.7

Ii have some questions:

1) How do I import the SQLite database fro 1.0.90 into the PostgreSQL for 3.1.7 database? I keep getting errors regarding the database structure being very different between version 1.0.90 and 3.1.7.

2) Which version should I install, 1.0.90 or 3.1.7, which one does have the most up to date features, and which is the future of the app?

Best,

Corrado

[ma...@anche.no]

Answering from a phone, without checking references...
To migrate from a version to the next one, we produce a CSV export, crunch it through a script, import it in a new empty database.
I think I have described the process somewhere...
The script should be named like 1.0-to-3.1 in its name.

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[Mario Frasca]

Hola Corrado,

On 07/01/2021 13:56, Corrado Topi wrote:
> 2) Which version should I install, 1.0.90 or 3.1.7, which one does
> have the most up to date features, and which is the future of the app?

sorry I had not answered this part.

the 1.0 line is not abandoned, but the only reason why it's still there
is that I am not sure whether the 3.1 line is stable enough.  also: I
have not set up the automatic compiler for the 3.1 package on Windows,
so if you are on Microsoft, you are still stuck with the 1.0.90 all-in
installer.  if you're not tied to Microsoft Windows, and you are on a
recent installation where Python2 is not easily available, then you
should use the 3.1 line.

if I understood you correctly, you have 3.1 running and you're happy
with it?

it would be nice to receive more feedback about it.

I was having timing problems with it, speed issues, the software
freezing between any two queries. it was so weird that I did not know
where to start looking into it.  if you can confirm you do not see
anything like that, I will check outside ghini, maybe it's something in
my system.

the "future of the app", I think it's the ghini-reloaded, the django app
which is now in prototype form and for which I intend to rewrite the
javascript front end.  it's a lot of work, and I can't do that alone.  I
need users who can provide feedback, and it would help if I could get
some financial resources to support development.

ciao,

Mario

[Corrado Topi]

Ciao Mario,

Long time no see! How are things?

3.1.7 works well on ubuntu 20.04.1 and I am trying to pull together a short guide on github, because you need to create virtual environmental and install through pip3. You also need to install quite a few additional packages through pip3 to make it work.

If you try to use 1.0.xx together wit 3.1.7 on the same database it does not work.

Can you tell me more about the django based solution? Is it the same as ghini server? I would like to test it, and it maybe the way to go, at the moment there are too many apps to maintain .... 

Talk to you soon

[Mario Frasca]

hola Corrado!

I invited you to our Ghini group on Telegram.

nel gruppo ci sono un paio di nuovi utenti, tutti GNU/Linux, che fatto un po' di salti mortali per riuscire ad installare la 1.0.  magari ci convinci che la 3.1 è una buona opzione?

ovviamente non si può usare lo stesso database con le due versioni.  e non c'è modo di convertire 3.1 in 1.0.

ciao, see you there, or chat on IRC,

Mario

--
You received this message because you are subscribed to the Google Groups "ghini/bauble" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bauble+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bauble/e25cec7a-2188-4ddf-a8e0-f034dfd90294n%40googlegroups.com.

[Corrado Topi]

Ciao Mario,

Non ho ancora ricevuto il tuo invito Telegram, ma mi unisco volentieri .... io sto abbandonando 1.0.xx che non mi funziona su ubuntu 20.04.1. Secondo me una strategia sarebbe portarlo a python3, ma ho profavo durante le vacanze con un sistema automatico e dopo averci speso qualche ora mi sono fermato.

3.1.7 sembra stabile su python3 e non sembra radicalmente diverso dal precedente. Non so bene che cosa cambi a livello di database ....

Fammi poi sapere per la versione django!

A presto

[Mario Frasca]

On 09/01/2021 09:45, Corrado Topi wrote:
> Secondo me una strategia sarebbe portarlo a python3

but that's precisely what I did with ghini.desktop-3.1

it was not an easy task, not one that I would expect you to be able to
solve in a week.  moving Python2→3 had impact on several other parts of
the software.

I had been collecting issues for which the solution would impact the
database and I addressed them in the same porting process.

the other main issue that I addressed when porting to Python3 was moving
from GTK2 to GTK3.

also the Glade files were impacted.

it was a big task, and you can find it openly documented in the git
repository.

Mario

[Brett]

FWIW, I've also been working on a spiritual successor to Bauble that I call Sepal.  It's web app with a Python based API and a Typescript/React front end.  Its in the pretty early stages but you can view the code in the Github organization (https://github.com/sepal-app).  I have an initial version where you can signup at https://development.sepal.app/ but it's not really worth using at the moment.  I just work on this in my spare time so I don't know if anything will ever come of it but I thought it might be worth mentioning here.

Brett (original creator of Bauble)

[Mario Frasca]

On 09/01/2021 09:45, Corrado Topi wrote:

> 3.1.7 sembra stabile su python3

good to hear.  my doubt was about speed.

> e non sembra radicalmente diverso dal precedente.

it's a port, and a "next generation".  it's still a Glade/Gtk desktop
program.

> Non so bene che cosa cambi a livello di database ....

check the conversion script, if you can read bash, you will understand
the differences.

now that I check it myself, I remember something which I had repressed,
apparently:

before the announcement that Python2 was reaching "end-of-life" I had a
new 1.1 line, where I had addressed more significant database changes.

I have not taken them into 3.1, apparently because I thought I would
first do the 2-3 port, then address the other database issues.

Mario

[Mario Frasca]

Hi Brett, nice to see you still on botany.

what's the license for your software?

care to join forces on the AGPL ghini-reloaded project?

documented at: taxasoft-ghini.readthedocs.io/

I need help for rewriting the front end, and I would like to use vue.js, or any other framework you might like.  I have it in pure javascript and I know it's not a sustainable practice on the long term.

best regards,

Mario Frasca

--
You received this message because you are subscribed to the Google Groups "ghini/bauble" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bauble+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bauble/e1f701c3-723c-4c2f-98a4-008da5381e54n%40googlegroups.com.

[Brett]

Hi Brett, nice to see you still on botany.

I've mostly moved on to full time software development but I'm half hoping Sepal will bring me back into the botanic garden world. :)

what's the license for your software?

I was planning on releasing it as AGPL. 

care to join forces on the AGPL ghini-reloaded project?

I'm using this project to experiment with a lot of new ideas and I'm not sure if our goals would align. Some of the things I've been thinking about have been:

- Simplify taxa naming, e.g. make names more free form instead of separate fields for each part of the name

- Create stronger parent -> child relationships between taxa where any taxa can have children, .e.g a genus can be made the child if a family but both are still just taxa

- Allow accession to be associated with any taxa not just species

- Using accessions + accession items instead of accessions + plants in Bauble

- Create a multitenant app based on organizations and role base access so that the app can host multiple organizations and members more like a SaaS platform.

- Keeping a better audit log of all changes for an organizations records that can be viewed in a timeline.

- Embracing Postgresql and all of its features instead of trying to be cross-database compatible

...and more.

documented at: taxasoft-ghini.readthedocs.io/

I need help for rewriting the front end, and I would like to use vue.js, or any other framework you might like.  I have it in pure javascript and I know it's not a sustainable practice on the long term.

At this point I think I'm too far along with Sepal to switch gears and too committed to these news ideas.  I'm certainly willing to see if there could be any cross collaboration.

Take care,

Brett

[Mario Frasca]

Hi again, I'll comment on a couple of your items, for which I have an
opinion:

On 09/01/2021 10:22, Brett wrote:
> - Simplify taxa naming, e.g. make names more free form instead of
> separate fields for each part of the name

I'm not sure I understand you here, but I think I do not agree.

you may try to explain better, and convince me.

> - Create stronger parent -> child relationships between taxa where any
> taxa can have children, .e.g a genus can be made the child if a family
> but both are still just taxa

indeed, this is how I made it in the django app.

it's all just taxa, and they have a rank.

https://cuaderno.ghini.me/browse/#page=results;q=taxon%20where%20epithet=Otoba%20and%20rank.name=genus

synonyms also solved this way.  it becomes very flexible and much closer
to reality.

Sterculiaceae → Sterculoideae → Malvaceae (s.l.);

Malvaceae (s.s.) → Malvoideae → Malvaceae (s.l.);

> - Allow accession to be associated with any taxa not just species

absolutely right.  this is how it is now in my django app.

verifications take a fundamental role in associating an accession to a
taxon.

a taxon might be called Orchidaceae (family) and an accession can be
associated straight to it, and will be mentioned as Orchidaceae sp.

> - Using accessions + accession items instead of accessions + plants in
> Bauble

this is just a naming issue in my opinion.  call it plant-group, plants,
or accession item, I don't mind.  your accession_item is possibly easier
to understand for seed banks or herbaria, once the reader grasps the
term "accession".  ;-)  "plant" is shorter, this is why I never
considered changing the term.

I moved to django in order to stay database-agnostic, but I think that
some of the geographic features I need are only available on PostgreSQL.

ciao,

Mario

[Corrado Topi]

Mario, from what I understand taxasoft the new django based ghini server. It looks interesting but it does not seem yet to be at a stage where it is ready to use to cover the same functionalities as ghini.desktop .... am I correct?

What is the relationship between the "old" ghini.server in the "old" ghini map and taxasoft? (Sorry if it is already clear and I am just not understanding). It seems to me that this is similar ....

OK for everything else, and thanks you so much for explaining.

PS: I am not sure I see radical differences at this stage? I may be completely wrong of course!

PS: If we could have the choice to install postgresSQL at installation that would be great. SQLITE is a great database, if everything works, but if you have to go in the database to adjust issues, it is a nightmare, at least for me.

Best,

[Mario Frasca]

hi again!

On 09/01/2021 14:06, Corrado Topi wrote:
> Mario, from what I understand taxasoft the new django based ghini
> server. It looks interesting but it does not seem yet to be at a stage
> where it is ready to use to cover the same functionalities as
> ghini.desktop .... am I correct?

ghini-reloaded, the "taxasoft" ghini, (I have to get rid of that
'taxasoft' part … :-D) is absolutely not ready for prime use.  I have
been looking for users who want to help me develop it, and I don't seem
to manage.  without users willing to experiment, dare use it for
managing their collection online, and able to provide some feedback,
there is no development there.

the documentation tells you how to participate, isn't it?

there's no reporting tool, no propagations management, to mention whole
missing sections.  but what is there is not guaranteed to work.  and I
need to rewrite the front end.

so:

if you just want to use a program, you have ghini.desktop.

> What is the relationship between the "old" ghini.server in the "old"
> ghini map and taxasoft? (Sorry if it is already clear and I am just
> not understanding). It seems to me that this is similar ....

the old ghini server or ghini web was showing stuff on a map, and the
map was the same as the new reloaded ghini, being leaflet showing the
standard OSM tiles.

the software around the map is completely different.  my first attempts
were using a node.js server, now it's django.  the first attempts were
exploring several options, each quite different than the other.  this
one is "simply" aiming at redoing the same ghini.desktop with a web
interface.

the map looking the same does not mean that the map offers the same
functionality, and it does not.

> PS: If we could have the choice to install postgresSQL at installation
> that would be great.

no way :-D

you want PostgreSQL, you need to be able to install, configure, manage
it.  there's so many variables you have to take into account, please
refer to the PostgreSQL documentation, there's no shortcuts I can offer you.

[Corrado Topi]

Ciao Mario,

I am using PostgresSQL 13 now as default, it is not a problem for me as you can imagine. I had started with SQLite because it was the default.

Migration from ghini version 1 to ghini version 3 is feasible if you remain in SQLite.  If you migrate from SQLite to PostgreSQL it is not feasible, apparently there are some keys missing.

Version 3.1.7 seems to work well on ubuntu 20.04.1. Unfortunately, the taxonomic check does not work on ghini 3.1.7.  I am happy to make modifications to use a different taxonomic database, but I do not want to end up with a modified version of ghini that cannot be upgraded.

The new server: for me I could test it when I have time, unfortunately as you know managing the database for the garden is a side completely unpaid endeavor, and can only happen when I have time. I am sorry of not being of more help. I think the idea of going only server with django + a javascript framework is great.

Best,

[Corrado Topi]

Dear Mario, dear Brett,

I do not see radical differences between the two approaches .... Am I completely wrong?

Best,

[Mario Frasca]

Dear Corrado,

you are mentioning two problems that I would like to examine.

can you open two issues on the github site?

- migration 1-to-3 works in general as far as I am concerned

- taxonomic check troubles?  I haven't tried.  what doesn't work?  which way doesn't it work?  please don't followup here, but in the issue you're going to open.

thank you!

Mario

--
You received this message because you are subscribed to the Google Groups "ghini/bauble" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bauble+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bauble/6d2ba10e-9a5d-49d7-8438-f2e46d93b55dn%40googlegroups.com.