looking for a ZODB browser

[Soppy bear]

Hello

is there any ZODB 5.x browser or web interface we can use to manage a zodb database with python 2.7 and gtk2 preferably ? i have found zodbbrowser but i think its based on zodb 3 and the last update is from 2017....

thank u in advance

tk

--
Soppy bear <tka...@yandex.com>

[Marius Gedminas]

On Sat, Mar 30, 2019 at 07:53:36PM -0400, Soppy bear wrote:
> is there any ZODB 5.x browser or web interface we can use to manage a
> zodb database with python 2.7 and gtk2 preferably ?

What do you mean by "manage"? E.g. zodbbrowser is a read-only browser
(with the exception of one dangerous experimental rollback feature that
is disabled by default).

> i have found
> zodbbrowser but i think its based on zodb 3 and the last update is
> from 2017....

zodbbrowser works with the latest ZODB (I've daily builds on Travis CI
to make sure I catch any regressions). You're right that a release's
overdue -- it's so easy to lose track of time!

It's mostly in maintenance mode -- I'll respond to people's issues or
PRs on GitHub, but I don't plan to do any active feature development.

HTH,
Marius Gedminas
--
Star Trek promised me computers I could talk to but what I got was a telephone
I type on.
-- "Sarcastro"

[tfl...@gmail.com]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I can confirm that I currently use zodbbrowser under Pyramid with Python 3
and ZODB 5.3 and it also works perfectly!

Regards,
Thierry
-----BEGIN PGP SIGNATURE-----
Version: FlowCrypt 6.7.1 Gmail Encryption
Comment: Seamlessly send and receive encrypted email

wsBcBAEBCAAGBQJco3hLAAoJEB4Qyhnx2DUmRKYH/3pYXN1e65UvFffY4GP/
nQGMeMURaEfscRdgiJUDAX0c4rjH7EVAGU1x8/fcEJcCFa7KPjBOVaPjT9UU
t330yTvfV3P7L++lws/+OEhI37lSVrP3n44H3quHlMGLkvyYt9PmOFR2Uqbv
t4PKsu62zxjcOHEjgbrZdXqCH1JnvwSYB7XaFgOPkOYq5Hqe065a1WR+Jth3
u6jiSuc8i4lbwYdCZa+G8O/FWWMHfitCOqdvZHWjYHQM4lzDFK+awM4IuBtV
PvYiMbPUNciASVhdJ+/Nm5IEsa2IWbJ087l5Og5FYdcMv6ljD4B1zEeXAluZ
LZM/DpruAzTrcrGaMgb8g64=
=qMjZ
-----END PGP SIGNATURE-----

[Christopher Lozinski]

> On Apr 2, 2019, at 4:04 PM, Marius Gedminas <mar...@gedmin.as> wrote:
>
> What do you mean by "manage”?

There is a difference between a browser, and a manager.

A browser just lets you browse the objects. A manager AKA ZMI the Zope Management Interface
lets you browse, but also rename, retitle, cut, copy, paste and delete objects. I think that a ZMI is so important because it lets you manipulate your graph of objects.

There is a very modern ZMI. Fully Javascript enabled. Kind of the 4th generation of what I have been working on.

You can see a video at
https://hub.docker.com/r/pythonlinks/distributedwiki

Or you can login and play with it at
https://demo.pythonlinks.info//
user: admin
password: admin

Click on Node->Manage

Feedback is hugely appreciated. Most people I talk to have no idea what a ZMI is.

Warm Regards
Chris

[Soppy bear]

On Tue, 2 Apr 2019 17:04:36 +0300
Marius Gedminas <mar...@gedmin.as> wrote:

> Star Trek promised me computers I could talk to but what I got was a telephone
> I type on.
> -- "Sarcastro"

Right... :-)

Thanks but i need a way to delete/create and update database objects (aka basic db management) in python 3 and zodb 5.5.1 . Ideally i was hoping to use gtk2 but since it never going to be ported to python 3.7 i will look for something else..


tk

--
tka...@yandex.com | Twitter: @wise_project
https://www.isotoperesearch.ca/
Not everyone who wander are lost.

[Christopher Lozinski]

One of the limitations of the ZODB is how big the data can be. 

PostrgeSQL helps, but you are still limited to the size of a single table. 

Usually PostgreSQL table size is limited to 16 to 64 Terabytes

https://link.springer.com/content/pdf/bbm%3A978-1-4302-0018-5%2F1.pdf

Well now there is PostgreSQL sharding from Citus Data, now owned by Microsoft.   You get a PostgreSQL api, 

but they split the data across multiple computers.  One hashes on a single column.  In this case it would be the object id, and they do the two phase commit. 

Do you think this will work with the ZODB?  Anyone tried it?

I introduced them to the Plone foundation people.  Maybe they will be sponsoring that conference. 

Comments?

Chris

[Joni Orponen]

On Fri, May 10, 2019 at 4:28 PM Christopher Lozinski <lozi...@specialtyjobmarkets.com> wrote:

One of the limitations of the ZODB is how big the data can be. 

PostrgeSQL helps, but you are still limited to the size of a single table. 

Usually PostgreSQL table size is limited to 16 to 64 Terabytes

Are there even any known production ZODB installations exceeding one terabyte in size?

--

Joni Orponen

[Christopher Lozinski]

On May 10, 2019, at 4:34 PM, Joni Orponen <j.or...@4teamwork.ch> wrote:

Are there even any known production ZODB installations exceeding one terabyte in size?

As of about a year ago Neo, a close cousin of the ZODB, had 80 TB databases in production and 160TB in test.  By now that number must be larger.  But it is GPL. 

It is also a chicken and egg problem.   One reason why people decide against the ZODB, is because it does not scale.  And so there are no huge databases.  

The more interesting question is whether this setup would actually work or not. 

Chris

[Julien Muchembled]

Le 05/10/19 à 16:34, Joni Orponen a écrit :

> On Fri, May 10, 2019 at 4:28 PM Christopher Lozinski <lozi...@specialtyjobmarkets.com <mailto:lozi...@specialtyjobmarkets.com>> wrote:
>
> One of the limitations of the ZODB is how big the data can be. 
> PostrgeSQL helps, but you are still limited to the size of a single table. 
>
> Usually PostgreSQL table size is limited to 16 to 64 Terabytes
>
>
> Are there even any known production ZODB installations exceeding one terabyte in size?

Yes, we have 2: currently 1.7TB & 2TB, growing quite fast.
And this is after compression.

A 3rd production instance was migrated to NEO recently. It powers a website for ~1 million customers. The FileStorage would exceed 1TB this month; we packed it quite often. The NEO DB currently takes 600GB on-disk and it is foreseen to exceed 10TB in a few years.

> As of about a year ago Neo, a close cousin of the ZODB, had 80 TB databases in production and 160TB in test. By now that number must be larger. But it is GPL.

Hmm, we don't have so big production DB yet, but that may be the case in near future.

We made 2 tests:
- 100TB: https://www.nexedi.com/blog/NXD-Document.Blog.NEO.100T
- 1PB, with 10% compressible data (200TB on-disk with 1 replica, i.e. something like RAID1 but at NEO level)

Julien