CodernityDB ?

[Bino Oetomo]

Dear All.

For the sake of speed on persistence storage, I'm not satisfied with sqlite3.
I know that snakeMQ also come with mongoDB storage driver, but since I need to port anything to openwrt .... porting MongoDB server is not easy enough for my lame brain.

I found CodernityDB
I did a simple test (1091 insert) ... and got below 9 secs.
So I think I will try to make yet another storage 'driver' for snakeMQ, based on CodernityDB.

But before I start pulling my hair, kindly please give me enlightenment :
Is a 'transactional' feature a must for snakeMQ persistence storage ?

Sincerely
-bino-

[David Siroky]

> I found CodernityDB <http://labs.codernity.com/codernitydb/>

Most NoSQL databases are primarily designed as a key:value storage. Its main
goal is not a consistency. You might have problems with ordering. I'd suggest
some relational database.

I'll look into Gadfly. It might be a good candidate. Stay tuned.

> But before I start pulling my hair, kindly please give me enlightenment :
> Is a 'transactional' feature a must for snakeMQ persistence storage ?

Yes and no. If a message is about to be stored then the Messaging calls
QueuesStorage.push(). What is exactly going to happen in the push() is
absolutely up to the storage driver. So if you can store the message with a
single e.g. SQL command then you don't need a transaction. If you have to call
more commands to read or modify the database then you need atomicity and even
some database locking. But again it depends on the database mechanism design.

Maybe a little bit uselessly complicated answer :-) Current snakemq storage
handlers does every operation with a single database command. If you follow the
same approach then you don't have to care about transactions.

David

[David Široký]

I'll look into Gadfly. It might be a good candidate. Stay tuned.

Code on github. Test it please and let me know if it works.
David

[Bino Oetomo]

Ah .. you are to fast for my brain.

I'll test it today

Sincerely
-bino-

[Bino Oetomo]

Dear David

On Sunday, May 25, 2014 6:11:16 PM UTC+7, David Široký wrote:

I think Gadfly still got problem

2014-05-26 09:11:29,604 snakemq.link DEBUG add_listener fd=6 ('127.0.0.1', 4000)
Traceback (most recent call last):
  File "./xcache", line 39, in <module>
    storage = snakemq.storage.gadfly.GadflyQueuesStorage('./','storage')
  File "/usr/local/lib/python2.7/dist-packages/snakeMQ-1.2-py2.7.egg/snakemq/storage/gadfly.py", line 29, in __init__
    self.conn = gadfly.gadfly()
  File "/usr/local/lib/python2.7/dist-packages/gadfly/database.py", line 26, in __init__
    from gadfly import sql, bindings
  File "/usr/local/lib/python2.7/dist-packages/gadfly/bindings.py", line 64
    [create, view, name, namelist, as, selection] = l
                                    ^
SyntaxError: invalid syntax

Sincerely
-bino-

[David Siroky]

> |
> 2014-05-2609:11:29,604snakemq.link DEBUG add_listener fd=6('127.0.0.1',4000)
> Traceback(most recent call last):

> File"./xcache",line 39,in<module>

> storage =snakemq.storage.gadfly.GadflyQueuesStorage('./','storage')

>
> File"/usr/local/lib/python2.7/dist-packages/snakeMQ-1.2-py2.7.egg/snakemq/storage/gadfly.py",line
> 29,in__init__
> self.conn =gadfly.gadfly()
> File"/usr/local/lib/python2.7/dist-packages/gadfly/database.py",line 26,in__init__
> fromgadfly importsql,bindings
> File"/usr/local/lib/python2.7/dist-packages/gadfly/bindings.py",line 64
> [create,view,name,namelist,as,selection]=l
> ^
> SyntaxError:invalid syntax

You must have some old gadfly version. I have 1.0.0 and it does not contain this
bug.

David

[Bino Oetomo]

Dear David

On Monday, May 26, 2014 1:06:00 PM UTC+7, David Široký wrote:

You must have some old gadfly version. I have 1.0.0 and it does not contain this
bug.

David

No. mine is also 1.0.0

Sincerely
-bino-

[David Siroky]

> You must have some old gadfly version. I have 1.0.0 and it does not contain
> this
> bug.
>
> David
>

> <https://lh4.googleusercontent.com/-63-bDf6nOpw/U4L-mtC3e1I/AAAAAAAAADQ/N5_DJ0uGLOg/s1600/Screenshot+-+05262014+-+03%3A41%3A57+PM-my-gadfly-version.png>

>
>
> No. mine is also 1.0.0

It seems that the official package is broken. I have a Ubuntu package and it is
fine.
http://packages.ubuntu.com/raring/python-gadfly

D

[Bino Oetomo]

On Monday, May 26, 2014 5:11:11 PM UTC+7, David Široký wrote:

It seems that the official package is broken. I have a Ubuntu package and it is
fine.
http://packages.ubuntu.com/raring/python-gadfly

D

Hmm
Ok .. meanwhile I'll look for another storage route.
I'm learning codernity and redis.

Sincerely
-bino-