Motor: what happened when it finds mongodb locked?

[aliane abdelouahab]

hi
what happened when motor tries to get results from mongodb and mongodb
is in lock mode? did it back and forget or return, so must i always
make is_locked() as a condition to predict if mongodb locks, or motor
handles this?

[Waldecir Santos]

it will depend on write concern, if it is setted as w=0, so you will need to call. IMHO  

Grato,

Waldecir

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[aliane abdelouahab]

and what about the default behavior, the default is acknowledged? it
will stay until mongod releases the lock?

On 6 avr, 00:26, Waldecir Santos <walde...@gmail.com> wrote:
> it will depend on write concern, if it is setted as w=0, so you will need
> to call. IMHO
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 8:21 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

[Waldecir Santos]

the default behaviour is basic, internal-operation-of-write-concern, so you will see only network and socket errors, i dont get this lock too, im researching but this appears to be another type of lock, because you have to use unlock() im not sure.

Grato,

Waldecir

[aliane abdelouahab]

yes, as an example when creating indexes mongodb locks, so what if
when creating indexes a client asks for a document?
so do i make a habit to begin with an assertion that is_lock() returns
false to let clients asks for documents?

On 6 avr, 00:52, Waldecir Santos <walde...@gmail.com> wrote:
> the default behaviour is basic,

> internal-operation-of-write-concern<http://docs.mongodb.org/manual/core/write-operations/#internal-operat...>,

> so you will see only network and socket errors, i dont get this lock too,
> im researching but this appears to be another type of lock, because you
> have to use unlock() im not sure.
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 8:46 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

[Waldecir Santos]

i think setting write_concern to 1 is way to go

A number greater than 1:
Guarantees that write operations have propagated successfully to the specified number of replica set members including the primary. If you set w to a number that is greater than the number of set members that hold data, MongoDB waits for the non-existent members to become available, which means MongoDB blocks indefinitely

Grato,

Waldecir

[aliane abdelouahab]

this why i was asking this question, with pymongo, since it is as
blocking calls, it can wait, but what about a non-blocking driver?
does it serve other clients and wait untill the lock is released?

On 6 avr, 01:03, Waldecir Santos <walde...@gmail.com> wrote:
> i think setting write_concern to 1 is way to go
>
> A number greater than 1:
> Guarantees that write operations have propagated successfully to the
> specified number of replica set members including the primary. If you set w
> to a number that is greater than the number of set members that hold data,
> MongoDB waits for the non-existent members to become available, which means
> MongoDB blocks indefinitely
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 9:00 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

[Waldecir Santos]

Pymongo will definitely lock i think, about motor i donk have idea, need to test, do you have any environment to test ? or think in some scenario ? maybe i can try.

Grato,

Waldecir

[aliane abdelouahab]

thank you, because i'm learning the behavior of non blocking drivers,
and found that even motor has is_locked() so it should have a
technique or put a flag to return back again.

On 6 avr, 01:19, Waldecir Santos <walde...@gmail.com> wrote:
> Pymongo will definitely lock i think, about motor i donk have idea, need to
> test, do you have any environment to test ? or think in some scenario ?
> maybe i can try.
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 9:11 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

[Waldecir Santos]

motor actually is pymongo but with greenlets to lets it do thinks without block, so motor will not block. this is a nice post about motor-internals-how-i-asynchronized-a-synchronous-library

Grato,

Waldecir

[aliane abdelouahab]

in the documentation here what i found:

@gen.engine
def lock_unlock():
c = yield motor.Op(motor.MotorClient().open)
locked = yield motor.Op(c.is_locked)
assert locked is False
yield motor.Op(c.fsync, lock=True)
assert (yield motor.Op(c.is_locked)) is True
yield motor.Op(c.unlock)
assert (yield motor.Op(c.is_locked)) is False

it seems that Motor will leave the request if it find that mongod is
locked, this is why he used the assertion?

On 6 avr, 01:24, Waldecir Santos <walde...@gmail.com> wrote:
> motor actually is pymongo but with greenlets to lets it do thinks without
> block, so motor will not block. this is a nice post about

> motor-internals-how-i-asynchronized-a-synchronous-library<http://emptysquare.net/blog/motor-internals-how-i-asynchronized-a-syn...>

>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 9:22 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

[Waldecir Santos]

This is a test code, imo, assert will stop code execution so if the first assertion failt it will raise AssertionException.

Grato,

Waldecir

[aliane abdelouahab]

it is like he is forcing to unlock mongodb by using yield
motor.Op(c.unlock) ? the idea is to force the unlock?

On 6 avr, 01:37, Waldecir Santos <walde...@gmail.com> wrote:
> This is a test code, imo, assert will stop code execution so if the first
> assertion failt it will raise AssertionException.
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 9:32 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

[Waldecir Santos]

Nope, this will wait the unlock so execute the callback when finished 

Grato,

Waldecir

[aliane abdelouahab]

ah! so i guess it will work like that:
-ask mongod, if it is locked, then:
-if no one is asking for other requests wait, else go serve other
untill the unlock happens then back to ask again mongod?

On 6 avr, 01:43, Waldecir Santos <walde...@gmail.com> wrote:
> Nope, this will wait the unlock so execute the callback when finished
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 9:42 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

[Waldecir Santos]

when you set a callback it will be called when the call can be made, if i can made myself clean, so when you call unlock, the callback will be executed when mongo send the response back, until that Tornado/Motor will serve another requests

Grato,

Waldecir

[aliane abdelouahab]

ah! that is what i'm asking about, now it becomes clear, thank you :)
so i should always make the is_locked() condition explicitly to avoid
lost of requests.

On 6 avr, 01:54, Waldecir Santos <walde...@gmail.com> wrote:
> when you set a callback it will be called when the call can be made, if i
> can made myself clean, so when you call unlock, the callback will be
> executed when mongo send the response back, until that Tornado/Motor will
> serve another requests
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 9:51 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

> ...
>
> plus de détails »

[Waldecir Santos]

For now i just use w=1 without any use, i dont see a good reason to keep calling is_locked(), i have loked inside motor-blog and it never use is_locked.

Grato,

Waldecir

[aliane abdelouahab]

i think the lock is used when there is replicated data to ensure
eventual consistence,

On 6 avr, 02:01, Waldecir Santos <walde...@gmail.com> wrote:
> For now i just use w=1 without any use, i dont see a good reason to keep
> calling is_locked(), i have loked inside motor-blog and it never use
> is_locked.
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 9:57 PM, aliane abdelouahab

> <alabdeloua...@gmail.com>wrote:

> ...
>
> plus de détails »

[Waldecir Santos]

so you must use w=1 and you dont need to control it by hand.

Grato,

Waldecir

[aliane abdelouahab]

so if w>1 then i must see if mongodb is not locked, from the
documentation:
w: (integer or string) If this is a replica set, write operations will
block until they have been replicated to the specified number or
tagged set of servers. w=<int> always includes the replica set primary
(e.g. w=3 means write to the primary and wait until replicated to two
secondaries).

On 6 avr, 02:06, Waldecir Santos <walde...@gmail.com> wrote:
> so you must use w=1 and you dont need to control it by hand.
>
> Grato,
>
> Waldecir
>

> On Fri, Apr 5, 2013 at 10:05 PM, aliane abdelouahab <alabdeloua...@gmail.com

> ...
>
> plus de détails »

[Waldecir Santos]

yes, he do all the work.

Grato,

Waldecir

[aliane abdelouahab]

now i got it, thank you :)
so i guess if w>1 and there is no locking verification, motor will try
to handle this alone?
because what i want to know is where to use is_locked() and if it is
crutial when replicating data?

On 6 avr, 02:15, Waldecir Santos <walde...@gmail.com> wrote:
> yes, he do all the work.
>
> Grato,
>
> Waldecir
>

> On Fri, Apr 5, 2013 at 10:13 PM, aliane abdelouahab <alabdeloua...@gmail.com

> ...
>
> plus de détails »

[Waldecir Santos]

You just need to set the w and j variable to get the behaviour you want

write_concern

Grato,

Waldecir

> ...
>
> plus de détails »

[aliane abdelouahab]

yes but what about motor behavior if someone dont verify the lock
state of the server and asks for writing data? it will make a normal
non-blocking operation by waiting another request from another client
untill the server is onlocked? so then is_locked() is to change the
default behavior of motor?

On 6 avr, 02:19, Waldecir Santos <walde...@gmail.com> wrote:
> You just need to set the w and j variable to get the behaviour you want
>

> write_concern<http://api.mongodb.org/python/current/api/pymongo/collection.html?hig...>
>
> Grato,
>
> Waldecir
>
> On Fri, Apr 5, 2013 at 10:17 PM, aliane abdelouahab <alabdeloua...@gmail.com

> ...
>
> plus de détails »

[Waldecir Santos]

you dont need to lockat, with you call a async without the w our set w=0 for exemple, the code will not wait the result so it will go on even on error. you dont need think mongodb as async he will responde for a async_driver.

Grato,

Waldecir

> ...
>
> plus de détails »

[aliane abdelouahab]

yes i got the idea, but what i need:
i did a project about nosql, and took mongodb as my dbms, then, i must
talk all about cases, and from thoses cases there is the one where
the write must be safe (example e-commerce) and the data is replicated
in different servers, and because the driver used is a non blocking
(motor), so i wanted to understand how motor will behave if there is
no explicit verification on mongod lock? (the admin forgot to add the
is_locked() assertion or statement)

On 6 avr, 02:35, Waldecir Santos <walde...@gmail.com> wrote:
> you dont need to lockat, with you call a async without the w our set w=0
> for exemple, the code will not wait the result so it will go on even on
> error. you dont need think mongodb as async he will responde for a
> async_driver.
>
> Grato,
>
> Waldecir
>

> On Fri, Apr 5, 2013 at 10:28 PM, aliane abdelouahab <alabdeloua...@gmail.com

[Waldecir Santos]

We need split this.

Mongodb.

Is ATOMIC and have many mechanisms to work on it, like all DBMS, and ahve some nice feature lie replicaset and etc.

Motor Driver, even other sync driver

have actions, and many connections options, we dont set w=1 if you dont want to wait the successful write, a normal opertion non-ecomerce i think j=1 and w=0 good enough

what you are, so mongo will grant you like any other DBMS, you only change behaviour in driver to take actions about, like socket exception and etc. You dont need to be care about lock. imo 

Grato,

Waldecir

[A. Jesse Jiryu Davis]

Wow, guys. Don't worry about any of this, please. Forget everything you just said. Pretend you never had this conversation.

1. MongoDB's "lock" feature is for systems administrators to take backups. It has nothing to do with you. Ignore it. Forget that it exists. (There is an idea of "database-level locking" in which MongoDB locks its own data files for a few microseconds on every insert, update, or delete, to ensure consistency. Now, forget you heard that, too. It also has nothing to do with you.)

2. PyMongo and Motor both set the default write concern to w=1, which is the proper write concern for you. Leave it alone. Forget about write concern, the driver handles it for you.

Good? No questions, please. Just walk away.

[aliane abdelouahab]

i was just curious ;)

On 6 avr, 02:47, Waldecir Santos <walde...@gmail.com> wrote:
> We need split this.
>
> Mongodb.
>
> Is ATOMIC and have many mechanisms to work on it, like all DBMS, and ahve
> some nice feature lie replicaset and etc.
>
> Motor Driver, even other sync driver
>
> have actions, and many connections options, we dont set w=1 if you dont
> want to wait the successful write, a normal opertion non-ecomerce i think
> j=1 and w=0 good enough
>
> what you are, so mongo will grant you like any other DBMS, you only change
> behaviour in driver to take actions about, like socket exception and etc.
> You dont need to be care about lock. imo
>
> Grato,
>
> Waldecir
>

> On Fri, Apr 5, 2013 at 10:40 PM, aliane abdelouahab <alabdeloua...@gmail.com

> ...
>
> plus de détails »

[aliane abdelouahab]

i thought lock also works when two people tried to modify the same
data, so it must wirte lock for A to let B modify it? so it is the
case where i must verify locks?
i just want where to use is_locked(), it is necessary when data is
replicated?

On 6 avr, 03:02, "A. Jesse Jiryu Davis" <je...@emptysquare.net> wrote:
> Wow, guys. Don't worry about any of this, please. Forget everything you
> just said. Pretend you never had this conversation.
>
> 1. MongoDB's "lock" feature is for systems administrators to take backups.
> It has nothing to do with you. Ignore it. Forget that it exists. (There is
> an idea of "database-level locking" in which MongoDB locks its own data
> files for a few microseconds on every insert, update, or delete, to ensure

> consistency. Now, forget you heard that, too. It *also* has nothing to do

> ...
>
> plus de détails »

[A. Jesse Jiryu Davis]

No! No! =)

The database locks itself for a few microseconds to perform a write, then unlocks. It uses a mutex internally. If two writes are attempted at the same time, one of them succeeds, then the other. is_locked() is only true if you have deliberately locked the database by calling fsync(lock=True). Do not call it, however. It is for systems administrators, not for you.

Do not call is_locked, Aliane.

> ...
>
> plus de détails »

[aliane abdelouahab]

ah now i get it more, thank you Jesse and Waldecir
:)

On 6 avr, 03:11, "A. Jesse Jiryu Davis" <je...@emptysquare.net> wrote:
> No! No! =)
>
> The database locks itself for a few microseconds to perform a write, then
> unlocks. It uses a mutex internally. If two writes are attempted at the
> same time, one of them succeeds, then the other. is_locked() is only true
> if you have deliberately locked the database by calling fsync(lock=True).
> Do not call it, however. It is for systems administrators, not for you.
>
> Do not call is_locked, Aliane.
>

> On Fri, Apr 5, 2013 at 10:07 PM, aliane abdelouahab <alabdeloua...@gmail.com

> ...
>
> plus de détails »

[aliane abdelouahab]

can you please update the documentation:
https://motor.readthedocs.org/en/latest/api/motor_client.html?highlight=read%20preference#motor.MotorClient.read_preference
the link to See ReadPreference. is absent
and
http://api.mongodb.org/python/current/api/pymongo/index.html#pymongo.read_preferences.ReadPreference
MongoClient is duplicated?

On 6 avr, 03:11, "A. Jesse Jiryu Davis" <je...@emptysquare.net> wrote:
> No! No! =)
>
> The database locks itself for a few microseconds to perform a write, then
> unlocks. It uses a mutex internally. If two writes are attempted at the
> same time, one of them succeeds, then the other. is_locked() is only true
> if you have deliberately locked the database by calling fsync(lock=True).
> Do not call it, however. It is for systems administrators, not for you.
>
> Do not call is_locked, Aliane.
>

> On Fri, Apr 5, 2013 at 10:07 PM, aliane abdelouahab <alabdeloua...@gmail.com

> ...
>
> plus de détails »

[A. Jesse Jiryu Davis]

Hmm. The first link shows a problem with how Motor's docs are generated: I've opened https://jira.mongodb.org/browse/MOTOR-6 to track the issue.

The second link: there's a distinction between MongoClient connected to one mongod, and MongoClient connected to a mongos in a sharded cluster.

> ...
>
> plus de détails »

[aliane abdelouahab]

ah, thank you :D

On 6 avr, 03:33, "A. Jesse Jiryu Davis" <je...@emptysquare.net> wrote:
> Hmm. The first link shows a problem with how Motor's docs are generated:

> I've openedhttps://jira.mongodb.org/browse/MOTOR-6to track the issue.

>
> The second link: there's a distinction between MongoClient connected to one
> mongod, and MongoClient connected to a mongos in a sharded cluster.
>

> On Fri, Apr 5, 2013 at 10:27 PM, aliane abdelouahab <alabdeloua...@gmail.com

>
>
>
>
>
>
>
> > wrote:
> > can you please update the documentation:
>

> >https://motor.readthedocs.org/en/latest/api/motor_client.html?highlig...

> > the link to See ReadPreference. is absent
> > and
>

> >http://api.mongodb.org/python/current/api/pymongo/index.html#pymongo....

> ...
>
> plus de détails »