proposal for implementing triggers in mongodb

[David Schere]

Hello,

I have a rough draft for a strategy on how to implement triggers in
mongodb. For clarity, by 'trigger' I mean a system by which the end
user registers a filter expression that is applied to collections as
they are manipulated within the mongo database. When a filter
expression returns true, the object and the operation that was
executed (insert,update,delete), are passed to a user specified
callback function. This callback is refered to as the trigger
function. .

The trigger system would be service daemon running outside of mongodb.
This has the following advantages:
1. It is possible for the trigger function to make calls to the
mongo database without the chance of deadlocks
2. Does not result in a performance hit to the mongo database
since filtering is taking place within a separate process
3. Minimize code changes and maintenance for mongodb
4. Can be written in a language other than C++, I was going to use
python and if needed use pyrex if performance becomes an issue.

Strategy:

1. Modifications to the mongo database:

Create a hook inside this routine in db/oplog.cpp to forward
logging events to a daemon.

/*@ @param opstr:
c userCreateNS
i insert
n no-op / keepalive
d delete / remove
u update
*/
void logOp(const char *opstr, const char *ns, const BSONObj& obj,
BSONObj *patt, bool *b, bool fromMigrate) {
if ( replSettings.master ) {
_logOp(opstr, ns, 0, obj, patt, b, fromMigrate);
}
>>>>>>>>>>>
Propose forwarding the 'opstr' and 'obj' to a TCP socket
interface were by processing can take place within in the trigger
daemon.
>>>>>>>>>>>

logOpForSharding( opstr , ns , obj , patt );
}

Add new user defined functions to mongodb:

AddTrigger( name, filterExpression, action )
filterExpression: a javascript code that will be executed on
incoming objects, if the expression tests true then action is called.
action = {
language:
callback
}

I plan on implementing the trigger daemon in python so the
language would equal 'python' and callback would be a string
"<module>.<method>" that would load the specified python
module and execute the method. All callbacks would have
the following syntax

triggerCallback( operation, obj )

RemTrigger( name )
Unregister the named trigger.

2. The trigger daemon:

short and simple for now.

Listen for connection
parse message
if Register add entry to a dispatch table
else if Unregister remove entry from a dispatch table
else
apply expressions and forward objects that pass the
expression to the user specified callback.


3. Integrate daemon creation and destruction into mongodb ( I could
use daemon tools )


Feedback requested and badly needed.


I can be better reached at DSc...@skylinenet.net

[Eliot Horowitz]

Once you have a daemon, why not just tail the oplog?

> --
> You received this message because you are subscribed to the Google Groups "mongodb-dev" group.
> To post to this group, send email to mongo...@googlegroups.com.
> To unsubscribe from this group, send email to mongodb-dev...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/mongodb-dev?hl=en.
>

[Shane Spencer]

One reason I can think of is to modify an operation before it hits the
initial storage stages.

[Shane Spencer]

Well sorry.. I mean for direct triggers inside the MongoDB process.

[Eliot Horowitz]

You wouldn't that to go through a daemon anyway, as your write would
then be blocking for the entire duration.

[Scott Hernandez]

This proposal doesn't seem to allow for modification (before
storage/writing), like before insert/update/delete. It is more of a
event notification which is generally different than traditional
database trigger which can reject/change the data (sometimes referred
to as "before" triggers) and are synchronous. It sounds like the goal
of this proposal is to decouple the core server from the event
notification mechanism.

Please correct me if I'm wrong about your plan. If I'm not, please
let's call them event notifications or something other than triggers
so we don't confuse too many people.