[Upcoming Releases] Modularizing Casbah (as well as other upcoming features and ideas)

16 views
Skip to first unread message

Brendan W. McAdams

unread,
Aug 21, 2010, 11:45:54 AM8/21/10
to mongodb-casbah-users
Fellow Casbah Users,

A heads up on something Max and I are currently working on (and is
currently reflected in master, albeit incomplete).

For a variety of reasons (which to some degree I'll go into below) we
are starting to split Casbah off into modules. This will allow
separation out of only the features people want / need for more
modular usage. It also allows us to build things which can be
leveraged in other projects without needing the whole shebang.

The module breakdown is currently:

* casbah-util
Dependent only upon Configgy and the mongo-java-driver.
Contains any general utilities (like our Logging trait which we swiped
from Akka) and most importantly, our Scala 2.8 Collections style
DBObject (MongoDBObject). i.e., the 2.8 collections trait
implementation of Factory and Builder patterns for DBObjects and the
Implicit conversions for converting Maps to and from DBObjects, etc.
By splitting these out, those who need just improved DBObject
utilities but don't want the rest of Casbah can take advantage of
them. I am, for example, looking at improving Akka's STM. It doesn't
make sense to pull all of Casbah into Akka but having the DBObject
utilities would be huge.

Has a com.novus.casbah.mongodb.util implementation of implicits and
imports (Which other modules can build on).

* casbah-query
Purely the bareword and core operator query syntax. This is
dependent only upon casbah-util (and transitively, mongo-java-
driver). This way if you don't want the query syntax (or want JUST it
and our DBObject utils, to create DBObjects for your own Mongo
interaction w/o our other toys) you don't have to.

Like util, has it's own standalone implicits and imports; query's
versions automatically extend Util's however so if you're using just
query you get the util imports / implicits automatically (limit the
number of extra import statements)

* casbah-core
This is, essentially, everything that Casbah is now from a user's
standpoint. It is dependent upon query, so you get a transitive
dependency on util, mongo-java-driver, etc. All the wrappers for the
mongo-java-driver such as the Cursor, Connection etc are in core.
Most of the things we don't want to externalize will continue to be
built in core.

This has the Imports/Implicits import everyone is already used to, and
builds upon the ones from query and util so you only need one import
statement.

* casbah-gridfs

We're splitting our GridFS implementation out into it's own module.
This module is dependent upon core, and will require it's own separate
import to activate. This is partly because of the bson encode/decode
safety issues that our GridFS implementation imposes; using it should
be explicitly asked for rather than automatically provided.

Uses it's own import/implicits separate from the core / util/ query
ones so oyu have to import it manually.

Max Afonov has also been working on ORM (Java 5 annotations - we've
also talked a bit about active record pattern and query integration)
for Casbah currently in the mapper branch. When that's available it
will be in a casbah-mapper module.

Originally Mapper was going to be 1.1, but depending on timing we'll
probably make that 1.2, and do the modularized release as 1.1.

1.1 will also have support for all of the MongoDB 1.6 features - $or
operator, replication sets and, if any code is needed to support it,
sharding.

This will give everyone the ability to use only the parts of casbah
they want - I'm hoping to focus a lot of energy on perfecting casbah-
util and -query so that other projects can find them useful. If you
don't want mapper, you won't get it. But thanks to the beauty of
SBT's subproject system it will be easy to build and we will continue
to publish a top level pom (Which will contain *ALL* the modules).

Another item of note is we have started to talk, in some cases about
using ONLY the wire format from the Java driver. We have run into
some inheritance issues inheriting from the Mongo objects because of
Java limitations. This is the reason MongoDBObject is a pimped
library rather than a BSONObject.

It also causes some limitations in our implicit conversions in the
Collection implementation. If you look at the java driver code
Collection etc are *VERY VERY* thin wrappers on the DBAPI.java code
which is the actual communication with MongoDB.

We are considering, for performance, flexibility and yak shaving
reasons, starting to call DBAPI directly and focusing on pure Scala
collections implementations of the MongoDBCollection code. We may do
similar with DBObject.

This will be a boon for everyone, and we will of course provide
implicit conversions to and from the native Java versions.

I know this is a lot to take in, I am hoping some of you Casbah users
can give me feedback and thoughts. We will also be adding a whole new
unit test suite which gets maximum possible coverage before these
releases; moving off of Scalatest and onto Specs.

I don't have timing on these things - the majority of Casbah features
is driven at the moment by my actual needs at work. This refactor is
more based on best practices and desire to improve things.

I am working currently on a MongoDB Book which, apart from my job, is
my primary focus. So this may mean Casbah development slows a bit.
Bug fixes DO take a priority, but new features may be slower to
evolve. Max is working on development as well and hopefully the
influx of new ideas will keep things going.

It should also go without saying that we welcome new contributors.
Give us your patches, ideas, complaints or rants. Casbah grew from my
own usage patterns of MongoDB but now I'm hoping it finds a place in
other people's environments.

Regards,

Brendan

Philip Wills

unread,
Aug 23, 2010, 11:08:21 AM8/23/10
to mongodb-ca...@googlegroups.com
The modularisation sounds sensible, but can I suggest something other
than casbah-util as a name. Util always suggests to me that someone
hasn't really thought out where to put something and util is just
acting as an accommodating bucket. Perhaps casbah-collections? Also
having something called core which isn't the minimum set to be useful
seems a bit strange to me.

With regards to contributing, is the best thing to have a bash at
something in the lighthouse tracker and then make a pull request, or
is that likely to lead to treading on the toes of Max and yourself?

Look forward to the book,

Phil Wills

Max Afonov

unread,
Aug 23, 2010, 11:17:07 AM8/23/10
to mongodb-ca...@googlegroups.com
On Mon, Aug 23, 2010 at 11:08, Philip Wills <othe...@gmail.com> wrote:
> With regards to contributing, is the best thing to have a bash at
> something in the lighthouse tracker and then make a pull request, or
> is that likely to lead to treading on the toes of Max and yourself?

I can only speak for myself, but I'm lost in the "mapper" branch[1]
most of the time. Until the new object -> DBObject code is merged into
master and/or released, you're unlikely to interfere with any of my
work.

In general though, the most valuable patches are those that address
someone's pressing need. That said, our pressing needs (i.e. those
expressed using Lighthouse tickets) aren't necessarily representative
of anyone else's needs. If there's anything that you personally find
unacceptable or broken, feel free to fork master on Github & send us a
pull request. All contributions are welcome, even if you only submit a
failing test case.

[1]: http://github.com/novus/casbah/tree/mapper

Brendan W. McAdams

unread,
Aug 23, 2010, 5:44:33 PM8/23/10
to mongodb-ca...@googlegroups.com
I don't disagree on the casbah-util name.  I couldn't think of anything better at the time ;)

Maybe casbah-bson-util

or casbah-dbobject-util

Or simply one of those without -util?

The idea being it is tools for working with dbobjects or bson objects and nothing else.

I'll revisit the core name as well - I think you're right in that the name is confusing as to it's purpose.

As for contributing, Max is correct.  The things in Lighthouse are items we have tracked as 'things we should fix/implement' but aren't necessarily priority.  You are of course welcome to hack on those but the best bet is to build some things which are specifically useful to YOU. Whatever you find useful, work on in a fork and send us a pull request detailing the changes.

And even failing unit tests, as Max says, are hugely helpful.  They give us a reproducible test case to work with.


Thanks!

-b
Reply all
Reply to author
Forward
0 new messages