MongoEngine Roadmap

235 views
Skip to first unread message

Ross Lawley

unread,
Mar 13, 2012, 8:01:23 AM3/13/12
to mongoeng...@googlegroups.com
Hi all,

I've been maintaining MongoEngine now for the last 12 months and with the help of the community I think we've made some great progress.  People keep on asking me whats on the roadmap?  So its hightime I created one; I think there is a lot of potential for MongoEngine to improve and make developers lives easier and as such there are two main areas I want to focus on going forward:

    1) MongoEngine core
    2) MongoEngine framework support

MongoEngine Core

The key goal is to stabilise MongoEngine, improve performance and make dropping into raw pymongo easier.  There will be some backward breaking changes, mainly to provide the best possible defaults - ones that don't hurt you as you scale and as always I will provide full upgrade documentation.

Roadmap

0.7

- Performance - Dirty data tracking
- Allow inheritance default to off
- Chainable QuerySet slice
- Serialization / deserialization
- Raw method for pyMongo

See: https://github.com/hmarr/mongoengine/issues?milestone=4&state=open for all the issues assigned to the 0.7 (not all will make it).

0.8

- Performance - Configurable lazy references
- Transparent pymongo connections - lazy ones at that :)
- Bug fixes
- Code quality

0.9

- Python 3 support
- Test refactor
- Polish and clean
- Website

1.0 - Party! :)

Anything missing or you would like in - let me know!

MongoEngine framework support

MongoEngine is used with various different frameworks and there are numerous projects out there for integration in varying states of completeness.  I would like to bring framework support into the project - so that we can provide similar levels of functionality to whichever framework you are using.

At the weekend I did a quick review and found:

Django has 6 projects on Github providing Forms, Views, Admin and debug toolbar support.
Flask has 2 projects providing Generic Views, Forms and Debug toolbar support
Pyramid has a project providing project scaffolding.
Bottle has a project / example for integration

I will need help from the community with this and I will be reaching out to people who have already published code on github.  If you are interested let me know!

To help with this I have added a new organisation in Github: http://github.com/mongoengine - this will host the code going forward (don't worry any changes to the core will be pushed upstream to http://github.com/hmarr/mongoengine/) I'm hoping that all the new framework libraries will be hosted under this organisation so that they are easy to find and cross repository issues can be tracked.

I'm really excited about pushing towards getting MongoEngine to 1.0 and with your help we'll get there soon, I would also love some feedback - anything you think MongoEngine could do better / shouldn't be doing let me know!

Ross

Roy Smith

unread,
Mar 13, 2012, 9:15:30 AM3/13/12
to mongoeng...@googlegroups.com
On Mar 13, 2012, at 8:01 AM, Ross Lawley wrote:

> Hi all,
>
> I've been maintaining MongoEngine now for the last 12 months and with the help of the community I think we've made some great progress.

This is a really awesome project. It hugely simplifies things compared to using the raw pymongo calls, and the data validation is really helping to keep our database in good shape.

> 0.7


> - Raw method for pyMongo

Yeah, this would be good. We've got a few calls where we've had to go around behind mongoengine's back to do something in pymongo which wasn't available in the mongoengine interface. Having a clean way to do this (as opposed to mucking about in mongoengine's internals and finding the connection object) would be an improvement

> I will need help from the community with this and I will be reaching out to people who have already published code on github. If you are interested let me know!

One of the things I've found frustrating about using mongoengine is that the docs are kind of skimpy. I'd be happy to pitch in and help work on that.

By the way, as of about 4 weeks ago, all of songza.com is running on top of mongoengine (and django 1.3).

One tool that might be useful is a something to automate creation of models. It shouldn't be that hard to look at an existing collection, figure out what fields exist, what data types are in those fields, what indexes exist, etc, and produce a skeleton of a Document subclass. It would probably need some fine-tuning to match reality, but would be a start and save a lot of grunt work converting an existing project.

I did some of that songza.com already (extracting the existing index specs). If you think this would be a generally useful tool, I'd be happy to flesh it out and get it into shape to release.

--
Roy Smith
r...@panix.com

Roy Smith

unread,
Mar 13, 2012, 9:44:48 AM3/13/12
to mongoeng...@googlegroups.com
On Mar 13, 2012, at 8:36 AM, Ross Lawley wrote:

Before we get to 1.0, I'm hoping to to update the website with some examples of who is using MongoEngine.  Right now I don't have any visibility about who is using it - so if your company is using MongoEngine in production and potentially are happy for that to be promoted could you email me the company name and url?



Website: songza.com
Backend done by s7labs.com (with the obligatory 'we're hiring' announcement --  see http://www.s7labs.com/jobs/)

The entire production backend has been running on mongoengine since early February 2012.  We're currently running mongoengine 0.5.2, django 1.3, MongoDB 2.0.2, python 2.6.5, on Ubuntu Linux.

Songza's publicity blurb:

"Songza is the only streaming service that finds you the right tunes at the right time by matching your day, time, device and situation to the right expertly-curated playlist -- all without audio ads, and for free"

We're more than happy to get any publicity you can give us.

--
Roy Smith

Raviteja Dodda

unread,
Mar 14, 2012, 3:10:09 AM3/14/12
to mongoeng...@googlegroups.com
Ross, the roadmap is great.

We have a pyramid backend, which uses MongoEngine to make all the calls to MongoDB. Our Product is in final stages of development, and will let you know once we are in production.

I can help in creating a scaffold for mongoengine with pyramid in future, but not now as my hands are full.

Btw, thanks for the replica-set support in MongoEngine 0.6.

-Ravi

Roy Smith

unread,
Mar 14, 2012, 3:39:56 PM3/14/12
to mongoeng...@googlegroups.com
On Mar 13, 2012, at 9:15 AM, Roy Smith wrote:

One tool that might be useful is a something to automate creation of models.   It shouldn't be that hard to look at an existing collection, figure out what fields exist, what data types are in those fields, what indexes exist, etc, and produce a skeleton of a Document subclass.  It would probably need some fine-tuning to match reality, but would be a start and save a lot of grunt work converting an existing project.

I cleaned this up a little bit and pushed it out to https://bitbucket.org/roysmith/python-tools.  Feedback appreciated.


---
Roy Smith



Ross Lawley

unread,
Mar 14, 2012, 6:06:28 PM3/14/12
to mongoeng...@googlegroups.com
Roy,

Thats great - I've only really had experience of people using mongoengine from the outset, rather than coming to it from an existing mongo setup.  I'll make sure I devote some time to look into it in more depth soon.

cheers,

Ross

Jake Peters

unread,
Mar 24, 2012, 2:00:12 PM3/24/12
to mongoeng...@googlegroups.com

On Mar 13, 2012, at 8:36 AM, Ross Lawley wrote:

Before we get to 1.0, I'm hoping to to update the website with some examples of who is using MongoEngine.  Right now I don't have any visibility about who is using it - so if your company is using MongoEngine in production and potentially are happy for that to be promoted could you email me the company name and url?


PayPerks (www.payperks.com) is also using MongoEngine with Django 1.3.  When you're ready to update the website, let me know (ja...@payperks.com) and I will give you a blurb, logos, testimonials, etc. We will look to contribute more to the project as we become more familiar with it -- we are recent users.

Anyone who has experience using Tastypie with MongoEngine, please let me know -- am looking for a bit of help making Tastypie and MongoEngine/Mongo work well together.

Cheers,
Jake

Reply all
Reply to author
Forward
0 new messages