Mongoengine 0.5

[Ross Lawley]

Hi all,

I'm happy with the shape of mongoengine and I'm hoping that we should get a release out in the near future.

I think the multi database support is relatively simple and I'll add that in soon.

However, I'm undecided about dynamic documents / fields and would like to start a conversation about what people are after!

Firstly, why dynamic fields?  Isn't the point of using an Document Object Mapper to map what fields that might be there?  To me it seems counter intuitive to explicitly name fields for a document (which may or may not be required) and then arbitrarily be able to add extra fields.  It seems to me these extra fields would be undiscoverable..

I understand the strength of mongodb is in its flexibility - but isn't the point of mongoengine to explicitly declare the data structures you want to use in your documents?

Anyway let me know what you think and if you need dynamic fields - let me know your use case to help me better understand.

Ross

 

[Amorano]

I know I am new! =)

But I disagree.

The strength of a No-SQL database is the ability to add Ad Hoc data.

While I do appreciate being able to restrict or define some of the
columns (with smething like MongoEngine) I still need the ability to
add data without a schema.

If everything was to be concrete, i.e. schema, I may pause and wonder
why I would need a NoSQL DB over a traditional relational model.

Perfect example of mixing:

We use SQL to store session and hard asset entries.

We use Mongo to provide schema-less access to the asset. We don't care
what is in the asset on the sql side, only that we have an asset, and
we can relate it to other assets.

At the top level, the traditional model is blazing fast to find
parents/children/edges

At the lower access points we can get full data about an asset and
query the sub-sets from the top level model to find correlating data
without having tons of bridge tables or gobs of columns in the asset
table.

The best of both (for us).

BUT, that also requires that at the low level we are free to define
anything we want in the asset's document.

Locking it into a schema would sandbox us just as if we were using a
traditional model.

HTH

Cheers.

[Ross Lawley]

Hi, 

I understand your initial reaction.  However, don't DictFields and ListFields already allow for this level of flexibility?  With them you can build complex datasets but you have to define the possibility of their presence in the document.  After all if its not declared in the document then you can't use the ODM to query that data without having to resort to __raw__ queries.

I suppose my point is you are defining a schema when you define the document and by doing so you are defining how you expect that document to be used.  That doesn't necessarily mean its inflexible on the contrary, it just means at the top level you have explicitly described the type of data the document can store.

If anyone has some real world examples, where mongoengine is too inflexible without dynamic fields I'd love to hear them!  Lets keep this conversation going as it helps me better understand how people want to use mongoengine.

Ross

[Matías Iturburu]

In our app we have a series of background processes making scrapping and mining. 

We do a series of euristics with the values retrieved and store them in mongodb.

We were using pymongo directly, since you have the posibility to just obj['key'] = value anything, but the syntax gets too cryptic too soon.

I did some refactoring to adopt mongoengine giving this feature for granted and shoot myself on the foot (http://stackoverflow.com/q/7335521/53468).

Supporting this feature would be a convenient plus over using bareback pymongo.

If the use case described in the question is not clear enough I'd love to elaborate.

Regards