The design of new c++11 bson lib

185 views
Skip to first unread message

li ning

unread,
Apr 8, 2016, 6:44:39 AM4/8/16
to mongodb-user
I use this new driver 2 days, found some design problems about libbson. I think those flaws will make this new lib useless in a real development. (forgive me if I have not found the correct usage)

I read the examples, there are 2 ways to build a bson object, one is stream, another is basic. 
The example problem is in a real development, it is few chance to build a bson like that. Most building way in example are in compile time.
But the real usage has not considered: Object BSON Mapping: How to convert an object to a bson object?
For example:

class A {
public:
    bsoncxx::document::value to_bson() const;  // {"data":"a"}
};
class C {
public:
    bsoncxx::document::value to_bson() const;  // {"data":"c"}
};
class D {
public:
    bsoncxx::document::value to_bson() const;  // {"data":"d"}
};
class B : public A {
public:
    bsoncxx::document::value to_bson() const; //  {"data":"a","data_more":"b","c":{"data":"c"},"ds":[{"data":"d1"},{"data":"d2"}]}
private:
    C c;
    std::vector<D> ds;
};

To do the mapping, the bson lib must support:
1. embed an bson to another bson, this represent "C object as a member in B"
Both stream and basic can not do this, but 26compat can do:
mongo::BSONObjBuilder().append("c", c.to_bson()).obj()


2. embed all elements to another bson, this represent "B derived from A". Both stream and basic can do this by concatenate
stream:
bsoncxx::document::value B::to_bson() const {
    auto doc = bsoncxx::builder::stream::document{};
    doc << bsoncxx::builder::concatenate(A::to_bson().view());
    return doc.extract();
}
basic:
bsoncxx::document::value B::to_bson() const {
    auto doc = bsoncxx::builder::basic::document{};
    doc.append(bsoncxx::builder::concatenate(A::to_bson().view()));
    return doc.extract();
}
26 compat can do also:
mongo::BSONObjBuilder().appendElements(A::to_bson())

3. build complex bson according to container whos size can change in runtime like std::vector
stream:
can not do this. There is no way to embed a array stream to doc stream like
doc_stream<<array_stream<<other

 And impossible write a loop in doc stream like:
doc_stream<< open_array<<loop(ds) {add D's content} <<close_array<< others

basic can do this by lambda, but the problem is I feel use lambda here is too fancy. Look this code (from my project):
c++11 bson:
        auto doc = bsoncxx::builder::basic::document{};
doc.append(bsoncxx::builder::basic::kvp("quest", [this](bsoncxx::builder::basic::sub_document d) {
d.append(bsoncxx::builder::basic::kvp("quests", [this](bsoncxx::builder::basic::sub_array a) {
for(auto i : _done_quests)
a.append([&](bsoncxx::builder::basic::sub_document d) {
d.append(bsoncxx::builder::basic::kvp("id", i));
d.append(bsoncxx::builder::basic::kvp("cv", -1));
});
for(const auto& i : _quests)
a.append([&](bsoncxx::builder::basic::sub_document d) {
d.append(bsoncxx::builder::basic::kvp("id", i.second->id()));
d.append(bsoncxx::builder::basic::kvp("cv", i.second->cv()));
});
}));
d.append(bsoncxx::builder::basic::kvp("chapter", [this](bsoncxx::builder::basic::sub_array a) {
for(const auto& t : _chapter_info)
a.append([&](bsoncxx::builder::basic::sub_document d) {
d.append(bsoncxx::builder::basic::kvp("t", std::get<0>(t)));
d.append(bsoncxx::builder::basic::kvp("i", std::get<1>(t)));
});
}));
d.append(bsoncxx::builder::basic::kvp("n", _daily_times));
}));

26 compat:
mongo::BSONArrayBuilder q;
for ( auto i : _done_quests )
q.append( BSON( "id" << i << "cv" << -1 ) );
for ( const auto& i : _quests)
q.append( BSON( "id" << i.second->id() << "cv" << i.second->cv() ) );
mongo::BSONArrayBuilder qc;
for ( const auto& t : _chapter_info )
qc.append( BSON( "t" << std::get<0>( t ) << "i" << std::get<1>( t ) ) );

mongo::BSONObj quest = mongo::BSONObjBuilder()
.append( "quests", q.arr() )
.append( "chapter", qc.arr() )
.append( "n", _daily_times )
.obj();
mongo::BSONObj res = mongo::BSONObjBuilder()
.append("quest",quest)

Which one do you prefer? The lambda way is hard to read and hard to write. I feel bsoncxx design have not consider a real usage too much. Hope it can improve the usage, some basic function is missing in Object BSON mapping

Allan Bazinet

unread,
Apr 8, 2016, 7:20:04 PM4/8/16
to mongodb-user
Not sure if this is the best solution either, but given an iterable collection, the following function template technique is what we use to embed a BSON array in a document using the stream technique.

template<typename Collection, typename Function>
bsoncxx::array::value convert(Collection const & collection,
                              Function           function)
{
    bsoncxx::builder::stream::array array;
    
    for (auto && entry : collection)
    {
        function(array, std::forward<decltype(entry)>(entry));
    }
    
    return array.extract();
}

using bsoncxx::builder::stream::document;
using bsoncxx::builder::stream::finalize;

struct Foo
{
    Foo(int a, int b) : _a{a}, _b{b} {}
    int _a;
    int _b;
};

using FooVec = std::vector<Foo>;

FooVec fooVec = { {1, 2}, {3, 4} };

const auto doc = (document{}
                  << "Array"
                  << bsoncxx::types::b_array
                  {
                      convert(fooVec,
                              [](bsoncxx::builder::stream::array & array,
                                 FooVec::value_type        const & foo)
                              {
                                  array << bsoncxx::types::b_document
                                  {
                                      (document{}
                                       << "A" << foo._a
                                       << "B" << foo._b
                                       << finalize)
                                  };
                              })
                  }
                  << finalize);


Jason Carey

unread,
Apr 11, 2016, 5:12:57 PM4/11/16
to mongod...@googlegroups.com
It's not terribly obvious, but the streaming api has a magical overload for doing this kind of subobject iteration.  Specifically, you're allowed to pass in Callable's with specific signatures, then those callable's will be invoked with the stream at the point where they were passed.

I.e.

std::vector<int> ints;
...

document{} << "Array" << open_array << [&](array_context<> ac){
    for (auto&& x : ints) {
        ac << x;
    }
} << close_array;


For an example of adapter types that wrap arbitrary iterator pairs to array/document appending.



--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/d79f5b05-16a6-4cfc-a7a7-b49de20ae6db%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

{ name     : "Jason Carey",
  title    : "Software Engineer",
  location : "New York, NY",
  twitter  : ["@MongoDB", "@MongoDBInc"],
  facebook : ["MongoDB", "MongoDB, Inc."] }

Allan Bazinet

unread,
Apr 11, 2016, 9:05:25 PM4/11/16
to mongodb-user
That is indeed quite magical.

Allan Bazinet

unread,
Apr 12, 2016, 12:41:34 AM4/12/16
to mongodb-user
In fact, it took me a while to fully comprehend the pure awesomeness of this, and frankly, my response when comparing this lambda to the, well, somewhat more convoluted example in the source tree, is along the lines of 'sheesh, well, why didn't you just tell me that in the first place'.

li ning

unread,
Apr 12, 2016, 6:05:23 AM4/12/16
to mongodb-user
In Allan Bazinet's reply, I found can do this:

doc<<bsoncxx::types::b_document{ obj.to_bson() }<<...

and

doc<<bsoncxx::types::b_array{ bsoncxx::builder::stream::array object }<<...

Now bson object can assemble, this solve my problem. For my personal view, lambda is cool but impractical.

在 2016年4月12日星期二 UTC+8上午5:12:57，Jason Carey写道：

Andrew Morrow

unread,
Apr 12, 2016, 9:04:01 AM4/12/16
to mongod...@googlegroups.com

Hi Allan -

The best answer is that we are all still learning about the right way to use the new BSON library. Unlike the older BSON library, the design of this one is much closer to that of a BSON toolkit rather than end-user library. Our expectation is that higher level abstractions will develop around it. We exposed all of the layers so that abstractions could target the layer appropriate for the needs.

We really appreciate your thoughts and comments on the driver. We have a 3.0.2 on the roadmap that includes many of the improvement suggestions and and bug reports that you have filed over the past several weeks.

Thanks,
Andrew

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.org/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user...@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.

Andrew Morrow

unread,
Apr 12, 2016, 9:08:59 AM4/12/16
to mongod...@googlegroups.com

As I mentioned to Allan in another email in this thread, the new BSON library is intended as a foundation on which you can build your own abstractions, if you don't like ours. Though it sounds like the lambda approach is not to your liking, I'm happy that you found something that works for you.

Thanks,
Andrew

Jason Carey

unread,
Apr 15, 2016, 11:32:19 AM4/15/16
to mongodb-user
The source tree example was trying to highlight the full power of the facility in a library author context, rather than for a user.

For what it's worth, almost all of that boilerplate can be stripped away with C++14:

auto make_range_array_appender = [](auto begin, auto end){
    return [begin, end](array_context<> ac) {
        for (iter = begin; iter != end; ++iter) {
            ac << *iter;
        }
    };
};

generic lambdas make everything easier
Reply all
Reply to author
Forward
0 new messages