mongocxx bsoncxx new c++11 driver: streambulder, bulding a bson array in a loop

[javantamt]

Hey there,

I'm going to build a bson in a loop.
The first snippet is a single call.
The second snipped are separate calls. In my opinion the reason why compiler says no to no 2 is, that open_array and close_array are not in the same command.

Can I do a loop in the stream builder? If not, which other possibilities are out there?
There could be a work around in building a array with the basic builder and say array.append(...) and than adding the array at once to the document. ( did't tested it yet)

Somebody a hint?
all the best!
javantamt

//working:
        fuuu << "MyArray"
            << open_array
                  << open_document << "vala" << 1 << close_document
                  << open_document << "valb" << 2 << close_document
                  << open_document << "valc" << 3 << close_document
            << close_array;
        bsoncxx::document::value fub =fuuu << finalize;

// not working
// ../src/mongoDBTreiber.cpp:173:16: error: no match for ‘operator<<’
// (operand types are ‘bsoncxx::v_noabi::builder::stream::document’ and
// ‘const bsoncxx::v_noabi::builder::stream::open_document_type’)

          fuuu << "MyArray"
              << open_array;
// my loop begin
          fuuu << open_document << "vala" << 1 << close_document;
          fuuu << open_document << "vala" << 1 << close_document;
// my loop end
          fuuu<< close_array;

[javantamt]

Hmmm does sombody has a good idea how to solve this problem?

[David Golden]

On Monday, June 6, 2016 at 10:50:38 PM UTC-4, javantamt wrote:

//working:
        fuuu << "MyArray"
            << open_array
                  << open_document << "vala" << 1 << close_document
                  << open_document << "valb" << 2 << close_document
                  << open_document << "valc" << 3 << close_document
            << close_array;
        bsoncxx::document::value fub =fuuu << finalize;

// not working
// ../src/mongoDBTreiber.cpp:173:16: error: no match for ‘operator<<’
// (operand types are ‘bsoncxx::v_noabi::builder::stream::document’ and
// ‘const bsoncxx::v_noabi::builder::stream::open_document_type’)

          fuuu << "MyArray"
              << open_array;
// my loop begin
          fuuu << open_document << "vala" << 1 << close_document;
          fuuu << open_document << "vala" << 1 << close_document;
// my loop end
          fuuu<< close_array;

 How about assigning the result of open_array to a variable to pass into the loop?

auto guuu = fuuu << "MyArray"<< open_array;

// loop begin
    guuu << open_document << "vala" << 1 << close_document;
    guuu << open_document << "vala" << 1 << close_document;
// loop end


fuuu << close_array;

Regards,

David

[Oliver S]

It is interesting, has anybody a workaround?  I think there is more which is going wrong here... it rquires  a "string" which contains like $or $and or something else otherwise you cant open_document's in loops.

[Oliver S]

I've tried everything i could, it seems not to be possible to use array_open or array_close before or after the loop. Can somebody fix this? Or has this been covered in versions after 3.0?

[David Golden]

Could you please post an SSCCE of what isn't working for you?

Thanks,

David

[Oliver S]

David, i understand your request on one side but seriously, this is a real design flaw and makes me personally feel like these concepts haven't been used and thought through! The compiler will stop with an error if close_array is not within the same "scope". It's also what he clearly stated , Loop begin / Loop end... 

Did you guys never use a "builder" statement.. .where you have an array opening and adding ... a dynamic amount of elements to it ? It's a shame i can't use mongo db like that, it's a daily thing for any programmer to do this.

Sorry about ranting... but it's a pain!

[Andrew Morrow]

The bsoncxx::builder::stream facility is intended primarily for simple one-shot inline construction of BSON. If you have more complex needs, you can use either the bsoncxx::builder::basic API, or the even more primitive bsoncxx::builder::core API.

These APIs are intended to be used as building blocks to construct a BSON generation facility that is idiomatic for your usage.

I will also note that both the bsoncxx::builder::stream and the bsoncxx::builder::basic APIs can in fact support the scenario you envision, since each can accept a generic callable as an argument to operator<< or append. I'm using C++14 in a few places here when it simplifies the typing:

With the basic builder:

        using bsoncxx::builder::basic::kvp;

        using bsoncxx::builder::basic::sub_array;

        const auto elements = {1, 2, 3};

        auto doc = builder::basic::document{};

        doc.append(

            kvp("foo", [&elements](sub_array child) {

                for (const auto& element : elements) {

                    child.append(element);

                }

            }));

        std::cout << bsoncxx::to_json(doc.view()) << "\n";

With the stream builder:

        using builder::stream::document;

        using builder::stream::open_array;

        using builder::stream::close_array;

        using builder::stream::finalize;

        const auto elements = {1, 2, 3};

        auto result = document{} << "foo" << open_array << [&elements](auto ctx) {

            for (auto&& element : elements) {

                ctx << element;

            }

        } << close_array << finalize;

        std::cout << bsoncxx::to_json(result) << "\n";

Note that the expression ctx << element discards its result, which implies that the result of appending element to ctx must yield an object with the same type as ctx, so I personally do not recommend this approach as it is fragile. The basic builder is more likely to be what you want for this sort of work.

It is also useful to see that since the callable can be anything, you can hoist the lambda into a callable type with a factory method, giving you a re-useable object which will append any sequence container:

    template<typename T>

    struct range_appender {

        const T begin;

        const T end;

        void operator()(builder::basic::sub_array array) const {

            std::for_each(begin, end, [&](const auto& value) {

                array.append(value);

            });

        }

    };

    template<typename T>

    range_appender<T> make_range_appender(T begin, T end) {

        return {begin, end};

    }

    template<typename C>

    auto make_range_appender(const C& c) {

        using std::begin;

        using std::end;

        return make_range_appender(begin(c), end(c));

    }

Using make_range_appender makes the example for the basic builder even simpler:

    using bsoncxx::builder::basic::kvp;

    const auto elements = {1, 2, 3};

    auto doc = builder::basic::document{};

    doc.append(kvp("foo", make_range_appender(elements)));

    std::cout << bsoncxx::to_json(doc.view()) << "\n";

Potentially, one could do something similar to extend the range_appender type to handle the stream scenario as well by adding an overload for operator() that takes the appropriate type. Additionally, it is almost certainly possible to make an appender object that will operate on the associative container types to generate (sub)documents, though it is important to keep in mind issues related to ordering.

The overall point is that the library is layered and extensible. If the stream builder doesn't do what you want, you can customize it by writing new types. If the type-stack aspect is interfering with that, use the basic builder, which is less specialized but similarly extensible. If neither of those represents the model that you want, start with the core builder and layer your own abstractions. Beyond that, you can always make use of libbson directly, if you need even more direct control for some reason.

I'm happy to answer any additional questions.

Thanks,

Andrew

--
You received this message because you are subscribed to the Google Groups "mongodb-user"
group.
 
For other MongoDB technical support options, see: https://docs.mongodb.com/manual/support/
---
You received this message because you are subscribed to the Google Groups "mongodb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mongodb-user+unsubscribe@googlegroups.com.
To post to this group, send email to mongod...@googlegroups.com.
Visit this group at https://groups.google.com/group/mongodb-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/mongodb-user/35bb0a1e-1991-4bb2-b2ef-00b9629481b1%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Oliver S]

Just as a reference, i think it's this issue: http://stackoverflow.com/questions/40885030/mongodb-c-failed-to-add-an-array-to-document

Using bsoncxx::builder::stream::document as type will stop the compiler, with auto it should work, i'm just trying it out now.

Am Montag, 7. November 2016 16:43:42 UTC+1 schrieb David Golden:

[David Golden]

I think it's an unfortunately accident of history that the stream builder uses the "<<" operator as it gives the impression that the original stream object is returned from the operator.  Instead, a document-context specific object (of a contextual type) is returned.  This happens so that many types of invalid documents are detected at compile time rather than run time.

Imagine instead if the "stream operator" used a less obvious operator.  E.g.:

    builder{} | "key" | "value" | open_array;

Would you conclude that the return value of that expression might be important to the continuing the chain in a subsequent loop?  I think that's more likely than with "<<".  I think with a different operator, people might be (perhaps only slightly) more naturally inclined to save the intermediate state:

    auto intermediate = builder{} | "key" | "value" | open_array;

In any case, the API is set, so the important insight is that you just need to pay attention to the intermediate contextual objects.  (Or you need to use the lambda approaches that Andrew mentioned.)

Here's a trivial example -- notice how the array context is established before the loop and closed afterwards:

using namespace bsoncxx;

int main() {
    builder::stream::document builder{};

    auto in_array = builder << "subdocs" << builder::stream::open_array;
    for (auto&& e : {1, 2, 3}) {
        in_array << builder::stream::open_document << "key" << e << builder::stream::close_document;
    }
    in_array << builder::stream::close_array;

    builder << "another_key" << 42;

    document::value doc = builder << builder::stream::finalize;

    std::cout << to_json(doc.view()) << std::endl;
    // { "subdocs" : [ { "key" : 1 }, { "key" : 2 }, { "key" : 3 } ], "another_key" : 42 }

    return EXIT_SUCCESS;
}

But as Andrew points out, this can be brittle.  There's no way to know that the builder is ready for "another_key" except by careful reasoning.  Passing a builder around for composition is therefore probably an anti-pattern, and other forms of composition are preferable for anything more that the sort of simple example shown.

Here's another example that is more careful to preserve the types all the way through as if the stream were constructed in a single statement:

    builder::stream::document builder{};

    auto in_array = builder << "subdocs" << builder::stream::open_array;
    for (auto&& e : {1, 2, 3}) {
        in_array << builder::stream::open_document << "key" << e << builder::stream::close_document;
    }
    auto after_array = in_array << builder::stream::close_array;

    after_array << "another_key" << 42;

    document::value doc = after_array << builder::stream::finalize;

I hope this explanation is helpful as an alternative to the lambda approach if your case supports it.

Regards,
David