libpmemobj++ arrays

[jann...@gmx.de]

I want to use the C++ bindings with arrays. But I am not sure that consistency is guaranteed. The documentation presents a way for allocating an array in persistent memory:

auto a = make_persistent<entry[]>(3); /* allocate an array of three entries */

Aside from allocation and deallocation, there is little documentation on arrays. Only one of the examples uses it. Consider the following snippets from pman.cpp:

persistent_ptr<field[]> board; /* current state of board */

// in a tx:
board = make_persistent<field[]>(SIZE * SIZE);

// in another tx:
board[y * SIZE + x] = f;

I would expect that the old state of the board is written to a log before it is modified. But the [] operator of a persistent pointer calls `get()` directly and uses  the returned pointer: 

operator[](std::ptrdiff_t i) const noexcept

{
    assert(i >= 0 && (i < pmem::detail::sp_extent<T>::value ||
              pmem::detail::sp_extent<T>::value == 0) &&
           "persistent array index out of bounds");


    return this->get()[i];
}

I cannot find where anything is written to the log. Can you help me spotting the missing part?

Or is it actually missing and the array's consistency has to be preserved manually?

Best Regards, 

Jana

[Eduardo Berrocal]

Dear Jana,

What do you mean by consistency? Are you referring to write atomicity? If that is the case, write atomicity for anything larger than 8 bytes needs to be manually enforce through transactional writes (otherwise, torn writes could occur). Transactions do maintain write logs in order to roll back crashed transactions.

For example, you can find those in the code you mentioned:

{
	transaction::manual tx(pop);
	intro_p->clear();
	transaction::commit();
	}

Cheers,

Eduardo.

[Eduardo Berrocal]

If what you ask is how do transactions work, that is a good question for the library developers (I am too interested to know).

How do transactions capture writes to added memory regions?

E

[Eduardo Berrocal]

Although I am not a library developer (they may respond better than me), I think the way it works is that the object adds itself to the transaction:

This is the overloaded '=' operator for persistent_ptr_base:

        /**
         * Nullptr move assignment operator.
         *
         * @throw pmem::transaction_error when adding the object to the
         *      transaction failed.
         */
        persistent_ptr_base &
        operator=(std::nullptr_t &&)
        {
                detail::conditional_add_to_tx(this);
                this->oid = {0, 0};
                return *this;
        }

[jann...@gmx.de]

Thanks for your interest. Setting up a minimal example for illustrating my question gave me hint on the solution. We should not use an ordinary data type in arrays, but always wrap them.

The source code snipped shown below is a very simple form of the board used by pman. It contains two persistent arrays. One of them demonstrates an incorrect usage.

Of interest is the main function. At first, it prints current values in freshly allocated arrays. They should be 0.

Next, a transaction is started. The values are changed, but then an abort is triggered manually. Due to the abort, the final output should show that everything was set back to 0.

Here is the interesting part. I am going to attach the whole file for a working example.

enum enum_field
{
    FREE = 0,
    WOOD
};

// pool's root object
struct Root
{
    // array of fields - this is probably wrong
    pmdk::persistent_ptr<enum_field[]> enum_board;
    // array of persistent fields - this should be correct
    pmdk::persistent_ptr<pmdk::p<enum_field>[]> p_enum_board;
};

using pool_t = pmdk::pool<Root>;

// forward declaration. creates a pool and allocats objects
void setup(pool_t& pool);

// actual interesting part
int main()
{
    pool_t pool;
    setup(pool);
   
    auto root = pool.get_root();

    // at this point, everything should be 0
    std::cout << "intially: enum_board[0]: " << root->enum_board[0] << "\n";
    std::cout << "intially: p_enum_board[0]: " << root->p_enum_board[0] << "\n";

    try
    {
        // change state
        pmdk::transaction::exec_tx(pool, [&] {
                root->enum_board[0] = WOOD;
                root->p_enum_board[0]= WOOD;

                // abort manually
                pmdk::transaction::abort(EINVAL);
        });
    } catch (pmem::manual_tx_abort e)
    {
        std::cerr << e.what() << "\n";
    }

    // everything should be back at the old state
    std::cout << "after abort: enum_board[0]: " << root->enum_board[0] << "\n";
    std::cout << "after abort: p_enum_board[0]: " << root->p_enum_board[0] << "\n";

    pool.close();
}

My output shows that the contents of the first array are not rolled back.

It is because the p<> wrapper is missing  and, correctly, nothing is logged:

intially: enum_board[0]: 0
intially: p_enum_board[0]: 0
explicit abort 22
after abort: enum_board[0]: 1
after abort: p_enum_board[0]: 0

[Piotr Balcer]

Hi Jana,
It is entirely possible that the pman example has a bug with the usage of arrays. Sorry about that.

Your code is correct. You need to wrap your array types in the p<> template to enable snapshotting.
Here's the relevant documentation:
http://pmem.io/pmdk/cpp_obj/master/cpp_html/make__persistent__array_8hpp.html

Arrays are surprisingly difficult to get right and we've spent considerable amount of time on making sure that the semantics we provide are natural and intuitive.
But I'll freely admit that we have not achieved the desired ease of use and flexibility, and to address that we intend on providing a pmem equivalent of std::array.

Piotr