InlineSkiplist::InsertConcurrently

[Dimitra Giantsidi]

Hi everyone,

In my understanding, rocksdb::InlineSkiplist supports concurrent inserts as long as the parallel keys to be inserted are not compared as equal.

I have extended the memtablerep_bench to also provide a "ParallelWrite" benchmark.

Threads are inserting to the memtable -- provided by the SkipLIstFactory using the InsertConcurrently() method.

However, there is apparently a faulty behavior -- many assertions are triggered.
(for example:

Assertion `splice->next_[i] == nullptr || compare_(x->Key(), splice->next_[i]->Key()) < 0' failed

and

internal_key.size() >= 8)

Note that I am always assuring that no equal keys are inserted and the code works as expected with 1thread!

I am attaching the method FillOne() that I have implemented.

```

void FillOne() {

// Format of an entry is concatenation of:
                                 //  key_size     : varint32 of internal_key.size()
                                 //  key bytes    : char[internal_key.size()]
                                 //  value_size   : varint32 of value.size()
                                 //  value bytes  : char[value.size()]
                                 SequenceNumber s = index;
                                 ValueType type = kTypeValue;
                                 Slice key(std::to_string(id*num_ops_*160 + index)); // id is unique per thread and num_ops_ equals to total_ops/threads_num

                                 index++;
                                 Slice value = generator_.Generate(FLAGS_item_size);
                                 uint32_t key_size = static_cast<uint32_t>(key.size());
                                 uint32_t val_size = static_cast<uint32_t>(value.size());
                                 uint32_t internal_key_size = key_size + 8;
                                 const uint32_t encoded_len = VarintLength(internal_key_size) +
                                         internal_key_size + VarintLength(val_size) +
                                         val_size;
                                 char* buf = nullptr;
                                 KeyHandle handle = table_->Allocate(encoded_len, &buf);
 

                                 char* p = EncodeVarint32(buf, internal_key_size);
                                 memcpy(p, key.data(), key_size);
                                 Slice key_slice(p, key_size);
                                 p += key_size;
                                 uint64_t packed = PackSequenceAndType(s, type);
                                 EncodeFixed64(p, packed);
                                 p += 8;
                                 p = EncodeVarint32(p, val_size);
                                 memcpy(p, value.data(), val_size);
                                 assert((unsigned)(p + val_size - buf) == (unsigned)encoded_len);
                                 table_->InsertConcurrently(handle);
}


I would really appreciate some help on this!

Thank you all in advance for your time,

Best,
Dimitra

[Shubham Vashisth]

Hi Dimitra,

I face the  issue, were you able to resolve this?

Best,
Shubham

[Dimitra Giantsidi]

Hi  Shubham,

I was able to solve the issue but I cannot recall how anymore ..

Maybe try the most up-to-date rocksdb version?

Best,
Dimitra

--
You received this message because you are subscribed to the Google Groups "rocksdb" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rocksdb+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/rocksdb/eb642077-c2e9-48d4-811e-049750f4bf45n%40googlegroups.com.

--

Giantsidi Dimitra

Research Scientist at The University of Edinburgh, UK

e-mail:dimitra....@gmail.com

site: https://dgiantsidi.github.io/

[Shubham Vashisth]

Hi Dimitra,

Thank you for replying. The problem still persists in the latest RocksDB version. Do you remember if it’s the benchmark that needs to updated or the skiplist itself?

Thank you very much! 

Best,
Shubham

[Dimitra Giantsidi]

Hi,

I think we eventually changed the benchmark :-) The skiplist was already a bit modified in our case but the error had nothing to do with this modification. Sorry I cannot be of any help but I have not even access to this code anymore.

Best,
Dimitra

[Shubham Vashisth]

No worries, thank you for replying.