Foreign pointer motivation in source code

[Vanegicloh J]

Hello.

I'am a bit confused about the use cases of seastar::foreign_ptr in ScyllaDB or Seastar itself.

I understand what foreign pointer is and why should we use it in theory, but i can't figure out how can we use it in practice.

Let's assume we have to pass object to the other shard via foreign_ptr. It will be destroyed in it's own shard and all methods that we may call, should be called in it's own shard too (eg.: smp::submit_to(fp. get_owner_shard(), [this] { return fp->do_some_work(); }); )

What could be the motivation to pass that object to the other shard? It makes no sense, because all operations with the object we do in the owner shard.

I would be very grateful if someone help me to clear it up using the real example from the seastar or scylla src code. I tried to do it myself, exploring multishard.cc and foreign_reader class, but it's a bit complicated for a such newbee like me.

Thank you!

[Raphael S. Carvalho]

Hi,

A good example of foreign_ptr usage in ScyllaDB is when opening a
sstable that belongs to shard S at shard T. SSTable is immutable, so
the components can be easily shared and accessed by more than one
shard[1]. Now a different use case: multishard_writer is a single
producer multi consumer mechanism, where each consumer belongs to a
different shard. foreign_ptr was used there for allowing the mechanism
to store a reference to the consumer (writer) of each shard, but the
operation in the consumer indeed has to happen in the owner shard. See
multishard_writer::consume() in mutation_writer/multishard_writer.cc.

[1]: relevant sources:
https://github.com/scylladb/scylla/blob/master/sstables/sstables.hh#L479,
https://github.com/scylladb/scylla/blob/master/sstables/sstables.cc#L1417,
https://github.com/scylladb/scylla/blob/master/sstables/sstable_directory.cc#L126,
https://github.com/scylladb/scylla/blob/master/sstables/sstable_directory.cc#L233

>
> --
> You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/d4010ecd-a820-4722-ada7-59be7ef92698n%40googlegroups.com.

[Avi Kivity]

On 03/06/2022 16.27, Vanegicloh J wrote:
> Hello.
> I'am a bit confused about the use cases of seastar::foreign_ptr in
> ScyllaDB or Seastar itself.
> I understand what foreign pointer is and why should we use it in
> theory, but i can't figure out how can we use it in practice.
>
> Let's assume we have to pass object to the other shard via
> foreign_ptr. It will be destroyed in it's own shard and all methods
> that we may call, should be called in it's own shard too (eg.:
> smp::submit_to(fp. get_owner_shard(), [this] { return
> fp->do_some_work(); }); )
>
> What could be the motivation to pass that object to the other shard?
> It makes no sense, because all operations with the object we do in the
> owner shard.
>

This question is more suitable to the developer list.


In multishard_writer,  foreign_ptr is used to avoid leaks. In case the
containing object is destroyed, foreign_ptr ensures the child object is
also destroyed, and on the correct shard.

[Vanegicloh J]

Thanks for the answer.

But what kind of leaks it helps to avoid? Memory leaks?

Nadav Har'El in discussion below wrote that:

>Yes, it's not about the safety of "freeing memory" (free(), delete(), etc.) which Seastar ensures is always safe - even if called on the wrong CPU.

https://groups.google.com/g/seastar-dev/c/lqrjjDMW1zw/m/z-28CIKmBgAJ

 

[bdenes@scylladb.com]

On Mon, 2022-06-06 at 04:03 -0700, Vanegicloh J wrote:

понедельник, 6 июня 2022 г. в 11:21:47 UTC+3, Avi Kivity:

Thanks for the answer.

But what kind of leaks it helps to avoid? Memory leaks?

Nadav Har'El in discussion below wrote that:

>Yes, it's not about the safety of "freeing memory" (free(), delete(), etc.) which Seastar ensures is always safe - even if called on the wrong CPU.

https://groups.google.com/g/seastar-dev/c/lqrjjDMW1zw/m/z-28CIKmBgAJ

Destructors expect to be run on the shard the object was created on (where the constructor was ran). This, more than anything else, is the task of foreign_ptr<>. Also, it is an important marker in the code: "watch out, this object lives on another shard, handle with care".

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/7a8b2f91-77af-4f53-aea8-02c8abddc800n%40googlegroups.com.

[Benny Halevy]

seastar memory can be read-shared across shards safely, as long as the object lifetime permits so.

The thing with shared pointers' lifetimes is that they are managed by a reference counter, that, for performance reasons, is maintained by non-locking/atomic instructions, and as such, must happen on the shard of origin.

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/7a8b2f91-77af-4f53-aea8-02c8abddc800n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "ScyllaDB users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to scylladb-user...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/scylladb-users/fd42ea544ae75df444e849d3a6173a8eeeb4c211.camel%40scylladb.com.