Initializing seastar for using with alien::submit_to

[Immanuel Litzroth]

I'm trying to integrate seastar with an existing codebase the code I'm using is roughly this:
>>> cpp

#include <seastar/core/alien.hh>

#include <seastar/core/smp.hh>

#include <seastar/core/sleep.hh>

#include <thread>

#include <boost/log/trivial.hpp>

#include <sys/eventfd.h>

#include <seastar/core/app-template.hh>

#include <iostream>

class Seastart

{

public:

  Seastart(

    int argc,

    char** argv)

  {

    eventfd_start_ = ::eventfd(0,0);

    thread_ = std::make_unique<std::thread>(

      [this,

       argc,

       argv]

      ()

      {

        std::cout << "running app" << std::endl;

        app_.run(

          argc,

          argv,

          [this]

          ()

          {

            std::cout << "inside seastar now" << std::endl;

            using namespace std::chrono_literals;

            uint64_t buf = 1;

            ssize_t written = ::write(eventfd_start_, &buf, sizeof(uint64_t));

            std::cout << "just before sleep, written " << written

                      << " bytes to " << eventfd_start_ << std::endl;

            return seastar::sleep(10s).then(

              []

              ()

              {

                std::cout << "tired of this, exiting now" << std::endl;

                return seastar::make_ready_future<int>(0);

              });

          });

      });

    // synchronize here

    std::cout << "blocking on read from eventfd " << eventfd_start_ << std::endl;

    uint64_t buf;

    read(eventfd_start_, &buf, sizeof(buf));

    std::cout << "continuing" << std::endl;

    for(unsigned shard = 0; shard < seastar::smp::count; ++shard)

    {

      auto fut = ::seastar::alien::submit_to(

        shard,

        [shard]

        ()

        {

          std::cout << "inside alien submit " << shard << std::endl;

          return seastar::make_ready_future<>();

        });

      fut.get();

      std::cout << "got the future of " << shard << std::endl;

    }

  }

  ~Seastart()

  {

    std::cout << "joining thread" << std::endl;

    thread_->join();

  }

private:

  std::unique_ptr<std::thread> thread_;

  int eventfd_start_;

  seastar::app_template app_;

};

int

main(

  int argc,

  char** argv)

{

  Seastart the_seastart(argc,

                        argv);

}

This seems to work with when starting up with 2 or more threads but *not* with 1 thread
>> Ouput with 1 thread (args = -c 1 --default-log-level trace )

blocking on read from eventfd running app3

WARNING: debug mode. Not for benchmarking or production

DEBUG 2019-12-31 13:13:18,549 seastar - smp::count: 1

DEBUG 2019-12-31 13:13:18,549 seastar - latency_goal: 0.00075

DEBUG 2019-12-31 13:13:18,549 seastar - num_io_queues: 1

WARN  2019-12-31 13:13:18,621 seastar - Seastar compiled with default allocator, heap profiler not supported

WARN  2019-12-31 13:13:18,641 [shard 0] seastar - Unable to set SCHED_FIFO scheduling policy for timer thread; latency impact possible. Try adding CAP_SYS_NICE

DEBUG 2019-12-31 13:13:18,641 [shard 0] seastar - generate_config dev_id: 0

inside seastar now

just before sleep, written 8 bytes to 3

continuing

tired of this, exiting now
>>>

and then it just hangs...
>> Output with 2 threads (args = -c 2 --default-log-level trace)

blocking on read from eventfd running app3

WARNING: debug mode. Not for benchmarking or production

DEBUG 2019-12-31 13:12:28,429 seastar - smp::count: 2

DEBUG 2019-12-31 13:12:28,429 seastar - latency_goal: 0.00075

DEBUG 2019-12-31 13:12:28,429 seastar - num_io_queues: 2

WARN  2019-12-31 13:12:28,502 seastar - Seastar compiled with default allocator, heap profiler not supported

WARN  2019-12-31 13:12:28,510 seastar - Seastar compiled with default allocator, heap profiler not supported

DEBUG 2019-12-31 13:12:28,530 [shard 1] seastar - generate_config dev_id: 0

WARN  2019-12-31 13:12:28,530 [shard 0] seastar - Unable to set SCHED_FIFO scheduling policy for timer thread; latency impact possible. Try adding CAP_SYS_NICE

DEBUG 2019-12-31 13:12:28,530 [shard 0] seastar - generate_config dev_id: 0

inside seastar now

just before sleep, written 8 bytes to 3

continuing

inside alien submit 0

got the future of 0

inside alien submit 1

got the future of 1

joining thread

tired of this, exiting now
>>

I'm a bit in the dark why I see this behaviour. I am running inside a docker environment, not that that should matter much.

Immanuel

[Immanuel Litzroth]

This code in core/reactor.cc seems to be the culprit:

int reactor::run(){

....

  if (smp::count > 1) {

        smp_poller = poller(std::make_unique<smp_pollfn>(*this));

    }

...

Immanuel