Is it possible to run Seastar application with thread sanitizer?

[Калашников Максим]

Dear Seastar developers,

I tried to enable tsan for our project but had no success. After searching, I didn't find any mention of tsan in the seastar github repository (in the issues and the code itself).

Please tell me, have you tried using tsan?

If it can help, I could prepare a reproducible example.

With best regards,

Max

[Kefu Chai]

On Thursday, June 6, 2024 at 10:29:18 PM UTC+8 Калашников Максим wrote:

Dear Seastar developers,

Hi Max,

 

I tried to enable tsan for our project but had no success. After searching, I didn't find any mention of tsan in the seastar github repository (in the issues and the code itself).

Please tell me, have you tried using tsan?

i searched in the commit history, but didn't find any traces that the TSan had been used in this project.

 

If it can help, I could prepare a reproducible example.

this is a good question actually. but i think Seastar encourages a share-nothing design, so if a Seastar application uses primitives like std::mutex, std::condition_variable in its core, it's probably not following the paradigm. also, IIRC, address sanitizer and leak sanitizer cannot be used along with thread sanitizer. see https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html#index-fsanitize_003dthread

so if we have to pick sacrifices, i think it would be TSan. what do you think?

 

With best regards,

Max

[Калашников Максим]

We are using only seastar synchronization mechanisms, nothing from pthread or stl. 

The main reason we wanted to enable tsan was that we do have intershard communication to carry some operations to one dedicated shard and business logic on others. We aware of seastar::foreign_ptr but as we recently discover we are not using it everywhere we should use it. And there may be other problems that we have not noticed yet. To detect these problems, we tried to use tsan. Do you think this could help? Maybe there is a better way to solve problems like this?

As for sacrifices, I'm sure it's possible to build seastar with tsan as a separate library and run tests against it in a separate CI pipeline. But this only makes sense if it brings at least some help.

We had some success with running seastar application with tsan - after modifying exception_hack.cpp and thread_impl.hh files. But tsan outputs a lot of messages even in simplest examples. I guess it detects false positives and that messages can be suppressed by using suppression files. It seems that integrating tsan correctly into seastar is quite a difficult task. And it is not entirely clear whether this will bring any useful results. What do you think about this?