SAT get stuck without power spikes

[Shundong Zhou]

Hi,

There is a known issue, SAT will get stuck by power spikes. I add the flag --pause_delay > runtime, but SAT still get stuck when DIMM CECC errors are detected.

Running SAT at the beginning with high CPU/MEM usage:

 PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                               

 490660 root      20   0  0.219t 0.217t  11512 S  7612 88.4 736:01.22 sat__py_binary_  

After some time, SAT gets stuck and CPU/MEM usage are very low, but the process is still there.

gdb attach to the process:

(gdb) info threads 

  Id   Target Id         Frame 

* 1    LWP 490660 "sat__py_binary_" 0x00000000013c694d in pthread_join (threadid=139653716875008, thread_return=0x0) at pthread_join.c:92

[Shundong Zhou]

(gdb) thread apply all bt

Thread 1 (LWP 747631):

#0  0x00000000013c694d in pthread_join (threadid=139848209434368, thread_return=0x0) at pthread_join.c:92

#1  0x0000000001320bcd in Logger::StopThread() ()

#2  0x000000000130d825 in Sat::Cleanup() ()

#3  0x0000000001305c93 in stressapptest::RealMain(std::__g::function<Sat* ()>, std::__g::function<OsLayer* (std::__g::map<std::__g::basic_string<char, std::__g::char_traits<char>, std::__g::allocator<char> >, std::__g::basic_string<char, std::__g::char_traits<char>, std::__g::allocator<char> >, std::__g::less<std::__g::basic_string<char, std::__g::char_traits<char>, std::__g::allocator<char> > >, std::__g::allocator<std::__g::pair<std::__g::basic_string<char, std::__g::char_traits<char>, std::__g::allocator<char> > const, std::__g::basic_string<char, std::__g::char_traits<char>, std::__g::allocator<char> > > > >)>, int, char**) ()

#4  0x0000000000ab7340 in main ()

[Shundong Zhou]

(gdb) thread apply all bt

Thread 1 (LWP 806450):

#0  pthread_cond_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185

#1  0x00000000013c1545 in Logger::QueueLogLine(std::__u::basic_string<char, std::__u::char_traits<char>, std::__u::allocator<char> >*) ()

#2  0x00000000013c1465 in Logger::VLogF(int, char const*, __va_list_tag*) ()

#3  0x00000000013c211e in logprintf(int, char const*, ...) ()

#4  0x00000000013ae3e8 in Sat::Run() ()

#5  0x00000000013a6669 in stressapptest::RealMain(std::__u::function<Sat* ()>, std::__u::function<OsLayer* (std::__u::map<std::__u::basic_string<char, std::__u::char_traits<char>, std::__u::allocator<char> >, std::__u::basic_string<char, std::__u::char_traits<char>, std::__u::allocator<char> >, std::__u::less<std::__u::basic_string<char, std::__u::char_traits<char>, std::__u::allocator<char> > >, std::__u::allocator<std::__u::pair<std::__u::basic_string<char, std::__u::char_traits<char>, std::__u::allocator<char> > const, std::__u::basic_string<char, std::__u::char_traits<char>, std::__u::allocator<char> > > > >)>, int, char**) ()

#6  0x0000000000ad57a0 in main ()

On Monday, July 15, 2019 at 3:07:13 PM UTC-7, Shundong Zhou wrote: