Sniper on RHEL 9 SDE using -s energystats

[Jake Merritt]

I recently went through a lengthy debugging process for an error that I saw posted here but not answered, and I was hoping that my work might save someone a fair bit of time. I built snipersim on a RHEL 9 virtual machine and I used the SDE frontend. When I tried to run this command:

./run-sniper \

-c /home/jmerri07/HotGauge/examples/skylake_14nm.cfg -n 8 \

-d output/test-fft \

-s scripts/energystats:40000 \

-s stop-by-icount:200000000 \

-- test/fft/fft 

I got this error:

[SNIPER] Start

[STOPBYICOUNT] Starting in ROI (detail)

[STOPBYICOUNT] Then stopping after simulating at least 200000000 instructions in detail

[SNIPER] --------------------------------------------------------------------------------

[SNIPER] Sniper using SIFT/trace-driven frontend

[SNIPER] Running full application in DETAILED mode

[SNIPER] --------------------------------------------------------------------------------

[SNIPER] End

[SNIPER] Elapsed time: 0.59 seconds

[RECORD-TRACE] Using the SDE frontend (sift/recorder)

[SIFT_RECORDER:0:0] Error:

Unable to open the output file /tmp/tmp3lh07_5k/run_benchmarks.app0.th0.sift

without -s energystats, sniper ran. 

this was my debugging trace:

1. Run-sniper does Popen starting lib/Sniper

2. This brings us to common/system/simulator.cc which sets up the sim

3. During setup (the start() function) we hit: m_stats_manager->recordStats("start");

4. recordStats func is in common/misc/stats.cc This calls: Sim()->getHooksManager()->callHooks(HookType::HOOK_PRE_STAT_WRITE, (UInt64)prefix.c_str());

5. callHooks (inside common/system/hooks_manager.cc) calls the function callPythonFunction with this line: SInt64 result = it->func(it->arg, arg);

6. This brings us to callPythonFunction in common/scripting/hooks_py.cc where I put a GIL check that returned 1

7. callPythonFunction calls PyObject_CallObject(pFunc, pArgs)

8. That calls the python function hook_pre_stat_write(self, prefix) which is in scripts/energystats.py

9. hook_pre_stat_write(self, prefix) sends us into update(), also inside energystats

10. Update() returns because sim.stats.time() == self.time_last_power = 0

11. We return back to callPythonFunction and a GIL check returns 1

12. We return back to callHooks and after this line SInt64 result = it->func(it->arg, arg); I inserted a GIL check that returns 0

13. At the end of the recordStats there are three nested for loops and inside those loops this is called: sqlite3_bind_int64(m_stmt_insert_value, 4, it3->second->recordMetric());

14. recordMetric (defined in common/misc/stats.h) calls statsCallback in common/scripting/py_stats.cc

15. We fail silently in statsCallback at the line PyObject *pResult = HooksPy::callPythonFunction(pFunc, Py_BuildValue("(sls)", objectName.c_str(), index, metricName.c_str()));

the issue was losing the GIL in step 12. I'm not sure why it gets lost, but the solution is these two edits:

1. In the functions callPythonFunction (in snipersim/common/scripting/hooks_py.cc) and statsCallback (in snipersim/common/scripting/py_stats.cc)

Add PyGILState_STATE gstate = PyGILState_Ensure(); at the very beginning

And PyGILState_Release(gstate); right before the return

2. I changed scripts/sim/__init__.py to (additions in bold)

import os, sqlite3, threading

stats.db = sqlite3.connect(

    os.path.join(config.output_dir, 'sim.stats.sqlite3'),

    check_same_thread=False,

    timeout=30.0,

)

stats.db_lock = threading.Lock()

3. I also edited this chunk of the db_delete function:
   
   This is in: scripts/sim/util.py

def db_delete(prefix, in_sim_end = False):

  global have_deleted_stats

  with sim.stats.db_lock:

    cursor = sim.stats.db.cursor()

    try:

      prefixid = sim.stats.db.execute('SELECT prefixid FROM prefixes WHERE prefixname = ?', (prefix,)).fetchall()

      if prefixid:

        cursor.execute('DELETE FROM prefixes WHERE prefixid = ?', (prefixid[0][0],))

        cursor.execute('DELETE FROM `values` WHERE prefixid = ?', (prefixid[0][0],))

      sim.stats.db.commit()

    

    except sqlite3.Error:

        sim.stats.db.rollback()

        Raise

I hope this helps! Good luck to everyone.

[Naman Jain]

Thank you for posting the fix.

I was facing the same error : "sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 128221677053504 and this is thread id 128220947019328"

Seems like sqlite3 is stricter in python3 compared to older version