INFO: Sniper Updates - DynamoRIO support in Sniper x86 - Issues remain for modern glibc versions (Ubuntu 22.04+)

124 views
Skip to first unread message

Trevor E. Carlson

unread,
Nov 2, 2022, 12:54:23 AM11/2/22
to snip...@googlegroups.com
All,

I’ve been trying to get Sniper working with Ubuntu 22.04 / 22.10, but have been running into issues because of how modern versions of glibc do things differently. These changes seem to break our front-end tools (SDE 9.7, Pin and Pinplay). I thought that we might be able to solve things by using DynamoRIO, but their tools are also currently not working with the new glibc versions [1,2,3]. In the end, I’d recommend that you use Ubuntu 20.04 while we work with both DynamoRIO and Pin to get things working.

Getting started with DynamioRIO on x86 is now more straight-forward than before. You can build it (not enabled by default), by setting some environment variables, and run it with a command line option.

make BUILD_DYNAMORIO=1 -j

This updated version of Sniper [4,5], built on 8.0, will automatically download and compile the dependencies for you (DynamoRIO and capstone) when enabled. If you’d like to run the DynamoRIO front-end, you can specify it this way:

cd test/fft ; make ; ../../run-sniper --frontend=dr -n 2 --roi -- ./fft -p 2

Let us know if you run into any issues.

Trevor


P.S., I recently added a rewrite rule to our Sniper website which should now redirect requests from the old Sniper Git Repo directory to the new GitHub git repo [6].




../../run-sniper --frontend=dr -n 1 -c gainestown --roi -- ./fft -p 1
[SNIPER] Start
[SNIPER] --------------------------------------------------------------------------------
[SNIPER] Sniper using SIFT/trace-driven frontend
[SNIPER] Running pre-ROI region in  CACHE_ONLY mode
[SNIPER] Running application ROI in DETAILED mode
[SNIPER] Running post-ROI region in FAST_FORWARD mode
[SNIPER] --------------------------------------------------------------------------------
[RECORD-TRACE] Using DynamoRIO frontend

FFT with Blocking Transpose
   1024 Complex Doubles
   1 Processors
   65536 Cache lines
   16 Byte line size
   4096 Bytes per page

[SNIPER] Enabling performance models
[SNIPER] Setting instrumentation mode to DETAILED
[SNIPER] Disabling performance models
[SNIPER] Leaving ROI after 2.56 seconds
[SNIPER] Simulated 1.2M instructions, 1.6M cycles, 0.74 IPC
[SNIPER] Simulation speed 456.9 KIPS (456.9 KIPS / target core - 2188.4ns/instr)
[SNIPER] Sampling: executed 46.60% of simulated time in detailed mode
[SNIPER] Setting instrumentation mode to FAST_FORWARD
[TRACE:0] -- DONE --

                 PROCESS STATISTICS
            Computation      Transpose     Transpose
 Proc          Time            Time        Fraction
    0           2294988         374652       0.16325

                 TIMING INFORMATION
Start time                        :        824196411
Initialization finish time        :        825129869
Overall finish time               :        827424857
Total time with initialization    :          3228446
Total time without initialization :          2294988
Overall transpose time            :           374652
Overall transpose fraction        :          0.16325

[SNIPER] End
[SNIPER] Elapsed time: 5.52 seconds


Optional: Run '../../tools/cpistack.py' in this directory to generate cpi-stack output for this run
Optional: Run '../../tools/mcpat.py' in this directory to generate power output for this run
Optional: Run '../../tools/dumpstats.py' in this directory to view detailed statistics for this run
Optional: Run '../../tools/gen_topology.py' in this directory to view the system topology for this run

Kleber Kruger

unread,
Nov 6, 2022, 10:31:15 AM11/6/22
to Sniper simulator

I can no longer natively compile Sniper using Ubuntu 20.04 after this update. Not even with Dockerfile-20.04 in the docker folder. It seems to me that the versions of the tools downloaded by the Sniper are incompatible with the simulator, so much so that if I updated mbuild before, the sniper would not compile.

A part of the error is described below:

In file included from /usr/include/x86_64-linux-gnu/c++/9/bits/c++config.h:528,
                 from /usr/include/c++/9/string:38,
                 from /home/kleber.kruger/sniper/sde_kit/pinkit/source/include/pin/pin.H:11,
                 from /home/kleber.kruger/sniper/sift/recorder/recorder_control.h:5,
                 from /home/kleber.kruger/sniper/sift/recorder/recorder_control.cc:1:
/usr/include/x86_64-linux-gnu/c++/9/bits/os_defines.h:44:19: error: missing binary operator before token "("
   44 | #if __GLIBC_PREREQ(2,15) && defined(_GNU_SOURCE)
      | ^
In file included from /usr/include/stdlib.h:25,
                 from /usr/include/c++/9/cstdlib:75,
                 from /usr/include/c++/9/ext/string_conversions.h:41,
                 from /usr/include/c++/9/bits/basic_string.h:6496,
                 from /usr/include/c++/9/string:55,
                 from /home/kleber.kruger/sniper/sde_kit/pinkit/source/include/pin/pin.H:11,
                 from /home/kleber.kruger/sniper/sift/recorder/recorder_control.h:5,
                 from /home/kleber.kruger/sniper/sift/recorder/recorder_control.cc:1:
/usr/include/x86_64-linux-gnu/bits/libc-header-start.h:56:17: error: missing binary operator before token "("
   56 | #if __GLIBC_USE (IEC_60559_BFP_EXT) || __GLIBC_USE (ISOC2X)
      | ^
/usr/include/x86_64-linux-gnu/bits/libc-header-start.h:73:17: error: missing binary operator before token "("
   73 | #if __GLIBC_USE (IEC_60559_FUNCS_EXT) || __GLIBC_USE (ISOC2X)
      | ^
In file included from /usr/include/c++/9/cstdlib:75,
                 from /usr/include/c++/9/ext/string_conversions.h:41,
                 from /usr/include/c++/9/bits/basic_string.h:6496,
                 from /usr/include/c++/9/string:55,
                 from /home/kleber.kruger/sniper/sde_kit/pinkit/source/include/pin/pin.H:11,
                 from /home/kleber.kruger/sniper/sift/recorder/recorder_control.h:5,
                 from /home/kleber.kruger/sniper/sift/recorder/recorder_control.cc:1:
/usr/include/stdlib.h:133:35: error: missing binary operator before token "("
  133 | #if __HAVE_FLOAT16 && __GLIBC_USE (IEC_60559_TYPES_EXT)
      | ^
/usr/include/stdlib.h:139:35: error: missing binary operator before token "("
  139 | #if __HAVE_FLOAT32 && __GLIBC_USE (IEC_60559_TYPES_EXT)

Trevor E. Carlson

unread,
Nov 6, 2022, 8:56:27 PM11/6/22
to snip...@googlegroups.com, kleber...@gmail.com
Kleber,

Can you try to compile Sniper with Pinplay instead of SDE? It seems that there are some issues with the SDE compile, but Pinplay is working fine (this is inside the Ubuntu 20.04 docker image):

$ git log --format='%h' -n 1
3bcb35b

$ grep DESC /etc/lsb-release    
DISTRIB_DESCRIPTION="Ubuntu 20.04.5 LTS”

$ make USE_PINPLAY=1 -j
Building for x86 (intel64) with Pinplay
[DOWNLO] Pin 3.18-98332
[DOWNLO] mbuild
[DOWNLO] xed
[DOWNLO] Python intel64
<...>

$ make -C test/fft
<...>
[SNIPER] Start
<...>
[TRACE:0] -- DONE --
[SIFT_RECORDER:0] ROI End

                 PROCESS STATISTICS
            Computation      Transpose     Transpose
 Proc          Time            Time        Fraction
    0               249             31       0.12450

                 TIMING INFORMATION
Start time                        :      -1844408305
Initialization finish time        :      -1844407905
Overall finish time               :      -1844407656
Total time with initialization    :              649
Total time without initialization :              249
Overall transpose time            :               31
Overall transpose fraction        :          0.12450

[SIFT_RECORDER:0:0] Recorded 1170063 (out of 1341880) instructions
[SNIPER] End
[SNIPER] Elapsed time: 4.57 seconds

Trevor

--
--
--
You received this message because you are subscribed to the Google
Groups "Sniper simulator" group.
To post to this group, send email to snip...@googlegroups.com
To unsubscribe from this group, send email to
snipersim+...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/snipersim?hl=en

---
You received this message because you are subscribed to the Google Groups "Sniper simulator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snipersim+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/snipersim/2d4ed7a1-a886-4dd3-8ac0-a2016e9f05can%40googlegroups.com.

Trevor E. Carlson

unread,
Nov 6, 2022, 9:54:38 PM11/6/22
to snip...@googlegroups.com, Kleber Kruger
Kleber (reposting to the mailing list),

Great to hear that this is working now. I’ve just pushed a new update to Sniper (56148244) to return the default to Pinplay until we fix the SDE compilation issue. I was also thinking that it might be nice to support the compilation of multiple target Pin-based frontends simultaneously (as we currently support compiling DynamoRIO at the same time). But, that is currently a low priority for now.

Trevor

On Nov 7, 2022, at 10:24 AM, Kleber Kruger <kleber...@gmail.com> wrote:

Trevor, thanks for helping me out again. Worked perfectly!

Sincerely, 
Kleber.
--
Kleber Kruger


Reply all
Reply to author
Forward
0 new messages