Running Python 3 programs on Sniper 7.2 using SIFT traces generated with Pin tool

[Shailja Pandey]

Hello everyone,

I am trying to simulate Python 3 programs which use PyTorch deep learning libraries on Sniper 7.2. Currently I am doing the simulation by giving Python source code as input to Sniper. I use a Makefile which contains the below command -

../../run-sniper -v -n 1 -c gainestown -- python xyz.py -p 1

The simulation fails if the python 3 program xyz.py uses some python libraries that are not compatible with Python 2.7 (which Sniper 7.2 uses).

Next, I am thinking to use a trace of xyz.py generated by pin tool as input to Sniper (instead of python source code as input).  If this is feasible, can we use Python 3 (and deep learning libraries compatible with Python 3) for writing the application while Python 2 is used by Sniper?

I have come across a post on this group where Wim mentions that "since Sniper 7 and the changes needed to support Pin 3, Sniper is no longer a pintool but a standalone regular Linux executable (sniper/lib/sniper) while the pintool is much smaller (sniper/lib/sift_recorder) and contains just some interfacing code." Link to this post - https://groups.google.com/forum/#!searchin/snipersim/tensorflow%7Csort:date/snipersim/aCNP1-o8AYg/6TLWfdt3AQAJ

I am not sure if we can generate and use a trace using pin tool and use it to perform simulation on Sniper 7.2.  Also, can we successfully generate traces for a Python program using Pin tool?

Please let me know if this would work and also some pointers on how to do this.

Any leads would be appreciated.

Thanks,

Shailja

[john....@bsc.es]

Hi Shailja,

I don't know if this helps you, but in my case, I am using Sniper 7.3 and PyTorch 1.4 (Python 3). And it is mainly working. For instance, I can launch PyTorch code with Sniper to do inference using AlexNet or ResNet networks and works as expected. The only difference, in this case, is that I'm using Sniper 7.3 and the PyTorch version I use is entirely compiled from sources.

I have some issues simulating some RNN using torchtext and spacy libraries, the simulation runs but never get into ROI for instance.

I hope this helps you.

Have a good day,

_____

John

[Shailja Pandey]

Hi John,

That seems to be a very useful input about Sniper version and compiling PyTorch from sources. Thanks!

I am also interested in simulating AlexNet and other CNNs. I have been working with Sniper 7.2 and installed PyTorch using Conda. I will try and run AlexNet as you suggested and see how it goes.

Thanks,

Shailja

[Kwang Eun Byun]

Hi shail.

I'm having a similar problem with you.

I trying to simulate conda environment python3  program which use Pytorch too.

but, python version of sniper-sim is 2.7 and pytorch is 3.8.

so, i use "conda deactivate" and try running pytorch code, of course, the package cannot be found and an error occurs.

I search solution like me, and found this post.

If you solving your problem, Can you tell me how you solved the problem?

Thanks.

2020년 7월 23일 목요일 오후 7시 16분 26초 UTC+9에 shail...@gmail.com님이 작성:

[Shailja Pandey]

Hi Kwang,

This error of python version mismatch between Sniper and Pytorch persists. As far as I remember, you should try switching between python 2 and 3 using update-alternatives. Once Sniper binary is generated using python_kit that Sniper recommends, you can set your system's python to be python3 and run Pytorch programs and use pytorch packages. I was able to use basic pytorch packages such as import torch etc, but I did not go further into using PyTorch with Sniper so I am not sure if everything will go smooth when you go deeper into it.

Thanks,

Shailja

--
--
--
You received this message because you are subscribed to the Google
Groups "Sniper simulator" group.
To post to this group, send email to snip...@googlegroups.com
To unsubscribe from this group, send email to
snipersim+...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/snipersim?hl=en

---
You received this message because you are subscribed to the Google Groups "Sniper simulator" group.
To unsubscribe from this group and stop receiving emails from it, send an email to snipersim+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/snipersim/335a11fd-a0e2-44d4-8b10-f7ff4dfe1208n%40googlegroups.com.

[Joe Joe]

Hi John,

I am new with sniper and currently trying to combine Pytorch or Tensorflow or Keras with sniper. 
I also tried to build packages from soucse using python setup.py install. However I can't find any executable python program in folder ./pyhton_kit/ . Could you please tell me how did you build it? 

Thank you very much!

Best,

Joe