Encountering (Terra) Error "unknown target CPU 'generic' " with Regent

[Markus Hunner]

Hi,

I am encountering the following error with regent:

hunner@ubuntu-desktop:~$ regent /usr/local/legion/tutorial/00_hello_world/hello_world.rg
error: unknown target CPU 'generic'
/usr/local/legion/language/terra/terra(_ZN5clang7Builtin7Context16InitializeTargetERKNS_10TargetInfoEPS3_+0xf) [0x1f38c7f]

I'm working on a fresh installation of Ubuntu 16.04 and used the docker file at "Legion/docker/Dockerfile.cuda" as a kind of guideline how to setup my working environment. This means I build llvm 3.8.1 by hand and used apt to install the other packages mentioned in this file.

I'm working with a AMD Ryzen 5 1600X CPU and I fear that older versions of llvm are not supporting the ryzen chips very well.
In the Terra issues tracker I found someone with the same error and a similar CPU: https://github.com/zdevito/terra/issues/175
So I guess this is first and foremost an issue with Terra, nevertheless I wanted to ask if anyone knows about possible leads to solve this issue.

[Michael Bauer]

I think you've likely root-caused this accurately to being a Terra issue. I don't know of anyone that's tried to run Regent code on a Ryzen CPU before. If we find anyone else that's tried it and gotten it to work we'll let you know. If you also happen to find a work-around please let us know and we'll add any patches to our fork of Terra that we maintain.

Mike

--
You received this message because you are subscribed to the Google Groups "Legion Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to legionusers+unsubscribe@googlegroups.com.
To post to this group, send email to legio...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/legionusers/9c7ac9b0-8426-4fbb-8621-c05cb71938c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Elliott Slaughter]

It should be possible to manually set the target CPU for Terra. Off the top of my head, I can't say of a general way to do this, but you can certainly do it for saveobj:

http://terralang.org/api.html#saving-terra-code

You could probably get code running searching for an example that uses regentlib.saveobj (there are several in the repository) and adding the target CPU flag there. You don't need to generate perfect code for Ryzen, you can probably run generic x86 or x86_64 code.

To view this discussion on the web visit https://groups.google.com/d/msgid/legionusers/CA%2BuQj1z%3DDyG673PAss77TyxRjbrfWo3XS4J%2BsFQmsXo4U1eDTQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Elliott Slaughter

"Don't worry about what anybody else is going to do. The best way to predict the future is to invent it." - Alan Kay

[Markus Hunner]

Unfortunately using Terra's saveobj() function with a user defined target object didn't work.

However, it set me on the right path to find a solution.

 

Adding the following lines to Terra programs allowed me to execute them:

terralib.nativetarget = terralib.newtarget {
   Triple = "x86_64-pc-linux-gnu"; -- LLVM target triple
   CPU = "x86-64";  -- LLVM CPU name,
}

Though adding the " import "regent" "-statement brought the error back again.

Eventually I was able to pinpoint the problem to the cpu name and discard the target triple as a problem.

It seems that llvm just uses the name "unknown" if it is unable to recognize the host's cpu.

hunner@ubuntu-desktop:~$ llc --version
LLVM (http://llvm.org/):
  LLVM version 3.8.1
  Optimized build.
  Built Jun  5 2018 (14:58:11).
  Default target: x86_64-pc-linux-gnu
  Host CPU: (unknown)

A list of cpu names recognized by llvm can be found by calling "llc -mattr=help". To further add to the confusion, this command even lists a available cpu target called "generic" for my system.

I found the following discussion in the Issue tracker of the Portable Computing Language project:

https://github.com/pocl/pocl/issues/512

User franz:

There was a commit to pocl's CMake to add a fallback to "generic" cpu and print a warning but it seems "generic" doesn't work at all (although llc does list it in the recognized cpu list).

So I guess something similar happens somewhere in the Terra codebase at the moment: Substituting "unknown" with the target "generic" in hopes of enabling code generation for unsupported cpus.

Here is a fix to this from the Portable Computing Language project:

https://github.com/franz/pocl/commit/8f52ac27cd94aac1f6ba22c2da55aa5f8231c436

Before the fix, they set the cpu target to "generic" when llc reported a cpu name of "unknown", causing a similar issue to the one I encountered.

As you can see, the way they choose to fix the problem was to emit an error during the building process if an unknown cpu was encountered and ask the user to provide a fitting substitution value. Maybe this could also be a easy solution for the Legion Project.

But now to my dirty quick fix:

I cloned Terra from github and added the following two lines to terra/src/tcompiler.cpp before line 309 to overwrite the assignment in line 271:

    TT->external = new Module("external",*TT->ctx);
    TT->external->setTargetTriple(TT->Triple);
    lua_pushlightuserdata(L, TT);
   
    // inserted code begins here
    if(TT->CPU == "generic")    
        TT->CPU = "x86-64";    
    // inserted code ends here
    return 1; // line 309 in the unmodified tcompiler.cpp

After this step I just built Terra with make and executed legion/language/install.py using the --with-terra flag.

As far as I can tell this allows the Terra compiler to use "valid" target information to generate generic x86_64 code.

I am now able to execute the regent example programs, as well as my own code.

Hopefully this hack can be helpful to other users facing problems with llvm not recognizing their CPU.

Markus Hunner

On Monday, June 4, 2018 at 3:32:25 PM UTC+2, Elliott Slaughter wrote:

It should be possible to manually set the target CPU for Terra. Off the top of my head, I can't say of a general way to do this, but you can certainly do it for saveobj:

http://terralang.org/api.html#saving-terra-code

You could probably get code running searching for an example that uses regentlib.saveobj (there are several in the repository) and adding the target CPU flag there. You don't need to generate perfect code for Ryzen, you can probably run generic x86 or x86_64 code.

On Mon, Jun 4, 2018 at 2:53 AM, Michael Bauer <meb...@cs.stanford.edu> wrote:

I think you've likely root-caused this accurately to being a Terra issue. I don't know of anyone that's tried to run Regent code on a Ryzen CPU before. If we find anyone else that's tried it and gotten it to work we'll let you know. If you also happen to find a work-around please let us know and we'll add any patches to our fork of Terra that we maintain.

Mike

On Sat, Jun 2, 2018 at 3:29 PM, Markus Hunner <hunner...@gmail.com> wrote:

Hi,

I am encountering the following error with regent:

hunner@ubuntu-desktop:~$ regent /usr/local/legion/tutorial/00_hello_world/hello_world.rg
error: unknown target CPU 'generic'
/usr/local/legion/language/terra/terra(_ZN5clang7Builtin7Context16InitializeTargetERKNS_10TargetInfoEPS3_+0xf) [0x1f38c7f]

I'm working on a fresh installation of Ubuntu 16.04 and used the docker file at "Legion/docker/Dockerfile.cuda" as a kind of guideline how to setup my working environment. This means I build llvm 3.8.1 by hand and used apt to install the other packages mentioned in this file.

I'm working with a AMD Ryzen 5 1600X CPU and I fear that older versions of llvm are not supporting the ryzen chips very well.
In the Terra issues tracker I found someone with the same error and a similar CPU: https://github.com/zdevito/terra/issues/175
So I guess this is first and foremost an issue with Terra, nevertheless I wanted to ask if anyone knows about possible leads to solve this issue.

--
You received this message because you are subscribed to the Google Groups "Legion Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to legionusers...@googlegroups.com.

To post to this group, send email to legio...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/legionusers/9c7ac9b0-8426-4fbb-8621-c05cb71938c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Legion Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to legionusers...@googlegroups.com.

To post to this group, send email to legio...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/legionusers/CA%2BuQj1z%3DDyG673PAss77TyxRjbrfWo3XS4J%2BsFQmsXo4U1eDTQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Elliott Slaughter]

Hi Markus,

Thanks for posting your solution.

Would you mind testing to see if this works for you (without your other modification to Terra):

terralib.nativetarget = terralib.newtarget {
   Triple = "x86_64-pc-linux-gnu"; -- LLVM target triple
   CPU = "x86-64";  -- LLVM CPU name,
}

terra.jitcompilationunit = terra.newcompilationunit(terra.nativetarget,true)

Otherwise I think the jitcompilationunit is not regenerated and will continue to use the old target.

To unsubscribe from this group and stop receiving emails from it, send an email to legionusers+unsubscribe@googlegroups.com.

To post to this group, send email to legio...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/legionusers/30ec2302-cc2c-4548-9cb4-0d29306080f4%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.