AMDGPU-PRO on NixOS

[Matthew Robbetts]

Hi list,

Apologies if this isn’t an appropriate place for such a question, but I have a question about how to configure NixOS to use the AMDGPU-PRO driver.

Specifically, I’m trying to use the OpenCL part, with a Radeon Vega FE graphics card. (I would actually prefer to use the open-source driver if possible, but my understanding is that support for this doesn’t exist yet for this card, and won’t until linux 4.17. However, I’m having a lot of trouble understanding the structure of the open-source driver’s various pieces – I think the names they chose for everything make it super hard to Google for – and if anyone has any advice there I’d be really grateful)

By grepping around the package repo, I’ve found that AMDGPU-PRO has a package:

nixpkgs/pkgs/os-specific/linux/amdgpu-pro/default.nix

and a module:

/etc/nixos/pkgs_me_2/nixpkgs/nixos/modules/hardware/video/amdgpu-pro.nix

The module code implies to me (I’m definitely not fluent in nixlang!) that if I add “amdgpu-pro” to my config.services.xserver.drivers list, that this will enable the module and trigger everything.

However, I’ve tried this, and I’m not sure it’s working correctly. Upon reboot, both commands   dmesg -T | grep amd  and   lsmod | grep amd   are empty. Running modprobe manually does move things forward:

# modprobe amdgpu

# lsmod | grep amd
amdgpu               2097152  0
amdttm                 90112  1 amdgpu
amdkcl                 20480  1 amdgpu
drm_kms_helper        122880  2 amdgpu,i915
drm                   286720  6 amdttm,amdgpu,amdkcl,i915,drm_kms_helper
agpgart                36864  3 amdttm,intel_gtt,drm
i2c_algo_bit           16384  2 amdgpu,i915
i2c_core               53248  5 i2c_algo_bit,amdgpu,i915,drm_kms_helper,drm

after which I get:

# dmesg -T | grep amd
[Mon Feb 12 10:07:49 2018] amdkcl: loading out-of-tree module taints kernel.
[Mon Feb 12 10:07:49 2018] [drm] amdgpu kernel modesetting enabled.

So, I thought that possibly, this could be it (other than the annoyance of having to manually modprobe). But, I also get:

# clinfo
clinfo: clinfo: no version information available (required by clinfo)
clinfo: clinfo: no version information available (required by clinfo)
clinfo: clinfo: no version information available (required by clinfo)
clinfo: clinfo: no version information available (required by clinfo)
clinfo: clinfo: no version information available (required by clinfo)
clinfo: clinfo: no version information available (required by clinfo)
clinfo: clinfo: no version information available (required by clinfo)
clinfo: clinfo: no version information available (required by clinfo)
clinfo: clinfo: no version information available (required by clinfo)
clinfo: symbol lookup error: clinfo: undefined symbol: 

# opencl-info 
ERROR: clGetPlatformIDs(-1001)

So, I’m stuck! Possibly it’s actually working already, but clinfo/opencl-info aren’t good tests...

If this is an actual bug, let me know and I’ll move this into the issue tracker. I am rather assuming it’s a PEBKAC issue though… any and all help very much appreciated!

Best regards,

Matthew Robbetts

[Peter Amidon]

On Monday, February 12, 2018 at 3:53:54 PM UTC-8, Matthew Robbetts wrote:

Hi list,

Apologies if this isn’t an appropriate place for such a question, but I have a question about how to configure NixOS to use the AMDGPU-PRO driver.

Specifically, I’m trying to use the OpenCL part, with a Radeon Vega FE graphics card. (I would actually prefer to use the open-source driver if possible, but my understanding is that support for this doesn’t exist yet for this card, and won’t until linux 4.17. However, I’m having a lot of trouble understanding the structure of the open-source driver’s various pieces – I think the names they chose for everything make it super hard to Google for – and if anyone has any advice there I’d be really grateful)

Hi Matthew,

I don't have AMDGPU-PRO working with NixOS, but I do have a nixpkgs overlay that should make it easy to set up the Radeon Open Compute (ROCm) graphics stack---which I believe is supposed to be better for high-performance OpenCL-style computation, although I haven't been able to test it against AMDGPU-PRO yet.

The overlay, together with instructions on how to use it, is here: https://github.com/peter-sa/nixos-rocm.

I hope this is helpful!

Best regards,

Peter Amidon

[Matthew Robbetts]

Holy moly, many thanks for sharing this!! It looks like lovely work (and would have taken me somewhere between many weeks and forever to figure out on my own). Please pursue upstreaming this! I’m sure many people will find it useful.

Unfortunately, I’m not going to be able to try it out for a few more days, but I’ll let you know how it goes when I get there!

Thanks again,

Matt

Best regards,

Peter Amidon

--
You received this message because you are subscribed to the Google Groups "nix-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nix-devel+...@googlegroups.com.
To post to this group, send email to nix-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nix-devel/7e5e0125-4281-4b6c-9c51-d7f8f9de1712%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aeron Tynes Hammack]

Just got this working Peter. Big thanks for your work!

Haven't had a chance to run it through the ropes yet, but everything looks good at least from a diagnostic list the hardware perspective.

FYI, I'm using on a server with two EPYC 24 processor CPUs and three S9150 GPUs. I'll let you know if I hit any hiccups on this configuration or if it runs with flying colors. New to NixOS, but loving it so far.

I have two more similar servers and the idea that I can transparently provision a full OpenCL stack on each in a declarative manner is a lovely thing to contemplate.

Best,

Aeron