Austin Shafer
home contact

Announcing nvidia-drm port and Call for Testing

This post marks the beginning of shipping something that's been in the works for a very long time: a FreeBSD port of the nvidia-drm kernel module. This is exciting because it allows FreeBSD NVIDIA users to run workloads that depend on DRM to perform rendering and share buffers, most importantly Wayland compositors and PRIME rendering offload.

https://github.com/amshafer/nvidia-driver

This is also a Call for Testing: while developing this nvidia-drm port I ran a variety of workloads, but I'm sure there will still be issues as more people run nvidia-drm in scenarios I haven't tested. As such, I'm looking for members of the FreeBSD community to give it a go and let me know what they think.

As mentioned above, there are two main workload types nvidia-drm enables:

  • Wayland - compositors use DRM to flip buffers to the screen. They also use it heavily to share hardware dmabuf buffers to avoid copies. One of the most popular compositors is sway, which is available on FreeBSD and is very stable.

  • PRIME - NVIDIA Optimus refers to a machine which has an integrated GPU and a NVIDIA GPU. PRIME render offload refers to a feature where the desktop runs on the integrated GPU while certain applications run on the NVIDIA GPU. DRM is used to share buffers across these two GPUs.

If you use setups that involve one of these on NVIDIA hardware or are interested in contributing please consider helping out the project and taking nvidia-drm for a test drive.

How can I report issues?

The preferred way of reporting things is in the issues tab on git. Please make sure to include relevant information such as what caused the issue, panic summaries, system information, etc. If you just have casual questions, or want to clarify things before filing a real issue I am active in the FreeBSD Discord. Feel free to ping, the #desktop channel is probably the most appropriate.

Installation

At the moment nvidia-drm is not present in the ports tree. That will eventually change, but for now installing is done by cloning the git repository and installing from there. This can be done with:

$ cd nvidia
$ DRMKMODDIR=/path/to/drm-kmod/ BSDSRCTOP=/usr/src make
$ DRMKMODDIR=/path/to/drm-kmod/ BSDSRCTOP=/usr/src make install
# You may have to kldunload the nvidia-modeset and nvidia kernel modules here
$ kldload nvidia-drm

Please see the README for what the DRMKMODDIR and BSDSRCTOP variables are and what they default to. You may also wish to pkg lock nvidia-driver to make sure that pkg does not overwrite the driver you installed.

At this point you should have the nvidia-drm.ko kernel module loaded, and should see the /dev/dri/card0 path present. You're now ready to run DRM applications. I usually test with a Wayland desktop environment such as sway.

Please also see the Known Issues section of the README.

How does this work anyway?

NVIDIA drivers are shipped with two components: the closed source blobs and an open source platform layer. The platform layer abstracts away calls that interact with the kernel to handle running the same closed source blobs on FreeBSD and Linux. The DRM portion of the driver is an open source platform layer, so we can modify the nvidia-drm files shipped in the Linux driver to run on FreeBSD. Thanks to FreeBSD LinuxKPI compatibility layer this can be done with very minimal changes.

The reason this took so long is because we (meaning NVIDIA) still needed to build the closed source libraries with DRM support, since there's no point having a working kernel module if the userspace libraries aren't going to perform DRM ioctls. As of the 525.xx driver series, all internal changes are present and nvidia-drm.ko can fully be utilized.

Full details of how this project works are in the how to update this port documentation in git.

Supported Cards

Please note that due to the above reasons for using driver series 525, nvidia-drm only supports graphics cards which are supported by 525. The list of supported cards can be found here.

NVIDIA Optimus

Until now, on multi-GPU systems FreeBSD users have to take advantage of the nvidia-hybrid-graphics port, which uses VirtualGL to perform copies between the main X server and apps running on the NVIDIA GPU. nvidia-drm allows for "proper" PRIME support, which lets the NVIDIA driver directly handle cross-GPU buffer sharing. nvidia-drm should be more efficient than the copies VirtualGL performs.

Once you have configured your X server to handle PRIME you can offload apps to the NVIDIA GPU like so:

__NV_PRIME_RENDER_OFFLOAD=1 vkcube
__NV_PRIME_RENDER_OFFLOAD=1 __GLX_VENDOR_LIBRARY_NAME=nvidia glxinfo | grep vendor

What is the plan going forward?

The immediate goal is to get some community feedback, both from users and developers. I need to make sure that this project is usable by the average community member, is well documented, and is easy to use. Please consider giving it a spin, any feedback is welcome!

The medium-term goal is to get this into the ports tree so that it's very trivial to use. This is going to involve writing a ports Makefile, and the goal is to package the nvidia-drm files separately from the main driver. That way a user can just install the nvidia-driver package, and then also install the nvidia-drm-driver package alongside it when they want nvidia-drm.ko.

For the time being I'm going to get some user feedback, make sure things are in order on the development side, and start talking to package maintainers about what they would like a port to look like.

Acknowledgements

A special thanks to everyone who helped me with this along the way. Everyone on the NVIDIA and FreeBSD sides have been terrific, and I'm looking forward to the beginning of a fun journey.