Configuring PRIME in 2023 (on FreeBSD)
There's essentially two ways to configure PRIME setups on X: manually and
automatically. Manually usually means running nvidia-xconfig
with the
-prime
argument and correcting it if needed (so not totally "manual"). In my
opinion the better way of configuring PRIME is by using the X server's
auto-configuration, by not specifying an xorg.conf
at all.
This article covers auto-configuration and, since this is motivated by nvidia-drm.ko being available on FreeBSD, a number of FreeBSD-specific PRIME setup answers. This will be a bit of dive into what PRIME is, since there's plenty of out-of-date information and most of it is new to the FreeBSD community.
If you'd like to skip the explanation and speed-read an actual config, then skip to the "Full Configuration" section at the end.
Terminology
- iGPU - shorthand for integrated GPU. This is the higher efficiency GPU, usually Intel or AMD.
- dGPU - shorthand for discrete GPU. This is the higher power GPU, usually NVIDIA or sometimes AMD.
- PRIME - refers to the technology for running a display server and applications on hybrid graphics machines (one iGPU and one dGPU).
- PRIME Provider - a GPU in the hybrid graphics system that supports sharing with PRIME.
- NVIDIA Optimus - This is the branding that many laptops using hybrid graphics are sold under. It refers to the physical device itself.
- X driver - this refers to the
<vendor>_drv.so
binary that is a bit of vendor-specific code that runs in the X server. These are usually found somewhere like/usr/local/lib/xorg/modules/drivers/
. They are technically part of the device driver for your card, a userspace component that helps drive the display server.
FreeBSD-specific Setup
To use PRIME on FreeBSD you'll need a couple things:
- nvidia-drm - The nvidia-drm kernel module.
- Also available as a port now, although it's not yet in-tree.
- Note that you'll have to install the
nvidia-driver
port from this repo as well.
- An X server with commit Add DRM platform for BSD
- This is easiest done by installing the X server from gitlab.
libudev-devd
version0.5.1
. This is necessary for X autoconfiguration to get udev properties for DRM devices.
PRIME
PRIME is a feature for solving the problem of presentation in hybrid graphics environments. In an Optimus laptop there are two GPUs, but (normally) only the iGPU is connected to the internal display. This means that when the desktop/applications are rendered on the dGPU they need to find a way to be shared with the iGPU to be displayed. Likewise, the external display ports are only connected to the dGPU, so displaying anything on a second monitor requires possibly sharing iGPU data with the dGPU.
As covered here there are two methods for using PRIME to present across the iGPU and dGPU: Output and Offload. Output is where everything happens on the dGPU and the iGPU presents it, Offload is where everything happens on the iGPU except for specific high-performance tasks running on the dGPU.
When configured (more on that later), there will be two "Providers" present that are advertising support for PRIME. You can check this with the following command:
$ xrandr --listproviders
You can also manually change the current PRIME setup with xrandr
, for example
sometimes when configuring manually you still have to tell the X server that
it should be running in Output mode and specify the dGPU as the source:
$ xrandr --setprovideroutputsource modesetting NVIDIA-0
The above tells the X server to run in "Output" mode, running everything on the
NVIDIA-0
provider and presenting from the modesetting
provider. These names
come from the --listproviders
call. Without the above command the X server
doesn't know to use PRIME to share the dGPU desktop through the iGPU so the screen
is black.
How Auto-configuration works
In its most basic form the X server is configured by statically
specifying all devices present in a system in an xorg.conf
file. If you do
not specify an xorg.conf
, then the X server will look in the xorg.conf.d
directory for additional configurations. This is where the OutputClass
X
configuration section comes into play.
Instead of manually specifying a device and assigning a driver, we let the X
server detect which driver corresponds to a device and then assign a config
rule to it. This is basically what the OutputClass
section does, it specifies
a rule to be applied if the driver in MatchDriver
is in use, and allows
assigning an actual X driver to it.
Driver detection is done through platform-specific code in the X server detecting DRM nodes present on a system. This is why earlier in this article we talked about needing an up-to-date Xorg which contains the FreeBSD platform change.
Why does this matter for PRIME? Well in hybrid GPU systems we would normally
have to check which is the iGPU, which is the dGPU, create entries for both,
mark one of them as inactive, etc. This is actually what nvidia-xconfig -prime
does for you. It can usually work but I find a high percentage of the
time it's very brittle and when things go wrong they get confusing very
quickly.
Auto-configuring is much easier and allows the package manager to install the necessary files and the user to simply run X. It is also the method currently recommended by the NVIDIA README. Notice that older driver's READMEs mention manual PRIME setup but these days auto-configuration is the recommended way.
As an example let's look at the most trivial setup for PRIME:
Section "OutputClass"
Identifier "intel"
MatchDriver "i915"
Driver "modesetting"
EndSection
Section "OutputClass"
Identifier "nvidia"
MatchDriver "nvidia-drm"
Driver "nvidia"
Option "AllowEmptyInitialConfiguration"
Option "PrimaryGPU" "yes"
EndSection
The above is an example of something that would be placed in
/usr/local/etc/X11/xorg.conf.d/
, it defines two output classes: one for the
iGPU and one for the dGPU. In this case the Intel OutputClass
is the iGPU,
and when a device being driven by i915 appears the modesetting
X driver will
be initialized for it. You can also add files to /etc/X11/xorg.conf.d/
, which
is useful for when files like this are installed by a package manager
automatically.
The PrimaryGPU
option is also important, it defines which GPU the desktop
itself will run on. If the dGPU is set as the primary (as it is above) then
PRIME Output is configured, since everything is always running on the dGPU.
If the iGPU is set as the primary then PRIME Render Offload is configured,
since everything runs on the iGPU unless the user explicitly requests an app to
run on the dGPU.
Wait but what is this nvidia-prime tool I keep reading about?
nvidia-prime
and prime-select
are essentially tools for automatically
configuring the auto-configurer. It sounds confusing, but basically those tools
define files in xorg.conf.d
to specify the primary device that the user
requested. This keeps the user from ever having to deal with the specifics, they
can just run the tool to set their desired primary card and then reboot.
If the user does something like prime-select intel
then the intel.conf
file
in xorg.conf.d
will be updated to have the PrimaryGPU
option enabled. And
vice versa. nvidia-prime
does the same but is accessible through the NVIDIA
Control Panel, allowing GUI configuration.
Full Configuration
Okay enough learning, time to actually set up an NVIDIA Optimus machine. In this case I'll be running on a laptop with an AMD iGPU and an NVIDIA dGPU.
I'm going to start by making sure that I have the following files in
/usr/local/etc/X11/xorg.conf.d/
to allow autoconfiguration to take place:
NVIDIA OutputClass
:
$ cat /usr/local/etc/X11/xorg.conf.d/20-nvidia-drm-outputclass.conf
Section "OutputClass"
Identifier "nvidia"
MatchDriver "nvidia-drm"
Driver "nvidia"
ModulePath "/usr/local/lib/nvidia/xorg"
ModulePath "/usr/local/lib/xorg/modules"
EndSection
AMD OutputClass
:
$ cat /usr/local/etc/X11/xorg.conf.d/20-amdgpu.conf
Section "OutputClass"
Identifier "AMD"
MatchDriver "amdgpu"
Driver "amdgpu"
Option "PrimaryGPU" "yes"
EndSection
Note that in this case we are making the iGPU the primary, since you probably want to save some battery by using the more power efficient GPU.
If your machine has an Intel card, then you'll also need the Intel
OutputClass
from earlier:
$ cat /usr/local/etc/X11/xorg.conf.d/20-intel.conf
Section "OutputClass"
Identifier "intel"
MatchDriver "i915"
Driver "modesetting"
Option "PrimaryGPU" "yes"
EndSection
Also, at this point you'll need to make sure you don't have any xorg.conf
files in /etc/X11/
or /usr/local/etc/X11/
. You may have to create
/usr/local/etc/X11/xorg.conf.d/
yourself if it doesn't already exist.
As we covered in the previous sections you should define the PrimaryGPU
Option in the OutputClass
of the GPU that you like to enable as the primary.
You should only define that option in one of the outputclasses in use.
What about configuring other things in X like input devices?
Graphics cards aren't the only thing that can be autoconfigured, anything can.
You'll just add a configuration file to xorg.conf.d
like we did for the GPUs.
Here's an example InputClass
for keyboards from the 40-libinput.conf
file
shipped with Linux distros:
Section "InputClass"
Identifier "libinput keyboard catchall"
MatchIsKeyboard "on"
MatchDevicePath "/dev/input/event*"
Driver "libinput"
EndSection
A full example 40-libinput.conf
can be found
here.
More details can be found in the INPUTCLASS
section in the xorg.conf(5)
manpage.
You can add files like this to customize anything additional in X that you
would have normally put into xorg.conf
.
Next steps for PRIME straight out of the box on FreeBSD
Thanks to everyone who has given nvidia-drm a test on FreeBSD! You have all been extremely helpful with reporting issues and asking questions, I really appreciate everyone's support. Please continue to file issues for anything you find.
The main two things that FreeBSD needs to continue with are:
- waiting for an up-to-date release of Xorg that contains the necessary patch.
- Adding default
xorg.conf.d
files that define OutputClasses
Point 1 can be worked around by adding
this
commit as a port-local patch in the xorg-server
port. Point 2 likewise just
involves installing the necessary files to xorg.conf.d
when the xorg-server
port is installed, which should be pretty trivial.
With these autoconfiguration files installed through ports a user could simply
use pkg
to install X, nvidia-drm, etc, start X, and have it all work.