Austin Shafer
home contact

Configuring PRIME in 2023 (on FreeBSD)

There's essentially two ways to configure PRIME setups on X: manually and automatically. Manually usually means running nvidia-xconfig with the -prime argument and correcting it if needed (so not totally "manual"). In my opinion the better way of configuring PRIME is by using the X server's auto-configuration, by not specifying an xorg.conf at all.

This article covers auto-configuration and, since this is motivated by nvidia-drm.ko being available on FreeBSD, a number of FreeBSD-specific PRIME setup answers. This will be a bit of dive into what PRIME is, since there's plenty of out-of-date information and most of it is new to the FreeBSD community.

If you'd like to skip the explanation and speed-read an actual config, then skip to the "Full Configuration" section at the end.

Terminology

  • iGPU - shorthand for integrated GPU. This is the higher efficiency GPU, usually Intel or AMD.
  • dGPU - shorthand for discrete GPU. This is the higher power GPU, usually NVIDIA or sometimes AMD.
  • PRIME - refers to the technology for running a display server and applications on hybrid graphics machines (one iGPU and one dGPU).
  • PRIME Provider - a GPU in the hybrid graphics system that supports sharing with PRIME.
  • NVIDIA Optimus - This is the branding that many laptops using hybrid graphics are sold under. It refers to the physical device itself.
  • X driver - this refers to the <vendor>_drv.so binary that is a bit of vendor-specific code that runs in the X server. These are usually found somewhere like /usr/local/lib/xorg/modules/drivers/. They are technically part of the device driver for your card, a userspace component that helps drive the display server.

FreeBSD-specific Setup

To use PRIME on FreeBSD you'll need a couple things:

PRIME

PRIME is a feature for solving the problem of presentation in hybrid graphics environments. In an Optimus laptop there are two GPUs, but (normally) only the iGPU is connected to the internal display. This means that when the desktop/applications are rendered on the dGPU they need to find a way to be shared with the iGPU to be displayed. Likewise, the external display ports are only connected to the dGPU, so displaying anything on a second monitor requires possibly sharing iGPU data with the dGPU.

As covered here there are two methods for using PRIME to present across the iGPU and dGPU: Output and Offload. Output is where everything happens on the dGPU and the iGPU presents it, Offload is where everything happens on the iGPU except for specific high-performance tasks running on the dGPU.

When configured (more on that later), there will be two "Providers" present that are advertising support for PRIME. You can check this with the following command:

$ xrandr --listproviders

You can also manually change the current PRIME setup with xrandr, for example sometimes when configuring manually you still have to tell the X server that it should be running in Output mode and specify the dGPU as the source:

$ xrandr --setprovideroutputsource modesetting NVIDIA-0

The above tells the X server to run in "Output" mode, running everything on the NVIDIA-0 provider and presenting from the modesetting provider. These names come from the --listproviders call. Without the above command the X server doesn't know to use PRIME to share the dGPU desktop through the iGPU so the screen is black.

How Auto-configuration works

In its most basic form the X server is configured by statically specifying all devices present in a system in an xorg.conf file. If you do not specify an xorg.conf, then the X server will look in the xorg.conf.d directory for additional configurations. This is where the OutputClass X configuration section comes into play.

Instead of manually specifying a device and assigning a driver, we let the X server detect which driver corresponds to a device and then assign a config rule to it. This is basically what the OutputClass section does, it specifies a rule to be applied if the driver in MatchDriver is in use, and allows assigning an actual X driver to it.

Driver detection is done through platform-specific code in the X server detecting DRM nodes present on a system. This is why earlier in this article we talked about needing an up-to-date Xorg which contains the FreeBSD platform change.

Why does this matter for PRIME? Well in hybrid GPU systems we would normally have to check which is the iGPU, which is the dGPU, create entries for both, mark one of them as inactive, etc. This is actually what nvidia-xconfig -prime does for you. It can usually work but I find a high percentage of the time it's very brittle and when things go wrong they get confusing very quickly.

Auto-configuring is much easier and allows the package manager to install the necessary files and the user to simply run X. It is also the method currently recommended by the NVIDIA README. Notice that older driver's READMEs mention manual PRIME setup but these days auto-configuration is the recommended way.

As an example let's look at the most trivial setup for PRIME:

Section "OutputClass"
    Identifier "intel"
    MatchDriver "i915"
    Driver "modesetting"
EndSection

Section "OutputClass"
    Identifier "nvidia"
    MatchDriver "nvidia-drm"
    Driver "nvidia"
    Option "AllowEmptyInitialConfiguration"
    Option "PrimaryGPU" "yes"
EndSection

The above is an example of something that would be placed in /usr/local/etc/X11/xorg.conf.d/, it defines two output classes: one for the iGPU and one for the dGPU. In this case the Intel OutputClass is the iGPU, and when a device being driven by i915 appears the modesetting X driver will be initialized for it. You can also add files to /etc/X11/xorg.conf.d/, which is useful for when files like this are installed by a package manager automatically.

The PrimaryGPU option is also important, it defines which GPU the desktop itself will run on. If the dGPU is set as the primary (as it is above) then PRIME Output is configured, since everything is always running on the dGPU. If the iGPU is set as the primary then PRIME Render Offload is configured, since everything runs on the iGPU unless the user explicitly requests an app to run on the dGPU.

Wait but what is this nvidia-prime tool I keep reading about?

nvidia-prime and prime-select are essentially tools for automatically configuring the auto-configurer. It sounds confusing, but basically those tools define files in xorg.conf.d to specify the primary device that the user requested. This keeps the user from ever having to deal with the specifics, they can just run the tool to set their desired primary card and then reboot.

If the user does something like prime-select intel then the intel.conf file in xorg.conf.d will be updated to have the PrimaryGPU option enabled. And vice versa. nvidia-prime does the same but is accessible through the NVIDIA Control Panel, allowing GUI configuration.

Full Configuration

Okay enough learning, time to actually set up an NVIDIA Optimus machine. In this case I'll be running on a laptop with an AMD iGPU and an NVIDIA dGPU.

I'm going to start by making sure that I have the following files in /usr/local/etc/X11/xorg.conf.d/ to allow autoconfiguration to take place:

NVIDIA OutputClass:

$ cat /usr/local/etc/X11/xorg.conf.d/20-nvidia-drm-outputclass.conf
Section "OutputClass"
    Identifier "nvidia"
    MatchDriver "nvidia-drm"
    Driver "nvidia"
    ModulePath "/usr/local/lib/nvidia/xorg"
    ModulePath "/usr/local/lib/xorg/modules"
EndSection

AMD OutputClass:

$ cat /usr/local/etc/X11/xorg.conf.d/20-amdgpu.conf
Section "OutputClass"
    Identifier "AMD"
    MatchDriver "amdgpu"
    Driver "amdgpu"
    Option "PrimaryGPU" "yes"
EndSection

Note that in this case we are making the iGPU the primary, since you probably want to save some battery by using the more power efficient GPU.

If your machine has an Intel card, then you'll also need the Intel OutputClass from earlier:

$ cat /usr/local/etc/X11/xorg.conf.d/20-intel.conf
Section "OutputClass"
    Identifier "intel"
    MatchDriver "i915"
    Driver "modesetting"
    Option "PrimaryGPU" "yes"
EndSection

Also, at this point you'll need to make sure you don't have any xorg.conf files in /etc/X11/ or /usr/local/etc/X11/. You may have to create /usr/local/etc/X11/xorg.conf.d/ yourself if it doesn't already exist.

As we covered in the previous sections you should define the PrimaryGPU Option in the OutputClass of the GPU that you like to enable as the primary. You should only define that option in one of the outputclasses in use.

What about configuring other things in X like input devices?

Graphics cards aren't the only thing that can be autoconfigured, anything can. You'll just add a configuration file to xorg.conf.d like we did for the GPUs.

Here's an example InputClass for keyboards from the 40-libinput.conf file shipped with Linux distros:

Section "InputClass"
        Identifier "libinput keyboard catchall"
        MatchIsKeyboard "on"
        MatchDevicePath "/dev/input/event*"
        Driver "libinput"
EndSection

A full example 40-libinput.conf can be found here.

More details can be found in the INPUTCLASS section in the xorg.conf(5) manpage.

You can add files like this to customize anything additional in X that you would have normally put into xorg.conf.

Next steps for PRIME straight out of the box on FreeBSD

Thanks to everyone who has given nvidia-drm a test on FreeBSD! You have all been extremely helpful with reporting issues and asking questions, I really appreciate everyone's support. Please continue to file issues for anything you find.

The main two things that FreeBSD needs to continue with are:

  1. waiting for an up-to-date release of Xorg that contains the necessary patch.
  2. Adding default xorg.conf.d files that define OutputClasses

Point 1 can be worked around by adding this commit as a port-local patch in the xorg-server port. Point 2 likewise just involves installing the necessary files to xorg.conf.d when the xorg-server port is installed, which should be pretty trivial.

With these autoconfiguration files installed through ports a user could simply use pkg to install X, nvidia-drm, etc, start X, and have it all work.

References