Linux, AMD Ryzen 3900X, X570, NVIDIA GTX 1060, AMD 5700XT, Looking Glass, PCI Passthrough and Windows 10

2019-10-18 2019-10-18 1644 words 8 minutes

Contents

Now that Windows is installed we can change a few hardware details in libvirt. Back in your VM settings you can remove the SATA CDROM 1/2/3 now. Also Display Spice and afterwards Video QXL needs to be removed (otherwise you won’t see and graphics card output from the NVIDIA graphics card). I also removed USB Redirector 1/2:

Now the Nvidia GPU + HD Audio controller that we isolated in the former parts can be finally passed through. Click Add hardware, select PCI Host Device on the left side. This shows a list of all PCI devices. In my case I select the NVIDIA GeForce GTX 1060 6GB:

Now the same procedure for the High Definition Audio Controller:

To make the NVIDIA graphics driver work later there is one more thing to do. Since version 337.88, NVIDIA drivers on Windows check if an hypervisor is running and fail if it detects one, which results in an Error 43 in the Windows device manager. Starting with QEMU 2.5.0 and libvirt 1.3.3, the vendor_id for the hypervisor can be spoofed, which is enough to fool the NVIDIA drivers into loading anyway. All one must do is add hv_vendor_id=1234567890ab by adding the following line to their libvirt domain configuration. Click on Overview in the VM settings on the left and then the XML tab. Add the line <vendor_id state='on' value='1234567890ab'/> in the <hyperv> section accordingly and click Apply:

We also need

xml

<kvm>
  <hidden state='on'/>
</kvm>

in the features section:

I had an old Apple USB keyboard and mouse around (alternative way see below). So I connected them to my USB hub and passed it through to the VM. Having a dedicated keyboard and mouse may reduce latency a bit which might become important for gaming. But I’ve figured out that it’s not really needed as my normal keyboard/mouse passed through via evdev works also very well (see next session). But for a quick test passing through a USB keyboard and mouse is quite handy. So click Add hardware. Select USB Host Device on the left side and select the keyboard you want to pass through:

And the same needs to be done for the mouse:

If you don’t have a spare keyboard and mouse passing your keyboard and mouse that you use for Linux via evdev is also possible (you can do this even additionally if you already have configured a spare USB keyboard or mouse). If you don’t want to do this just skip everything until we passing the sound card. So to make this work it’s important to close virt-manager and quit it (It may be also visible in the system tray. Quit it there too.). We need to change the XML configuration file for the Windows VM with the virsh command now. virt-manager doesn’t support the changes we need. You may also not be able to edit the VM configuration with virt-manger anymore in the future as it would remove the settings we add now. But you can still use virt-manager to start and monitor the Windows VM.

So be sure to make a backup of the XML configuration now. By default the files are in /etc/libvirt/qemu directory. Since I called my VM Windows10 the configuration file is now /etc/libvirt/qemu/Windows10.xml accordingly.

First we need the keyboard and mouse device. You find them in /dev/input/by-id/ directory. In my case it looks like this:

bash

ls -al /dev/input/by-id/
total 0
drwxr-xr-x 2 root root 280 Oct 24 21:30 .
drwxr-xr-x 4 root root 720 Oct 24 21:30 ..
lrwxrwxrwx 1 root root  10 Oct 23 20:50 usb-041e_30d3_150413000592-event-if03 -> ../event18
lrwxrwxrwx 1 root root  10 Oct 23 20:50 usb-046d_HD_Pro_Webcam_C920_48ED9A6F-event-if00 -> ../event19
lrwxrwxrwx 1 root root  10 Oct 24 21:30 usb-Apple__Inc_Apple_Keyboard-event-if01 -> ../event26
lrwxrwxrwx 1 root root  10 Oct 24 21:30 usb-Apple__Inc_Apple_Keyboard-event-kbd -> ../event25
lrwxrwxrwx 1 root root   9 Oct 23 20:50 usb-Logitech_USB_Receiver-if02-event-mouse -> ../event2
lrwxrwxrwx 1 root root   9 Oct 23 20:50 usb-Logitech_USB_Receiver-if02-mouse -> ../mouse0
lrwxrwxrwx 1 root root  10 Oct 23 20:50 usb-Microsoft_Microsoft_Wireless_Optical_Desktop®_1.00-event-if01 -> ../event22
lrwxrwxrwx 1 root root  10 Oct 23 20:50 usb-Microsoft_Microsoft_Wireless_Optical_Desktop®_1.00-event-kbd -> ../event20
lrwxrwxrwx 1 root root  10 Oct 23 20:50 usb-Microsoft_Microsoft_Wireless_Optical_Desktop®_1.00-if01-event-mouse -> ../event21
lrwxrwxrwx 1 root root   9 Oct 23 20:50 usb-Microsoft_Microsoft_Wireless_Optical_Desktop®_1.00-if01-mouse -> ../mouse1
lrwxrwxrwx 1 root root  10 Oct 24 21:30 usb-Mitsumi_Electric_Apple_Optical_USB_Mouse-event-mouse -> ../event24
lrwxrwxrwx 1 root root   9 Oct 24 21:30 usb-Mitsumi_Electric_Apple_Optical_USB_Mouse-mouse -> ../mouse3

Since I’ve a Logitech mouse one of the two usb-Logitech_USB_Receiver-if02* entries are of interest. To find out which one you need open a terminal window. Execute cat /dev/input/by-id/usb-Logitech_USB_Receiver-if02-event-mouse and move your mouse. If you see some cryptic output then it’s the one we need ;-) In my case both entries produced this cryptic output but it turned out that both work so I just picked one. For now just copy the whole path /dev/input/by-id/... somewhere as we need it shortly.

You do basically the same for the keyboard. But here you need two terminal windows. In one execute the cat /dev/input/by-id/... command and in the other see again if some cryptic output shows up if you type something. Also copy the path somewhere.

Now run sudo virsh edit [vmname] (change [vmname] accordingly of course). Then we need to replace the first line

xml

<domain type='kvm'>

with

xml

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

After </devices> (this closes the </devices> section) add this (of course replace the path to your mouse input-linux,id=mouse1,evdev=/dev/input/by-id/... and keyboard input-linux,id=kbd1,evdev=/dev/input/by-id/... accordingly):

xml

<qemu:commandline>
  <qemu:arg value='-object'/>
  <qemu:arg value='input-linux,id=mouse1,evdev=/dev/input/by-id/usb-Logitech_USB_Receiver-if02-event-mouse'/>
  <qemu:arg value='-object'/>
  <qemu:arg value='input-linux,id=kbd1,evdev=/dev/input/by-id/usb-Microsoft_Microsoft_Wireless_Optical_Desktop®_1.00-event-kbd,grab_all=on,repeat=on'/>
</qemu:commandline>

In part 2 we already added our user to the input and kvm group. That’s important now so maybe verify it again.

Next we need to include this devices to our QEMU config /etc/libvirt/qemu.conf. You might need sudo vi /etc/libvirt/qemu.conf to be able to edit it. Make sure that you have this settings (of course replace username with your real username):

plain

user = "username"
group = "kvm"

And further (of course again replace the path to your mouse and keyboard in the two /dev/input/by-id/... lines accordingly):

plain

cgroup_device_acl = [
    "/dev/kvm",
    "/dev/input/by-id/usb-Microsoft_Microsoft_Wireless_Optical_Desktop®_1.00-event-kbd",
    "/dev/input/by-id/usb-Logitech_USB_Receiver-if02-event-mouse",
    "/dev/null", "/dev/full", "/dev/zero",
    "/dev/random", "/dev/urandom",
    "/dev/ptmx", "/dev/kvm", "/dev/kqemu",
    "/dev/rtc","/dev/hpet", "/dev/sev"
]

Next we restart libvirtd:

bash

sudo systemctl restart libvirtd.service

You can swap control of the keyboard and mouse between Linux and Windows later by pressing both the left and right control keys of your keyboard at the same time.

[TODO] Reminder for myself: Implement switching from PS/2 to Virtio inputs.

And finally we also want sound. While there are a few ways to do it two possibilities makes most sense IMHO. The fastest and easiest way is to just use a USB soundcard and just pass it through to the Windows VM as we already did with the keyboard and mouse. I tried it with a Asus Xonar U3. That one isn’t that expensive and has good sound quality. That even delivers good enough quality for a beyerdynamic’s headset.

The second reasonable option is passing VM audio to host via PulseAudio. For this we need our user id. Just type id -a. In my case it’s 1000. The we run again sudo virsh edit [vmname] (again replace [vmname] with your virtual machine name). If might already have a <qemu:commandline> ... </qemu:commandline> block at the end of the file if you configured keyboard and mouse passing as described above. In this case we just add two new lines after <qemu:commandline> (replace 1000 with your user id which we figured out above):

xml

  <qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
  <qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/>

If you don’t already have the <qemu:commandline> ... </qemu:commandline> block add the following after </devices> (and again replace 1000 with your user id which we figured out above):

xml

<qemu:commandline>
  <qemu:env name='QEMU_AUDIO_DRV' value='pa'/>
  <qemu:env name='QEMU_PA_SERVER' value='/run/user/1000/pulse/native'/>
</qemu:commandline>

And finally restarting two services (in this case pulseaudio is a user service so we don’t use sudo and we need to supply the --user option):

bash

sudo systemctl restart libvirtd.service
systemctl --user restart pulseaudio.service

To control the sound later make sure to install pavucontrol and/or pavucontrol-qt.

Now start the VM. If everything went well you should now see the Windows login screen on your second monitor or on a different input port if you have connected the GPU used for Linux and the GPU used for Windows to the same monitor. In my case the Linux graphics card (AMD 5700XT) is connected via DisplayPort and the Windows graphics card (NVIDIA 1060 GTX) is connected via HDMI to the same monitor. So I just switch the different inputs on my monitor to either work with Linux or with Windows.

Also the USB keyboard and mouse should work as expected otherwise you would have a hard time to log into Windows ;-)

If you passed keyboard and mouse via evdev as described above you should now also be able to swap control of the keyboard and mouse between Linux and Windows by pressing both the left and right control keys of your keyboard at the same time. You’ll immediately recognize that your mouse and keyboard “don’t work anymore” as soon as you start the Windows VM. But pressing both control keys as described before makes it work again. And if you switch to Windows do the same again.

Now the next thing for the Windows VM is to start your browser, download the latest NVIDIA graphics card driver and install it. After a Windows reboot the driver hopefully gets initalized correctly.

That’s basically it. There is one final thing we can do: Install and using Looking Glass so that we can have the Windows screen in a Window in KDE, Gnome or whatever desktop environment you prefer - at nearly native speed even sufficient for 3D gaming! And you also don’t have to switch monitor inputs anymore if you want with either Linux or Windows. Looking Glass enables you to have all in one screen. There’re of course other solutions that provides this feature but not nearly at that speed. So see you next time!