Log in

Hans de Goede
At the end of 2013 I've spend 2 full months working on getting XHCI streams support and the UAS driver in the Linux kernel, which uses streams into shape. With the release of the 3.15 kernel this work now is available for end users to use.

This is good news for anyone who cares about performance of USB connected harddisks / ssds. The old usb mass-storage protocol is well known for its poor performance. UAS however allows NCQ and thus allows effectively using the full USB-3 bandwidth. If you've an UAS capable harddisk enclosure then all you need is a 3.15 kernel build with the UAS driver enabled and you should instantly get better performance. Note that most harddisk enclosures, including USB-3 enclosures do not support UAS, so if you want to use UAS double check before buying a harddisk enclosure.

One use-case of UAS I love is to have a 2.5" ssd with a full Fedora rawhide install with me so that when people ask me about hardware compatilibty issues they are having, I can simply plug in the ssd to there laptop boot rawhide and see if having the latest kernel + xorg fixes things. For a decently priced UAS capable 2.5" hdd enclosure search ebay for: "sedna usb 3.0 2.5 inch hdd enclosure". when shopping for UAS enabled devices, always check that the device description mentions UAS or UASP.

Besides UAS support, the 3.15 kernel also features support for using USB-3 bulk streams from userspace through usbfs. To use this you need the just released libusb-1.0.19 release. One use case of USB-3 bulk streams from userspace is using these for usb-redirection in qemu. With the upcoming 2.1 qemu release this will be supported allowing the use if usb-3 redirection with uas devices from within a qemu vm. This will be supported with both qemu's host usb redirection, as well as spice's network usb redirection. This means that starting with qemu-2.1 qemu's USB redirection features full USB-3.0 compatibility.
Hans de Goede
Xorg without root rights is now available in Fedora rawhide. Currently the suid root wrapper which is intended for ums compatbility is configured to run the server as root by default even on kms using setups, because starting Xorg as a normal user requires the display manager to set up its session and tty as it can no longer do that itself, and non of the display managers are ready to do this yet.

Still you can test running Xorg as user if you want, create a file called /etc/X11/Xwrapper.config with the following line in there: "needs_root_rights = auto" the default for "needs_root_rights" currently is "yes" rather then "auto" because of the display managers not being ready yet. Then switch to a text console (ctrl + alt + F2) log in as a regular user and run: "startx". Now do "ps aux" and you should see a (second) Xorg process running as a normal user.

If you're running rawhide and have some time to spare I would appreciate it if you can give this a try, and let me know if you see any issues running Xorg as normal user. Don't forget to remove the /etc/X11/Xwrapper.config file you've created after testing so that you don't break your graphical login screen.
Hans de Goede
06 May 2014 @ 02:02 pm
The last few months I've been working on making Xorg run without root rights. Xorg has traditionally always been suid root because it needed direct hardware access. With the advent of kms all hardware access is done by the kernel, so the primary reason for the X server running with root rights is no longer relevant.

Yet the current xserver still need root rights for 3 reasons:

1. Access to /dev/input/event* and /dev/dri/card*

Wayland needs this to, so a while ago systemd-logind has grown an API to allow a process inside a session to become the session controller, and get filedescriptors for these device nodes over dbus.

Some people may think that this is overkill and that simply opening up the permissions on the device nodes would be enough, but that is not true. For /dev/input/event* nodes we want a manager process to revoke the access of a session becoming inactive, so that it cannot stick around and snoop input events destined for another session. And for /dev/dri/card* nodes root rights are necessary to become the drm master.

2. Setting up a session and a tty, ie VT_SETMODE VTPROCESS

Since creating a new session requires root rights this is left to the process starting the X server, usually the display manager. When the X server is started from a text login on a virtual terminal using startx, it simply takes over the existing session on that vt. As for VT_SETMODE VTPROCESS that will work as a normal user too as long as that user owns the tty (and setting tty ownership is part of setting up a proper session).

3. Logging to /var/log/Xorg.#.log

Xorg 1.16 will log to ~/.local/share/xorg/Xorg.#.log when started as non root.

Driver changes

With systemd-logind now managing device node access, we also need a bunch of driver changes as many Xorg drivers used to directly open device nodes themselves. Xorg 1.16 introduces server-managed fds, when:

  1. the X server is compiled with systemd-logind support; and

  2. systemd-logind is available; and

  3. the driver used supports server-managed fds

Then the server will manage the filedescriptor for the relevant /dev/... device node, and pass it into the driver at init time. The necessary driver changes are quite minimal, see this video driver example and this input driver example.

VT switching

For switching away (vt-leave) from the VT on which the X server is running, the X server will keep using the VT_PROCESS VT mode, so everything can be shutdown safely before the VT switch happens. For server-managed fds the server will do a DRM_DROP_MASTER for video devices, and close the fd for input devices.

When switching back to the X server VT, systemd-logind will pass in new fds for input devices and call DRM_SET_MASTER for video devices asynchronously from the VT enter. So when systemd-logind is used the VT enter sequence is a bit differently, the VT enter signal the X server receives is ignored. Instead the X server waits till systemd-logind has signalled it is the drm master again for all the server-managed video device nodes. Filedescriptors received before the server is drm master again are cached, and the input devices are activated immediately after the video nodes are, fds received after this are activated immediately.

User-space modesetting (UMS) compatibility

When the Xorg binary is no longer suid root, UMS drivers will not work. To solve this there is a new suid-wrapper called Xorg.wrap which is suid root, when installed this wrapper will get called in stead of the real X server, and it will check if there are KMS capable cards in the system, if KMS capable cards are found it will drop all elevated rights and execute the X server as a normal user. If no KMS capable cards are found it will execute the X server with root rights, allowing old UMS drivers, such as the proprietary drivers for some cards, or the vesa driver to work.

Use of this wrapper is optional and security concious users who have a KMS capable system will be able to do "yum remove xorg-x11-server-wrapper", or their distro's equivalent.
Tags: , ,
Hans de Goede
02 May 2014 @ 02:12 pm
I've been debugging various backlight brightness problems lately, and I've learned a lot in the process. First lets take a look at how all the bits fit together. There are 3 parts involved in changing the brightness:

  1. Detecting brightness up / down keypresses

  2. Processing the keypresses

  3. Controlling the actual backlight

Detecting brightness up / down keypresses

The brightness keys can be hooked up in a number of ways. Sometimes they function as normal keyboard keys generating extended keycodes. On other cases they are reporting events through some firmware interface (ie ACPI) and they require a driver to be loaded listening for these firmware events.

Processing the keypresses

This is normally done by your desktop environment (Gnome, KDE, etc.) some part of your DE will listen to the events and react to them.

If the acpi video driver is active it may be processing its own events and changing the brightness itself, this usually is underiable as now the
brightness gets changed twice for each keypress. Fedora has this disabled by default. For other distros it is advisable to disable this by adding
video.brightness_switch_enabled=0 to your kernel commandline.

Controlling the actual backlight

This unfortunately is quite complicated. Most laptops have multiple ways to control the backlight, some of which sometimes do not work. You can see
which interfaces are currently available on your laptop by doing:

ls /sys/class/backlight

There are usually 1 or more firmware interfaces, usually the standard ACPI video interface, as well as a vendor specific interface. On a Lenovo Yoga 2 11, I counted 3 firmware interfaces known to Linux, acpi-video, ideapad-laptop and thinkpad_acpi all could create a backlight interface (and none of them
worked). Besides the firmware interface there will often also be a direct hardware (raw) interface, ie /sys/class/backlight/intel_backlight.

Detecting where the problem is

The first step is to figure out in which of the 3 parts listed above the problem is. If you get an on screen display (osd) showing the brightness
setting when you press your brigthness up / down setting, but the brightness does not change, then detecting and processing the keypresses works fine, and there is a problem with controlling the actual backlight.

If the osd does not show then install evemu, and as root run evemu-record, this will show you a list of input devices. You can select capturing raw events from one of them by entering the number for the device. Try these one by one, and for each press the brightness keys and see if they generate events. Likely candidates for generating brightness key events are "Video Bus", something with your laptop model / manufacturer name in it, something with WMI in it, and the actual keyboard device. If one of these generates events for the brightness keys, and the events are KEY_BRIGHTNESSUP/DOWN then the detection of the keypresses works fine and there is a problem in the processing of them by your DE. If there are no events, or the events have the wrong key-codes, then file a bug against the kernel.

Note in some rare cases the OSD will not show, but the backlight brightness will change, this means all steps are done purely in the laptops firmware, and any DE brightness control / change notification will not be possible. There is nothing which can be done to fix this.

Debugging backlight control problems

Backlight control not working usually indicates a problem with one of the firmware interfaces. Note that if more then one backlight interface is present that DE-s will just pick one, DE-s prefer the firmware interfaces.

There are a number of kernel commandline options which influence the various firmware backlight drivers. To debug backlight control problems try these *one* by *one*, please do not mix and match! For each boot with a new kernel option do "ls /sys/class/backlight" and save the output + which kernel option it belongs to. The options below are listed from most likely to help to least likely. Once you've found an option which makes the backlight work properly, make sure to test it also works with suspend and resume, and then you can stop going through the list. Here are the options to try;

  • video.use_native_backlight=1

  • acpi_backlight=vendor

  • acpi_osi="!Windows 2012"

  • acpi_osi="!Windows 2009"

If you've found an on option that helps, please still file a bug, so that we can add a quirk to the kernel for your laptop model to do the right thing automatically. This way you will both help yourself when you install a new Linux version in the future, as well as others with the same model laptop.

Filing bugs

When filing a bug please for brightness problems please always put your exact laptop model in the Summary of the bug. Also please always run the following 2 commands:

  • dmesg > dmesg.log

  • grep '.*' /sys/class/dmi/id/*_* 2> /dev/null > dmi.log

And attach the 2 generated .log files to the bug-report.

When the bug is a problem about backlight control please also add the output of "ls /sys/class/backlight" for each boot with different kernel commandline
options you've tried, including the output for a boot without any kernel options.
Hans de Goede

Sunday May 4th Revspace is organizing a workshop on reverse engineering USB protocols of devices with proprietary protocols.

Reverse engineering protocols is fun! It's like a puzzle, and if there's software out there that can solve the puzzle, your brain can do it too.

We'll be taking a look at reverse engineering the protocols on USB-connected devices. Too many of these devices still come with windows drivers only, and use an undocumented protocol. Figuring out those protocols is often a lot easier than you might think, and so is writing a simple driver.

If you want to participate in this workshop, bring a Linux laptop, preferably with a virtual machine with a Windows guest OS. If you have an exotic USB device that only has a Windows driver, please bring that too.

For more details see: https://revspace.nl/UsbProtoReverseEngineeringWS2
Hans de Goede
30 May 2013 @ 12:50 pm
Today is spice test day! For details see: https://fedoraproject.org/wiki/Test_Day:2013-05-30_Spice

Now stop reading Fedora planet, and go and test Spice!
Tags: , ,
Hans de Goede
Hi all,

Spice by default is intended for use over the network, and as such uses image compression, which is not really useful when only using a vm locally. So if you've vms which you only use locally you may want to turn the image-compression off. To do this, do: "virsh edit <vm-name>", then locate the
section starting with "<graphics", ie in my case this is a single line section looking like this:

   <graphics type='spice' autoport='yes'/>

To disable image-compression, add the following 2 lines too this section, changing it to a multi-line section if necessary:

     <image compression='auto_glz'/>  
      <streaming mode='filter'/>       

So in my example this changes the graphics section into:

   <graphics type='spice' autoport='yes'>
      <image compression='off'/>  
      <streaming mode='off'/>      

Note that the "/>" at the and of the line starting with "<graphics" has been changed into just a ">", since this is a multi-line section now!

Then shut off the vm, power it up again and enjoy your faster local graphics.
Tags: , ,
Hans de Goede
Hi all,

So far if you wanted to configure a Linux guest with multiple monitors this required adding multiple qxl cards to the vm and manually putting an xorg.conf in place.

Since Fedora-18 however Spice supports having multiple monitors on a single qxl device, so with any Linux vm you can now easily add / remove / resize monitors on the fly, depending on your configuration you may need some small tweaks to enable this on the guest and host side.

  1. On the guest side F-19 should work out of the box, in F-18 the xorg-x11-drv-qxl driver was build against an older spice-protocol package, causing it to not have support for this new feature enabled, this is fixed in this build: http://koji.fedoraproject.org/koji/buildinfo?buildID=422914 , which will hit updates-testing soon: https://admin.fedoraproject.org/updates/xorg-x11-drv-qxl-0.1.0-2.fc18?_csrf_token=bef3ee718fbf6ddde4d40197a7f4c9c83a5d5110 in the mean time you can install it inside your Fedora-18 guest from koji if you want to test this feature

  2. On the host side the qxl device in vms created in F-18 report itself with a pci-revision of 3, where as a revision of 4 indicates support for this new feature. But qemu in F-18+ does support this feature, we simply need to tell it to advertise this to the guest. Do to this do "virsh edit <vm-name>" and then at the top of the xml change the "<domain type='...  >" line to:

        <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>

          and add the following at the end, above the "</domain>" line:

              <qemu:arg value='-global'/>
              <qemu:arg value='qxl-vga.revision=4'/>


          Note this is not necessary for vms created in F-19

To use this new feature, shut off and then power on the vm, then connect to the vm using remote-viewer. ie: "remote-viewer spice://localhost:5900", and log into an X-session. Once logged into the X-session, go to the "View -> Displays" menu, you can now enable additional monitors there. Also note that you can freely resize all the client windows and the guest monitors will be adjusted to match the client window size(s).

Enjoy! And please let us know about any issues you find.
Tags: , ,
Hans de Goede
Only 10 days left till the Linux Packaging Workshop on Saturday June 16th in The Hague, The Netherlands. For more
details see: https://revspace.nl/Debrpm (warning Dutch). So if you live in the vicinity and you've ever considered becoming a Fedora packager, this is an excellent opportunity for taking the first steps!

If you plan to attend please send an email to debrpm@echtehackers.nl.
Hans de Goede
I'm happy to announce the existence of libusbx, a fork of libusb,. a bit late I must admit, as libusbx has been available for a while now, but as one of the people behind it I still wanted to mention it on my blog.

The libusbx fork was started by various libusb-1.0 developers (almost all of them), because the official maintainer of libusb-1.0 refused to do any stable / tarbal releases for over 18 months and in general was very slow with merging new developments and fixes.

Anyways, lets not look backwards, but forwards towards what looks to be a bright new future with libusbx :)

For more info on libusbx see the libusbx wiki.

I'm currently working on moving Fedora over to libusbx, you can find the Rename Review Request here.
Tags: ,