me

Adventures in fixing suspend/resume on a HP x2 Detachable 10-p0XX

I got contacted by a user with a HP X2 10 p018wm 2-in-1 about the device waking up 10-60 seconds after suspend. I have access to a HP X2 10 p002nd myself which in essence is the same HW and I managed to reproduce the problem there. This is when the fun started:

1. There were a whole bunch of ACPI related errors in dmesg. It turns out that these affect almost all HP laptop models and we have a multiple bugs open for this. Debugging these pointed to the hp-wmi driver. I wrote 2 patches fixes 2 different kind of errors and submitted these upstream. Unfortunately this does not help with the suspend/resume issue, but it does fix all those errors people have been complaining about :)

2. I noticed some weird messages in dmesg with look like a PCI bus re-enumeration is started during suspend when suspending by closing the lid and then the re-enumeration continues after resume. This turns out to be triggered by this piece of buggy AML code which
is used for monitor hotplug notification on gfx state changes (the i915 driver ACPI opregion also tracks the lid state for some reason):

                Method (GNOT, 2, NotSerialized)
                {
                    ...
                    CEVT = Arg0
                    CSTS = 0x03
                    If (((CHPD == Zero) && (Arg1 == Zero)))
                    {
                        If (((OSYS > 0x07D0) || (OSYS < 0x07D6)))
                        {
                            Notify (PCI0, Arg1)
                        }
                        Else
                        {
                            Notify (GFX0, Arg1)
                        }
                    }
                    ...
                }

Notice how "If (((OSYS > 0x07D0) || (OSYS < 0x07D6)))" is always true, the condition is broken the "||" clearly should have been a "&&" this is causing the code to send a hotplug notify to the PCI root instead of to the gfx card, triggering a re-enumeration. Doing a grep for this on my personal DSDT collection shows that 55 of the 93 DSDTs in my collection have this issue!

Luckily this can be easily fixed by setting CHPD to 1 in the i915 driver, which is something which we should do anyways according to the
opregion documentation. So I wrote a patch doing this and submitted it upstream. Unfortunately this also does not help with the suspend/resume issue.

3. So the actual spurious wakeups are caused by HP using an external embedded controller (EC) on the "legacy-free" platform which they use for these laptops. Since these are not designed to use an external EC they lack the standard interface for this, so HP has hooked the EC up over I2C and using an ACPI GPIO event handler as EC interrupt.

These devices use suspend2idle (s2idle) instead of good old firmware handled S3, so the EC stays active during suspend. It does some housekeeping work which involves a round-trip through the AML code every minute. Normally EC wakeups are ignored durin s2idle by some special handling in the kernel, but this is only done for ECs using the standardized ACPI EC interface, not for this bolted on the
side model. I've started a discussion on maybe extending our ACPI event handling to deal with this special case.

For now as a workaround I ended up writing 2 more patches to allow blacklisting wakeup by ACPI GPIO event handlers on select models. This breaks wakeup by opening the LID, the user needs to wake the laptop with the powerbutton. But at least the laptop will stay suspended now.
me

Creating an USB3 OTG cable for the Thinkpad 8

The Lenovo Thinkpad 8 and also the Asus T100CHI both have an USB3 micro-B connector, but using a standard USB3 OTG (USB 3 micro-B to USB3-A receptacle) cable results in only USB2 devices working. USB3 devices are not recognized.

Searching the internet learns that many people have this problem and that the solution is to find a USB3 micro-A to USB3-A receptacle cable. This sounds like nonsense to me as micro-B really is micro-AB and is supposed to use the ID pin to automatically switch between modes dependent on the used cable.; and this does work for the USB-2 parts of the micro-B connector on the Thinkpad. Yet people do claim success with such cables (with a more square micro-A connector, instead of the trapezoid micro-B connector). The only problem is such cables are not for sale anywhere.

So I guessed that this means is that they have swapped the Rx and Tx superspeed pairs on the USB3 only part of the micro-B connector, and I decided to cut open one of my USB3 micro-A to USB3-A receptacle cables and swap the superspeed pairs. Here is what the cable looks like when it it cut open:



If you are going to make such a cable yourself, to get this picture I first removed the outer plastic isolation (part of it is still there on the right side in this picture). Then I folded away the shield wires till they are all on one side (wires at the top of the picture). After this I removed the metal foil underneath the shield wires.

Having removed the first 2 layers of shielding this reveals 4 wires in the standard USB2 colors: red, black, green and white. and 2 separately shielded cable pairs. On the picture above the separately shielded pairs have been cut, giving us 4 pairs, 2 on each end of cable; and the shielding has been removed from 3 of the 4 pairs, you can still see the shielding on the 4th pair.

A standard USB3 cable uses the following color codes:

  • Red: Vbus / 5 volt

  • White:  USB 2.0 Data -

  • Green: USB 2.0 Data +

  • Black: Ground

  • Purple: Superspeed RX -

  • Orange: Superspeed RX +

  • Blue: Superspeed TX -

  • Yellow: Superspeed TX -

So to swap RX and TX we need to connect purple to blue / blue to purple and orange to yellow / yellow to orange, resulting in:



Note the wires are just braided together here, not soldered yet. This is a good moment to carefully test the cable. Also note that the superspeed wire pairs must be length matched, so you need to cut and strip all 8 cables at the same length! If everything works you can put some solder on those braided together wires, re-test after soldering, and then cover them with some heat-shrink-tube:



And then cover the entire junction with a bigger heat-shrink-tube:



And you have a superspeed capable cable even though no one will sell you one.

Note that the Thinkpad 8 supports ACA mode so if you get an ACA capable "Y" cable or an ACA charging HUB then you can charge and use the Thinkpad 8 USB port at the same time. Typically ACA "Y" cables or hubs are USB2 only. So the superspeed mod from this blogpost will not help with those. The Asus T100CHI has a separate USB2 micro-B just for charging, so you do not need anything special there to charge + connect an USB device.
me

Xwayland randr resolution change emulation now available in Fedora 31

As mentioned in an earlier blogpost, I have been working on fixing many games showing a small image centered on a black background when they are run fullscreen under Wayland. In that blogpost I was moslty looking at how to solve this for native Wayland games. But for various reasons almost all games still use X11, so instead I've ended up focussing on fixing this for games using Xwayland.

Xwayland now has support for emulating resolution changes requested by an app through the randr or vidmode extensions. If a client makes a resolution change requests this is remembered and if the client then creates a window located at the monitor's origin and sized to exactly that resolution, then Xwayland will ask the compositor to scale it to fill the entire monitor.

For apps which use _NET_WM_FULLLSCREEN (e.g. SDL2, SFML or OGRE based apps) to go fullscreen some help from the compositor is necessary. This is currently implemented in mutter. If you are a developer of another compositor and have questions about this, please drop me an email.

I failed to get this all upstream in time for Fedora 31 final. But now it is all upstream, so 've backported the changes and created an update with the changes. This update is currently in updates-testing, to install this update run the following command:

sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2019-103a594d07
me

Capslock and keyboard layout indicator for plymouths diskcrypt password screen

As some of you running Fedora 31 may already have noticed, I have some good news to share. As part of my recent work on plymouth I've implemented a feature request which was first requested in 2012: support for an indicator that capslock is active while entering the disk unlock password for machines using full diskencryption. Besides the capslock indicator I've also added support for an indicator of the configured keyboard layout, since this sometimes also causes confusion:



And here is what it looks like when capslock is pressed:



If you're running Fedora 31 with full diskencryption then you may notice that the above screenshots are slightly different then what you have now. I've pushed an update to Fedora 31 updates-testing today which implements a few small tweaks to the theme after feedback from the design-team on the initial version. For those of you still on Fedora 30, this is coming to Fedora 30 too, it should show up in updates-testing with the next updates push.
me

Disney+ streaming uses draconian DRM, avoid

First of all, as always my opinions are my own, not those of my employer.

Since I have 2 children I was happy to learn that the Netherlands would be one of the first countries to get Disney+ streaming.

So I subscribed for the testing period, problem all devices in my home run Fedora. I started up Firefox and was greeted with an "Error Code 83", next I tried Chrome, same thing.

So I mailed the Disney helpdesk about this, explaining how Linux works fine with Netflix, AmazonPrime video and even the web-app from my local cable provider. They promised to get back to me in 24 hours, the eventually got back to me in about a week. They wrote: "We are familiar with Error 83. This often happens if you want to play Disney + via the web browser or certain devices. Our IT department working hard to solve this. In the meantime, I want to advise you to watch Disney + via the app on a phone or tablet. If this error code still occurs in a few days, you can check the help center ..." this was on September 23th.

So I thought, ok they are working on this lets give them a few days. It is almost a month later now and nothing has changed. Their so called help-center does not even know about "Error Code 83" even though the internet is full of people experiencing this. Note that this error also happens a lot on other platforms, it is not just Linux.

Someone on tweakers.net has done some digging and this is a Widevine error: "the response is: {"errors":[{"code":"platform-verification-failed","description":"Platform verification status incompatible with security level"}]}". Widevine has 3 security levels and many devices, including desktop Linux and many Android devices only support the lowest security setting (software encryption only). In this case e.g. Netflix will not offer full HD or 4k resolutions, but otherwise everything works fine, which is a balance between DRM and usability which I can accept. Disney+ OTOH seems to have the drm features kranked up to maximum draconian settings and simply will not work on a lot of android devices, nor on Chromebooks, nor on desktop Linux.

So if you care about Linux in any way, please do not subscribe to Disney+, instead send them a message letting them know that you are boycotting them until they get their Linux support in order.
me

Better support for running games under Wayland (with GNOME3/mutter as compositor)

First of all I do not want people to get their hopes up about $subject of this blogpost. Improving gaming support is a subjects which holds my personal interest and it is an issue I plan to spend time on trying to improve. But this will take a lot of time (think months for simple things, years for more complex things).

As I see it there are currently 2 big issues when running games under Wayland:

1. Many games show as a smal centered image with a black border (letterbox) around the image when running fullscreen.

For 2D games this is fixed by switching to SDL2 which will transparently scale the pixmap the game renders to the desktop resolution. This assumes that 2D games in general do not demand a lot of performance and thus will not run into performance issues when introducing an extra   scaling step. A problem here is that many games still use SDL1.2 (and some games do not use SDL at all).

I plan to look into the recently announced SDL1.2 compatibility wrapper around SDL2. If this works well this should fix this issue for all SDL1.2 2D games, by making them use SDL2 under the hood.

For 3D games this can be fixed by rendering at the desktop resolution, but this might be slow and rendering at a lower resolution leads to the letterbox issue.

Recently mutter has has grown support for the WPviewport extension, which allows Wayland apps to tell the compositor to scale the pixmap the app gives to the compositor before presenting it. If we add support to SDL2's Wayland backend for this then, this can be used to allow rendering 3D apps at a lower resolution and still have them fill the entire screen.

Unfortunately there are 2 problems with this plan:

  1. SDL2 by default uses its x11 backend, not its wayland backend. I'm not sure what fixes need to be done to change this, at a minimum we need a fix at either the SDL or mutter side for this issue, which is going to be tricky.

  2. This only helps for SDL2 apps, again hopefully the SDL1.2 compatibility wrapper for SDL2 can help here, at least for games using SDL.

2. Fullscreen performance is bad with many games.

Since under Wayland games cannot change the monitor resolution, they need to either render at the full desktop resolution, which can be very slow; or they render at a lower resolution and then need to do an extra scaling step each frame.

If we manage to make SDL2's Wayland backend the default and then add WPviewport support to it then this should help by reducing an extra memcpy/blit of a desktop-sized pixmap. Currently what apps which use scaling do is:

  1. render lower-res-pixmap;

  2. scale lower-res-pixmap to desktop-res-pixmap

  3. give desktop-res-pixmap to the compositor;

  4. compositor does a hardware blit of the desktop-res-pixmap to the framebuffer.

With viewport support this becomes:

  1. render lower-res-pixmap;

  2. give low-res-pixmap to the compositor;

  3. compositor uses hardware to do a scaling blit from the low-res-pixmap to the desktop-res framebuffer

Also with viewport support, the compositor could in the case of there only being the one fullscreen app even keep the framebuffer in lowres and use a hardware scaling drm-plane to send the low-res framebuffer scaled to desktop-res to the output while only reading the low-res framebuffer from memory saving a ton of memory bandwidth. But this optimization is going to be a challenge to pull off.
me

Wayland itches summary

Thank you all for the large amount of feedback I have received after my previous Wayland Itches blog post. I've received over 40 mails, below is an attempt at summarizing all the mails.

Highlights

1. Middle click on title / header bar to lower the Window does not work for native apps. Multiple people have reported this issue to me. A similar issue was fixed for not being able to raise Windows. It should be easy to apply a similar fix for the lowering problem. There are bugs open for this here, here and here.

2. Running graphical apps via sudo or pxexec does not work. There are numerous examples of apps breaking because of this, such as lshw-gui and usbivew. At least for X11 apps this is not that hard to fix. But sofar this has deliberately not been fixed. The reasoning behind this is described in this bug. I agree with the reasoning behind this, but I think it is not pragmatic to immediately disallow all GUI apps to connect when run as root starting today.

We need some sort of transition period. So when I find some time for this, I plan to submit a merge-requests which optionally makes gnome-shell/mutter start Xwayland with an xauth file, like how it is done when running in GNOME on Xorg mode. This will be controlled by a gsettings option, which will probably default to off upstream and then distros can choice to override this for now, giving us a transition period

Requests for features implemented as external programs on X11

There are various features which can be implemented as external programs
on X11, but because of the tighter security need to be integrated into the
compositor with Wayland:

  • Hiding of the mouse-cursor when not used à la unclutter-xfixes, xbanish.

  • Rotating screen 90 / 270 degrees à la "xrandr -o [left|right]" mostly used through custom hotkeys, possible fix is defining bindable actions for this in gsd-media-keys.

  • Mapping actions to mouse buttons à la easystroke

  • Some touchscreen's, e.g. so called smart-screens for education, need manual calibration. Under X11 there are some tools to get the callibration matrix for the touchscreen, after which this can be manually applied through xinput. Even under X11 this currently is far from ideal but at least it is possible there.

  • Keys Indicator gnome-shell extension. This still works when using Wayland, but only works for apps using Xwayland, it does not work for native apps.

  • Some sort of xkill and xdotool utility equivalents would be nice

  • The GNOME on screen keyboard is not really suitable for use with apps which are not touch-enabled, as it lacks a way to send ctrl + key, etc. Because of this some users have reported that it is impossible to use alternative on screen keyboards with Wayland. Not being able to use alternative on screen keyboards is by design and IMHO the proper fix here is to improve GNOME's on screen keyboard.

App specific problems


  • Citrix ICA Client does not work well with Xwayland

  • Eclipse does not work well with Xwayland

  • Teamviewer does not work with Wayland. It needs to be updated to use pipewire for screencapturing and the RemoteDesktop portal to inject keyboard and mouse events.

  • Various apps lack screenrecording / capture support due to the app not having support for pipewire: gImageReader, green-recorder, OBS studio, peek, screenrecorder, slack

  • For apps which do support pipewire, there is not an option to share the contents of a window, other then the window making the request. On Xorg it is possible to share a random window and since pipewire allows sharing the whole desktop I see no security reason why we would not allow sharing another window.

  • guake window has incorrect size when using HiDPI scaling, see this issue

Miscellaneous problems


  • Mouse cursor is slow / lags

  • Drag and drop sometimes does not work, e.g. dragging files into file roller to compress or out of file roller to extract.

  • Per keyboard layouts. On X11 after plugging in a keyboard, the layout/keymap for just that one keyboard can be updated manually using xinput, allowing different keyboard layouts for different keyboards when multiple keyboards are connected

  • No-title-bar shell extension, X button can be hit unintentionally, see this issue

  • Various issues with keyboard layout switching

Hard to fix issues


  • Alt-F2, r equivalent (restart the gnome-shell)

  • X11 apps running on top of Xwayland do not work well on HiDPI screens

  • Push-to-talk (passive key grab on space) does not work in Mumble when using native Wayland apps, see this issue

Problems with other compositors then GNOME3 / mutter

I've also received several reports about issues when using another Wayland compositor as GNOME / mutter (Weston, KDE, Sway). I'm sorry but I have not looked very closely into these reports. I believe that it is great that Linux users have multiple Desktop Environments to choose from and I wish for the other DEs to thrive. But there are only so many hours in a day so I've chosen to mainly focus on GNOME.
me

Improved Logitech wireless device support in kernel 5.2

The just released 5.2-rc1 kernel includes improved support for Logitech wireless keyboards and mice. Until now we were relying on the generic HID keyboard and mouse emulation for 27 MHz and non-unifying 2.4 GHz wireless receivers.

Starting with the 5.2 kernel instead we actually look at the devices behind the receiver. This allows us to provide battery monitoring support and to have per device quirks, like device specific HID-code to evdev-code mappings where necessary. Until now device specific quirks where not possible because the receivers have a generic product-id which is the same independent of the device behind the receiver.

The per device key-mapping is especially important for 27MHz wireless devices, these use the same HID-code for Fn + F1 to Fn + F12 for all devices, but the markings on the keys differ per model. Sofar it was impossible for Linux to get the mapping for this right, but now that we have per device product-ids for the devices behind the receiver we can finally fix this. As is the case with other devices with vendor specific mappings, the actual mapping is done in userspace through hwdb.

If you have a 27 MHz device (often using this receiver, keyboard marked as canada 210 or canada 310 at the bottom). Please give 5.2 a try. Download the latest 60-keyboard.hwdb file and place it in /lib/udev/hwdb.d (replacing the existing file) and then run "sudo udevadm hwdb --update", before booting into the 5.2 kernel. Then run "sudo evemu-record" select your keyboard and try Fn + F1 to Fn + F12 and any other special keys. If any keys do not work, edit 60-keyboard.hwdb, search for Logitech and add an entry for your keyboard, see the existing Logitech entries. After editing you need to re-run "sudo udevadm hwdb --update", followed by "sudo udevadm trigger" for the changes to take effect. Once you have a working entry, submit a pull-req to systemd to get the changes upstream. If you need any help drop me an email.

We still have some old code for the generic HID emulation for 27 MHz receivers with a product-id of c50c, these should work fine with the new code, but we've been unable to test this. I would really like to move the c50c id over to the new code and remove all the old code. If you've a 27 MHz Logitech device, please run lsusb, if your device has a product-id of c50c and you are willing to test, please drop me an email.

Likewise I suspect that 2.4GHz receivers with a product-id of c531 should work fine with the new support for non-unifying 2.4 GHz receivers, if you have one of those also please drop me an email.
me

The Wayland Itches project

Now that GNOME3 on Wayland is the default in Fedora I've been trying to use this as my default desktop, but until recently I've kept falling back to GNOME3 on Xorg because of various small issues.

To fix this I've switched to using GNOME3 on Wayland as day to day desktop now and I'm working on fixing any issues which this causes as I hit them, aka "The Wayland Itches project". So far I've hit and fixed the following issues:

1. TopIcons

The TopIcons extension, which I depend on for some of my workflow, was not working well under Wayland with GNOME-3.30, only the top row of icons was clickable. This was fixed in GNOME-3.32, but with GNOME-3.32 using TopIcons was causing gnome-shell to go into a loop leading to a very high CPU load. The day I wanted to start looking into fixing this I was chatting to Carlos Garnacho and he pointed out to me that this was fixed a couple of days ago in gnome-shell. The fix for this is in gnome-shell 3.32.2 .

2. Hotkeys/desktop shortcuts not working in VirtualBox Virtual Machines

When running a VirtualBox VM under GNOME3 on Wayland, hotkeys such as alt+tab go to the GNOME3 desktop, rather then being forwarded to the VM as happens under Xorg. This can be fixed by changing 2 settings:

  gsettings set org.gnome.mutter.wayland xwayland-allow-grabs true
  gsettings set org.gnome.mutter.wayland xwayland-grab-access-rules "['VirtualBox Machine']"

This is a decent workaround, but we want things to "just work" of course, so we have been working on some changes to make this just work in the next GNOME version.

3. firefox-wayland

I've been also trying to use firefox-wayland as my day to day browser, this has lead to me filing three firefox bugs and I've switched back to regular
firefox (x11) for now.


If you have any Wayland Itches yourself, please drop me an email at hdegoede@redhat.com explaining them in as much detail as you can and I will see what I can do. Note that I typically get a lot of emails when asking for feedback like this, so I cannot promise that I will reply to every email; but I will be reading them all.
me

Lenovo Ideapad and Yoga laptops and wifi on/off switches

Once upon a time a driver was written for the Lenovo Ideapad firmware interface for handling special keys and rfkill functionality. This driver was written on an Ideapad laptop with a slider on the side to turn wifi on/off, a so called hardware rfkill switch. Sometime later a Yoga model using the same firmware interface showed up, without a hardware rfkill switch. It turns out that in this case the firmware interface reports the non-present switch as always in the off position, causing NetworkManager to not even try to use the wifi effectively breaking wifi.

So I added a dmi blacklist for models without a hardware rfkill switch. The same firmware interface is still used on new Ideapad and Yoga models and since most modern laptops typically do not have such a switch this dmi blacklist has been growing and growing. Just in the 5.1 kernel alone 5 new models were added. Worse as mentioned not being on the list for a model without the hardware switch leads to non working wifi, pretty much leading to any new Ideapad model not working with Linux until added to the list.

To fix this I've submitted a patch upstream turning the huge blacklist into a whitelist. This whitelist is empty for now, meaning that we define all models as not having a rfkill switch. This does lead to a small regression on models which do actually have a hardware rfkill switch, before this commit they would correctly report the wifi being disabled by the hw switch and e.g. the GNOME3 UI would report "wifi disabled in hardware", where as now users will just get an empty list of available wifi networks when the switch is in the off position. But this is a small price to pay to make sure that as of yet unknown and new Ideapad models do not have non-working wifi because of this issue.

As said the whitelist for models which do actually have a hardware rfkill switch is currently empty, so I need your help to fill it. If you have an Ideapad or Yoga laptop with a wifi on/off slider switch on it. Please run "rfkill list" if this contains "ideapad_wlan" in the output then you are using the ideapad-laptop driver. In this case please check that the "Hard blocked" setting for the "ideapad_wlan" rfkill device properly shows no / yes based on the switch position. If this works your model should be added to the new whitelist. For this please run: "sudo dmidecode &> dmidecode.log" and send me an email at hdegoede@redhat.com with the dmidecode.log attached.

Note the patch to change the list to a whitelist has been included in the Fedora kernels starting with kernel 5.0.10-300, so if you have an
Ideapad or Yoga running Fedora and you do see "ideapad_wlan" in the "rfkill list" output, but the "Hard blocked" setting does not respond, try with a kernel older then 5.0.10-300, let me know if you need help with this.