me

Logitech G15 and Z-10 LCD-screen support under Linux

A while ago I worked on improving Logitech G15 LCD-screen support under Linux. I recently got an email from someone who wanted to add support for the LCD panel in the Logitech Z-10 speakers to lcdproc, asking me to describe the process I went through to improve G15 support in lcdproc and how I made it work without requiring the unmaintained g15daemon code.

So I wrote a long email describing the steps I went through and we both thought this could be interesting for others too, so here it is:

1. For some reason I decided that I did not have enough projects going on at the same time already and I started looking into improving support for the G15 family of keyboards.

2. I started studying the g15daemon code and did a build of it to check that it actually works. I believe making sure that you have a known-to-work codebase as a starting point, even if it is somewhat crufty and ugly, is important. This way you know you have code which speaks the right protocol and you can try to slowly morph it into what you want it to become (making small changes testing every step). Even if you decide to mostly re-implement from scratch, then you will likely use the old code as a protocol documentation and it is important to know it actually works.

3. There were number of things which I did not like about g15daemon:

3.1 It uses libusb, taking control of the entire USB-interface used for the gaming functionality away from the kernel. 

3.2 As part of this it was not just dealing with the LCD, it also was acting as a dispatches for G-key key-presses. IMHO the key-press handling clearly belonged in the kernel. These keys are just extra keys, all macro functionality is handled inside software on the host/PC side. So as a kernel dev I found that these really should be handled as normal keys and emit normal evdev event with KEY_FOO codes from a /dev/input/event# kernel node.

3.3 It is a daemon, rather then a library; and most other code which would deal with the LCD such as lcdproc was a daemon itself too, so now we would have lcdproc's LCDd talking to g15daemon to get to the LCD which felt rather roundabout.
So I set about tackling all of these

4. Kernel changes: I wrote a new drivers/hid/hid-lg-g15.c kernel driver:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/hid/hid-lg-g15.c
Which sends KEY_MACRO1 .. KEY_MACRO18, KEY_MACRO_PRESET1 .. KEY_MACRO_PRESET3, KEY_MACRO_RECORD_START, KEY_KBD_LCD_MENU1 .. KEY_KBD_LCD_MENU4 keypresses for all the special keys. Note this requires that the kernel HID driver is left attached to the USB interface, which required changes on the g15dameon/lcdproc side.

This kernel driver also offers a /sys/class/leds/g15::kbd_backlight/ interface which allows controller the backlight intensity through e.g. GNOME kbd-backlight slider in the power pane of the control-center. It also offers a bunch of other LED interfaces under /sys/class/leds/ for controlling things like the LEDs under the M1 - M3 preset selection buttons. The idea here being that the kernel abstract the USB protocol for these gaming-kbd away and that a single userspace daemon for doing gaming kbd macro functionality can be written which will rely only on the kernel interfaces and will thus work with any kbd which has kernel support.

5. lcdproc changes

5.1 lcdproc already did have a g15 driver, which talked to g15daemon. So at first I started testing with this (at this point all my kernel work would not work, since g15daemon would take full control of the USB interface unbinding my kernel driver). I did a couple of bug-fixes / cleanups in this setup to get the code to a starting point where I could run it and it would not show any visible rendering bugs in any of the standard lcdproc screens

5.2 I wrote a little lcdproc helper library, lib_hidraw, which can be used by lcdproc drivers to open a /dev/hidraw device for them. The main feature is, you give the lib_hidraw_open() helper a list of USB-ids you are interested in; and it will then find the right /dev/hidraw# device node (which may be different every boot) and open it for you.

5.3 The actual sending of the bitmap to the LCD screen is quite simple, but it does need to be a very specific format. The format rendering is done by libg15render. So now it was time to replace the code talking to g15daemon with code to directly talk to the LCD through a /dev/hidraw# interface. I kept the libg15render dependency since that was fine. After a bit of refactoring, the actual change over to directly sending the data to the LCD was not that big.

5.4 The change to stop using the g15daemon meant that the g15 driver also lost support for detecting presses on the 4 buttons directly under the LCD which are meant for controlling the menu on the LCD. But now that the code is using /dev/hidraw# the kernel driver I wrote would actually work and report KEY_KBD_LCD_MENU1 .. KEY_KBD_LCD_MENU4 keypresses. So I did a bunch of cleanup / refactoring of lcdproc's linux_input driver and made it take over the reporting of those button presses.

5.5 I wanted everything to just work out of the box, so I wrote some udev rules which automatically generate a lcdproc.conf file configuring the g15 + linux_input drivers (including the key-mapping for the linux_input driver) when a G15 keyboard gets plugged in and the lcdproc.conf file does not exist yet.

All this together means that users under Fedora, where I also packaged all this can now do "dnf install lcdproc", pluging their G15 keyboard and everything will just work.



p.s.

After the email exchange I got curious and found a pair of these speakers 2nd hand for a nice price. The author of the initial email was happy with me doing the work on this. So I added support for the Z-10 speakers to lcdproc (easy) and wrote a set of kernel-patches to support the display and 1-4 keys on the speaker as LCD menu keys.

I've also prepared an update to the Fedora lcdproc packages so that they will now support the Z-10 speakers OOTB, if you have these and are running Fedora, then (once the update has reached the repos) a "sudo dnf install lcdproc" followed by unplugging + replugging the Z-10 speakers should make the display come alive and show the standard lcdproc CPU/mem usage screens.
me

Linux tool for Logitech 27MHz wireless keyboard encryption setup

For a long time Logitech produced wireless keyboards using 27 MHz as communications band. Although these have not been produced for a while now these are still pretty common and a lot of them are still perfectly serviceable.

But when using them under Linux, there is one downside, since the communication is one way by default the wireless link is unencrypted by default, which is kinda bad from a security pov. These keyboards do support using an encrypted link, but this requires a one-time setup where the user manually enters a key on the keyboard.

I've written a small Linux utility to do this under Linux, which should help give these keyboards an extra lease on life and stop them unnecessarily becoming e-waste. Sometimes these keyboards appear to be broken, while the only problem is that the key in the keyboard and receiver are not in sync, the README also contains instructions on howto reset the keyboard, without needing the utility, restoring (unencrypted) functionality.

The 'lg-27MHz-keyboard-encryption-setup' utility is available on Fedora in the 'logitech-27mhz-keyboard-encryption-setup package.
me

Fixing the Sierra Wireless EM7345-LTE modem not working on Linux

I spend quite a bit of time on getting a Sierra Wireless EM7345-LTE modem to work under Linux. So here are some quick instructions to help other people who may hit the same problem.

These modems are somewhat notorious for shipping with broken firmware. They work fine after a firmware upgrade, but under Windows they will only upgrade to "carrier approved" firmware versions, which requires to be connected to the mobile-network first so that the tool can identify the carrier. And with some carriers connecting to the network does not work due to the broken firmware (ugh). There are a ton of forum-threads on how to work around this under Windows, but they all require that you are atleast able to register with the mobile-network.

Luckily someone has figured out how to update these under Linux and posted instructions for this. The procedure is actually much more straight forward under Linux. The hardest part is extracting the firmware from the Windows driver download.

One problem is that the necessary Intel FlashTool download is no longer available for download from Intel. I needed this tool a while ago for something else, and back then I used the PhoneFlashTool_5.8.4.0.rpm file from https://androiddatahost.com/nm466 . The rpm-file in the zip there has a sha256sum of: c5b964ed4fae470d1234c9bf24e0beb15a088a73c7e8e6b9369a68697020c17a

I now see that it seems that Intel is again offering this for download itself, you can find it here: https://github.com/projectceladon/tools and projectceladon seems to be an official Intel project. Note this is not the version which I used, I used the PhoneFlashTool_5.8.4.0.rpm version.

Once you have the Intel FlashTool installed just follow the posted instructions and after that your model should start working under Linux.
me

Acer Aspire Switch 10 E SW3-016's and SW5-012's and S1002's horrible EFI firmware

Recently I acquired an Acer Aspire Switch 10 E SW3-016, this device was the main reason for writing my blog post about the shim boot loop. The EFI firmware of this is bad in a number of ways:

  1. It considers its eMMC unbootable unless its ESP contains an EFI/Microsoft/Boot/bootmgfw.efi file.

  2. But it will actually boot EFI/Boot/bootx64.efi ! (wait what? yes really)

  3. It will only boot from an USB disk connected to its micro-USB connector, not from the USB-A connector on the keyboard-dock.

  4. You must first set a BIOS admin password before you can disable secure-boot (which is necessary to boot home-build kernels without doing your own signing)

  5. Last but not least it has one more nasty "feature", it detect if the OS being booted is Windows, Android or unknown and it updates the ACPI DSDT based in this!

Some more details on the OS detection mis feature. The ACPI "Device (SDHB) node for the MMC controller connected to the SDIO wifi module contains:

        Name (WHID, "80860F14")
        Name (AHID, "INT33BB")


Depending on what OS the BIOS thinks it is booting it renames one of these 2 to _HID. This is weird given that it will only boot if EFI/Microsoft/Boot/bootmgfw.efi exists, but it still does this. Worse it looks at the actual contents of EFI/Boot/bootx64.efi for this. It seems that that file must be signed, otherwise it goes in OS unknown mode and keeps the 2 above DSDT bits as is, so there is no _HID defined for the wifi's mmc controller and thus no wifi. I hit this issue when I replaced EFI/Boot/bootx64.efi with grubx64.efi to break the bootloop. grubx64.efi is not signed so the DSDT as Linux saw it contained the above AML code. Using the proper workaround for the bootloop from my previous blog post this bit of the DSDT morphes into:

        Name (_HID, "80860F14")
        Name (AHID, "INT33BB")


And the wifi works.

The Acer Aspire Switch 10 E SW3-016's firmware also triggers an actual bug / issue in Linux' ACPI implementation, causing the bluetooth to not work. This is discussed in much detail here. I have a patch series fixing this here.

And the older Acer Aspire Switch 10 SW5-012's and S1002's firmware has some similar issues:

  1. It considers its eMMC unbootable unless its ESP contains an EFI/Microsoft/Boot/bootmgfw.efi file

  2. These models will actually always boot the EFI/Microsoft/Boot/bootmgfw.efi file, so that is somewhat more sensible.

  3. On the SW5-012 you must first set a BIOS admin password before you can disable secure-boot.

  4. The SW5-012 is missing an ACPI device node for the PWM controller used for controlling the backlight brightness. I guess that the Windows i915 gfx driver just directly pokes the registers (which are in a whole other IP block), rather then relying on a separate PWM driver as Linux does. Unfortunately there is no way to fix this, other then using a DSDT overlay. I have a DSDT overlay for the V1.20 BIOS and only for the v1.20 BIOS available for this here.

Because of 1. and 2. you need to take the following steps to get Linux to boot on the Acer Aspire Switch 10 SW5-012 or the S1002:

  1. Rename the original bootmgfw.efi (so that you can chainload it in the multi-boot case)

  2. Replace bootmgfw.efi with shimia32.efi

  3. Copy EFI/fedora/grubia32.efi to EFI/Microsoft/Boot

This assumes that you have the files from a 32 bit Windows install in your ESP already.
me

How to fix Linux EFI secure-boot shim bootloop issue

How to fix the Linux EFI secure-boot shim bootloop issue seen on some systems.

Quite a few Bay- and Cherry-Trail based systems have bad firmware which completely ignores any efibootmgr set boot options. They basically completely reset the boot order doing some sort of auto-detection at boot. Some of these even will given an error about their eMMC not being bootable unless the ESP has a EFI/Microsoft/Boot/bootmgfw.efi file!

Many of these end up booting EFI/Boot/bootx64.efi unconditionally every boot. This will cause a boot loop since when Linux is installed EFI/Boot/bootx64.efi is now shim. When shim is started with a path of EFI/Boot/bootx64.efi, shim will add a new efibootmgr entry pointing to EFI/fedora/shimx64.efi and then reset. The goal of this is so that the firmware's F12 bootmenu can be used to easily switch between Windows and Linux (without chainloading which breaks bitlocker). But since these bad EFI implementations ignore efibootmgr stuff, EFI/Boot/bootx64.efi shim will run again after the reset and we have a loop.

There are 2 ways to fix this loop:

1. The right way: Stop shim from trying to add a bootentry pointing to EFI/fedora/shimx64.efi:

rm EFI/Boot/fbx64.efi
cp EFI/fedora/grubx64.efi EFI/Boot


The first command will stop shim from trying to add a new efibootmgr entry (it calls fbx64.efi to do that for it) instead it will try to execute grubx64.efi from the from which it was executed, so we must put a grubx64.efi in the EFI/Boot dir, which the second command does. Do not use the livecd EFI/Boot/grubx64.efi file for this as I did at first, that searches for its config and env under EFI/Boot which is not what we want.

Note that upgrading shim will restore EFI/Boot/fbx64.efi. To avoid this you may want to backup EFI/Boot/bootx64.efi, then do "sudo rpm -e shim-x64" and then restore the backup.

2. The wrong way: Replace EFI/Boot/bootx64.efi with a copy of EFI/fedora/grubx64.efi

This is how I used to do this until hitting the scenario which caused me to write this blog post. There are 2 problems with this:

2a) This requires disabling secure-boot (which I could live with sofar)
2b) Some firmwares change how they behave, exporting a different DSDT to the OS dependending on if EFI/Boot/bootx64.efi is signed or not (even with secure boot disabled) and their behavior is totally broken when it is not signed. I will post another rant ^W blogpost about this soon. For now lets just say that you should use workaround 1. from above since it simply is a better workaround.

Note for better readability the above text uses bootx64, shimx64, fbx64 and grubx64 throughout. When using a 32 bit EFI (which is typical on Bay Trail systems) you should replace these with bootia32, shimia32, fbia32 and grubia32. Note 32 bit EFI Bay Trail systems should still use a 64 bit Linux distro, the firmware being 32 bit is a weird Windows related thing.

Also note that your system may use another key then F12 to show the firmware's bootmenu.
me

Plug and play support for (Gaming) keyboards with a builtin LCD panel

A while ago as a spin-off of my project to improve support for Logitech wireless keyboards and mice I have also done some work on improving support for (Gaming) keyboards with a builtin LCD panel.

Specifically if you have a Logitech MX5000, G15, G15 v2 or G510 and you want the LCD panel to show something somewhat useful then on Fedora 31 you can now install the lcdproc package and it will automatically recognize the keyboard and show "top" like information on it. No need to manually write an LCDd.conf or anything, this works fully plug and play:

sudo dnf install lcdproc
sudo udevadm trigger


If you have a MX5000 and you do not want the LCD panel to show "top" like info, you may still want to install the mx5000tools package, this will automatically send the system time to the keyboard, after which it will display the time.

Once the 5.5 kernel becomes available as an update for Fedora you will also be able to use the keys surrounding the LCD panel to control the lcdproc menu-s on the LCD panel. The 5.5 kernel will also export key backlight brightness control through the standardized /sys/class/leds API, so that you can control it from e.g. the GNOME control-center's power-settings and you get a nice OSD when toggling the brightnesslevel using the key on the keyboard.

The 5.5 kernel will also make the "G" keys send standard input-events (evdev events), once userspace support for the new key-codes these send has landed, this will allow e.g. binding them to actions in GNOME control-center's keyboard-settings. But only under Wayland as the new keycodes are > 255 and X11 does not support this.
me

Adventures in fixing suspend/resume on a HP x2 Detachable 10-p0XX

I got contacted by a user with a HP X2 10 p018wm 2-in-1 about the device waking up 10-60 seconds after suspend. I have access to a HP X2 10 p002nd myself which in essence is the same HW and I managed to reproduce the problem there. This is when the fun started:

1. There were a whole bunch of ACPI related errors in dmesg. It turns out that these affect almost all HP laptop models and we have a multiple bugs open for this. Debugging these pointed to the hp-wmi driver. I wrote 2 patches fixes 2 different kind of errors and submitted these upstream. Unfortunately this does not help with the suspend/resume issue, but it does fix all those errors people have been complaining about :)

2. I noticed some weird messages in dmesg with look like a PCI bus re-enumeration is started during suspend when suspending by closing the lid and then the re-enumeration continues after resume. This turns out to be triggered by this piece of buggy AML code which
is used for monitor hotplug notification on gfx state changes (the i915 driver ACPI opregion also tracks the lid state for some reason):

                Method (GNOT, 2, NotSerialized)
                {
                    ...
                    CEVT = Arg0
                    CSTS = 0x03
                    If (((CHPD == Zero) && (Arg1 == Zero)))
                    {
                        If (((OSYS > 0x07D0) || (OSYS < 0x07D6)))
                        {
                            Notify (PCI0, Arg1)
                        }
                        Else
                        {
                            Notify (GFX0, Arg1)
                        }
                    }
                    ...
                }

Notice how "If (((OSYS > 0x07D0) || (OSYS < 0x07D6)))" is always true, the condition is broken the "||" clearly should have been a "&&" this is causing the code to send a hotplug notify to the PCI root instead of to the gfx card, triggering a re-enumeration. Doing a grep for this on my personal DSDT collection shows that 55 of the 93 DSDTs in my collection have this issue!

Luckily this can be easily fixed by setting CHPD to 1 in the i915 driver, which is something which we should do anyways according to the
opregion documentation. So I wrote a patch doing this and submitted it upstream. Unfortunately this also does not help with the suspend/resume issue.

3. So the actual spurious wakeups are caused by HP using an external embedded controller (EC) on the "legacy-free" platform which they use for these laptops. Since these are not designed to use an external EC they lack the standard interface for this, so HP has hooked the EC up over I2C and using an ACPI GPIO event handler as EC interrupt.

These devices use suspend2idle (s2idle) instead of good old firmware handled S3, so the EC stays active during suspend. It does some housekeeping work which involves a round-trip through the AML code every minute. Normally EC wakeups are ignored durin s2idle by some special handling in the kernel, but this is only done for ECs using the standardized ACPI EC interface, not for this bolted on the
side model. I've started a discussion on maybe extending our ACPI event handling to deal with this special case.

For now as a workaround I ended up writing 2 more patches to allow blacklisting wakeup by ACPI GPIO event handlers on select models. This breaks wakeup by opening the LID, the user needs to wake the laptop with the powerbutton. But at least the laptop will stay suspended now.
me

Creating an USB3 OTG cable for the Thinkpad 8

The Lenovo Thinkpad 8 and also the Asus T100CHI both have an USB3 micro-B connector, but using a standard USB3 OTG (USB 3 micro-B to USB3-A receptacle) cable results in only USB2 devices working. USB3 devices are not recognized.

Searching the internet learns that many people have this problem and that the solution is to find a USB3 micro-A to USB3-A receptacle cable. This sounds like nonsense to me as micro-B really is micro-AB and is supposed to use the ID pin to automatically switch between modes dependent on the used cable.; and this does work for the USB-2 parts of the micro-B connector on the Thinkpad. Yet people do claim success with such cables (with a more square micro-A connector, instead of the trapezoid micro-B connector). The only problem is such cables are not for sale anywhere.

So I guessed that this means is that they have swapped the Rx and Tx superspeed pairs on the USB3 only part of the micro-B connector, and I decided to cut open one of my USB3 micro-A to USB3-A receptacle cables and swap the superspeed pairs. Here is what the cable looks like when it it cut open:



If you are going to make such a cable yourself, to get this picture I first removed the outer plastic isolation (part of it is still there on the right side in this picture). Then I folded away the shield wires till they are all on one side (wires at the top of the picture). After this I removed the metal foil underneath the shield wires.

Having removed the first 2 layers of shielding this reveals 4 wires in the standard USB2 colors: red, black, green and white. and 2 separately shielded cable pairs. On the picture above the separately shielded pairs have been cut, giving us 4 pairs, 2 on each end of cable; and the shielding has been removed from 3 of the 4 pairs, you can still see the shielding on the 4th pair.

A standard USB3 cable uses the following color codes:

  • Red: Vbus / 5 volt

  • White:  USB 2.0 Data -

  • Green: USB 2.0 Data +

  • Black: Ground

  • Purple: Superspeed RX -

  • Orange: Superspeed RX +

  • Blue: Superspeed TX -

  • Yellow: Superspeed TX -

So to swap RX and TX we need to connect purple to blue / blue to purple and orange to yellow / yellow to orange, resulting in:



Note the wires are just braided together here, not soldered yet. This is a good moment to carefully test the cable. Also note that the superspeed wire pairs must be length matched, so you need to cut and strip all 8 cables at the same length! If everything works you can put some solder on those braided together wires, re-test after soldering, and then cover them with some heat-shrink-tube:



And then cover the entire junction with a bigger heat-shrink-tube:



And you have a superspeed capable cable even though no one will sell you one.

Note that the Thinkpad 8 supports ACA mode so if you get an ACA capable "Y" cable or an ACA charging HUB then you can charge and use the Thinkpad 8 USB port at the same time. Typically ACA "Y" cables or hubs are USB2 only. So the superspeed mod from this blogpost will not help with those. The Asus T100CHI has a separate USB2 micro-B just for charging, so you do not need anything special there to charge + connect an USB device.
me

Xwayland randr resolution change emulation now available in Fedora 31

As mentioned in an earlier blogpost, I have been working on fixing many games showing a small image centered on a black background when they are run fullscreen under Wayland. In that blogpost I was moslty looking at how to solve this for native Wayland games. But for various reasons almost all games still use X11, so instead I've ended up focussing on fixing this for games using Xwayland.

Xwayland now has support for emulating resolution changes requested by an app through the randr or vidmode extensions. If a client makes a resolution change requests this is remembered and if the client then creates a window located at the monitor's origin and sized to exactly that resolution, then Xwayland will ask the compositor to scale it to fill the entire monitor.

For apps which use _NET_WM_FULLLSCREEN (e.g. SDL2, SFML or OGRE based apps) to go fullscreen some help from the compositor is necessary. This is currently implemented in mutter. If you are a developer of another compositor and have questions about this, please drop me an email.

I failed to get this all upstream in time for Fedora 31 final. But now it is all upstream, so 've backported the changes and created an update with the changes. This update is currently in updates-testing, to install this update run the following command:

sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2019-103a594d07
me

Capslock and keyboard layout indicator for plymouths diskcrypt password screen

As some of you running Fedora 31 may already have noticed, I have some good news to share. As part of my recent work on plymouth I've implemented a feature request which was first requested in 2012: support for an indicator that capslock is active while entering the disk unlock password for machines using full diskencryption. Besides the capslock indicator I've also added support for an indicator of the configured keyboard layout, since this sometimes also causes confusion:



And here is what it looks like when capslock is pressed:



If you're running Fedora 31 with full diskencryption then you may notice that the above screenshots are slightly different then what you have now. I've pushed an update to Fedora 31 updates-testing today which implements a few small tweaks to the theme after feedback from the design-team on the initial version. For those of you still on Fedora 30, this is coming to Fedora 30 too, it should show up in updates-testing with the next updates push.