Log in

No account? Create an account
22 May 2019 @ 06:44 pm
Better support for running games under Wayland (with GNOME3/mutter as compositor)  
First of all I do not want people to get their hopes up about $subject of this blogpost. Improving gaming support is a subjects which holds my personal interest and it is an issue I plan to spend time on trying to improve. But this will take a lot of time (think months for simple things, years for more complex things).

As I see it there are currently 2 big issues when running games under Wayland:

1. Many games show as a smal centered image with a black border (letterbox) around the image when running fullscreen.

For 2D games this is fixed by switching to SDL2 which will transparently scale the pixmap the game renders to the desktop resolution. This assumes that 2D games in general do not demand a lot of performance and thus will not run into performance issues when introducing an extra   scaling step. A problem here is that many games still use SDL1.2 (and some games do not use SDL at all).

I plan to look into the recently announced SDL1.2 compatibility wrapper around SDL2. If this works well this should fix this issue for all SDL1.2 2D games, by making them use SDL2 under the hood.

For 3D games this can be fixed by rendering at the desktop resolution, but this might be slow and rendering at a lower resolution leads to the letterbox issue.

Recently mutter has has grown support for the WPviewport extension, which allows Wayland apps to tell the compositor to scale the pixmap the app gives to the compositor before presenting it. If we add support to SDL2's Wayland backend for this then, this can be used to allow rendering 3D apps at a lower resolution and still have them fill the entire screen.

Unfortunately there are 2 problems with this plan:

  1. SDL2 by default uses its x11 backend, not its wayland backend. I'm not sure what fixes need to be done to change this, at a minimum we need a fix at either the SDL or mutter side for this issue, which is going to be tricky.

  2. This only helps for SDL2 apps, again hopefully the SDL1.2 compatibility wrapper for SDL2 can help here, at least for games using SDL.

2. Fullscreen performance is bad with many games.

Since under Wayland games cannot change the monitor resolution, they need to either render at the full desktop resolution, which can be very slow; or they render at a lower resolution and then need to do an extra scaling step each frame.

If we manage to make SDL2's Wayland backend the default and then add WPviewport support to it then this should help by reducing an extra memcpy/blit of a desktop-sized pixmap. Currently what apps which use scaling do is:

  1. render lower-res-pixmap;

  2. scale lower-res-pixmap to desktop-res-pixmap

  3. give desktop-res-pixmap to the compositor;

  4. compositor does a hardware blit of the desktop-res-pixmap to the framebuffer.

With viewport support this becomes:

  1. render lower-res-pixmap;

  2. give low-res-pixmap to the compositor;

  3. compositor uses hardware to do a scaling blit from the low-res-pixmap to the desktop-res framebuffer

Also with viewport support, the compositor could in the case of there only being the one fullscreen app even keep the framebuffer in lowres and use a hardware scaling drm-plane to send the low-res framebuffer scaled to desktop-res to the output while only reading the low-res framebuffer from memory saving a ton of memory bandwidth. But this optimization is going to be a challenge to pull off.