Optimus and Ubuntu 12.10 (Part 4)

This post is the fourth of a series of posts on tweaking Ubuntu 12.10 to exploit Optimus technology on my Lenovo W530 to the extent I need. Make sure you are familiar with the context, especially the objectives and constraints as described in Part 1, Part 2 and Part 3.

Moving windows from the primary screen to the external VGA screen

As it has been explained before, windows cannot cross screen boundaries, and the GNOME desktop cannot span multiple X screens even in the case when the these screens belong to the same display (X instance). It should be obvious that this is also the case with screens that belong to different X servers.

The objectives defined in the previous parts require the ability to extend the GNOME desktop to the monitor attached to the external VGA port, to run presentations using two monitors and to clone the primary monitors content to the external monitor.

In order to meet one of the objectives, one could mirror, or better said clone the content of one screen to another screen. To be even more precise, only a given portion of the screen has to be cloned to other viewport, the area of the other screen that is displayed by the external monitor. There is a userspace tool called hybrid screenclone to perform exactly this task - find more on this tool below.

Extending the desktop to a screen of the other X server

Thinking further, in order to be able to extend the desktop and thereby meet other objectives, one could set up a mock monitor in the first X server, and then, clone the content to the screen of the other X server so it would show up on the external monitor.

My first approach was to examine the video outputs of the integrated graphics device, and found VGA[12] that is wired to /dev/null (see Part 2). With various xrandr commands I could force the unused, thus always disconnected output to be configured with a fixed resolution, right of the primary monitor. A screenshot confirmed I was half way through: it contained a black are next to the primary desktop. I could even drag windows onto this black area, however, the desktop would not extend to this portion of the X screen. Also, libreoffice refused to start the slideshow in multi-monitor mode, as it only detected one connected monitor. While the idea was not completely useless, this approach did turn out not to be usable in production.

A bit of googling revealed, that the author of hybrid screenclone also maintains a patch against the intel video driver which adds a dynamically configurable virtual output - enabling exactly the scenario I was targeting.

Intel driver hack

The listing below takes the reader through the process of creating and installing a package containing the patched version of the intel video driver.

$ mkdir /tmp/foo && cd /tmp/foo # we are going to compile in tmpfs/RAM
$ sudo aptitude build-dep --schedule-only  xserver-xorg-video-intel
$ sudo aptitude # review interactively what is going to be installed
$ apt-get source xserver-xorg-video-intel
$ cd xserver-xorg-video-intel-2.20.9/
$ wget https://raw.github.com/liskin/patches/master/hacks/xserver-xorg-video-intel-2.20.2_virtual_crtc.patch
$ # the newer patch did not match.
$ patch -p1 < xserver-xorg-video-intel-2.20.2_virtual_crtc.patch
$ # now update the version to show this is a patched package
$ # NEVER alter packages without making it clear in the package version!
$ # I prepend this to debian/changelog:
$ mv debian/changelog debian/changelog.old && cat <<EOF > debian/changelog
xserver-xorg-video-intel (2:2.20.9-0ubuntu2+virtual-crtc) quantal; urgency=low

  [ Tibor Bősze ]
  * Add xserver-xorg-video-intel-2.20.2_virtual_crtc.patch

 -- Tibor Bősze <tibor.boesze@gmail.com>  Sun, 13 Jan 2013 03:15:00 +0200

EOF
$ cat debian/changelog.old >> debian/changelog && rm debian/changelog.old
$ # now build and install the package
$ dpkg-buildpackage -b
$ sudo dpkg -i ../xserver-xorg-video-intel_2.20.9-0ubuntu2+virtual-crtc_amd64.deb
$ # as you see, the package version will clearly show that this is a patched package
$ # prevent the package to be automatically updated
$ sudo aptitude hold xserver-xorg-video-intel

After a reboot, the command below will active the virtual monitor. The graphical display manager will not show any second display, however, creating a screenshot quickly confirms that the second virtual monitor is active, and the desktop correctly extends to it. Also, libreoffice impress can finely use it for running the slideshow in dual monitor mode.

$ xrandr --output LVDS2 --auto --output VIRTUAL --mode 800x600 --right-of LVDS2

Screenclone

To render this post complete, below are listed the commands to download and compile the tool.

$ # we are still doing stuff in tmpfs/RAM
$ aptitude install git-core
$ git clone git://github.com/liskin/hybrid-screenclone.git
$ cd hybrid-screenclone && make
g++ -std=c++0x -g -Wall    screenclone.cc  -lpthread -lX11 -lXdamage -lXtst -lXinerama -lXcursor -o screenclone
screenclone.cc:18:33: fatal error: X11/Xcursor/Xcursor.h: No such file or directory
compilation terminated.
make: *** [screenclone] Error 1
$ apt-file search Xcursor.h
libxcursor-dev: /usr/include/X11/Xcursor/Xcursor.h
$ aptitude install libxcursor-dev
$ make
g++ -std=c++0x -g -Wall    screenclone.cc  -lpthread -lX11 -lXdamage -lXtst -lXinerama -lXcursor -o screenclone
screenclone.cc:24:37: fatal error: X11/extensions/Xinerama.h: No such file or directory
compilation terminated.
make: *** [screenclone] Error 1
$ aptitude install libxinerama-dev libxdamage-dev libxtst-dev
$ make
$ mv screenclone ~/optimus/

By default the tool will clone the first screen of the first display to the first screen of the second one, that is, :0.0 to :1.0. This almost completely fits my use case, I added the parameter -x 1 which limits the content to be copied to the area of the screen that is displayed by the second monitor, the VIRTUAL output in my case.

$ # clone the viewport of the VIRTUAL output from :0.0 to the top left corner of :1.0
$ ~/optimus/screenclone -x 1 

Part 5 will revisit the issue of changing framebuffer number assignment and provide a more elegant solution to fixing usplash.

Optimus and Ubuntu 12.10 (Part 3)

This post is the third of a series of posts on tweaking Ubuntu 12.10 to exploit Optimus technology on my Lenovo W530 to the extent I need. Make sure you are familiar with the context, especially the objectives and constraints as described in Part 1 and Part 2.

My previous post on the topic describes how to control the power state of the DIS with vgaswitheroo that is part of the stock Ubuntu 12.10 kernel. It explains key terms related to X also shows alternative ways to use both the DIS and IGD within the same X server.

As the "single X server with 2 screens" approach is not an option until the related bugfix is available in the official repositories, I investigated how a second X server could be used to reach my goals. As a first step I analysed existing solutions in the area. Starting a second X server is core concept of Bumblebee, that enables rendering on the DIS, and then uses VisualGL to copy the content of individual windows back to the screen handled by the primary X server that uses the IGD only. Unfortunately, this project does not support external monitors in the case when ports are wired to DIS only. Also, it currently does not seem to be mature and stable enough for my production thinkpad. Nevertheless, it gave me a starting point...

Two X servers with one screen each

Starting a separate X instance to handle the DIS enables better isolation/sandboxing but also introduces additional issues.

• First of all, the primary X instance has to be configured in a way that it will not grab any resources of the DIS, else the secondary instance fails to starts up with the message "No screens found".
• Similarly, the second instance has to be configured explicitly to only use the DIS and related monitors.
• I used configuration to override the actual connection status of the external VGA port and always enable VGA internally. Without an enabled monitor X would not start up.
• The desktop cannot be extended to a monitor of the second X instance.
• One could use a separate window manager on the second X instance - twm is very lightweight one. I stayed with a naked X for reasons described below.
• With two X instances, I get a cursor on both displays. Without a pointing device, X would not start either. There are solution to use a mock input device, but anyway, having a two cursors that move in tandem on the two monitors is not critical. With my current configuration, the touchpad only controls the cursor on my primary X, while the trackpoint controls both.
• Finally, the second X server will run as root, essentially with access control disabled so mortals can open windows on the second X as well.

$ cd ~
$ mkdir optimus && cd optimus
$ cat >>xorg.conf.nouveau<<EOF
Section "Modes"
 Identifier "FallbackModes" # Mode to use if External-VGA is diconnected
 Modeline "1024x768"   65.00  1024 1048 1184 1344  768 771 777 806 -hsync -vsync
EndSection

Section "ServerLayout"
 Identifier "Layout0"
 Screen "Screen0"
 Option "AutoAddDevices" "false"
 Option "AutoEnableDevices" "false"
 Option "AutoAddGPU" "false"
EndSection

Section "Monitor"
 Identifier "External-VGA"
 UseModes "FallbackModes"
 Option "Enable" "true" # always enabled
 Option "PreferredMode" "1024x768"
EndSection

Section "Monitor"
 Identifier "LCD"
 Option "Enable" "false" # always disabled
EndSection

Section "Device"
 Identifier "DIS"
 Driver "nouveau"
 BusID "PCI:1:0:0"
 Option "HWCursor" "true"
 # The numbers in output names change based on whether IGD or DIS is           
 # initialized first by the kernel. This tweak takes care of both cases.
 Option "Monitor-VGA-1" "External-VGA"
 Option "Monitor-VGA-2" "External-VGA"
 Option "Monitor-LVDS-1" "LCD"
 Option "Monitor-LVDS-2" "LCD"
EndSection

Section "Screen"
 Identifier "Screen0"
 Device "DIS"
 Monitor "External-VGA"
 DefaultDepth 24
 SubSection "Display"
  Depth 24
 EndSubSection
EndSection
EOF

$ cat >>xorg.conf.intel<<EOF
Section "ServerLayout"
   Identifier "Layout0"
   Screen "Screen0"
   Option "AutoAddDevices" "true"
   Option "AutoEnableDevices" "true"
   Option "AutoAddGPU" "false"
EndSection

Section "Device"
   Identifier "IGD"
   Driver "intel"
   BusID "PCI:0:2:0"
EndSection

Section "Screen"
   Identifier "Screen0"
   Device "IGD"
   DefaultDepth 24
   SubSection "Display"
      Depth 24
   EndSubSection
EndSection
EOF
$ sudo cp xorg.conf.intel /etc/X11/
$ sudo rm /etc/X11/xorg.conf # this is the link to the 2 screen config
$ sudo ln -s /etc/X11/xorg.conf.intel /etc/X11/xorg.conf
$ sudo lightdm restart

If there is no explicit default xorg configuration, then the X server will hold both /dev/dri/card[01] in which case the second X instance could not start up.

Starting, disabling and restoring external VGA output

I used the following small script to ensure DIS is powered on, spawn the X server, decorate it with a random background. After pressing Enter, X is terminated and DIS powered off.

#!/bin/bash

msg() {
 echo "******* $1"
}

msg "Ensuring DIS is powered on."
echo ON | sudo tee /sys/kernel/debug/vgaswitcheroo/switch
msg "Launching X server on display :1."
sudo /usr/bin/X -ac -audit 0 -config /home/tibi/optimus/xorg.conf.nouveau -sharevts -verbose 1 -logverbose 9 -logfile /tmp/Xorg.1.log -nolisten tcp -noreset :1 &

PID=$!
sleep 2
msg "PID is $PID, log goes to /tmp/Xorg.1.log."
# bonus: get a random background
BACKGROUND=$(find /home/tibi/Pictures -maxdepth 1 -name '*.jpg' | sort --random-sort | head -1)
msg "Setting background: $BACKGROUND"
gm display -window root -display :1.0 $BACKGROUND
msg "DONE."

msg "Press Enter to terminate and clean up."
read
msg "Terminating X server..."
sudo kill $PID
sleep 2
echo OFF | sudo tee /sys/kernel/debug/vgaswitcheroo/switch
msg "Discrete graphics device powered off."

I can safely disable and restore the external display without stopping the X server with the following commands:

### DISABLE
# numbers change across reboot, one of the two will work, other will print an error.
xrandr -d :1 --output VGA-1 --off
xrandr -d :1 --output VGA-2 --off
echo OFF | sudo tee /sys/kernel/debug/vgaswitcheroo/switch

### RESTORE
echo ON | sudo tee /sys/kernel/debug/vgaswitcheroo/switch
# numbers change across reboot, one of the two will work, other will print an error.
xrandr -d :1 --output VGA-1 --auto
xrandr -d :1 --output VGA-2 --auto

Usplash and plymouth issues

As it has been stated earlier, the issue around missing usplash/plymouth turned out to be connected to the random order in which the kernel initialized the graphics devices.

$ lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09)
01:00.0 VGA compatible controller: NVIDIA Corporation GK107 [Quadro K1000M] (rev a1)
$ # framebuffers - the order (0 and 1) changes randomly across reboots
$ cat /proc/fb
0 inteldrmfb
1 nouveaufb

Usplash is hardcoded to render to /dev/fb0 which is fine if the IGD is initialized first by the kernel, but not good if /dev/fb0 is associated with the DIS framebuffer. Kernel boot fbcon=map:1 is of no help in solving this issue. One workaround to mediate this problem is blacklisting nouveau, and loading it later, after usplash is started on the IGD framebuffer.

$ cat <<EOF | sudo tee /etc/modprobe.d/blacklist-nouveau.conf
# blacklist nouveau to force IGD framebuffer to take precedence
blacklist nouveau
EOF
$ sudo update-initramfs -c -k all

### and later during the boot process...

modprobe nouveau # we need modprobe as it was blacklisted during boot
udevadm settle # wait until all udev events are handled
echo OFF > /sys/kernel/debug/vgaswitcheroo/switch

This approach is rather a quick and dirty workaround. Technically, it would be enough - and much more elegant - to merely ensure the proper order of module loading by tinkering around with initram scripts; this is subject to further investigation.

About Part 4

Part 4 shows how this second naked X server can be used to achieve the objectives outlined in the previous post.

Optimus and Ubuntu 12.10 (Part 2)

This post is a follow up on Optimus and Ubuntu 12.10. Please make sure you are familiar with the context.

Setting the scope and objectives

This series of posts takes the reader through Nvidia Optimus related tweaks I applied on my Lenovo W530. My goal is to get a very stable system with extended battery life, and the ability to connect an external projector to the VGA port and cover the following use cases:

• Extend the desktop to the external monitor.
• Get a cloned output of the primary monitor to the external monitor with panning support - this means, that in the case the external monitor's resolution is smaller, a smaller viewport will follow the mouse and show a cropped clone of the desktop's content. The viewport will follow the mouse to show the area of interest.
• Run LibreOffice presentations with the external monitor showing the current slide and the primary monitor showing the presentation overview, notes and time.
• Never ever get X freezes or kernel lockups on suspend/resume with or without an external monitor connected.
• Switching to virtual terminals should always work in a bulletproof manner. The box is a workhorse, cannot allow hiccups.
• Might sound like a small detail, but a properly displayed usplash/plymouth is also important, not only for cosmetic purposes.

All these features I am able to use today on my T400 (that includes a single Intel graphics device), with the help either the GNOME GUI, or xrandr scripts in special cases. However, on the W530 with it's two graphics devices, both the the external VGA and mini DisplayPort are wired to the nvidia chip so I will not able to hook up an external monitor with only the integrated graphics device enabled. The discrete device is a resource hog on one hand, and the W530 is not able to properly boot with only discrete graphics enabled in the case hardware virtualization support is also enabled.

I took the decision to push myself and tweak Ubuntu 12.10 until my goals and criteria is met. I started looking for existing solutions to get Optimus working, and educating myself on the topic. On the road I took the decision not to use Bumblebee or other unstable/immature components that could impose stability issues of additional risks of X or kernel lockups.

Further, I also set a secondary objective: trying to reach my goals with open-source components only. So, I tried to avoid the proprietary Nvidia driver as much as possible.

All these goals are met by now, with the open source nouveau driver and set of tricks and tweaks. This writing focuses more on the way of investigating alternatives and achieving the goals as opposed to merely providing the final solution.

Stock Ubuntu with Optimus enabled in firmware

Having applied the lightdm tweak from the earlier post, the blank screen issues disappeared completely even with Optimus enabled in firmware setup. The issue around missing usplash/plymouth turned out to be connected to the random order in which the kernel initialized the graphics devices.

$ lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09)
01:00.0 VGA compatible controller: NVIDIA Corporation GK107 [Quadro K1000M] (rev a1)
$ # framebuffers - the order (0 and 1) changes randomly across reboots
$ cat /proc/fb
0 inteldrmfb
1 nouveaufb
$ # vgaswitheroo state - the order (0 and 1) changes randomly across reboots
$ sudo cat /sys/kernel/debug/vgaswitcheroo/switch
0:IGD:+:Pwr:0000:00:02.0
1:DIS: :Pwr:0000:01:00.0

With Optimus enabled, the X sever starts up properly, but external monitors are not recognized out of the box. Power state of the discrete graphics device can be controlled through vgaswitheroo, but switching between the discrete (DIS) and integrated graphics device (IDG) cannot be performed. Why is that, and what is vgaswitcheroo at all?

vgaswitcheroo

It is a mechanism built into the kernel that allows to switch between multiple graphics devices if (and only if) the configuration is equipped with a hardware multiplexer.It is documented not to work with Nvidia Optimus, as it does not incorporate a hardware multiplexer.

Nevertheless, it automatically loaded (it is part of the stock Ubuntu kernel) and allowed me to change the power state of the discrete graphics device. There are other solutions for changing the power state either automatically or manually, such as acpi_call or bbswitch. Both of these other solution have one thing in common - they are kernel modules not considered stable or mature. So my choice is to go with vgaswitcheroo for power state management, using the commands as shown below:

$ sudo cat /sys/kernel/debug/vgaswitcheroo/switch # display status
$ echo OFF | sudo tee /sys/kernel/debug/vgaswitcheroo/switch # power off DIS
$ echo ON | sudo tee /sys/kernel/debug/vgaswitcheroo/switch # power on DIS

I would like to stress again, that it is used for power state management only, as actual switching cannot be done with Optimus. One can verify power state of the discrete device with lspci but also with the power led of a monitor connected to the VGA port.

About X servers, displays, screens and monitors

The core concept of Bumblebee is to start up a second X server to enable rendering on the DIS, and then use VisualGL to copy the content of individual windows back to the screens handled by the primary X server that uses the IGD only. This approach is not necessary the best one.

Contrary to what many might believe, one single X server can handle many screens and many video cards. Let me take a bottom up approach to define the terms and structure used by X - or at least my understanding thereof.

1. A Monitor is a graphical output device, such as a projector, an LCD panel, CRT or other kind of monitor. They are physically connected (in a static/persistent or dynamic/transient manner) to one of the video cards.
2. A Screen is a virtual area where applications can render their windows. A screen can be composed of multiple monitors running on the same video card, each monitor can show a portition/viewport of the screen - they can even overlap or show cloned content. (It should be noted that they can also be fully virtual as in the case of VNC.) This way it is obvious that windows within the same screen can overlap or span multiple monitors. Windows cannot overlap or span multiple screens. Window managers can run independently on each screen.
3. One instance of X server is associated with a display. It can handle multiple video cards and multiple screens, these screens share input devices such as keyboard and mouse. One screen can be active at a time, and the active screen can be changed with the mouse (although there also existed a small command line utility for this, called switchscreen).
4. Xinerama is an extension that allows Monitors even from multiple video cards to be combined into one screen. It does not support hardware acceleration nor dynamic reconfiguration.

Revisiting the core concept of Bumblebee in light of the information above, one intuitive alternative would be the use of a single X server, with one screen for the IDG, and another screen for the DIS. Monitors can be turned on and off dynamically, and based on that, the screen size can also be adjusted dynamically. One thing that needs to be investigated is the behaviour of the X server when one of the video devices is powered off.

How wiring affects what X identifies

DIS with nouveau driver:

• LVDS-1 or LVDS-2 which is a dead end and always shows as disconnected
• VGA-1 or VGA-2 which is the external VGA output wired to the nvidia card
• DP-1
• DP-2
• DP-3

IDG with the intel driver - note the differences in naming, no hyphen is used:

• LVDS2 or LVDS1 which is wired to the LCD panel
• VGA2 or VGA1 which would be supported by the card but is a dead end, not wired to anything

One X server with two screens

The configuration below does not address the change of output names caused by the non deterministic order in which the kernel initialized the graphics devices.

cat <<EOF | sudo tee /etc/X11/xorg.conf.2screens
Section "ServerLayout"
  Identifier "Layout0"
  Screen 0 "Screen0"
  Screen 1 "Screen1" RightOf "Screen0" # two screens side by side
  Option "AutoAddDevices" "true"
  Option "AutoEnableDevices" "true"
  Option "AutoAddGPU" "true"
EndSection

Section "Device"
  Identifier "IGD"
  Driver "intel"
  BusID "PCI:0:2:0" # this can be read from the lspci output above
EndSection

Section "Screen" # IDG screen will not have explicit monitor configuration
  Identifier "Screen0"
  Device "IGD"
  DefaultDepth 24
  SubSection "Display"
    Depth 24
  EndSubSection
EndSection

Section "Monitor"
  Identifier "External-VGA"
  Option "Enable" "true" # will always show as connected, so we have at least one active output
  Option "PreferredMode" "1024x768"
EndSection

Section "Monitor" 
  Identifier "LCD"
  Option "Enable" "false" # this is a dead end - could have used "Ignore" "true"
EndSection

Section "Device"
  Identifier "DIS"
  Driver "nouveau"
  BusID "PCI:1:0:0" # this can be read from the lspci output above
  Option "Monitor-VGA-2" "External-VGA" # connecting output names with monitor config
  Option "Monitor-LVDS-2" "LCD"
  Option "HWCursor" "true"
EndSection

Section "Screen" # DIS screen will only have one active monitor: External-VGA
  Identifier "Screen1"
  Device "DIS"
  Monitor "External-VGA"
  DefaultDepth 24
  SubSection "Display"
      Depth 24
  EndSubSection
EndSection
EOF
$ # now create a link xorg.conf that will point to our actual configuration.
$ sudo ln -s /etc/X11/xorg.conf.2screens /etc/X11/xorg.conf 
$ sudo service lightdm restart # ... and log in
$ DISPLAY=:0.1 xclock # start xclock on screen 1 and smile

The configuration above would work fine. Unfortunately, the mouse could not leave screen 0 when I was testing - it wrapped meaning when I pulled the pointer outside of screen 0 on the right side, it entered on the left side of screen 0 instead of screen 1. This is an upstream bug, marked as "fix released", however a working package is not available in the official Ubuntu 12.10 repositories yet.

Powering off the DIS without any provision while the screen was actively used froze the kernel, however powering it off once after turning off the output devices seemed stable.

echo OFF | sudo tee /sys/kernel/debug/vgaswitcheroo/switch
# freezes the kernel, requires hard reset

# if the output is disabled first like this:
xrandr --screen 1 -q # see which devices are connected
xrandr --screen 1 --output VGA-2 --off # switch off connected devices
echo OFF | sudo tee /sys/kernel/debug/vgaswitcheroo/switch
# works fine, and screen can be restored when the devices reactivated.

Unfortunately, this approach is not usable at in production until the bugfix making me unable to switch screens is resolved.

It should be also noted, that when running with a two screens configuration, windows cannot span or cross screens, and the desktop can not be simply extended either - without further magic to be covered later. However, any application can be opened on any of the two screens.

Xinerama

This old extension allows one to extend the desktop to screen 1 with a single line of configuration change. The following line has to be added to the ServerLayout section:

Option "Xinerama" "on"

Two downsides made me search for other alternatives: The performance with Xinerama is very very poor, but even more important, Xinerama does not support dynamic configuration changes, so resizing, rearranging the monitors or switching outputs on or off is simply not supported. With the current set of objectives this simply means Xinerama is not what I am looking for.

About Part 3

Part 3 explains setting up two X servers and various related tweaks to fulfill all objectives listed above.

Fixing the visual appearance of 32bit applications on Ubuntu 12.10 64bit

Both Lotus Notes and Lotus Sametime are available as 32 bit applications. Ubuntu 12.10 provides multiarch support out of the box, which refers to the ability of a system to install and run applications of multiple different binary targets on the same system, in this case i386-linux-gnu application on an amd64-linux-gnu system. Nevertheless, the visual appearance of 32 bit applications is not in-line with the look-and-feel of the amd64 ones. This is especially annoying as I am going to have Lotus Notes in front of me on a daily basis.

The following section describes how to find and install the set of 32 bit packages that will resolve the issue as much as possible. First some background information. Lotus notes uses an eclipse based framework, and eclipse uses SWT, their cross platform standard widgeting toolkit. SWT on linux uses GTK+, the gimp toolkit as the native backend for rendering widgets. Ubuntu 12.10 provides a desktop environment that mainly uses GTK3, a version of GTK+ available since 2011. GTK3 is backwards compatible with GTK2 which is available since 2002. However, one can install GTK2 and GTK3 themes and rendering engines separately...

To find out more about why the 32bit applications do not render as we would expect, let us install a 32bit GTK2 and GTK3 demo application.

$ sudo aptitude install gtk2.0-examples:i386 gtk-3-examples:i386 --without-recommends
$ gtk3-demo 
Gtk-Message: Failed to load module "overlay-scrollbar"

(gtk3-demo:5164): Gtk-WARNING **: Theme parsing error: gtk-widgets.css:62:17: Theming engine 'unico' not found
Gtk-Message: Failed to load module "canberra-gtk-module"
$ gtk-demo
Gtk-Message: Failed to load module "overlay-scrollbar"

(gtk-demo:4930): Gtk-WARNING **: Unable to locate theme engine in module_path: "murrine",
Gtk-Message: Failed to load module "canberra-gtk-module"

It can be seen from the output what modules and theming engines could not be loaded. Now let us take incrementally try to resolve the issues.

$ # let us first take a try at installing 32bit overlay-scrollbar
$ aptitude install overlay-scrollbar-gtk2:i386 overlay-scrollbar-gtk3:i386 --without-recommends --simulate
The following NEW packages will be installed:
  overlay-scrollbar-gtk2:i386{b} overlay-scrollbar-gtk3:i386{b} 
0 packages upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 79.7 kB of archives. After unpacking 250 kB will be used.
The following packages have unmet dependencies:
 overlay-scrollbar-gtk2:i386 : Depends: overlay-scrollbar:i386 which is a virtual package.
 overlay-scrollbar-gtk3:i386 : Depends: overlay-scrollbar:i386 which is a virtual package.
The following actions will resolve these dependencies:

     Keep the following packages at their current version:
1)     overlay-scrollbar-gtk2:i386 [Not Installed]        
2)     overlay-scrollbar-gtk3:i386 [Not Installed]        

Accept this solution? [Y/n/q/?] q
Abandoning all efforts to resolve these dependencies.
Abort.
$ # as it can be seen, there is a missing dependency, so it cannot be installed.

$ # let us take a try at unico theming engine
$ aptitude install gtk3-engines-unico:i386 --without-recommends --simulateThe following NEW packages will be installed:
  gtk3-engines-unico:i386{b} 
0 packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 9,066 B of archives. After unpacking 57.3 kB will be used.
The following packages have unmet dependencies:
 gtk3-engines-unico : Conflicts: gtk3-engines-unico:i386 but 1.0.2+r139-0ubuntu2 is to be installed.
 gtk3-engines-unico:i386 : Conflicts: gtk3-engines-unico but 1.0.2+r139-0ubuntu2 is installed.
The following actions will resolve these dependencies:

     Remove the following packages:
1)     gtk3-engines-unico          
2)     light-themes                
3)     ubuntu-artwork              
4)     ubuntu-desktop              

Accept this solution? [Y/n/q/?] q
Abandoning all efforts to resolve these dependencies.
Abort.
$ # This time, there is a conflict: 
$ # the 32bit and 64bit versions of unico are mutually exclusive.

$ # to make the long story short, the following packages could be installed:
$ #   - libcanberra-gtk3-module:i386
$ #   - libcanberra-gtk3-module:i386
$ #   - gtk2-engines-murrine:i386
$ sudo aptitude install libcanberra-gtk3-module:i386 libcanberra-gtk3-module:i386 gtk2-engines-murrine:i386 --without-recommends

$ # both the output and the visual appearance confirm things got better...
$ gtk-demo 
Gtk-Message: Failed to load module "overlay-scrollbar"
tibi@grizzly:~$ gtk3-demo 
Gtk-Message: Failed to load module "overlay-scrollbar"

(gtk3-demo:6039): Gtk-WARNING **: Theme parsing error: gtk-widgets.css:62:17: Theming engine 'unico' not found
$ gtk-demo 
Gtk-Message: Failed to load module "overlay-scrollbar"

After installing the packages as shows above, both Lotus Notes and Lotus Sametime render the widgets much nicer. The theme is not perfectly inline with the out of the box Ubuntu 12.10 theme, but I can live without having the overlay scrollbar in Notes and Sametime.

Lotus Notes migration to Ubuntu 12.10 64bit

Having made the first steps in securing the W530 (see previous posts) and having set up further security & compliance related components as dictated by corporate policy, I was ready to start migrate my data from the old t400 thinkpad to the new box. The most critical part of the data to be migrated is Lotus Notes data. I am proficient with the operating system and all other software components I have on my old laptop but I am not a Lotus Notes experts at all, it is mostly a black box for me and a potential source of trouble challenges. The old laptop is running Ubuntu 10.04 32bit while I set up the new one with Ubuntu 12.10 64bit as mentioned in previous posts.

Backing up Lotus Noted Data

To clarify the scope, I was migrating from my data from a Lotus Notes 8.5.2 environment on Ubuntu 32bit to a 8.5.3 one on 64bit. Additionally, I also decided to clean the cruft accumulated over the last 7 years... I checked for defunct workspace icons and any obsolete local replicas and removed those. Then sorted all email in my inbox to their appropriate folders (usually, I do this while waiting for connecting flights at airports) and archived all emails from before 2013. I also changes the archive settings to save any new items to a new archive file. After cleaning up, I was ready to migrate my data.

First and foremost, do know what you have to migrate. I wanted to migrate the minimum required amount of settings but not less. Lotus Notes stores data in the proprietary Network Storage Facility format, these files have nsf os NSF extension. These files are self contained document-oriented databases storing semi structured data - the application logic and view definitions are also incorporated. There are some files like cache.nsf or log.nsf that do not carry valuable information, others will have to be backed up.

My mail file is just a normal nsf file, however it is encrypted with the private key stored in my ID file. It is a good practice to keep a backup of your ID file anyway, I have seen people believe that remembering their Lotus Notes password would be enough to recover a corrupt or deleted ID file...

Workspace definitions and icons are stored in the file desktop8.ndk, which also needs to be backed up. I also made sure to backup my custom signatures.

There are some settings stored in the eclipse workspace, but I did not migrate any of those. I was trying hard to start with an clean as possible environment and ended up with using the following command to create the backup on the old machine:

# backup all databases except log.nsf, the id file, desktop file and signatures
find ~/lotus/notes \( -iname '*.nsf' -o -iname '*.id' \
  -o -iname '*.htm' -o -iname 'desktop8.ndk' \) \
  -not -iname 'log.nsf' -print0 | tar cspzf \
/media/SAMSUNG/backup/t400-20130103/lotus_notes_data-sparse.tar.gz --null -T -

Installation and migration

On the W530, before restoring the files, I installed Lotus Notes and let it create the data directory, and initialize the ~/lotus/notes/data folder with default files. To do so, I started it, waited until the splash screen disappears and the first application window appears asking me inputs for initial setup. At this very first screen, I pressed cancel to abort and exit. I made sure all Lotus Notes processes terminated - one can use the "Lotus Notes Zap" utility for this.

Then I extracted the data and issues some more tweaks using the commands listed below. Other minor customization like setting the theme and default font was done from within the graphical interface, but nothing worth to document...

cd ~ # make sure we are in the right directory
tar xzf /media/tibi/SAMSUNG/backup/t400-20130103/lotus_notes_data-sparse.tar.gz
# start Lotus Notes... and choose "office network" as current location, look around, then exit.

chmod -x ~/lotus/notes/data/*.gif # just some cosmetics...
# I use English language, but more with my preferred date format
cat <<EOF >> ~/lotus/notes/data/notes.ini
DateOrder=YMD
ClockType=24_HOUR
DateSeparator=-
TimeSeparator=:
EOF
# restart notes and verify the date format used in your inbox.

The result is a fully functional install with all the data migrated, however it is lacking a few more tweaks to improve the visual appearance of native widgets. These tweaks are going to be covered in the next post.

Optimus and Ubuntu 12.10

The W530 comes with an integrated intel GPU and a discrete graphics card from nvidia. Out of the box, the firmware is set up to run with Optimus enabled, but it provides option to enable only integrated or only discrete graphics.

I do not want to play any games. This thinkpad is a workhorse. I prefer a very stable system that is able run off battery for an extended duration as I am travelling a lot. I also need to be able to use the external VGA port, I am not so keen on using the mini DisplayPort at the moment.

Stock Ubuntu with Optimus enabled

I started using the laptop with the factory settings as far as graphics setup goes. Between Xmas and New Year's Eve I did not have any chance to test it with an external display, but I confirmed both integrated and discrete graphics are active.

$ lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation 3rd Gen Core processor Graphics Controller (rev 09)
01:00.0 VGA compatible controller: NVIDIA Corporation GK107 [Quadro K1000M] (rev a1)

Unfortunately, I experienced occasional blank screen on startup - black screen with a X mouse pointer to be precise - that could be worked around by switching to a virtual terminal (e.g. Ctrl-Alt-F1), logging in and restarting lightdm. Also, plymouth failed to show up randomly on startup and shutdown, which was a mostly cosmetic issue, but not a good sign at all. Although this situation was technically manageable, I was looking for a very deterministic, stable configuration, so I changed the firmware settings.

Stock Ubuntu with discrete graphics only

Switched to discrete graphics, which resulted into a garbled screen, an I was not even able to switch to a virtual terminal. This was with the opensource nouveau driver. At that time, I have not tested the proprietary nvidia drivers, but rather quickly changed to integrated graphics in firmware setup.

Stock Ubuntu with integrated graphics only

Switched to integrated graphics, which sometimes worked, but in the majority of case I ended up with an error telling me "The system is running in low-graphics mode". Here, too, I was able to switch to a virtual terminal and recover gracefully. After some investigation it turned out the situation was related to a race condition around lightdm that prevented proper graphical startup - see this bug report for further details. The W530 is up in 4-5 seconds after POST with the fast SSD and is more prone to race conditions therefore.

In the bug report referred to above I found multiple suggested workarounds mainly focused on local scripts restarting lightdm twice. I have applied a slightly different workaround and optimized it to fit my needs:

$ sudo patch /etc/init/lightdm.conf <<EOF
--- /etc/init/lightdm.conf    2012-10-09 17:00:02.000000000 +0200
+++ /tmp/init/lightdm.conf    2013-01-06 00:34:26.061236424 +0100
@@ -42,10 +42,12 @@
         plymouth quit || :
         exit 0
     fi
    fi

+    # line added to prevent race condition resulting into "The system is running in low-graphics mode"
+    sleep 0.5s
    exec lightdm
end script

post-stop script
     if [ "$UPSTART_STOP_EVENTS" = runlevel ]; then
EOF

This tweak introduces negligible delay at startup, and I have not experience any graphics issues ever since. With the discrete graphics card disabled in firmware I was able to run the laptop 12 hours on battery, of which about 3 hours were spent suspended, and the other 9 without heavy workloads, mostly.

There is one very important potential drawback of this approach. On the W530, both the VGA and mini DisplayPort are wired to the nvidia chip - at least this is what I conclude from the user guide and from what I have found on the internet. If this turns out to be the case I will not able to hook up an external display with only the integrated card enabled. This made me unable to use a projector. We'll see once I get access to an external display to test with.

To be continued...

The current writing is the first of a series of posts on the Optimus, W530 and Ubuntu 12.10 tuple. Make sure you do not miss Part 2!

Security & compliance related adjustments

After having installed the base system, it was time to go productive. This required migrating my work-related stuff from the T400 box to my new W530. But before copying any sensitive information to the thinkpad, a few security measures had to be taken...

According to ITCS 300, the information I need to store on my thinkpad makes whole disk encryption necessary. The purpose of disk encryption is to protect sensitive information even in the case when a physically present attacker gains physical control over your computer or hard drive. Encrypting critical files or even the whole home partition would not be enough - think about the swap partition where in-memory data is saved to. Even if software based full disk encryption is used, there are cold boot attack methods to gain access to encryption keys or other memory content. This is why Bitlocker of even LUKS based full disk encryption alone do not yield an universal solution.

Firmware setup

I acquired a self encrypting disk featuring FIPS certified hardware based full disk encryption with AES-256. It allows me to set up an ITCS 300 compliant thinkpad without any impact on performance or battery life imposed by software based full disk encryption such as LUKS. Moreover, it is more secure than LUKS, as the encryption key never has to leave the SSD, so is not even copied to main memory. All one has to do is enable the hard disk password in the firmware menu.

Physical disassembly of the device in order to bypass the hard disk password check would not impose a security risk to the extent present on traditional rotating disks, where replacing the electronic circuit board would simply allow the attacher to gain access to the data without the password. All data is encrypted with a 256 bit key, and the key resides on the SSD controller itself and never leaves it. When a HDD password is set and forgotten, the encryption key is not accessible and cannot be recovered in any way through disassembling the drive - at least this is true in the case the AES encryption key is not stored plain after setting the password, but scrambled/encrypted using the password or the hash thereof.

To make the long story short, I set both user and master passwords on the SSD from within the firmware setup menu. For a more detailed explanation on why both user and master passwords were set, or why the firmware setup menu was used, I recommend to read my post on the topic from December 2012.

Additionally, I also enabled the supervisor password of the thinkpad, but decided not to use a power-on password. In the way I configured the firmware, one can boot the thinkpad without any password, even select the boot device from a supervisor-controlled, preconfigured menu. The hard drive password will be asked anyways unless booting from another device. Configuring a power-on password with a value which is identical to the hard disk password would allow the thinkpad to only ask for the password once and re-use it, however this would not increase, but decrease the security level of the system. There are known attacks worth to be aware of that exploit this scenario to gain access to the hard disk password by cracking the power-on password, even if a security chip is present on the motherboard.

Disabling the guest session in lightdm

While this is not a real security concern at all, I opted to disable the guest session by altering the lightdm configuration file as follows:

echo “allow-guest=false” | sudo tee -a /etc/lightdm/lightdm.conf

Honestly, I do not see any benefits of this feature in a corporate environment...

First steps after installing Ubuntu 12.10

After setting up the base system on my W530, I also applied some initial customization to the desktop environment. Obviously, one of the first things was changing the background...

Keyboard shortcuts

There is a set of keyboard shortcuts I got used to. In the majority of cases, shortcuts are uncomplicated to customize - even for multimedia of other buttons. One just presses the Super button to active dash, types 'keyb' and hits 'enter' to active the gnome keyboard panel of the control center...

• First and foremost, under 'Layout Settings' I configured 'English international w/ AltGr dead keys' in addition to my native layout. I use English layout while coding.
• I selected 'Use same layout for all windows'.
• Under 'Options', I configured 'Alt-CapsLock' to change the keyboard layout.
• Back on the keyboard panel I moved to 'Shortcuts' and reconfigured the logout action to be triggered by 'Ctrl-Alt-Backspace'.
• Then I added a custom action 'Shutdown menu' that allowed triggering the command '/usr/bin/gnome-session-quit --power-off' on 'Ctrl-Alt-Del'. This command displays a menu allowing to select suspend, restart or shutdown. Another option would have been the command '/usr/lib/indicator-session/gtk-logout-helper --shutdown', but this does not allow choices other than shutdown.

I also wanted to reveal the same shutdown menu when the power button is pressed - I got used to it on my other thinkpads and liked it. I tried using xev and acpi_listen but found that pressing the power button does not send any key or acpi events... However, the unlabelled black button next to my 'mute mic' button was recognized as 'Launch1' so mapped it to trigger the same command as 'Ctrl-Alt-Del'.

Compiz settings

I quickly installed the package 'compizconfig-settings-manager' to tweak the window manager. I am rather conservative in this area, but applied the following changes:

• Enabled snapping windows to window edges.
• Disabled 'show desktop in switcher' in the Unity plugin configuration.
• Tuned the Grid plugin to enable corner and bottom half placement of windows.
• Set the launcher behaviour to autohide in the Unity plugin configuration. Note that this property can also be controlled from within System Settings > Appearance > Behavior

Fine tuning the touchpad

Many properties of the touchpad can be configured via the GUI, however, in fine tuned parameters related to palm detection to increase productivity. I prefer a sensitive palm detection, as I frequently hit the touchpad with the side of my thumbs during development.

I saved the following script under /home/tibi/.local/bin/touchpad-config:

#!/bin/sh
synclient PalmDetect=1 PalmMinWidth=5 TapButton3=2 HorizTwoFingerScroll=1

Running it made the touchpad behave according to my needs. To make the changes persistent I configured dconf as follows:

gsettings set org.gnome.settings-daemon.peripherals.input-devices hotplug-command /home/tibi/.local/bin/touchpad-config