Ubuntu Sound Gotcha - Missing Devices

I was laid up at home for a day so instead of using my laptop as a laptop, I went ahead and "docked" manually to my dual screen gear (that has largely been attached to my "work" Macbook Pro.) I used the thunderbolt and hdmi connectors on the side of my Spectre XT (and it was the first time I've ever gotten video to work out of the t-bolt. WIN.) The monitors are Acers with DVI connectors so I'm using some monoprice cabling to convert to T-bolt and HDMI.

That setup will probably explain to some of you what went wrong... but sadly it did not initially to me. I was using the setup and listening to music/youtube etc. After some point in time (and crucially, I'm not sure when exactly), I lost my audio output. The sound notify/menubar/whatever device was still present and still indicated I had sound output, but nothing was being produced.

I entered into the sound settings and lo and behold, there was no device listed in the "Play Sound Through" panel of the output setting. (And I didn't even notice that right away as my eyes were laser focused on the completely zeroed out volume indicator at the top that wouldn't move. Of course it wouldn't move as there was no output device.)

I gradually pieced together what may have happened--I suspect at some point AFTER connecting the monitors, I either suspended/resumed or actually rebooted. It appears that something in the audio detection algorithms determined I should be using HDMI audio (however, these Acer monitors have no speakers.) And by gradually pieced together, I mean I futzed around with this for a couple of hours.

Once it dawned on me that the only thing that I had really changed was the monitors, I went ahead and removed them (which I had done several times during the futzing around) and shutdown entirely and rebooted. VOILA. "Built-in Audio" device reappears and I'm another happy camper.

Using kdump on Ubuntu in Azure

This is another of my occasional posts that may help the next guy. I call them YAHTNG, yet another helping the next guy,t blog entry...

kdump

kdump is a tool that allows you to capture (in a file) the linux kernel state when it crashes (oops). It uses the kexec functionality that's long been part of the linux kernel (since 2004 if memory serves.) In order to use this on linux, you install the linux-crashdump metapackage that in turn depends on the right bits and pieces.

apt-get install linux-crashdump

On different versions of Linux, different bits and pieces get installed. Prior to Raring, 13.04, you get one set of packages and Raring and newer, you get a different set. In either case, on Microsoft's Azure cloud and elsewhere under the hyper-v hypervisor, you will get a hang if  you just install the linux-crashdump package and then experience a crash. This is due to some Azure-specific kernel modules that get loaded in the kexec/kdump kernel. You need to exclude these modules, i.e., blacklist them. Here's how.

Older Ubuntu Releases including Precise

In 12.04 (Precise) and 12.10 (Quantal) you want to edit /etc/init.d/kdump (this is the script that runs at boot time to configure the kdump kernel. The kdump kernel gets loaded into memory and configured via this script.)

--- /etc/init.d/kdump 2013-06-28 00:09:22.400504335 +0000
+++ kdump.nohyperv 2013-06-28 00:16:48.903733116 +0000
@@ -48,6 +48,7 @@ do_start () {
  # Append kdump_needed for initramfs to know what to do, and add
  # maxcpus=1 to keep things sane.
  APPEND="$APPEND kdump_needed maxcpus=1 irqpoll reset_devices"
+ APPEND="$APPEND ata_piix.prefer_ms_hyperv=0 modprobe.blacklist=hv_vmbus,hv_storvsc,hv_utils,hv_netvsc,hid_hyperv"

  # --elf32-core-headers is needed for 32-bit systems (ok
  # for 64-bit ones too).

As you can see, we are simply prohibiting the Azure kernel modules hv_vmbus, hv_storvsc, hv_utils, hv_netvsc, and hid_hyperv from loading in the kdump kernel. They still get loaded in the regular Azure kernel (and you will want to keep them there for performance and behavior reasons.) However, if they load in the kdump kernel, they won't actually work and will "hang" the kdump kernel while they try and connect to the Azure services (or hyper-v services.) Additionally, we  prefer NOT to load the hyper-v module setting for ata_piix by setting it to zero.

After you modify this init script, you will want to reboot. (But take note and read the last section on the crashkernel as you will likely want to make that change as well, prior to rebooting.

Newer Ubuntu Release (Raring and the upcoming Saucy)

The newest releases of Ubuntu include an additional package that handles kdump configuration called kdump-tools. This package manages the kernel modules in a simple config file /etc/default/kdump-tools. You can edit that file to blacklist the appropriate modules:

    67 #KDUMP_CMDLINE_APPEND="irqpoll maxcpus=1 nousb"
    68 KDUMP_CMDLINE_APPEND="irqpoll maxcpus=1 nousb  ata_piix.prefer_ms_hyperv=0 modprobe.blacklist=hv_vmbus,hv_storvsc,hv_utils,hv_netvsc,hid_hyperv "

In addition to preferring to NOT use the ata_piix for hyperv, it also blacklists the same kernel modules as previously mentioned.

Smaller Images

Low memory (extrasmall, small) Azure instances (well, really any small images including small physical machines) unfortunately run into bug #1206691, default crashkernel setting rarely works on a system with little memory. You will need to modify /etc/grub.d/10_linux and set the crashkernel to 128M for any size instance. Do this by simply altering the range here:

   74 # add crashkernel option if we have the required tools
    75 if [ -x "/usr/bin/makedumpfile" ] && [ -x "/sbin/kexec" ]; then
    76     GRUB_CMDLINE_EXTRA="$GRUB_CMDLINE_EXTRA crashkernel=384M-2G:64M,2G-:128M"


    74 # add crashkernel option if we have the required tools
    75 if [ -x "/usr/bin/makedumpfile" ] && [ -x "/sbin/kexec" ]; then
    76     GRUB_CMDLINE_EXTRA="$GRUB_CMDLINE_EXTRA crashkernel=384M-700M:64M,700M-:128M"



Once you have made this change, be sure to update grub:

sudo update-grub

so that the chnage will take effect. You will also want to reboot. Then you can validate that change by inspecting the boot command line:

cat /proc/cmdline

and see that the new value is now shown.

ubuntu@bug1195328-1210:~$ cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.5.0-36-generic root=UUID=39eb48d3-958a-48e0-896e-b6b03cc2342a ro crashkernel=128M console=tty1 console=ttyS0 earlyprintk=ttyS0 rootdelay=300

Reference Material

The official references for configuring Ubuntu for kdump are here:
https://wiki.ubuntu.com/Kernel/CrashdumpRecipe
https://help.ubuntu.com/lts/serverguide/kernel-crash-dump.html
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/785394
https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1206691

and you should refer to them for procedures for testing and verifying your crashdump setup.

Micosoft Azure has some notes on the kernel modules here:
http://support.microsoft.com/kb/2858695

YAHTNG

Yet Another Helping The Next Guy

I bought my son a shiny new Ivy Bridge based Asus laptop from dealnews for high school graduation. Of course, he only runs Ubuntu, but I wanted to keep the Win 7 64 around for flashing BIOS and related reasons (and someday he may need to run some Windows program....)

It's pretty straight-forward to install Ubuntu alongside windows in a dual-boot fashion as the Live installer knows how to do this. But apparently not on some UEFI based systems--or that's what I thought. It turns out, the installer knows how to work with UEFI systems as well. It just doesn't know how to do it on a system with a broken/corrupt/obscured GUID Partition Table (gpt).

I've used gpt for years as I used to work in HP's Itanium Linux lab. And I've seen some GPT tables corrupted before. The ASUS (at least this one) ships with the secondary GPT corrupted. The last partition (I think 5th) somehow extends INTO the secondary GPT. (Think of the secondary as a backup table in case the primary gets corrupted--a failsafe.) This corrupt secondary table makes the partition table unreadable or possibly unusable to Ubuntu (and moreover, to the underlying libparted and the parted utility.)  The installer just "determined" that the disk was unformatted so it kindly offered to just install Ubuntu to this blank disk (destroying all the existing partitions in the process.) Not so good.

I downloaded gdisk (from Universe) as it seems to be designed from the ground up for GPT tables and it was able to read the partition table and show me the partitions. These matched up with what /proc/partitions was showing me (which seemed to be right all along.) It also matched up with what I was seeing in Windows 7. ASUS ships with a C drive, a D drive, and a recovery drive along with the normal EFI system partition (ESP) and a Microsoft Reserved Partition (MSR). So the layout looked something like this:


Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          411647   200.0 MiB   EF00  EFI system partition
   2          411648          673791   128.0 MiB   0C01  Microsoft reserved part
   3          673792       339511295   161.6 GiB   0700  Basic data partition
   4       339511296       925575167   279.5 GiB   0700  Basic data partition
   5       925575168       976773167   24.4 GiB    2700  Basic data partition

Where 3 is C (OS) drive, 4 is D (DATA) drive, and 5 is the ASUS Recovery partition. It is partition 5 that seemed to be the source of the error:


Warning! Secondary partition table overlaps the last partition by
33 blocks!
You will need to delete this partition or resize it in another utility.


To resolve this issue, I simply deleted partition 4 and recreated it from the same start point but smaller and ending sooner. Then I dd'd partition 5 to another disk, deleted it and recreated it (same size) but now it ended before the end of the disk. I then restored it via dd.  (The partition Warning! went away as soon as 5 was deleted and did not reappear when it was recreated at a new start sector.)

I failed to grab a snapshot of the final partition table entries but I basically shortened 4 by a couple GiB (and didn't try to "nail" it to exactly 33 blocks shorter.)

Once the GPT table had a functioning secondary GPT, the Ubuntu installer was able to find it without problems. I just used gdisk to make the D partition into a Linux format partition (8300). (There was nothing on the D drive.) The installer did create a GRUB entry for the Win 7 but that doesn't really work. However, using ESC, I can select the Windows partition from the UEFI boot menu and it dual boots just fine.  After the installation was done, I upgraded from the Precise install release bits to the latest bits (including newer kernel) and the grub update worked fine as well--so it seems to be a rock solid dual boot machine now.

Holler if this helps you or if you know more about why ASUS formats the disk this way--there could be an underlying reasons. Oh, and one more thing, I actually did make the Win 7 recovery disks PRIOR to doing any Ubuntu at all and I've not tried to do anything since with the recovery partition.

You can reach me via the comments below or email me at asus DOT dowdberry AT medberry DOT net.

And some final words: Ubuntu Live Installer makes an awesome detective kit and this Ivy Bridge notebook truly rocks.

Helping the next guy

Just a bit of IRC discussion I thought I'd share. A friend was looking for a way to disable X on an Ubuntu install converted to be a mythtv backend. After a web discussion, another friend suggested passing "text" in on the cmdline. That's a darn good suggestion.

This works in both gdm and lightdm as they both search for this in their startup script (/etc/init.d/gdm and /etc/init/lightdm). Lightdm is the default display manager in Ubuntu 11.10 Oneiric and gdm was the default display manager in prior versions.

Flickr Uploading from Ubuntu

As a linux user, I've long been frustrated with the tools available for uploading pix or vids to Flickr. I thought I'd give using Wine a shot with the Windows uploadr today. No luck. It gives an error "No XPCOM".

I was stumped and frustrated at this point. Sadly, if I had really been paying attention at UDS-O in Budapest, I would have immediately tried the winetricks hacks. (Winetricks were shown as a way to get performance measures during Wednesday's plenary session.) But no, I wasn't that swift.

Fortunately, someone else was... even swifter: Thank you  Sandip Bhattacharya. Your blog post Using the official Flickr uploadr on Ubuntu made my day. Kudos Sandip!