Usually, upgrading Ubuntu goes well. I run the Upgrade tool which downloads new packages, installs them and then asks me to reboot. Some new versions had minor issues, for example Wayland not fully compatible with my NVIDIA card or an old version of MATE preventing the dist-upgrade, but nothing major, nothing that couldn’t be worked around. This time was different: no sound, and no way to easily get around this.
Cannot dist-upgrade
Sunday, October 21 2018, Ubuntu 18.10 was released since a couple of days. However, after I installed the updates through the Software update tool, I wasn’t proposed to upgrade. I had to dig into the software options and reconfigure the delivery of releases to « every release », not just the LTS. Then I reran the Software Update tool, and got the option to upgrade. However, clicking on the upgrade button did just… nothing. I tried two or three times: same result.
Before accepting that this time, I would have to go through the clean install route, I searched a bit on Google, and found out that sometimes, the upgrade doesn’t start when updates are still available. But updates were all installed. However, running the following on a terminal found and installed a couple of additional updates.
sudo apt update
sudo apt dist-upgradeI then retried the Upgrade button, and that worked! Yeah!
The upgrade went well, and then I was offered to reboot, which I did.
Slow and silent
First the new release felt a bit sluggish, then pretty flaky. First time I tried to reboot because I needed to go to Windows, to play Minecraft loaded by Twitch and recorded through OBS, the system hung up waiting for a stop job. I am getting this issue from time to time and when that happens, I have to wait for the 1min30s timeout to elapse before the stop job is forcibly killed. I don’t know what is a stop job, what job is frozen and how I could get rid of this issue for good.
I was in my living room turning off my TV and AV receiver while my new Ubuntu setup finally rebooted, and I didn’t have time to select Windows in GRUB, so it restarted Ubuntu, and trying to restart redid the « stop job » issue! Argh, don’t tell me I’ll get this each time I reboot or shutdown now? I’ll have to wait 1min30s, maybe even 3min, just to power off or reboot. What’s the point of having a SSD if timeouts like this counter its performance benefits? I got fed up and powered off my PC with the power button. I had to press and hold the button almost ten seconds for the machine to finally turn off, then I had to press the button 2-3 times for it to finally turn on. Maybe I’m heading towards the necessity of replacing my computer CASE, which sucks me to the point of thinking about getting rid of that damned thing and use just a laptop. But that case issue has nothing to do with Ubuntu, I should just have picked a better case, there’s nothing more to it, at least for this post.
After my Minecraft session, which was kind of fruitful, I wanted to check that my Ubuntu installation would boot again and be able to shutdown at least once without the « stop job » issue. I thus rebooted to Ubuntu, but the system froze before showing the welcome screen. I had to press CTRL-ALT-F2 to reach a console, log in, check the syslog, nothing of interest. Then the system finally booted into X. I wanted to switch back to the console to log off, but when coming back to X, it froze again on a black screen, this time no key was working. Then another hard reboot!
Second attempt worked: I reached the desktop. I launched the backup script for my Minecraft world, then the MKV->MP4 batch conversion script. OBS records in MKV, which is more robust against crashes, but VideoStudio doesn’t accept MKV, so I have to turn the recordings into MP4. Fortunately, FFMPEG does it without loss, by just repackaging the MPEG stream into another container. Then I wanted to organize my videos into directories, so I launched one to check it, and found THE thing: no sound!
Checking pavucontrol, I found that my sound card disappeared. Only detected devices were my HDMI port hooked up to my monitor, and a crappy USB Webcam that could serve as very basic and rudimentary mic. I don’t use that for my Minecraft recording; I have an AKG real microphone for that! I tried to reboot to no avail. Trying to run speaker-test just hung, again, needed to press ALT-F4 to shutdown the terminal.
Searching on Google for solutions lead to nothing except old stuff that didn’t work. Some people restored sound by reinstalling ALSA and PulseAudio. Others had to downgrade to the previous kernel. Others edited configuration files, commenting a line that is not there in my case, and adding another line. I tried to reinstall PulseAudio with sudo apt install –reinstall pulseaudio, to no avail. As a last resort, I tried to reboot with the 4.15 kernel of Ubuntu 18.04,
After almost an hour of searching, it was more and more obvious that I needed to give up on Ubuntu 18.10 and either downgrade (which essentially means reinstall) to Ubuntu 18.04, or switch to a new distribution. I was quite annoyed, as preceding upgrades went right, and then the hardware problems start again like in the past. Moreover, the system was freezing for a second each time I hit Tab while in a terminal, before displaying completions.
Can this work at all?
I had to go sleeping (it was past 11 PM), go to work the day after, but I thought about it. First I needed to test if that can work! For this, the simplest solution is to test using a live USB. Monday morning, I had to resist the temptation of testing that before going to work. Doing that would have made me start late and thus finish work late in the evening. So I did my workday first. So on Monday evening, I needed to download the Ubuntu 18.10 ISO; I picked the MATE one since that’s what I’m using now, instead of that GNOME3 thing which works so so. I got the ISO, and used the disk creator tool built into Ubuntu to write it into a USB drive. However, the tool refused to format my drive: too small for the new 2Gb ISO! That old 2Gb key was just a bit too small, which is kind of annoying. As a smart man, I should have extra empty, unused USB keys hanging around, but seems I’m not smart enough for that! So I had to use an existing key.
So I took the Clonezilla key, a 16Gb device, really too large for that small backup tool. I stored Ubuntu on it, and then I installed Clonezilla on the old 2Gb stick. Then I booted off my new Ubuntu medium, the system booted successfully, and I got sound! Ok, so at worst, if I clean install, I should get sound… except maybe if that is an incompatibility with the NVIDIA driver.
Unlocking ALSA
First I ironed out the NVIDIA hypothesis by uninstalling the NVIDIA proprietary graphic driver. After I reboot, I tested and still had no sound, and was back to a default VESA resolution. I thus reinstalled the driver, a bit annoyed that Nouveau cannot even basically handle my graphic card. I don’t expect full 2D/3D acceleration out of Nouveau, but at least, mode switching should work with a card bought in 2013, reaching a 1080p resolution. No, nothing. But at least, there was no incompatibility between the graphic and sound driver.
I then dug further into ALSA, which was still detecting my sound chip. Here is the list of devices it found:
eric@Drake:~$ aplay -L
default
Playback/recording through the PulseAudio sound server
null
Discard all samples (playback) or generate zero samples (capture)
pulse
PulseAudio Sound Server
sysdefault:CARD=PCH
HDA Intel PCH, ALC887-VD Analog
Default Audio Device
front:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
Front speakers
surround21:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
4.0 Surround output to Front and Rear speakers
surround41:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Digital
IEC958 (S/PDIF) Digital Audio Output
dmix:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
Direct sample mixing device
dmix:CARD=PCH,DEV=1
HDA Intel PCH, ALC887-VD Digital
Direct sample mixing device
dsnoop:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
Direct sample snooping device
dsnoop:CARD=PCH,DEV=1
HDA Intel PCH, ALC887-VD Digital
Direct sample snooping device
hw:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
Direct hardware device without any conversions
hw:CARD=PCH,DEV=1
HDA Intel PCH, ALC887-VD Digital
Direct hardware device without any conversions
plughw:CARD=PCH,DEV=0
HDA Intel PCH, ALC887-VD Analog
Hardware device with all software conversions
plughw:CARD=PCH,DEV=1
HDA Intel PCH, ALC887-VD Digital
Hardware device with all software conversions
hdmi:CARD=NVidia,DEV=0
HDA NVidia, HDMI 0
HDMI Audio Output
hdmi:CARD=NVidia,DEV=1
HDA NVidia, HDMI 1
HDMI Audio Output
hdmi:CARD=NVidia,DEV=2
HDA NVidia, HDMI 2
HDMI Audio Output
hdmi:CARD=NVidia,DEV=3
HDA NVidia, HDMI 3
HDMI Audio Output
dmix:CARD=NVidia,DEV=3
HDA NVidia, HDMI 0
Direct sample mixing device
dmix:CARD=NVidia,DEV=7
HDA NVidia, HDMI 1
Direct sample mixing device
dmix:CARD=NVidia,DEV=8
HDA NVidia, HDMI 2
Direct sample mixing device
dmix:CARD=NVidia,DEV=9
HDA NVidia, HDMI 3
Direct sample mixing device
dsnoop:CARD=NVidia,DEV=3
HDA NVidia, HDMI 0
Direct sample snooping device
dsnoop:CARD=NVidia,DEV=7
HDA NVidia, HDMI 1
Direct sample snooping device
dsnoop:CARD=NVidia,DEV=8
HDA NVidia, HDMI 2
Direct sample snooping device
dsnoop:CARD=NVidia,DEV=9
HDA NVidia, HDMI 3
Direct sample snooping device
hw:CARD=NVidia,DEV=3
HDA NVidia, HDMI 0
Direct hardware device without any conversions
hw:CARD=NVidia,DEV=7
HDA NVidia, HDMI 1
Direct hardware device without any conversions
hw:CARD=NVidia,DEV=8
HDA NVidia, HDMI 2
Direct hardware device without any conversions
hw:CARD=NVidia,DEV=9
HDA NVidia, HDMI 3
Direct hardware device without any conversions
plughw:CARD=NVidia,DEV=3
HDA NVidia, HDMI 0
Hardware device with all software conversions
plughw:CARD=NVidia,DEV=7
HDA NVidia, HDMI 1
Hardware device with all software conversions
plughw:CARD=NVidia,DEV=8
HDA NVidia, HDMI 2
Hardware device with all software conversions
plughw:CARD=NVidia,DEV=9
HDA NVidia, HDMI 3
Hardware device with all software conversionsThe speaker-test command was hanging, but if I waited enough time after pressing Ctrl-C, it returned. Ok, cool! Running pavucontrol while pseaker-test was playing its noise, I could see it in the PulseAudio applications. Ok, so speaker-test
What if I switch device? I tried speaker-test -D sysdefault:CARD=PCH. I got an error because ALSA couldn’t open the device. Searching on Google about that, not specific to Ubuntu, lead to clues: it could be a permission issue. Trying to check /dev/dsp failed: no such file. But I could fiund the following.
eric@Drake:~$ ls -ld /dev/snd/*
drwxr-xr-x 2 root root 60 oct 23 08:11 /dev/snd/by-id
drwxr-xr-x 2 root root 100 oct 23 08:11 /dev/snd/by-path
crw-rw----+ 1 root audio 116, 9 oct 23 08:11 /dev/snd/controlC0
crw-rw----+ 1 root audio 116, 15 oct 23 08:11 /dev/snd/controlC1
crw-rw----+ 1 root audio 116, 8 oct 23 08:11 /dev/snd/controlC2
crw-rw----+ 1 root audio 116, 6 oct 23 08:11 /dev/snd/hwC0D0
crw-rw----+ 1 root audio 116, 14 oct 23 08:11 /dev/snd/hwC1D0
crw-rw----+ 1 root audio 116, 3 oct 23 08:11 /dev/snd/pcmC0D0c
crw-rw----+ 1 root audio 116, 2 oct 23 19:23 /dev/snd/pcmC0D0p
crw-rw----+ 1 root audio 116, 4 oct 23 08:11 /dev/snd/pcmC0D1p
crw-rw----+ 1 root audio 116, 5 oct 23 08:11 /dev/snd/pcmC0D2c
crw-rw----+ 1 root audio 116, 10 oct 23 08:11 /dev/snd/pcmC1D3p
crw-rw----+ 1 root audio 116, 11 oct 23 08:11 /dev/snd/pcmC1D7p
crw-rw----+ 1 root audio 116, 12 oct 23 08:11 /dev/snd/pcmC1D8p
crw-rw----+ 1 root audio 116, 13 oct 23 08:11 /dev/snd/pcmC1D9p
crw-rw----+ 1 root audio 116, 7 oct 23 08:11 /dev/snd/pcmC2D0c
crw-rw----+ 1 root audio 116, 1 oct 23 08:11 /dev/snd/seq
crw-rw----+ 1 root audio 116, 33 oct 23 08:11 /dev/snd/timerOk, interesting, only root and users of group audio can access these devices, and thus play sounds. Am I part of the audio group? I tried groups eric and found out I wasn’t. But on my HTPC still running Ubuntu 18.04, I was! Lucky I had that HTPC, otherwise I would have been forced to reboot to the Live USB to check, or boot another machine with it, like my ultrabook.
Ok, so what if I add myself to the group?
sudo usermod -a -G audio ericI had to relog, and then got sound through ALSA using Speaker-Test! Yeah! But still no PulseAudio! I tried to reboot, not just relog, to no avail. Ok, at least I can reconfigure Audacious to play through ALSA, but I suspect I’ll get trouble for YouTube and Spotify for which I cannot configure audio output.
Unlocking PulseAudio
I tried all sorts of things to debug this. I was able to get some logs out of PulseAudio several ways, but the most useful way was through SystemD. In Ubuntu, PulseAudio is started by a userspace variant of SystemD. There is a command allowing to get the logs: journalctl. I couldn’t figure out a way to filter so I ended up calling journalctl -a -f on one console, and then pulseaudio -k on another to force PulseAudio to restart. Then I was checking the produced log. I ended up finding out errors. The system was not able to communicate through DBus using a certain socket. I started to think that each time the system freezes, this is because PulseAudio tries to emit a sound, has to wait and timeout.
I couldn’t find out how to reconfigure DBus, or understand how PulseAudio and DBus interact enough to troubleshoot this. I was quite stuck, and all I could find was deprecated information. There was a bug report that seemed to affect Ubuntu 18.10, but no solution. It was past 10 PM and I was on this for almost two hours, trying and searching to no avail, more and more pissed off and risking to be so angry that I would end up having trouble to sleep. I was about to give up or reinstall from scratch.
I ended up fed up and decided to completely get rid of PulseAudio using sudo apt purge pulseaudio. After a reboot, sound was WORKING! What??? How come MATE plays sound without PulseAudio? Audacious was working, and VLC as well. But not Firefox for YouTube, and not Spotify.
Before accepting the sacrifice of YouTube and Spotify to avoid a clean install, at least for now, I tried to reinstall PulseAudio. For I really don’t know why, the sound continued working, and PulseAudio displayed my internal audio device now. YouTube and Spotify resumed working. I read posts about some people that got the issue fixed and it came back the next reboot. Ok, let’s reboot and hammer that bug for good, then! I rebooted, and sound was still working. I still don’t fully understand why.
Either something messed into the groups I was member of and PulseAudio got screwed up because I lost permission to use ALSA, either some configuration files needed to be updated but APT decided to keep my current versions. Purging PulseAudio removed the configuration files and reinstalling reverted to sane defaults. At least, sound is working now.
Even better, the system seems more responsive now and didn’t freeze on startup. It really seems that PulseAudio and ALSA had trouble communicating, causing these hangups.
Why not a clean install?
Because my home directory lives on the same partition as my Ubuntu install. Any attempt to put my /home on a separate partition leads to insufficient disk space after a time. I have a 250Gb SSD shared with Windows so I cannot put 50Gb for my /home and 30Gb for just Ubuntu. One simple solution would be to move my /home to an hard drive. As long as Ubuntu is on a SSD, I’ll have a fine boot time. Or I would need a way to use SSD only as a cache and put everything on the hard drive. I could also come back to the dual SSD strategy: one SSD for Windows, one for Ubuntu. I’ll think about it, but at least I don’t have to do anything short term. Maybe I can wait and replace everything, and get a better case to fix the power button hickup at the same time…
If everything had failed, before the clean install, I could have tried to restore my Clonezilla image of my SSD, and that would have got me Ubuntu 18.04 back to normal. In case of failure, then I would have had to clean install 18.04, 18.10 or something else, or just give up on Linux for the moment. At least this is not necessary anymore.