System configuration

• Kernels >= 2.6.31 seem to work pretty good without RT patch, also for real-time pro audio usage. It's not strictly necessary anymore to install a real-time ('rt') kernel to get good results. Although the best results are still expected when using a real-time kernel. Try it, test it and decide for yourself.
• In the pre-2.6.39 kernel era rt kernels were indeed necessary in some cases where sound devices were sharing IRQ's with other peripherals. With the rt kernel and the rtirq script you could prioritize IRQ threads but since 2.6.39 it is possible to use the rtirq kernel with a generic kernel and the threadirqs kernel option.

• Download and install a Linux distribution specially tailored for Linux audio production.
• Get yourself an Indamixx device.

Download the realTimeConfigQuickScan.pl script and the modules on which it depends, to automatically analyze your current configuration. This script will make configuration suggestions based on your current setup and link to the relevant section on this page for background information.

Discuss this script at the LinuxMusicians Forum

This section describes installing a real-time ('rt') kernel on an existing Linux distribution. Some distributions contain packages you can install to get an rt kernel, for others you'll need to compile it yourself.

run uname -a to find out which kernel you're running - for example, for my Debian Sid machine, this returns:

Linux drollie 2.6.33.7.2-rt30-multimedia-amd64 #1 SMP PREEMPT RT Sat Jan 8 16:42:48 UTC 2011 x86_64 GNU/Linux

2.6.33.7.2-rt30-multimedia-amd64 is the version number of the currently running kernel. The configuration file with which this kernel was created can usually be found in /boot/config-2.6.33.7.2-rt30-multimedia-amd64. Open it, and check if it has the following characteristics:

CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_PREEMPT_RT=y
CONFIG_PREEMPT=y

If you can't find the config file for your kernel, or want to make extra-sure the currently running kernel is okay, read https://rt.wiki.kernel.org/articles/r/t/_/RT_PREEMPT_HOWTO_6bc9.html#Checking_the_Kernel

This step is included in the quickScan

Install the 'linux-rt' meta-package (or a specific one, like linux-image-2.6.24-19-rt). Bear in mind that this package is only available for Ubuntu 8.04 up to and including 10.04. Tango Studio also offers a very stable and good performing real-time kernel for 10.04 and for more recent releases there is a PPA available. The latter PPA is very much in flux and using this PPA is at your own risk.

There are real-time kernels available from Pengutronix and 64 Studio. The 64 Studio real-time kernels are in the backports apt repository, so for Debian Squeeze you will have to add the following line to your apt sources list:

deb http://apt.64studio.com/backports squeeze-backports main

See also: http://forums.debian.net/viewtopic.php?t=17035.

Arch has a 3.0-rt kernel available in AUR: https://aur.archlinux.org/packages.php?ID=51360

Linux frummie 3.0-rt #1 SMP PREEMPT RT Sun Nov 27 16:12:17 CET 2011 x86_64 AMD Athlon(tm) Processor L110 AuthenticAMD GNU/Linux

The patchset for the 3.0 kernel branch is pretty stable so a 3.0-rt kernel should be usable on a Linux audio set-up.

If your distribution isn't any help, you can compile and install an rt kernel manually.

See these links:

• https://rt.wiki.kernel.org/articles/r/t/_/RT_PREEMPT_HOWTO_6bc9.html
• http://proaudio.tuxfamily.org/wiki/index.php?title=Howto_RT_Kernel (gentoo-minded)

You should check if the following daemons, services or processes are running, and if so, consider stopping them and/or resort to alternatives:

• powersaved
• kpowersave
• CPU frequency scaling daemons that scale the frequency of the CPU depending on the CPU load could cause xruns in some cases. More recent versions of Jack1 (>= 0.118.0) are suffering less from xruns or run xrun free with a CPU frequency scaling daemon enabled that's set to a scaling governor like ondemand. A specialized CPU scaling daemon is in the works that depends on the DSP-load instead of the CPU load: jackfreqd
• Debian/Ubuntu: a default install will have the apt-xapian-index package installed that enables the Quick Search option in the Synaptic package manager. In order for this Quick Search functionality to work properly it needs to regularly update its database with the help of a cronjob. Especially on less powerful machines this cronjob could cause major xruns. Consider uninstalling the apt-xapian-index package.
• Desktop Environments (DE): especially on less powerful machines it is recommended to use a lighter DE than Gnome or KDE.
• Desktop Effects: same recommendation as for the DE you use. Disable Desktop Effects (which are being handled by the Compiz compositing manager) to lower the chance of running into xruns.
• Gnome NetworkManager and WiFi: NetworkManager keeps scanning for new wireless networks in the background and this might cause xruns. The best option is to not use a wireless network manager in a low-latency real-time audio environment. If you really can't do without WiFi you probably won't have another option than to use wpa_supplicant.

There are quite some processes and drivers running that you might not need when making music. Below an extremely simple sample Bash script to disable all unneeded things.

#!/bin/sh

modprobe -r ppdev # I don't have a parallel port
modprobe -r lp # I don't use a printer when making music
modprobe -r uvcvideo # I don't use a webcam when making music
modprobe -r videodev # Ditto
modprobe -r ath9k # Wireless driver
modprobe -r r8169 # NIC driver
modprobe -r btusb # Bluetooth USB

/etc/init.d/bluetooth stop & # Stop Bluetooth related processes
/etc/init.d/cups stop & # Stop cups, I don't use a printer when making music
/etc/init.d/networking stop & # Stop networking, the internet is a very distracting factor when making music
/etc/init.d/network-manager stop & # Ditto
/etc/init.d/ondemand stop # Prevent this script from running so my CPU won't use the ondemand governor

killall modem-manager # I don't have a modem
killall wpa_supplicant # See the network stuff above

modprobe snd-hrtimer # Load the ALSA high res timer for my MIDI stuff

echo -n performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor # Set the CPU scaling governor to performance

TASKLETPR=76 # Define a variable TASKLETPR and set it to 76

ps -eLo pid,cmd | grep [t]asklet | awk '{ system("chrt -f -p '$TASKLETPR' " $1)}' # Do some greppy awky stuff and set the prio of the tasklet daemon

echo -n "0000:00:13.0" > /sys/bus/pci/drivers/ohci_hcd/unbind # Unbind a USB port, this one interferes with my onboard soundcard

You could run this script once you're logged in (it needs root privileges though), run it from /etc/rc.local or even as an init script. A suggestion would be to run it only when a real-time kernel is loaded for example:

if [ $(uname -r | cut -d "-" -f 3) = "realtime" ]
then /path/to/the/above/script
fi

The modules and processes could differ of course on a per machine basis so first figure out which modules are loaded with lsmod for instance and then decide which ones you really need. Same for any processes running on your machine, ps -ef or even pstree -p could be helpful when looking for unneeded processes.

The above script also contains a line to unbind a rogue device that sits on the same IRQ as the sound card. Unbinding a device is basically a last resort if a device is interfering in such a way that all other possibilities, like using rtirq, fail to improve the performance of the sound card. In this particular case the thread created for the USB port started consuming so much CPU that at a certain moment xruns were all over the place. In order to prevent this from happening unbinding the USB driver from the physical USB port worked out well.

This is the situation on the affected machine:

16:        923   IO-APIC-fasteoi   ohci_hcd:usb2, hda_intel

So usb2 shares IRQ 16 with the HDA Intel soundcard. In order top be able to unbind the USB driver from usb2 we need to find the bus ID of usb2:

$ tree /sys/bus/usb/drivers/usb/
/sys/bus/usb/drivers/usb/
|-- 1-4 -> ../../../../devices/pci0000:00/0000:00:13.5/usb1/1-4
|-- bind
|-- uevent
|-- unbind
|-- usb1 -> ../../../../devices/pci0000:00/0000:00:13.5/usb1
|-- usb2 -> ../../../../devices/pci0000:00/0000:00:13.0/usb2
|-- usb3 -> ../../../../devices/pci0000:00/0000:00:13.1/usb3
|-- usb4 -> ../../../../devices/pci0000:00/0000:00:13.3/usb4
`-- usb5 -> ../../../../devices/pci0000:00/0000:00:13.4/usb5

6 directories, 3 files

So the ID of usb2 is 0000:00:13.0. If we write that to the unbind file in the /sys/bus/pci/drivers/ohci_hcd/ directory usb2 should get unbound from the USB driver:

# echo -n "0000:00:13.0" > /sys/bus/pci/drivers/ohci_hcd/unbind

Finding the right unbind file might be a bit trial and error, but /sys/bus/pci/drivers/ohci_hcd/ contains the right unbind file for USB1.1 root hubs such as usb2. Now we can check with the first tree command (or with lsusb) what the current status is:

$ tree /sys/bus/usb/drivers/usb/
/sys/bus/usb/drivers/usb/
|-- 1-4 -> ../../../../devices/pci0000:00/0000:00:13.5/usb1/1-4
|-- bind
|-- uevent
|-- unbind
|-- usb1 -> ../../../../devices/pci0000:00/0000:00:13.5/usb1
|-- usb3 -> ../../../../devices/pci0000:00/0000:00:13.1/usb3
|-- usb4 -> ../../../../devices/pci0000:00/0000:00:13.3/usb4
`-- usb5 -> ../../../../devices/pci0000:00/0000:00:13.4/usb5

5 directories, 3 files

As you can see, usb2 is gone, it doesn't show up with lsusb anymore either:

$ lsusb
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 002: ID 04f2:b175 Chicony Electronics Co., Ltd 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

usb2 used to correspond with Bus 002. That one seems to be gone. A final check:

$ cat /proc/interrupts | grep 16
 16:        924   IO-APIC-fasteoi   hda_intel

Great, the sound card is now sitting on its own on IRQ 16 and runs in a way more stable manner than before when it still shared an IRQ with the USB port. Major drawback of this method of course is that you loose an USB port.

More information: http://lwn.net/Articles/143397/

For audio use, it is desirable to use a filesystem that favours few big files over many small files and low-latency over long-term thoughput.

It is said that ReiserFS and fuseblk are bad choices from this perspective, while ext3 and ext4 are good. ext4 has had some performance issues but those are not directly related to processing audio and in the meanwhile most issues have been ironed out.

In any case, it is advisable to mount filesystems with the noatime option enabled.

This step is verified by the Quickscan, but could use more information about which filesystems are suitable.

It won't affect the quality of the audio, but it will change how many tracks the disk can support and because encryption requires CPU, it will affect how much signal processing you can do all of which is a long-winded way of saying “don't do it”

Both Jack1 and Jack2 now use /dev/shm so mounting /tmp to tmpfs is not necessary anymore.

Setting the noatime parameter in your fstab reduces the amount of disk I/O (the inode access times are not updated each time a file is read) which could improve the overall performance of your system.

/dev/sda1 / ext3 noatime,errors=remount-ro 0 1

Most modern distributions use either atime or relatime. See the mount manpage for more information on these parameters.

It is advisable to set up your /etc/security/limits.conf or /etc/security/limits.d/audio.conf file properly, for example:

 @audio - rtprio 90       # maximum realtime priority
 @audio - memlock unlimited  # maximum locked-in-memory address space (KB)

Setting memlock to unlimited shouldn't be strictly necessary since most apps would work fine with lower values (such as 500000).
However, certain applications have been reported to complain or even to crash with lower values than unlimited.
On the other hand, giving unlimited capability of locking memory can cause buggy applications to temporarily freeze the whole system.
See http://www.linuxmusicians.com/viewtopic.php?f=10&t=2193 for more details.

You could also allow the audio group to renice processes with the help of the limits.conf file, but since nice uses SCHED_OTHER it basically does nothing to increase the performance of a low-latency real-time audio environment that relies on SCHED_FIFO/SCHED_RR. Consult the sched_setscheduler manpage for more info on this subject.

See also: http://linuxmusicians.com/viewtopic.php?f=27&t=392

If you do not use PAM (which is rare), you might want to use set_rlimits instead

Suggested settings for /etc/sysctl.conf:

 vm.swappiness = 10

This setting changes the so-called swappiness of your system, or in other words, the moment when your system starts to use its swap partition. You can check the current value with cat /proc/sys/vm/swappiness, in most cases it is set to 60. This is too high, it will make your system start using swap too fast which might influence the overall performance.

There are references on the net on adjusting the fs.inotify.max_user_watches value also for enhanced performance. But it remains very unclear where these references come from and if adjusting this value actually does anything at all.

It is generally good practice to have an 'audio' group, and add any users that should be allowed to perform audio tasks to this group. This prevents some interference from non-audio-processes with audio tasks. To verify you're in the 'audio' group, run the groups command. Remember, after adding yourself to new groups, you need to log out and back in again. Be careful when adding an 'audio' group to your system: most systems come with a pre-configured 'audio' group, and do not warn you when you add another group with the same name, leading to much confusion.

This step is verified by the Quickscan.

MIDI sequencers and such will benefit from being able to use hardware timers like the real-time clock (/dev/rtc) or the High Precision Event Timer (/dev/hpet). Make sure the 'audio' group has read permissions on it.

A simple chgrp might not be persistent across reboots - to make the change last, create a new 40-timer-permissions.rules file in /etc/udev/rules.d with the following lines:

KERNEL=="rtc0", GROUP="audio"
KERNEL=="hpet", GROUP="audio"

… and reboot. See also: https://bugs.launchpad.net/ubuntu/+source/udev/+bug/306458

This step is verified by realTimeConfigQuickScan.pl.

For rtprio, higher values mean higher priority. You can list threads and their priorities with ps axHo user,lwp,pid,rtprio,ni,command or ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i "irq"

Most applications take care of running some threads with higher rtprio when possible themselves, like JACK with the real-time option set. JACK's watchdog has a higher prio than jackd itself (+10) so choose a sensible priority value for JACK (70 for example, this will assign prio 80 to JACK's watchdog thread), bearing in mind also that sound devices might get a priority in the 80-90 range if you use rtirq.

You can also set rtprio and scheduling settings manually using the chrt command-line tool, which should come with most distributions nowadays, or use the rtirq init script.

See also: http://subversion.ffado.org/wiki/IrqPriorities

rtirq is a script written by Rui Nuno Capela, the author of QjackCtl and Qtractor amongst others. This script allows to make use of the threaded IRQs as used by real-time kernels or kernels >= 2.6.39 with the threadirqs kernel option enabled. The term threaded IRQ is quite an abstract one but what it boils down to is that every peripheral gets an IRQ and the Linux kernel splits this IRQ in different parts of which the so-called bottom-halve is of concern in the context of the rtirq script. Bottom-halves are also known as softirqs which might make this matter a bit less abstract, 'hardirqs' (the top-halves) are the IRQs assigned by the system which can't be managed from within the OS as opposed to 'softirqs' (the bottom-halves) which can be managed.

These bottom-halves are managed through so-called tasklets that are part of the tasklet API. This functionality can be verified with the aforementioned ps -eLo pid,cls,rtprio,pri,nice,cmd | grep -i “irq” command. The process called sirq-tasklet (<= 2.6.38, kernels >= 2.6.39 do not have such a process simply because that specific part from the RT patchset has been merged into the main kernel) is the actual tasklet management tool that takes care of assigning the priorities to different bottom-halves, or actually the threads in which these bottom-halves run (hence the term threaded IRQs). The distinction between these three terms (softirq, bottom-halve and tasklet) is a bit vague but it's beyond the scope of this article to fully expand on this subject. More info can be found in this FFADO Wiki article: http://subversion.ffado.org/wiki/IrqPriorities

The rtirq tarball as availabe on the site of the author consists of two files, the script itself (rtirq.sh) and a configuration file (rtirq.conf). In most cases the rtirq.sh file is used as an init script (and hence stripped from the .sh extension) and rtirq.conf mostly ends up in /etc/default/ or /etc/sysconfig/, also stripped from its extension. Let's look at the contents of the configuration file:

#
# Copyright (c) 2004-2011 rncbc aka Rui Nuno Capela.
#
#   This program is free software; you can redistribute it and/or
#   modify it under the terms of the GNU General Public License
#   as published by the Free Software Foundation; either version 2
#   of the License, or (at your option) any later version.
#
#   This program is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#   GNU General Public License for more details.
#
#   You should have received a copy of the GNU General Public License along
#   with this program; if not, write to the Free Software Foundation, Inc.,
#   51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#
# /etc/sysconfig/rtirq
# /etc/default/rtirq
#
# Configuration for IRQ thread tunning,
# for realtime-preempt enabled kernels.
#
# This program is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License version 2 or later.
#

# IRQ thread service names
# (space separated list, from higher to lower priority).
RTIRQ_NAME_LIST="rtc snd usb i8042"

# Highest priority.
RTIRQ_PRIO_HIGH=90

# Priority decrease step.
RTIRQ_PRIO_DECR=5

# Whether to reset all IRQ threads to SCHED_OTHER.
RTIRQ_RESET_ALL=0

# On kernel configurations that support it,
# which services should be NOT threaded 
# (space separated list).
RTIRQ_NON_THREADED="rtc snd"

# Process names which will be forced to the
# highest realtime priority range (99-91)
# (space separated list, from highest to lower priority).
# RTIRQ_HIGH_LIST="timer"

The RTIRQ_NAME_LIST variable contains a list of space separated service names of which the first entry gets the highest priority. The term service seems to refer to module names and sound device designations (so the output of lsmod and aplay -l respectively) and doesn't have to correspond to the full output, part of the output may suffice as the rtirq script does the matching itself.

The RTIRQ_PRIO_HIGH variable sets the highest priority a service can get assigned.

The RTIRQ_PRIO_DECR speaks for itself, here you set the number with which the priorities for each consequent service mentioned in the RTIRQ_NAME_LIST variable should be decreased.

The RTIRQ_RESET_ALL is a legacy variable and can best be left to default.

The RTIRQ_NON_THREADED variable is another legacy option, your kernel configuration has to support it and in almost all cases it doesn't because the specific option, which was part of the CONFIG_PREEMPT_VOLUNTARY kernel config option and that allowed for setting non-threaded IRQs, simply doesn't exist anymore. So basically this variable doesn't do anything.

The RTIRQ_HIGH_LIST variable contains a list of space separated service names that get priorities in the 99-91 range, so above the value as set in the RTIRQ_PRIO_HIGH variable. Use this variable only for services of which you want to be 100% sure they don't get interrupted by anything else. You will mostly want to put timers in there like rtc or the ALSA high resolution timer (snd-hrtimer).

A common use case for rtirq is when a FireWire controller shares its IRQ with one or even more devices. With rtirq you can then give a higher priority to the FireWire bottom-halve which should lower the chance of running into xruns drastically or even enable the device to initialize properly. The cat /proc/interrupts | grep 1394 command outputs the IRQ that is in use by your FireWire controller (when using the legacy stack):

$ cat /proc/interrupts | grep 1394
16:       8201     318657   IO-APIC-fasteoi   uhci_hcd:usb3, ohci1394, mmc0, eth1, jmb38x_ms:slot0, nvidia

As you can see IRQ 16 is really crowded in this case and a FireWire device connected to this controller simply won't initialize. By adding the ohci1394 module to the list of separated values of the RTIRQ_NAME_LIST variable you can assign a higher priority to the softirq or bottom-halve of the FireWire controller:

RTIRQ_NAME_LIST="rtc ohci1394 usb i8042"

If RTIRQ_PRIO_HIGH is set to 90 and RTIRQ_PRIO_DECR to 5 then the real-time clock (rtc) will get prio 90 and the FireWire controller prio 85. All other bottom-halves of IRQ 16 will get a default prio of 50 so the FireWire controller will always take precedence. In this particular case (a HP DV7-1040) using rtirq this way enables the FireWire device (a Focusrite Saffire Pro 10 IO) to initialize properly and run in a stable manner at lower latencies (128 frames/48Khz samplerate * 3 periods = 8ms). And given the fact that the controller (an onboard JMicron chipset) sits on a very crowded IRQ shared by a NIC, a cardreader, an USB port and a GPU rtirq definitely shows off its strength here.

Conventional PCI has a shared bus topology and also suffers from only being able to address a limited number of IRQ's. So besides PCI devices sharing interrupts they could also get into each other's way because of “lengthy transactions by some devices, due to slow acknowledgments, long data bursts, or some combination, which could cause buffer underrun or overrun in other devices”¹⁾. It's successor, PCI-X, suffers less from PCI bus latency because it uses higher bandwidths, but it's still using a shared bus topology unlike the latest PCI standard, PCIe. PCIe is “based on point-to-point topology, with separate serial links connecting every device to the host”²⁾.

So if you're using conventional PCI sound cards, you might want to consider maximizing the PCI latency timer of the sound card and raising the latency timer of all other PCI peripherals a bit (default is 64). This may sound counter-intuitive but “the PCI latency timer determines how long a device may own the PCI bus for … Therefore, if your system allowed the video interface to hog the PCI bus for a relatively long time, but limited the audio interface to only short periods of bus ownership, then audio performance will suffer.

The script … resets *all* devices to an acceptable middle-of-the-road timer setting, then specifically allows the audio interfaces to own the bus for even longer.”³⁾

#!/bin/sh

case $1 in
     start)
           # "open up" the PCI bus by allowing fairly long bursts for all devices, increasing performance
           setpci -v -s "*:*.*" latency_timer=b0

           # maximise latency for RME Hammerfall, thus allowing
           # more data per PCI transfer and minimizing xruns
           setpci -v -s 01:04.0 latency_timer=ff
           # ditto for the onboard AC97 audio interface
           setpci -v -s 00:07.5 latency_timer=ff
esac

exit 0

Source of the script: http://lists.linuxaudio.org/pipermail/linux-audio-user/2007-April/044371.html

Question remains in how far adjusting the PCI latency timer affects performance for PCI-X devices. PCIe doesn't suffer from PCI bus latency because of its different topology. Besides, PCIe devices have their PCI latency timer set to 0 and the setpci command simply doesn't change anything, setting latency_timer=ff on PCIe devices still yields a setting of 0.

• Some more hints and tips on Esben-Stein
• Some tips on the Rosegarden wiki
• here You can find an example for Linux/Gentoo.
• Sound on Sound article on PCI problems: http://www.soundonsound.com/sos/dec04/articles/pcnotes.htm

Make sure the raw1394 module is loaded. You can have this module loaded automatically by adding it to your /etc/modules file. Just add the following line to this file:

raw1394

Loading this module will create the /dev/raw1394 node. Normal users do not have permissions to use this node but with an udev rule in /etc/udev/rules.d you can control these permissions. Most recent distributions also install the corresponding udev rules when installing FFADO packages but if your distro doesn't do so you can create a new file 50-raw-firewire.rules with the following line to allow the audio group to use this node:

KERNEL=="raw1394", GROUP="audio"

You will also need the FFADO drivers (most distros have packages for these drivers) and set the driver for JACK to 'firewire' in either QjackCtl or on the commandline with the -dfirewire argument.

When starting JACK it exits with a warning that looks like this:

cannot use real-time scheduling (FIFO at priority 10) [for thread 720017136, from thread 720017136] (1: Operation not permitted)

This means that the current user is not allowed to run JACK in real-time mode. A quick fix would be to disable the real-time option in QjackCtl. If you want to get rid of this warning and use JACK in real-time mode you will have to modify your limits.conf/audio.conf file and make sure you're a member of the audio group.

• The snd-hrtimer module
• The threadirqs kernel option
• New JuJu FireWire stack
• More on MIDI timing