An introduction to OpenAL in C#

in OpenAL, Technical

Graphics and gameplay are two important pillars to build a game. Audio – both music and sound effects – is another important part of games. Many players and reviewers do not focus on audio much and indeed, if the music and sound effects fit with what is happening on the screen, they will fit naturally in the player’s experience. If you would play a game without audio though, you would immediately feel that something is wrong.

While a lot of game developers – especially programmers – are at least vaguely aware of the workings of an update loop or graphics code, audio is often something that enters the equation at a very late stage. In this post I will introduce the general concepts of basic audio programming using OpenAL.

I was first introduced to audio programming when the requirement for an audio engine for a game I was working on arose. The engine was mostly built by trial and error, but I gained valuable experience by writing it. The audio engine is written in C# on top of the wrappers provided by OpenTK, is licensed under GPL, and can be found on GitHub. The library is scheduled for a complete rewrite, so keep an eye on my GitHub profile or this website to get notified about it.

In this blog I will work with code examples based on OpenTK wrappers. I will however only be using methods that are called on the base OpenAL library directly, so all code examples can be trivially translated to other wrapper libraries, or even the OpenAL library itself.

What is OpenAL?

OpenAL is an API for audio related functions. It sits on top of your sound driver and allows for several advanced features related to audio rendering. OpenAL is in principle hardware supported by many soundcards, but since soundcards have become less standard in computer setups, the OpenAL functionality is often performed by a software implementation run on the CPU. Furthermore, the original OpenAL is no longer open source, which is another reason that most people have switched to OpenAL Soft, an open-source software implementation of OpenAL.

The atoms of music

Before we can start discussing how to make the computer play audio, we have to quickly discuss how digital audio is built up at all. I will not go into the technical details (if there is any need for this, feel free to contact me or leave a comment and I will dedicate an entire blog post to this, but there are probably ample other sources to read up on this subject), but we can take a look inside an audio file. Below you can see a screenshot of one of my currently favourite tracks opened in the free audio editor Audacity.

audio waves

Unless you have some interesting skills I do not possess, it is unlikely that based on this image you can envision how it sounds. When I press the play button though, the computer knows exactly what to play based on this data. If we zoom in far enough, we see that in fact the data we are looking at consists of many small waves:

audio waves 2

This makes sense, because audio itself are just waves in the air. The amplitude and frequency of the waves decide the gain (volume) and pitch of the sound. We can however zoom in even further to see that the waves are actually formed by many points:

audio waves 3

We can compare these points to pixels in an image. Since it is difficult to store continuous things in computers, we have to divide or data in a finite number of discrete steps. The finer the subdivision, the closer we can stay to the original and thus the better the quality of the audio (or image).

Audio files contain exactly the values of the points. In addition, the audio file knows how far apart the points should be in time, since there is no standard distance between these sampling points, to allow for varying quality in audio files. The rate at which sample points are placed is specified by a property which we call sample rate; the higher the sample rate, the denser the points. A higher sample rate results in higher quality audio, but also larger filesizes.

A final detail that is important to keep in mind is the fact that there are two bars in the screenshots above. These bars represent the left and the right audio track, as the audio is recorded in stereo. It is also possible to have files that only contain one track (mono), in which the left and right track are considered identical. To store both tracks, the numbers representing the points are intertwined for the left and the right track. Of course we could first have the entire left track followed by the entire right track, but this makes it more difficult to stream the sound (more on streaming later).

Audio data is thus represented by a long list of numbers. The numbers are 16-bits integers (shorts in C#). Audio files do usually not store the numbers directly, but use a form of compression to reduce the filesize. Well known audio formats that use some form of compression are MP3, OGG Vorbis, and FLAC. Of these formats, only FLAC compresses the audio in a lossless manner. Audio data can also be stored in Wave files, which are generally uncompressed.

Types

Even though OpenAL itself is not object-oriented, we can distinguish two important types in OpenAL: buffers and sources. The functions that correspond to these types can be easily distinguished by their name starting with Buffer and Source respectively (with the exception of generating and disposing). Both types will be discussed in detail below.

Buffer

A buffer is comparable to an array of numbers and thus represents a short section of the audio track. It also stores the format in which the data is stored (e.g. mono or stereo) and sample rate. Computers used to have separate soundcards to play audio, but presently the audio is often handled by software that runs on the GPU. Buffers are still a remnant from the time soundcards were still widely used: buffers were stored on the soundcard directly to be played (compare vertexbuffers on graphics cards).

Data has to be written to a sound buffer before it can be played at all. Creating and filling a buffer in OpenAL is pretty straightforward. Consider the following piece of code:

int BufferData(short[] data, ALFormat format, int sampleRate)
{
    int handle = AL.GenBuffer();
    AL.BufferData(handle, format, data, data.Length * sizeof(short), sampleRate);
    return handle;
}

The first line of the method is pretty straightforward: we ask OpenAL to create a buffer for us. The function returns an integer handle we can use to access the buffer.

The second line then fills the buffer with the data we provided. We give the function the integer handle, the format, the actual data, the size of the data in bytes (to let OpenAL know how much memory to read, since internally the array is translated to a memory address), and the sample rate.

Finally we return the handle for use by the rest of the code.

When we are done using the buffer, we can use the AL.DeleteBuffer(int handle) method to dispose the buffer. Since the amount of buffers that can be used simultaneously is limited, it is important to dispose buffers if they are no longer being used.

Sometimes you will need to work with multiple buffers. OpenAL provides the functions AL.GenBuffers(int amnt) and AL.DeleteBuffers(int[] handles) to easily handle the use of multiple buffers. Sending data to a buffer still has to be done one by one however.

Source

When we have the data written to OpenAL, we still need a way to tell OpenAL to play sound from that data. This is where sources come in. One source corresponds to one “audio layer”. A source can only play one buffer at the time, so if you need overlapping sounds, more than one source will be needed. The amount of sources that you can use simultaneously differs greatly, but several implementations only allow up to 28 or 32 sources, so in general it is good practice to be conservative with the amount of sources used.

At any given point, the source contains a queue of buffers to play. We can start, pause and stop the playing of the sound from the source easily. We can also make the source automatically loop after it has finished playing all the buffers in its queue.

Working with sources is very similar to working with buffers. Consider the following code fragment:

void PlayData(short[] data, ALFormat format, int sampleRate)
{
    int buffer = BufferData(data, format, sampleRate);
    int source = AL.GenSource();
    AL.SourceQueueBuffer(source, buffer);
    AL.SourcePlay(source);

    // wait for source to finish playing

    AL.SourceStop(source);
    AL.DeleteSource(source);
    AL.DeleteBuffer(buffer);
}

This methods looks quite complicated on first sight, but we will see that it isn’t as bad as it looks.

In the first line we store the data in a buffer and retrieve the integer handle for the buffer. After that, we generate a new source from OpenAL and store the integer handle for the source as well. Next we queue the buffer in the source by using the integer handles. Now we’re all set to start playing the sound so we call the SourcePlay method to get it going.

After the source has finished playing, we stop it and dispose the source and the buffer, to free resources for other audio.

To actually detect when the source has finished playing, a loop can be used to check if the source has finished playing every few milliseconds:

    int processedBuffers;

    do
    {
        Thread.Sleep(10);
        AL.GetSource(source, ALGetSourcei.BuffersProcessed, out processedBuffers);
    } while (processedBuffers < 1);

The AL.GetSource method is basically a way to call get methods on properties of the source. The properties are defined in special enumerators. In this case we want to know how many buffers the source has finished playing, because if we know the source finished playing the source we queued, then the sound must have finished playing as well.

Of course we do not have to put this code in the function. Another solution would be to keep track of all currently used sources in a central solution, and check whether sources are finished in the main loop of the game.

The AL.GetSource method has a counterpart AL.Source (NB: so not AL.SetSource) to set properties on the source. We can for example set the volume at which the source should be playing with AL.Source(handle, ALSourcef.Gain, volume), where volume is a float between 0.0 and 2.0. Making the source loop is done by AL.Source(handle, ALSourceb.Looping, true). Note that the last letter of the name of the enum represents the expected type. All possible types are bool (b), float (f), integer (i), Vector3 (3f), or three integers (3i).

Remark that to get the amount of processed buffers we used an enum called AL.GetSourcei. This is a special enum for readonly integer properties.

The listener model.

The final object we should touch upon is the listener. For the purpose of playing music and sound this object is not very important, but it is a fundamental concept in understanding the underlying listener-source model of OpenAL: the listener represents the player in a three-dimensional environment together with sound sources (see image below).

listener

Sources can be provided with a position and a velocity. For stereo sound these properties are ignored, but for mono sounds the position of the source relative to the position and looking direction of the listener is used to determine what speakers should be used to play the sounds, allowing realistic full surround sounds for games. Further, the velocity of sources (again in relation to the position, direction and velocity of the listener) is used to create the famous Doppler effect.

It is not recommended to use these features for simple games, but they can provide a lot of immersion if used right in (first-person) 3D games. This means that for most applications, the listener can be completely ignored. The only exception is the possibility to set the volume of all sounds. The interface to get and set properties for the listener are very similar to those of the source. The following fragment for example changes the volume:

AL.Listener(ALListenerf.Gain, volume);

Bits and pieces

Before you can actually take a go at using buffers and sources, a connection to the sound driver has to be made first. In OpenTK this is done by creating an instance of AudioContext. The context has to be disposed when closing the application.

Because OpenAL consists of unmanaged code, your game/application will not halt when an error occurs in OpenAL occurs (or your computer will crash, that is always a possibility...). To actually catch errors, the method AL.GetError() can be used. If this method returns ALError.NoError (or '0'), everything is fine. If not, you can get a more specific error message using AL.GetErrorString(). It is recommended to check for errors after every OpenAL call. If you forget checking for errors, the error might not be detected until much later, and it because very difficult to debug the code that caused it.

Conclusion

There is much more to say about using OpenAL. In this post I have only scratched the service. The information should however suffice to get started with a simple audio engine. I will continue exploring features and possibilities of OpenAL in future blogposts, alongside the complete rewrite of my personal audio library. Feel free to leave a comment or get in contact with me if you have any specific requests or questions about this post.

Until next time!

Place comment

Your email address will not be published. Required fields are marked *