LINUX MULTIMEDIA WITH PD AND GEM

Overview

Miller Puckette's Pure Data (Pd) is a software sound synthesis environment in the form of a graphic patching language. It has a GUI composed of a canvas for placing and connecting various modules, including synthesis primitives such as oscillators and envelope generators as well as more complex modules such as looping samplers and unusual types of FM synthesis. When the modules are connected to an audio output object Pd can produce and control sound in an excellent realtime performance environment.

Basic Operation

One good way to get acquainted with Pd's basics is to browse through the /usr/local/lib/pd/doc directory where more than fifty neatly documented and fully explained examples await your inspection. Load and run some of the example and reference patches: the examples appear in order of increasing complexity, and opening them one by one provides an excellent "walk-through" of Pd's basic operations.

You will find generators, filters, reverb units, sample scrubbers, and more complex patches in there, enough to satisfy at least your initial curiosities about Pd's synthesis and processing capabilities. If this kind of audio patching were all Pd could do it would still be an impressive environment, but thanks to the GEM OpenGL library Pd can also manipulate video and image parameters in realtime. Pd's flexibility permits arbitrary connection and control between its audio and video streams, forming a powerful environment for controlling and coordinating multimedia presentations.

Start Pd by selecting it from the AGNULA Applications menu or by opening an xterm window and typing 'pd' at the command prompt. You should then see a Tcl/Tk window similar to Figure 1.

Figure 1: Pd's opening display

Pd thoughtfully supplies a patch to test your audio and MIDI I/O. After starting Pd select File/Open and launch the testone.pd file found in /usr/local/lib/pd/doc/7.stuff. If your hardware performs as expected you can then proceed to some of Pd's more interesting patches. For a first example, try running 01.sinewave.pd in the /usr/local/lib/pd/doc/3.audio.examples directory. As well as being another good audio test this simple patch demonstrates Pd's self-documentation at work, with every aspect of the patch clearly explained in its accompanying text (Figure 2).

Figure 2: sinewave.pd's self-documentation

Once you have a good understanding of this patch, you should continue to browse the examples that appear in the Pd doc directory.

From The Simple

Figure 3 demonstrates Pd's neat method of graphically patching various components into complex sonic networks.

Figure 3: A simple Pd patch

This patch also demonstrates some basic structural principles: The osc~ signal generator creates an audio waveform (a cosine wave), the slider controls the frequency (pitch) of the waveform, and the network around the dbtorms object modifies the amplitude (volume) of the generated signal. Finally the modified signal is sent to the dac~ object (the digital-to-analog converter) and the results are heard through the audio system.

Now let's proceed to a more complex example using the GEM external. In order to use Pd with GEM you must invoke it via the $HOME/.pdrc file or with a command string similar to this one:

	pd -rt -lib /home/dlphilp/gem-0.87_2/Gem

The -rt flag prioritizes Pd's performance to realtime status. When coupled with a low-latency kernel Pd's performance is quite acceptable for live shows and other realtime circumstances. You'll want all the help you can get when you're running Pd with the GEM library: the kernel latency patch is a godsend, but you'll still need a hardware-accelerated OpenGL installation to make the best use of GEM.

Figure 4: A simple Pd/GEM patch

In Figure 4 we see the basic structure of a simple Pd patch utilizing the GEM library functions. Note that the gemwin and gemhead objects are required for all other GEM-related actions. This patch provides the mechanisms for loading a movie (anim-3.mov in this case) and playing it back while rendering it to the surfaces of a cube. The cube size is controlled by the slider movement, and the film can be started and stopped by clicking on the smaller bang button (one of the two cyan-colored boxes). The next three figures demonstrate this patch in action. The first snapshot illustrates the simplest rendering, with one face of the cube displayed and expanded to fit most of the rendering window. The second figure shows how the rotate object manipulates the cube, and the third snapshot displays the curious effect that occurs when the slider value exceeds the rendering window size.

Figure 5: Default video display

Figure 6: Video on a cube

Fig 7: Exploded video display

Of course the still images can't convey the effect of the movie playing upon the surfaces of the animated cube, but if you've followed these examples you should be able to see for yourself that it is very cool.

To The Complex

Now let's look at the possibility of combining our two example patches. Using straightforward cut/copy/paste editing we can easily copy one patch's contents into another and start having some serious fun. Pd truly lives up to the promise of its name: Data is purely data here, any data stream can be routed anywhere within a patch (with some restrictions), and we can easily set up a system in which one set of controllers simultaneously controls audio and video parameters.

Fig 8: A complex Pd/GEM patch

Figure 8 illustrates our complex combined audio/video patch. As you can see, the two sliders each simultaneously control an aspect of the video along with an aspect of the audio. Adjusting the audio amplitude results in an adjustment of the cube size, while moving the slider for the audio frequency control will also rotate the cube on its x/y axes. Multimedia artists will find great possibilities in Pd's support for such arbitrary attachments and correspondences. I should also note that GEM includes many other fascinating pixel-based effects, but I leave their exploration as an additional exercise.

Additional Exercises