Paul White gets a sneak preview of an approach to synthesizer technology that could truly change the face of both synthesis and sampling as we know them...
It's a very poorly‑guarded secret that all the major synth companies are looking very closely at resynthesis as the next major evolutionary step in synthesizer technology. Conventional sample‑based synths don't 'need to know' how the original sound is created — they simply use a digital snapshot or sample of a sound, and then add further processing to change the pitch, envelope, spectral filtering, and so on. At the other end of the spectrum, we have physical modelling, where the synth 'needs to know' almost everything there is to know about the sound being synthesized, as it has to emulate all the physical characteristics of the instrument, and the player/instrument interface, in software.
Resynthesis fits neatly between these two extremes. The idea is that you take any sound, analyse it, and then recreate it from partials (sine waves) using Fourier resynthesis. The synth 'needs to know' nothing about how the sound was created in the first place, but does need to be able to break the sound down into its spectral components, and then recreate these components as exactly as possible. This idea has been around for almost as long as electronic synthesis itself, but at the 1994 AES show in San Francisco, Oberheim previewed a working resynthesis system which showed just how much potential this approach to sound creation has.
The first obvious question must be: if a resynthesized sound is so close to the original, why not just sample it and save yourself all the bother? As you may have guessed, there are several very good answers to this question. Once a sound has been analysed and resynthesized using a series of partials, you can extend the sound without the need to loop it (you simply play the series of partials for longer), change the sound's pitch (without it being subject to quantisation distortions), and, more importantly, you can modify the resynthesized elements in various ways to create new and interesting sounds. One practical example of this is true audio morphing, where the partials making up one sound can be transmutated over time into an entirely different sound. This is quite different from crossfading or filter morphing, and in an example demonstrated to me, a sax changed seamlessly into a mewing cat and back again before my very ears. So, how have Oberheim achieved all this?
The FAR Side
Working in collaboration with The University of California (Berkerley CA), IRCAM, Silicon Graphics, Lincoln Labs at MIT and CCRMA at Stanford University CA, Oberheim have set up what they call G‑Wiz Labs, a development facility dedicated to developing the hardware and software for 'real‑world' application of their FAR (Fourier Analysis and Resynthesis) technology. The FAR system appears to divide the original sound into several hundred short sections, or frames, per second, after which it undertakes a spectral analysis of each frame. The frames are then recreated using a series of partials, and a method of fading from one frame to the next makes the transition seamless. The completed resynthesized sound is then compared with the original sound, to identify any non‑pitched components which could not be reconstructed from partials, and these components are then recreated using filtered noise before being added to the resynthesized signal.
At the moment, the system is very costly, and runs only in conjunction with an Apple Mac computer, but it isn't hard to envisage more affordable spin‑offs based on this technology. The system is already capable of very convincing resynthesis — it sang me a complete line of the Susanne Vega song Tom's Diner, which was later changed in tempo, rhythm, pitch, and harmonic content for my amusement.
Sound morphing instruments based on preset partial sets are one way of utilising the technology at the affordable end of the price spectrum, while many of the sampler's traditional jobs could be made much easier and more flexible using resynthesis. For example, it will be possible to analyse just the highest and lowest note of an instrument, and then have the software interpolate the full range of notes in between, rather than having to resample every two or three semitones. Similarly, sounds can be sustained indefinitely without looping and without sounding stagnant, simply by meandering back and forth amongst the partial frames (not unlike wavetable synthesis).
By changing the relative balance of the partials, or by changing the pitch of some of them, sounds can be dramatically altered, while still retaining some of the characteristics of the original sound. I was shown a simple slider control that could change the balance between odd and even harmonics, which, linked to a performance control, could be very creative.
It's obvious that the surface of resynthesis has only just been scratched, but from the short time I spent with the system, numerous possiblities sprang to mind, all of them exciting. According to Oberheim, we can expect to see saleable units by mid‑1995, probably in the form of a 2U rack box with SCSI, Ethernet, a 0.5Gb hard drive and up to 128Mb of RAM. The unit will need a Mac to run, and will cost in the region of $10,000, so I don't expect we'll all be queuing up for one at once, but I for one am very much looking forward to the cheaper spin‑offs that will almost certainly follow. Perhaps more importantly, now that Oberheim have shown their hand, it won't be long before the other synth majors let on what they've been up to in this area.