Celemony Melodyne 5

The long note here is a good example of Melodyne 5's new weighted pitch centre algorithm in action. Note how it ignores the quiet, sharp section at the end of the note and positions it such that the first, loud section is centred.

Although it's now been around for two decades, Melodyne is still the stuff of science fiction. In our exclusive review, we explore version 5's new superpowers.

Since its launch 20 years ago, Celemony's Melodyne has joined Antares' Auto-Tune in that realm of ubiquity where people use its name as a verb. When producers talk of vocals as having been 'Melodyned', they are usually implying the use of non-invasive pitch‑correction. 'Auto-Tuning', by contrast, often suggests the use of pitch processing as a special effect.

Of course, both packages are adept in both fields, but Melodyne has some unique capabilities. It remains one of very few programs that can perform pitch‑correction on polyphonic sources, and the Sound Editor introduced in Melodyne 4 provides a huge palette of resynthesis and processing possibilities. Version 4 also brought forth some amazing tools for manipulating the tempo of entire multitracks. New versions of Melodyne don't come along that often, but when they do, they are usually worth the wait!

More than four years on from version 4, Melodyne 5 also offers an impressive selection of new features. In this review, I'll assume that you are familiar with the basic operation of Melodyne and with the main features of version 4: if you aren't, check out our review of that version in the February 2016 issue.

Switching To Manual

The basic architecture of the Melodyne product line is unchanged in version 5. There are still four editions, with Melodyne Studio at the top and Melodyne Essential the most affordable, and the program is still available both as a stand-alone application and as a native plug‑in. There are no major changes concerning the integration of Melodyne into your DAW of choice: if your host program supports the Audio Random Access (ARA) protocol, you can work with Melodyne within the arrange page, but the streamlined plug‑in developed for version 4 still works very well in other DAWs.

A slightly unheralded aspect of the program that took a huge leap forward in version 4 was the online documentation. This has now been developed even further by having the documentation tailor itself to your specific use case. Two pop-ups labelled Edition and I'm Working With let you specify which of the four Melodyne variants you own, and whether you're working stand-alone or in a particular DAW. You'll then see only those elements of the manual that relate to your specific circumstances. This is a brilliant idea and much more user-friendly than having to scroll past great screeds of text to find the relevant sections.

Death By 1000 Cuts

When it comes to invisibly putting that which was out of tune into tune, many devotees feel that Melodyne has no peers. However, if you've ever watched a Melodyne ninja at work, you'll know that this could be quite a labour-intensive process. Seasoned users often head straight to the Note Separation tool and step through a vocal take chopping the auto-detected notes into much finer divisions.

This is tedious, but improves the results in two ways. If there is pitch variation in the course of a sustained note, Melodyne defines its pitch centre as an average value; so when you snap the entire note to the pitch grid, there's no actual guarantee that any individual section of it will be perfectly in tune. Chop it into smaller pieces and quantise each of these separately, and they'll usually be closer to perfect.

The second reason for going behind Melodyne's back and chopping up notes manually is that not everything detected by Melodyne actually has much pitched content. Sibilants, fricatives and many other consonants are primarily noise-based, and if you start trying to 'correct' their pitch, you'll achieve nothing except an increasingly obvious degradation of the sound. For the most natural results, then, it was often worthwhile to snip them out as individual 'blobs' so that they could be excluded from pitch manipulation. This had the added benefit that the Amplitude tool could subsequently be used as a very effective, albeit painstaking, means of reducing any obtrusive esses in level.

Sibil Servant

Celemony have been paying close attention, and some of the biggest developments in Melodyne 5 are designed to do away with all this tedious mouse-clicking. The functionality I'm about to describe is available in all but the basic Essential version. If you're working in Melodic or the new Percussive Pitched algorithm (see box), Melodyne now automatically splits out sections of an otherwise pitched source that contain mainly noise, and categorises them as Sibilants. Zoom in on a sibilant, and you'll see that there's no pitch line drawn through it; if you have Show Sibilants engaged, Melodyne will also display vertical hatching to enhance the visual differentiation.

The vertical hatched area in the selected blob represents a sibilant; here I'm using the Sibilant Balance tool to reduce its level relative to the pitched portion of the same note.

By default, sibilants remain attached to the pitched note that precedes or follows them, but are immune from most of the operations that apply to pitched notes. Pitch changes, formant changes and time-stretching all apply only to the pitched element of the note, leaving the sibilant unaltered. The Amplitude tool adjusts sibilant and pitched elements alike, but de-essing is made vastly easier by the new Sibilant Balance tool. Click and drag this in one direction, and any sibilant elements in the selection are progressively reduced in level; drag in the other direction, and the pitched elements are reduced instead.

There's no way that I can see to force Melodyne to reclassify what it detects as pitched or sibilant, but on every vocal I tried it with, it got the distinction absolutely spot on. At first, I wondered why it keeps sibilants attached to the adjacent pitched note, and thought it might be more useful to have them always separated out as blobs in their own right. However, as I worked with the new tools I realised that there is a good reason for this. A vocal sibilant isn't usually an isolated burst of noise with absolutely no pitched content: it's a burst of noise that overlaps the start or end of a pitched note. What Melodyne 5 does — uniquely, in my experience — is take this into account. It's not that no pitch‑correction takes place for sibilants, just that it's only applied to the non-sibilant part of the sound.

Needless to say, this is a colossal time-saver, and makes you wonder how you got along without it! It also enables supremely natural de-essing. For a quick-and-dirty approach, such as might be useful on backing vocals, you can simply select everything, choose the Sibilant Balance tool and dial the consonants down to taste. For exposed lead vocals, it's child's play to get in there and adjust the balance of individual esses, tees and other sticky-outy sounds. The only frustration I encountered was that, occasionally, I wanted to actually increase the volume of a consonant where the singer had swallowed the end of a word, without changing the level of the pitched component. This isn't possible using the Sibilant Balance tool alone: you need to split out the closing consonant and apply the Amplitude tool. It should perhaps also be mentioned that the sibilant detection seems very much oriented towards vocal applications, as you'd expect, and typically ignores the noise elements of many instrumental recordings.

One of the really neat features of the Sibilant Balance tool is that you can easily go way beyond what's possible with conventional de-essers. For instance, you could duplicate the vocal track, then globally set the Sibilant Balance to +100 percent on one and -100 percent on the other. You would then have complete freedom to apply different processing to each, making it easy to create the sort of super-bright yet controlled vocal sound that's ideal for modern pop music. This isn't a revolutionary concept — it can be done with spectral editing packages such as Steinberg's SpectraLayers, and with Eiosis's neat E2 De-Esser — but Melodyne 5 achieves a balance of power, naturalness and ease of use I've not encountered before.

Worth Its Weight

What, then, of the other reason for chopping up notes by hand in previous versions? That too has been addressed, courtesy of a new "musically weighted algorithm to calculate pitch centres". In other words, rather than using an averaged pitch curve as the basis for pitch‑correction, Melodyne intelligently determines which part of a given note is most significant in terms of our perception of pitch, and weights that section of the note more strongly when deciding how much correction to apply. This new algorithm is implemented either when you double-click using the Pitch tool, or when you apply the Correct Pitch macro, and it works extremely well.

The keyboard shortcuts window is now searchable, either by command or by key.Finally, just in case you do still find yourself needing to do lots of chopping up or other repetitive actions, you may welcome the improvements that Celemony have made to the keyboard shortcut system. The Shortcuts dialogue now contains two search fields. One allows you to search by function, so for instance if you enter the word 'Note', you'll see all commands that contain the word Note in their name, with associated shortcuts if they are assigned. The other allows you to press a key or key combination to discover what, if anything, it's assigned to; entering a modifier such as Shift displays all functions that have a keyboard shortcut involving that modifier. When shipping, Melodyne 5 will also come with sets of shortcuts designed to mirror the defaults in different host DAWs, so that muscle memories need not be retrained.

On The Level

Experienced Melodyne users will know that the program isn't only useful for correcting pitch and time. The aforementioned Amplitude tool offers a very natural way to adjust the level of individual notes within a part. However, in a typical vocal take that contains hundreds of individual blobs, using this to tackle more than a handful of them isn't really practical. Enter a new feature called the Leveling Macro — which is, in effect, a sort of offline compression algorithm. Select a group of notes and open this, and Melodyne will calculate their average level, taking into account any edits you've already made with the Amplitude tool. The macro presents two sliders labelled Level Quiet Notes and Level Loud Notes, which do exactly what they say on the tin, pulling notes either side of the average towards it. Unlike conventional compression, it doesn't also pull up noise and unwanted sounds such as breaths: the Leveling Macro intelligently ignores this sort of low-level detritus.

These 'before and after' shots show the Leveling Macro in action. Even at this extreme setting it sounds perfectly natural on this tin whistle part, bringing up the soft lower register to match the piercing high notes. After.

The Leveling Macro — available in all four editions of Melodyne 5 — is another one of those Melodyne features that, initially, leaves you wondering where all the controls are. Surely, you might think, it would give you control over the average level, and the threshold below which noises are ignored? Nope. Such control is neither present nor, in my experience, is it needed. The Leveling Macro just works, and depending on how uneven the source recording was to start with, will almost always do nearly all of the legwork necessary to create an even, consistent performance.

Unlike the sibilant detection, the Leveling Macro works brilliantly on sources other than vocals, producing more transparent results than any compressor I know of. Unlike compression, it doesn't change the note envelope, but simply makes the entire note louder or quieter. Also unlike compression, it seems to take into account something like the Fletcher-Munson equal loudness curve, bringing up notes that are low in an instrument or singer's register to compete with those that have a lot of mid-range energy. Even setting both sliders to 100 percent will often give you results that sound perfectly natural in context.

The Leveling Macro works beautifully on lead instruments and is equally useful on bass guitars: sometimes you might have to go through and manually duck the occasional clank or thunk, but often the macro does its job perfectly without any additional work. On exposed lead vocals, you'll find the odd word still sticks out or gets lost even with heavy Leveling, but there are vanishingly few processing artifacts, and it's easy to deal with any remaining discrepancies either using the Amplitude tool or post-Melodyne fader automation. It even works remarkably well on polyphonic sources in those versions of the program that support polyphonic detection. The only source where I found it easy to push the Leveling Macro too far was on drums, where ratios of more than 25 percent or so tended to cause abrupt jumps in the level of the cymbal spill; even here, though, subtle application can still be very effective. The same is true of complete mixes, but frankly it's amazing that it works at all, let alone as well as it does.

Umbilical Chords

The other major innovation in Melodyne 5 is the introduction of chord detection and a corresponding Chord Track. Polyphonic editing of source material is available only in the Editor and Studio editions of Melodyne, but Celemony have enabled polyphonic analysis and thus chord detection in the Assistant and Essential versions, meaning that the chord tools form a universal addition to Melodyne 5. In the stand-alone version, chords seem to be detected as soon as you analyse something using a polyphonic algorithm; in the plug‑in versions, you need actively to display the Chord Track, then right-click in it and choose Analyse Chords from the pop-up.

You can freely edit the chord track or choose plausible alternatives from a pop-up list.Chords can be re-analysed at any time you choose and, crucially, are applied globally across all instances of Melodyne. In other words, there can be only one Melodyne Chord Track, and this is shared between all the Melodyne plug‑ins in your project. In the Studio edition, detection can also be based on multiple instances (or tracks, in the stand-alone version), so you could for instance ask Melodyne to take into account both the bass guitar and the piano when deciding what the chords are, or indeed to use just three or four monophonic sources such as the individual parts of a string quartet. It's very important that bar lines fall in the right places, so you need to perform any tempo detection and editing before your final chord analysis.

We are used to Melodyne apparently being able to perform magic without any human intervention, but I have to say the chord detection doesn't quite reach those heights. On the plus side, and unlike some other programs I've tried, it is musically intelligent, and has a good stab at understanding individual chords in the context of the key it detects. If your piece is in C-sharp minor, for example, it will never incorrectly note the dominant chord as A-flat major. On the down side, the detection is quite source-dependent, and a bit fussy: it often puts too many chord changes in, and is prone to identifying passing notes as chord variations. What is to the human ear obviously a repeated chord sequence isn't always detected the same way each time, though the variations are usually minor ones, such as a substitution of a diminished for a dominant seventh chord.

There are options to allow or suppress the detection of extended chords, to display inversions as slash chords, and to omit thirds where they are not prominent in the detected analysis. Even so, unless your song incorporates a very simple chordal pad or keyboard part with minimal ornamentation, the chances are you'll need to do some editing to get the chord track into shape. Fortunately, this is easy. Right-clicking on a chord symbol brings up a shortlist of possible alternatives, or you can simply click on a dubious chord and type the correct one. Most common systems for chord notation are supported, and you can type using any of them — so even if, for example, you like to have major sevenths displayed using the triangle symbol, that doesn't stop you entering them by typing 'Cmaj7'. You can even type in 'I', 'II' and so on and Melodyne will calculate the correct chord relative to the detected key, though there doesn't seem to be the option to display chords in this form (and you need to type 'IIm' or 'II-' rather than 'ii' to get the minor second, for example). In hosts that support ARA, it is also possible to tell Melodyne to follow the host DAW's chord track data rather than using its own analysis.

If you work with vocals, you'll get more done with fewer mouse-clicks and to a higher standard. I can't think of any good reason why you would not choose to upgrade.

Working Together

Once you have a plausible Chord Track, you can then enable Chord Snap from the Options menu. You'll see the pitch grid in the main editing window change so that notes outside the detected chords are greyed out, and when you drag blobs around, they'll snap to the nearest non-greyed note. In other words, assuming your chord map is correct, Chord Snap makes it impossible to accidentally generate any 'wrong' notes. More interestingly, it opens up some pretty neat remixing possibilities.

Here, I've analysed the chords of a ukulele track and engaged Chord Snap. Note how the 'permitted' white notes vary with the chords; note too Melodyne's rather exotic names for some chords — the one labelled F#o/C, for example, would probably be better described as a D7.

Melodyne Studio was already amazing when it came to conforming the tempo of audio files, allowing you to drag any old loop or part into a multitrack and have it automatically adapt even to a complex tempo map. Now, you can do something very similar with the pitch by enabling Chord Snap, whereupon any notes you drag or double-click will automatically be conformed to the chord map.

As you might expect, the results of this process can be a bit hit-or-miss, but that's mainly because it can be hard to predict in advance how well two pieces of music with different keys, scales, chords and tempos are likely to work together! Also, in the version I tested, the documentation covered the process of extracting and editing a Chord Track in very clear fashion, but rather stopped short when it came to explaining the applications of this feature. It's one of those areas that requires some experimentation if you want to get the best from it; you'll also get more from the new chord features if you already have a basic grasp of music theory, rather than expecting them to remove the need for one!

All In All

When Melodyne 4 came out back in 2016, it was probably the biggest single upgrade to a piece of music software since Steinberg invented their Virtual Studio Technology. It would hardly be fair to expect another quantum leap forward on the same scale, but version 5 isn't far short! What's more, whereas most of the really good stuff in version 4 was reserved for the flagship Studio edition, nearly all of the significant v5 improvements apply to Melodyne Editor and Melodyne Assistant as well.

Once again, Celemony have implemented major enhancements on both the creative and the practical fronts. On the former, the new chord detection system perhaps isn't quite in the same league as the tempo editing introduced in Melodyne 4, nor as open-ended as the Sound Editor, but its benefits are still pretty obvious when you need to combine audio from different sources. And when it comes to practical, ergonomic improvements, Melodyne 5's sibilant detection, Leveling Macro and improved pitch centre calculation are simply massive. If you work with vocals, you'll get more done with fewer mouse-clicks and to a higher standard. I can't think of any good reason why existing users would not choose to upgrade to Melodyne 5 right away.