Paul Hedges sitting in front of the audio‑for‑picture system used to record the Radio Times sting, with one of his beloved Lowden guitars and the Yamaha Promix 01 mixer used to combine keyboard and MIDI module sources.
One of the easiest ways of achieving synchronisation between music and pictures is to run both on the same computer.Hugh Robjohns talks to a writer of music for broadcast who does just that. This is the second article in a three‑part series.
As I mentioned last month, the normal way moving pictures are supplied to a music composer is on a VHS tape, either with the location sound on the audio track (and possibly some off‑the‑shelf guide music), or with longitudinal timecode (the audio kind we all know about) and a dialogue guide track. It is also usual to have the timecode numbers displayed on screen in a box in the bottom half of the picture. Those with rather grander budgets might have the facilities to replay the video on a professional Beta SP machine (a common broadcast VTR format) which has a dedicated LTC (Longitudinal TimeCode) track as well as VITC which is a version of timecode encoded as moving dots right at the very top of the picture. The advantage of VITC (Vertical Interval TimeCode) is that the tape position can be read when it is stationary or moving very slowly — something quite impossible for standard LTC readers.
The Simple Solution
The simplest approach to writing music to the pictures is simply to run the supplied tape and play the required music, live, whilst watching. Well, it was good enough for the massed public crowded into the cinemas watching the early silent movies of the '20s and '30s! OK, so that may not be terribly practical these days, given the complexity of modern music. The next easiest way is to always rewind the video to the same place and hit the Play button on both video and sequencer at the same time. The synchronisation will be fairly loose, but it may well be close enough to work acceptably with many background‑music compositions.
However, this technique is hardly an attractive, or efficient, way of working and the next step up is to start making use of the timecode as a method of synchronisation. The obvious approach is to take the LTC from the video and pass that to the sequencer, enter the required time offset (if any) between the two and switch the sequencer to chase the timecode. Hitting play on the video should now cause the sequencer to locate to the same point in the programme and play in absolute synchronism.
The problem, though, is that now the video player is the master for the whole system. From the technical point of view, the (typically) unstable speed stability of the video transport could inflict similar instability on the sequencer, depending on the sync method used and the quality of the synchronisation strategy employed within the software. Matters become even more complicated if you introduce digital recorders or consoles, where the sample clock has to be locked to the timecode and video frame rates — the whole thing quickly becomes an utter nightmare if you aren't careful.
From a more practical viewpoint, most video players are desperately slow to use and have no facilities for automatic looping around a specific section of a programme. The frustration alone quickly persuades most people who regularly use this kind of facility to find a better alternative. I know of one professional composer for feature films and big‑budget television dramas who reckoned that he wasted up to 10 minutes an hour just waiting for the video machine to spool back to the start of the sequence he was working on! The solution he eventually decided upon was to buy a professional hard‑disk video player, and initially he looked at a machine called the V‑Mod. The advantage of replaying the video from a hard disk, of course, is the instant locating to any point — no more spooling time wasted! However, it turned out that it was considerably cheaper to buy a 200MHz PCI Mac with a Miro video card and a 9Gb hard drive (giving around 40 minutes of high‑quality video) to run the videos from, with the added bonus of gaining a backup machine in case his main computer ever broke down at an inopportune moment! He chose to take a video output from the 'Video Mac' and project it onto a large screen in front of him while he was writing; this, he claimed, helped to ensure the music matched the grandeur of the pictures — a point I'll return to shortly
The one thing you have to factor in to the equation when using this technique is the time required to load the video from a VHS (or whatever) into the Mac in the first place, and the hard‑disk capacity required if you wish to have several projects archived, and on the go at the same time. But this is a minor inconvenience compared to the removal of one of the most frustrating aspects of working with a conventional video machine.
The really good news for anyone tinkering with the idea, but not wanting to make the major outlay for a second Mac, is that you don't have to. Most of the better known sequencing packages — Cubase, Logic and Opcode's Studio Vision, for example — all include facilities to integrate QuickTime movies, running pictures in a separate window in solid sync with the song file, all on the one machine. All you have to do is record the video into the Mac (or PC) using the QuickTime movie recorder program and then call up the appropriate movie into the sequencer for a synchronised playback controlled from the sequencer in the normal way. To illustrate the strengths (and weaknesses) of this approach, I asked a friend and colleague, Paul Hedges, to explain how he used this very system on a project he was commissioned to do for the BBC last year.
Writing A Daily Sting
Paul Hedges's music for the Radio Times sting. Note how the video picture in the smaller window changes with the position of the Cubase progress bar through the song.
The project in question was to write a six‑second 'musical logo' to accompany a computer‑generated graphics sequence used in a daily 'what's on' programme for a BBC cable and satellite channel called, appropriately enough, Radio Times. This was a brief visual punctuation mark with slightly different versions of the graphics to denote each day of the week (the version shown on the screen grabs is for Wednesday). Paul described his thoughts about his chosen method of working:
"With such a short sequence, involving a lot of musical hit points, running the video from a standard VHS machine (the pictures were sent on a VHS tape) would have been a complete nightmare in terms of the frustration and time involved in continually cycling around such a short video sequence. So I chose to load the video into my Mac using the Apple Video Player as a QuickTime Movie.
"Looking through the video, it was clear there were a number of hit points where the music had to enhance the graphics. The opening scene is of a set of coloured bars coming into the blank screen from each side in a blind effect. When complete, a 'light sheen' traverses the screen diagonally and then the blind opens to reveal the Radio Times logo. Next, the blind closes again, this time to present the day of the week which freezes and could remain on screen for some time.
"I use Cubase as my sequencer on the Mac and all I then had to do was go to the File menu, select Setup Movies, Add Movie, and open the appropriate video — radiotimes2, in this case. On the Edit menu of Cubase there is an option called Movies which accesses any QuickTime file already registered through the previous steps and opens it into a small sub‑window."By default, the start of the QuickTime file is synchronised to the beginning of the song in Cubase, but at the top of the movie window is a timecode offset box which can be used to redefine the relationship between the start of the movie and the song. For example, entering 00:01:00:00:00 (HH:MM:SS:FF:subframes) would force the movie to start one minute after the start of the song file.
"A check box to the left of the time‑offset display allows the movie to be switched on‑ or off‑line — in other words to operate as an integral part of Cubase, or not. When it is on‑line, playing Cubase causes the pictures to run synchronously with near‑instant locating to any appropriate point. Equally, clicking on the Play button on the movie window causes Cubase to follow and, best of all, dragging the scroll bar to locate a particular picture frame drags the Cubase song position along with it, so it is easy to set and check any hit points — soft or hard. The only thing you can't do is play back any audio associated with the movie such as a temporary score or guide dialogue — Cubase doesn't support that in the version I'm using (v3.5).
"One aspect to be aware of when using this Cubase Movie feature is the drain on the Mac's resources. This particular project was entirely created using MIDI — there were no real audio files involved at all, which made things easier. I also made sure the viewing window was relatively small and that the original movie was of low resolution (only 11.4Mb for the complete 15‑second clip) to minimise the amount of data that had to be dragged off the hard drive and the processor time to display it. I also loaded the movie file onto a different hard drive from the one used for the Cubase and MIDI data, to share the workload between the drives. I am using a PowerMac 8500/180, which is slow by current standards, but coped with this without any problems at all."
Writing To Movie
"Writing the music started with working out an appropriate tempo. Since there were six evenly spaced hit points on this project, I simply timed them and worked out a suitable tempo from that, initially laying cymbal crashes as an audio guide and setting loop points with a bar's breathing space either side of the video start and end. However, in a more complex piece you can use the Match points (M‑points) feature in Cubase to help establish a relationship between the hit points and MIDI tempo.
"The hit points in this project area all defined by a movement of some kind in the graphics — opening blinds, light sheens and so on — which all take a finite time to complete the transition. That is why they are 'soft' and require an equivalent‑sounding musical cue such as a bell tree or backwards cymbal, rather than a drum beat — something with an extended duration which obviates the need for absolute precision in the synchronisation. It is also a good idea for the musical cue to lead the visual effect slightly, as a kind of advanced warning — it seems to work better that way that being slightly late, anyway!
"Fortunately, all the cue points were full‑screen events and therefore easy to see in the small movie window, but this technique would not work as well in other applications. Also, the small scale of the picture can almost hypnotise you into writing small‑scale music, which is definitely a trap to beware of. This point became an issue on another television music project I was involved in with my musical collaborator, Jerry Smith, entitled The Two Thousand Mile Harvest. This was a television programme set in the vast plains of America, and we decided that the music must be equally expansive, epic and flowing. Consequently, the mini view screen in the corner of the Mac's monitor had to be dumped in favour of watching the video on a large television, which was a far more effective way to become inspired by the grandeur of the shots. The other technique we used was to make timings of the various 'soft hits' in the pictures relative to each other, rather than in absolute time values from the start of the programme. In this way we had a much freer hand with tempos and focused more on musical mood changes, rather than becoming fixated on minuscule timecode readouts whilst squinting at long shots of open prairies!
"This Radio Times project was a relatively straightforward composition, with everything sourced and controlled via MIDI. The finished track was laid to DAT via a Yamaha Promix 01 console with the faders set to a static balance, as the internal dynamics were taken care of by the MIDI. I put a note in with the tape explaining to the picture editor that the opening strain of the music was to be synchronised to the first sight of the coloured bars and, as they wanted the music to end in an 'indeterminate way', I left a long repeating section at the end so that they could run the music for as long as they wanted before fading it. That was a much more practical solution for them than producing a lot of different length versions.
"Working with pictures using the QuickTime system with the pictures on screen, and intimately linked with Cubase, is excellent and extremely effective. Some of the constraints of eating into the Mac's resources should be much less of an issue with the latest Macs, and the only real difficulty is the limited resolution of the movie window. If I did a lot of this kind of thing I think I would investigate putting the movie window onto another Mac screen so it could be bigger and not get in the way of the Cubase windows which can become tiresome with the arrangement I used here."
