Recording & Storing Large Projects On PC Hard Disks

Storing large PC‑based hard disk recording projects can become tricky, especially when you need to make regular backups too. Martin Walker looks at the options available.

As you probably know, digital audio recorded at a sample rate of 44.1kHz occupies about 5Mb of hard disk space per minute. An album lasting for one hour in its final CD stereo format will thus need about 600Mb. While you're doing your recording, though, you'll need more than two tracks, and the problem is that once you start adding tracks the amount of hard disk space required to store the audio expands rapidly. Fortunately, multitrack recording on hard disk, such as with MIDI + Audio sequencers, rarely requires each individual track to last the entire length of a song (unless you're dealing with a live performance), and data is only actually recorded when a track is active.

If we assume, for example, that on average, each track has content that lasts about 50% of the total recording time, for 4‑track use you'd need a minimum of 5Mb (amount per mono minute) x 60 minutes (total project length) x 4 (number of tracks), ÷ 2 (50% of each track filled), which once again gives a total hard disk storage requirement of 600Mb. For 8‑track use this expands to 1.2Gb, which is peanuts as modern hard drive sizes go. However, these must be viewed as absolute minimum sizes, since some extra space is bound to be occupied by alternative takes which will not be used in the final version, but which you won't want to delete until everything is finalised. In fact, I suspect that many people would back up such takes and keep them for posterity — you never know when they might come in handy during a remix.

Once you move to 16‑track, or if you find yourself filling the entire length of each project track, you'll quickly find yourself exceeding the 2Gb size limit imposed by DOS and Windows 95 for a single drive partition. At this point many people start to panic, imagining possible mayhem if recording and playing back across multiple partitions, or even multiple hard drives. But this needn't be a problem — in fact, using multiple drives may even improve overall performance compared to accessing a single huge one.

Channelling Your Effort

The 'Drive Size' box shows that a separate 3Gb hard drive for audio should be adequate for most people running 8‑track systems, since this would provide 71 minutes of recording time for each track, which should be enough for most album projects. If you have a 16‑track system, the popular 5.1Gb drive size will provide 60 minutes of storage, and since a single CD can run to 74 minutes, one of the newer 6.4Gb drives, yielding 76 minutes, would probably be a safer bet. If you want to go beyond 16 tracks, the largest drives that are easily available at the moment are 9.1Gb. A drive of this size would provide 72 minutes of 24‑track storage, and 54 minutes with 32 tracks.

Don't be tempted to use removable drives instead of backing up your data. This is asking for trouble.

However, it's not just the size of the drive that's important. Each time you double the number of tracks, you're asking the drive to work twice as hard just shifting data to and from the soundcard, and the performance of the drive can often be the limiting factor in the maximum number of tracks available. The most useful indication of hard drive performance is Average Access Time (my 'Bottleneck Blues' feature discussed this in greater length — see the 'Further Reading' box). Each audio track is stored in a different place on the drive, and in order for your MIDI + Audio sequencer to replay these multiple data streams, the drive must access the appropriate part of each stream in rotation.

First a small section of the first track will be grabbed, then the equivalent section of the second track, and so on. To replay 16 or more audio tracks, the read/write heads of the hard drive will be darting all over the place at extremely high speed. So if, for example, you're replaying 16 audio tracks (which means that a total of 1.3Mb of data must be read per second), it's not sufficient to buy a drive which can read data at 1.3Mb per second, since this transfer rate won't take into account the extra time needed to physically move the read/write heads of the drive to 16 different places during the process.

If you're using a MIDI + Audio sequencer which allows real‑time EQ and effects, each stream of data must also be processed before it is sent to the soundcard. Here it will emerge as either an analogue (audio) signal, or a digital signal which is sent to the outside world to be D/A‑converted (to again emerge as audio), or it will be digitally recorded direct to another medium, such as DAT. This is a huge amount of work for any computer system to do, and if the potential maximum number of tracks is to be achieved, every aspect of PC performance must be optimised. As always, buying a drive that has a performance in line with the requirements stated by your software package is a good starting point, but there is a limit beyond which other techniques must be explored.

Multiple Drives

Once you move beyond the 2Gb size limit for a single partition, as set by DOS and Windows 95 (see my 'Driving it Home' article for an explanation of this), you should find no problems in splitting an audio project across multiple drives. After all, you're not trying to split individual files, but simply placing additional files on a different drive or drives. In fact, running multiple drives means that the overall access time can actually decrease, as compared with accessing the same amount of data from one huge drive, since there are more read/write heads looking at the data. For people with huge storage requirements this is a sensible approach.

RAID (Redundant Array of Inexpensive Disks) is a technique for improving the performance of large storage systems by spreading the data across an array of smaller hard disks. If you have four drives working in harmony, data can be accessed from them in a quarter of the time it would have taken if you were using a solitary drive. However, it is the Redundancy aspect that is most appealing for anyone with valuable data. Although the Mean Time Between Failures (see 'Failure Guaranteed?' box) gives a statistical indication of hardware reliability, if you need many drives to accommodate a huge amount of data, the thought of even a part of your storage system suddenly becoming faulty is frightening indeed. The beauty of the RAID approach is that, by writing a certain amount of extra 'redundant' data onto an array of disk drives, which essentially duplicates data elsewhere on the array, you give yourself the chance to completely recover from a disk crash.

Depending on the type of RAID system used (there are seven levels, from 0 to 6), you can simply speed up access by using multiple drives, or at the higher levels, split each file save across them all, writing enough extra information that, in the event of one hard drive failure in the array, the system can simply carry on by retrieving the extra required information from the others. Although RAID technology offers security, this must be weighed against the increased cost of providing extra storage for the redundant data, and the performance penalty of writing it in the first place. For web servers and databases, the security aspect far outweighs these considerations, but at the cutting edge of digital audio, a long‑term backup policy onto a different drive or another medium will probably be more appropriate.

Saving It For Later

The more data you record to hard disk, the more important it is to make a backup. If you have your own project studio and are assembling an album your music may be on the hard drive for anything between a few months and a year or more, and it's dangerous to consign this much creative output to a single piece of hardware. If you run a commercial studio offering hard disk recording you'll obviously have to maintain backups of every project passing through.

There are many alternative backup methods. Once you have digital I/O, you can back up your audio data direct to an audio DAT tape, but this has the big disadvantage of operating in real time for stereo signals, and even more slowly for multitrack projects. Dedicated backup drives can 'archive' your data using compression software, or you can save the project data just as it is. If you need rapid access to a project, to quickly create another mix, fly in extra vocals or add extra tracks, you won't want to spend a long time archiving the current contents of your hard drives, and then reloading the archived version of the project in question.

The simplest solution, for smaller projects, is to use a removable drive like the 1Gb Iomega Jaz and 1.5Gb Syquest Syjet models. These store data on removable cartridges. If you only need eight tracks, these can even be used for direct recording, although many people prefer to use a fixed hard drive for this, in order to tweak more performance from the PC system, reserving the removable one for backup purposes. Of the two, the Syjet is slightly faster and offers greater capacity, but is more expensive (you can't have it both ways). Incidentally, the little Iomega Zip drive is wonderful for general‑purpose storage of samples and individual songs but, with a capacity of 100Mb, is not large enough (or fast enough) for multitrack use. Similarly, the Syquest EZFlyer, at 230Mb, is more suited to portable sample storage.

Each time you double the number of tracks, you're asking the drive to work twice as hard just shifting data to and from the soundcard, and the performance of the drive can often be the limiting factor in the maximum number of tracks available.

Of course, the easiest option is not to back up the data at all, but to swap the whole hard drive. Various options exist to do this — the Soundscape PC recording system (reviewed in the last issue of SOS) features a model with two drives that can be unplugged, allowing the musician to keep a project safely tucked away for future use, and the Glyph range has 'hot swap' options for its hard drives (more on this range later on). The advantage over using the more specialist drives with removable media is that you're not limited by the capacity of the cartridge — if you want another 9Gb storage at a moment's notice, you plug in another hard drive with this capacity. Of course, swapping the entire drive will prove rather more expensive than an equivalent‑sized removable cartridge (if available), but you do get the advantage of top‑flight performance if you buy a huge top‑flight drive, whilst still being able to plug in a range of smaller drives to suit other projects. However, don't be tempted to use removable drives instead of backing up your data, as this is asking for trouble.

Media Trends

Another new audio storage trend has been emerging of late — the CD‑R. Since most projects end up on CD anyway, and there are an increasing number of cheap CD‑R drives for the PC which record at 4x speed, an entire 'album in progress' can be recorded onto a blank CD in 15 minutes or so, and then played back on any audio CD player. The same data can also be read back at an even faster rate, using standard CD‑ROM drives. These are available at up to 24x speed, so even a full 74‑minute CD could be reloaded in only a few minutes, especially if the data is saved in CD‑ROM format, rather than standard CD Audio format (which can take rather longer to read in). The CD‑R discs can only be used once, but they are now down to about £2 each (if bought in quantity) so, for up to 74 minutes of stereo data, are even cheaper than DAT tapes. For multitrack use, they could still be viable if you are prepared to split a project across several discs, saving a track or two onto each — you should get about nine minutes of 16‑track audio per CD‑R.

One drawback of CD‑R for file backup in the past was that the discs were normally written on a track‑by‑track basis, rather than using the more versatile file‑based approach taken by computers. This makes 'burning a CD' a rather more specialised process. The latest software, such as Adaptec's Easy CD Pro and CeQuadrat's PacketCD, uses a technique known as 'packet writing', and this allows you to write smaller bursts of data to the disc. This uses the space more effectively. It also changes the writing process to a simple Windows 95 'drag and drop' procedure, using the 'Save' or 'Save As' options to write files in exactly the same way as to any other drive.

Shine On

Magneto‑Optical (MO) drives use a rather different method of recording and retrieving data, combining the more usual magnetic system, used by floppy and hard drives, with laser light. There are both magnetic and optical layers on the MO disk. The magnetic layer is used to store the information, but a laser is used to read it back from the optical layer, whose polarity is locally altered by the adjacent magnetic data. Data is written not by a magnetic field, but by using the laser at a higher power setting. Heating up the exact spot on the disk to around 200 degrees Centigrade causes the magnetic layer at this point to lose its strength. Then a low‑level magnetic field can be used to set the orientation of the magnetic layer, providing the normal '0' or '1' value.

MO technology has its pros and cons, just like any other format. Disadvantages are that writing is slower than a normal hard drive, since two passes are needed to overwrite any existing data — one to wipe the earlier data, and the second to write the new. However, advances have been made that speed this process, and MO drives are available in both 5.25‑inch and 3.5‑inch formats, with storage capacity of up to about 650Mb. One big advantage of MO drives is that their contents are not affected by stray magnetic fields. The same could not be said of traditional hard drives, floppy disks or DAT tapes! Also, for long‑term storage, the shelf life of MO disks is supposedly as high as 100 years! Several manufacturers, such as Panasonic, have combined a type of optical drive with a normal CD‑ROM player, and these use rewritable
5.25‑inch 650Mb media costing about £22 each, but can play back standard CD‑ROM disks as well. These are still popular, but something tells me that most musicians would probably prefer to buy a CD‑R drive, and live with its 'write‑once' approach.

A Reel Solution

For the most cost‑effective archive storage, tape drives are still very popular. The main difference between these and the other forms of storage already mentioned is exactly the same as that between analogue/digital tape recorders, and hard disk recording — the lack of random access. Although this means that it can take far more time to recover a single file from a complete computer backup, music data storage is normally on a complete project‑by‑project basis, containing fewer individual files, so this isn't such a disadvantage. The main advantages of tape are that the storage capacity is only limited by the maximum length of tape, and that since tape drives have been used for years to provide computer backup, they have an enviable reputation for long‑term reliability. The downside is that reading and writing is normally much slower than with other systems, since although compression methods work well for much computer data in speeding up these processes, music data rarely compresses as effectively.

The most familiar tape format for musicians is probably DAT, albeit in a slightly different set of formats known as DDS (Digital Data Storage). Many DDS drive mechanisms are available from both Sony and Hewlett‑Packard, and these can back up 4Gb of data (up to 8Gb when compressed) onto a single DDS2 format tape, and 12Gb data (up to 24Gb compressed) onto a single DDS3 tape. The drives themselves are much like audio DAT machines, but without the audio converters, and use identical‑looking 4mm tape cartridges with even more emphasis on data integrity. Data DAT machines are fast, with transfer rates of 60 to 120Mb per minute. However, the larger models tend to be priced at well over £1000, and will probably be more suitable for professional studio use, where time is money.

For those of us with a little more time than money, there are various other tape storage systems available. For those with more modest storage requirements, Iomega market the Ditto 2Gb drive, which uses Sony‑developed QW‑2Gb (compressed) tapes costing about £16 each. The transfer rate is only about 8Mb per minute (0.5Gb per hour), but, as they say, the price is right, at about £100 for the stand‑alone external version. There's also a bigger, 3.2Gb, version at about £140. The next stage up is to use one of the Travan mini‑cartridges, which were developed by the 3M company. These are a widely used standard within the computer industry, and can store between 4Gb (up to 8Gb compressed) on a single cartridge costing about £30. Transfer rates are between 32 and 65Mb per minute. A Travan‑based 8Gb backup unit can cost as little as £300, so this is a particularly cost‑effective solution.

Narrowing It Down

Most of these drives can be purchased in both internal and external versions. If you only have one PC to back up, the internal version will normally be a little cheaper, but for use with several PCs an external unit is more useful, as well as being portable. Although many external drives that plug into the parallel (printer) port of the PC are available, they can never be as fast as one that plugs into a SCSI port, so for the large file sizes used by hard disk recording, SCSI backup drives have to be the best bet. After all, many large hard disk recording systems will be SCSI‑based anyway, so you will already have the required SCSI host adaptor card.

There are several manufacturers (including Dynatek, Glyph, and Micropolis) who specialise in providing hard drives and backup drives specifically tailored for audio/visual use. A rack width can normally accommodate two drives side by side, which gives you plenty of options. The best course of action is to get further details of each range, and then discuss your specific requirements with the company concerned. You can, of course, also buy backup drives through mainstream computer retailers, but what you save in initial cost will probably be outweighed by the long‑term support provided by a specialist. It's your data: you decide how important it is.

Peering Into The Future

As more and more data needs to be squirted along the data busses, there are new standards poised to make this process ever faster. The USB (Universal Serial bus) is being much touted at the moment, and its strength lies in allowing a daisy chain of up to 127 peripherals to be attached to PCs, although its data rate is fairly slow by today's standards, at 12Mbps (mega bits per second). Since the USB connection provides a 5V power supply as well as signal connections, it can provide user‑friendly upgrades, by removing the need to open up the PC quite so often, and backup devices attached via USB should be faster than those attached to the PC parallel port.

Although PC motherboards have had USB hardware capability for some time, most manufacturers have neglected to fit the appropriate sockets, partly to save a few pennies, and partly because there is, as yet, little to plug into them. Windows 95 also has limited support for USB. However, the new TX motherboards (more about these in a future issue) provide more extensive USB support, and USB peripherals are expected to become more important within the next few months. Windows 98 (now expected in the second quarter of 1998) will also provide much more built‑in support for USB.

One of the most eagerly awaited developments is DVD (Digital Video Disc), which is a much more advanced form of CD‑ROM, with media of 4.7Gb single‑sided capacity and 9.4Gb double‑sided. CD‑ROMs will still play back on DVD drives, but CD‑R discs will not, since the laser required to read them is a different colour. Once blank DVD discs come down to a sensible price, this would seem to be the ideal medium for audio backup. However, with this much storage we still need a much faster means of transferring data. Firewire, or, to give it its proper title, 'IEEE‑1394 Firewire', is designed to do just this. It has sufficient speed (up to 50Mb per second) to allow multi‑channel digital audio streams and synchronised video to be ported in real time to external devices, and should provide us with a universal audio/visual standard. The cables have only six cores (two twisted pairs for data and signalling, and two pins for power), so we should also eventually see the end of the hugely confusing selection of SCSI cables currently on offer. Firewire is also supported by a huge range of companies, not only those from the computer industry, but also music giants such as Yamaha. Anything that simplifies the interconnection of digital signals is to be applauded, but if Firewire eventually becomes the universal standard that many people are expecting, musicians will be clapping more loudly than anyone else.