We investigate an experimental system that promises to retrospectively add Yamaha CS80-style expression control to any keyboard.
Paul White is a guitarist and I'm a keyboard player, so it's inevitable when we're travelling on SOS business that our friendly argumentative banter turns to the alleged superiority of the guitar as an expressive instrument. As Paul is always keen to remind me, a guitarist can choose how hard and where to pluck the strings, how they are damped, and how to introduce vibrato and bend notes with their other hand. All very expressive and magical in the right hands, of course, whereas the keyboard player — according to Paul at least — can either push the right keys down, or not! OK, so those of us who play keyboards know there is a little more to it than that, but the options for creative expression are certainly constrained in comparison.
A close up of MkIV Touchkeys pad surfaces (which have a slightly pink tinge on the white notes!).I owned (between 1985 and 1997) a beautiful Yamaha CS80 (with a complete set of legs and pedals), but foolishly, in retrospect, sold it when I needed the space for a console Hammond tonewheel organ! I really enjoyed playing the CS80 because it was easily the most expressive synthesizer ever built, thanks in large part to its fabulous polyphonic aftertouch system, and I've been looking for something as capable ever since! Discussions about polyphonic aftertouch, and which keyboards and virtual instruments support it, pop up quite often in SOS forum threads and that's where I first heard about something called 'TouchKeys' — an experimental DIY retro-fit system that adds a remarkably versatile form of touch-sensitivity to standard keyboards. Naturally, I was very intrigued and wanted to know more about this system, and that involved a trip to Queen Mary University of London (QML) to meet its maker, Dr Andrew McPherson.
The TouchKeys Concept
The basic concept of a touch-sensitive keyboard is not new, and in fact John Eaton and Bob Moog had an idea for such a device way back in 1968. However, the complexity of creating a practical touch-sensitive system is ably demonstrated by the fact that it took them 20 years to reveal a working prototype, and their system still hasn't entered production another 15 years on.
As is so often the case, the devil is in the engineering detail and the availability of affordable supporting technologies — and it's the latter that lies behind the amazing practicality of the TouchKeys system. In brief, TouchKeys employs full-length custom-made capacitive sensor pads fixed onto each key of a standard keyboard. The system can accommodate from 12 to 88 notes, and the individual sensor pads are just 1.6mm thick when mounted. The sensing technology is similar to trackpads on laptops, and is capable of sensing not only the absolute finger position, but also any up-down or side-to-side movement across the key surface. The pads are multi-touch, too, able to detect up to three fingers on the same key. All of this positional data is collected and processed 200 times a second, and used to trigger specific effects or actions via standard MIDI controller data. The implementation is simultaneously very complex, very clever, very elegant and very impressive.
Dr Andrew McPherson playing a Doepfer keyboard fitted with MkIV TouchKeys.
Dr Andrew McPherson is an American academic currently working at QML (see box), where his research combines his passions for music and technology. The TouchKeys system evolved from previous 'augmented instrument' experiments and the original intention was to sense as accurately as possible the way in which a player touches the keyboard during a performance, and then use that information in intelligent ways to make the keyboard a richer instrument to play. The idea is to provide musicians with new avenues of performance and creativity, by extending their existing instrument skills and techniques.
Development
TouchKeys development started in 2010 when Andrew was working at Drexel University in Philadelphia, and his first practical prototype emerged the following year. The current incarnation is the fifth-generation product, although there are only very minor cosmetic improvements over the fourth generation, which I played; basically the MkV version employs a much whiter surface (on the white notes), as opposed to a slightly pink-ish colour in the MkIV!
After three years of development the hardware and software arrived at a level where the system was both reliable and practical, and Andrew was keen to get it out of the laboratory and into the hands of real-world music-makers. Only in that way could he gather the feedback necessary to help develop TouchKeys further. However, to achieve sufficient cost-effectiveness to be attractive to end users Andrew needed to place a significant parts order to benefit from economy of scale. That, in turn, needed a substantial financial backing and the solution was a 35-day Kickstarter campaign beginning in late July 2013. Impressively, 198 backers signed up in total, and around 60 people from 20 countries acquired instruments through the campaign, with a further 10 purchasing TouchKeys systems subsequently.
The touch sensors, circuit boards and software were all designed by Andrew, and manufactured entirely in the UK. Andrew and his colleagues at the university took care of the QC and kit packaging, and they also built about 20 TouchKey-equipped instruments (by installing the kits on various Novation and Doepfer keyboards). The latter was a very time-consuming process, but it provided full turn-key systems for some of the less technical Kickstarter supporters who wanted to receive working instruments. Although currently sold out of packaged DIY kits from the original production run, anyone interested can contact Andrew to be added to the waiting list for a second production (there is no timetable or details as yet).
Andrew continues to take the TouchKeys project forward as part of his academic research, but one option currently being explored is to hand the project over to a commercial partner at some point. Apparently the TouchKeys technology could be integrated directly into a keyboard's individual keys quite easily if a major manufacturer wanted to invest in the injection-moulding set-up costs. Of course, all of the key intellectual property of the TouchKeys technology — including the sensor design, the firmware, and the main applications — are protected by various patent applications from Drexel University and QML. Andrew has also gained a lot of invaluable knowledge of the micro-engineering subtleties involved after his three years of development.
Technology
As mentioned, custom-made multi-touch-capable, touch-sensitive capacitive pads provide the TouchKeys user input, and these are pre-formed to match the exact shapes and sizes of the various keys across a standard keyboard. The individual note pads are attached with precise laser-cut double-sided tape which was selected specifically to provide a very secure fixing while leaving no residue or damage if subsequently removed. The surface of each pad has an array of separate touch zones (which can be seen as a faint diamond pattern), and several different designs and surface coatings were tried before arriving at the current Mk V configuration. While this diamond texture can just be felt when playing, I found the tactile response to be very similar to a standard plastic keyboard surface and it presented no difficulties for playing. The stick-on nature of the sensor pads is most obvious on the black keys as their edges are exposed. However, the MkV version has slightly narrower pads with rounded corners to ensure they don't impact the player's technique, and while I was aware of their presence they didn't detract from the playing experience.
A trio of keys viewed from below, revealing the embedded micro-controllers, ribbon cables and programmed note markings.
The touch pads on the black keys provide a resolution of 1536 x 256 contact points, while the longer white keys have more pads and achieve a resolution of around 2432 x 256. The visible dots scattered over the surface are circuit board 'vias', which provide electrical connections between tracks on the four-layer PCB that is integrated beneath the capacitive touch surface. The whole circuit assembly measures just 0.8mm thick, while the double-sided adhesive tape adds another 0.8mm to its overall height.
Astonishingly, each note's sensor pad incorporates its own embedded micro-controller chip which contains all the interface hardware and firmware necessary to generate meaningful data from the capacitive contacts. This is only possible through the use of miniature surface-mount technology, and the micro-controller is visible on the bottom surface of the touch pad. The surface contact data is transmitted over a digitalbus to a system controller board via a narrow ribbon cable emerging from the back of the pad. Helpfully, each key pad is marked on its underside with a specific note, and the micro-controller chip is programmed during manufacture to encode the data output as belonging to that specific note.
The ribbon cables from the sensor pads are dressed neatly along the backs of each key such that they don't impede the key action in any way, or stress the connectors on the touch pads. After disappearing inside the instrument's cabinet these ribbon cables plug into a controller board — larger kits link multiple controller boards together — that are generally placed on top of the keyboard action, under the instrument's case. The controller board electronics scan and gather the data from each key 200 times a second — the system is fully polyphonic — and the information is output as a proprietary data stream via a standard mini-USB 2 port. The TouchKeys hardware is entirely bus-powered which makes retro-fitting the system very simple and minimally invasive. Andrew estimates that anyone could install a full 88-note system in a single evening without too much difficulty.
Interestingly, when optimising the system for the first production run, Andrew discovered some unexpected physical complications. Apparently, some commercial keyboards have slightly shortened black keys, and on others the black keys have a sloping rear edge. It wasn't practical to make alternative-length black-key sensor pads, but the problem was solved by mounting the ribbon cable connector a short distance along the pad. That allows the installer to cut off any surplus material at the rear of the pad, if necessary, without damaging the sensor circuitry. Since the sensor pads can't be bent (because they are a multi-layer PCB) the sloping rear edge problem was solved by designing some optional laser-cut padding wedges.
Bespoke TouchKeys software — available for Mac, Linux and (very recently) Windows platforms — is open source so that anyone can tinker with it if they so wish. While developing the TouchKeys system Andrew's research revealed that only about 40 percent of a typical musician's finger contacts on the keys results in played notes. Consequently, the instrument's own MIDI note on/off data is integrated with the TouchKeys positional data to determine which are intentional contacts and movements, and which should be ignored. Having said that, since the touch data is actually being gathered all the time, it can also be used even if the keys aren't physically depressed. As an example, Andrew demonstrated how sliding a finger lightly across a specific set of white keys could be used like a traditional ribbon controller.
TouchKeys pads being installed on a keyboard and wired to controller boards outside of the instrument case.
After integrating the positional data and the keyboard's own note on/off data, the TouchKeys software processes the touch information to produce the required MIDI controller data and maps it in any way the user desires. The keyboard's own MIDI information can also be passed on or filtered out as required with complete flexibility, and consequently, TouchKeys control options are almost limitless. Thankfully, some default starting configurations and modes are provided for convenience, and the standard MIDI mappings currently built into the software include options to generate manual vibrato from side-to-side finger movements (like a viola player's!), and polyphonic pitch-bend and after-touch commands controlled by moving the fingers up and down on the keys. Placing two fingers on the same key can be used to generate micro-tones, and moving the finger over the key as it is depressed can trigger scoops. These are all just suggestions, though, and the system can be configured to respond to different playing techniques in almost any way imaginable. There is a dedicated Google Group for technical support and to exchange ideas.
In Use
Much to my surprise, the TouchKeys system doesn't seem to detract from the playability of a standard keyboard to any significant extent — at least not to an amateur musician like myself! The current MkV retro-fit touch-sensors fit extremely neatly and securely onto the keys, and the system works remarkably well despite its experimental and DIY nature. Undoubtedly, integrating the sensors directly into the body of traditionally shaped keys would be a more attractive solution, but that is understandably impractical at this stage of the research project. Hopefully a commercial partner will recognise the potential of this astonishing system and production-engineer it accordingly in the not too distant future.
Despite the relatively small production scale, the Kickstarter campaign kits were surprisingly affordable: a single-octave set cost just £195 and a full 88-note set was only £1200 (see the pricing box), so this isn't impractically expensive technology at all. Installing the TouchKeys system is relatively simple and while it requires some manual dexterity it doesn't need a degree in electronics engineering. The keyboard instrument's case does need to be opened up to allowthe controller boards to be located appropriately, but there is no soldering required, and no interference with the keyboard's own electronics. Once the software has been installed and the MIDI controller modes configured, it's good to go!
I played Andrew's developmental MkIV TouchKeys system installed in a Doepfer LMK2+ weighted keyboard in QML's C4DM green-walled studio. The MIDI controller data was allocated to drive five inceptions of Arturia's CS80 virtual instrument and the standard configuration's finger-wobble vibrato action was totally intuitive and instinctive (although I did play the 'cello in my formative years!).
Since I was apparently playing my beloved CS80 I wanted to try something akin to its expressive polyphonic aftertouch, making individual notes within a chord sing out in that classically Vangelis way! The original CS80's polyphonic aftertouch was activated by (quite heavy) over-pressure once the keys reached the bottom of their travel. However, this kind of technique won't work with TouchKeys because it isn't 'pressure-sensitive' in that way. Instead, Andrew had set up the system to introduce aftertouch effects (opening the filter cut-off, for example) by sliding fingers slightly towards the back of the keys, and it took me a while to get to grips with this unfamiliar technique. However, I started to master it after a while and the results were very expressive. By making more exaggerated up-down gestures, the system could also be persuaded to introduce pitch-bends and swoops, too.
Almost any deliberate finger movement across the keys can be detected and employed to provide some form of enhanced articulation — it just comes down to imagination, dexterity and technique. The configuration software allows countless parameters and their working ranges to be adjusted to fine-tune the way the system responds to the player's technique, and while the default starting positions are quite usable, most system owners will probably end up tweaking things to suit their own requirements as they develop and hone their own unique playing style and mannerisms.
Even in the relatively short time I spent
A TouchKeys-equipped keyboard and Arturia's CS80V software provides a pretty convincing imitation of an original CS80. with the system, its innate mechanical playability and the phenomenal potential for unparalleled expressive performance control was very obvious. The nice thing about TouchKeys is that it builds upon existing keyboard techniques — you're not learning from scratch, just adding new movements and actions to enhance the performance — and that means the learning curve is fairly shallow.
If I was a keyboard manufacturer I'd pick up the phone to Andrew's office as soon as possible; the potential of this system for high-end performance keyboards is absolutely phenomenal! In the meantime, anyone interested in acquiring a TouchKeys system can make contact with Andrew through the Kickstarter page: www.kickstarter.com/projects/instrumentslab/touchkeys-multi-touch-musical-keyboard. Further technical information, various demonstration videos, and links to Andrew's published academic papers on the technology can also be found on this Kickstarter web page.
Guide TouchKeys Kit Prices:
- One-octave set : £195.
- Two-octave set: £360.
- Four-octave set: £650.
- Five-octave set: £850.
- 88-note set: £1200.
- All prices include VAT and the TouchKeys software (Mac, Linux and Windows).
Dr Andrew McPherson
Andrew McPherson came to Queen Mary University of London (QML) about two and a half years ago to take up a post as a lecturer (Assistant Professor) in the Centre for Digital Music (C4DM), part of the university's School of Electronic Engineering and Computer Science. His previous educational background combined music and electronics with a double-major Undergraduate degree from MIT (2004) in music and electrical engineering, after which he went on to study a Masters in electronic engineering at MIT's Media Lab in 2005, and followed that with a PHD in classical composition at the University of Pennsylvania in 2009.
Andrew's primary area of research is the intersection between music and technology, and particularly in the way musicians interact with their instruments. "Musicians are creative and subversive people,” he told me. "They do things with instruments and equipment that you don't expect!” And to that end Andrew is interested in creating 'augmented' or 'hackable' instruments that musicians can develop, as well as exploring ways of introducing new performance techniques to familiar instruments. The idea is not to replace the role of the musician — Andrew is an accomplished viola player himself — but rather to extend the capabilities of different instruments to enhance the creative opportunities for the musicians.
Magnetic Resonator Piano
While studying for his PHD at the Music and Entertainment Technology Laboratory (MET-lab) at Drexel University in Philadelphia, USA, Andrew's first enhanced-instrument experiments resulted in what is essentially a giant multi-channel E-bow system for the Grand Piano. The resulting 'Magnetic Resonator Piano' (MRP) is capable of generating completely new sounds entirely acoustically — it's quite a freaky thing to hear and play!
As we all know, a conventional piano produces sound by striking the strings with key-activated hammers. However, once a note has been struck the pianist has no means of altering the sound before the key is released. The MRP was designed to overcome this limitation by allowing the player to continuously shape each note in terms of its dynamics, pitch and timbre, while still retaining the richness and nuance of the traditional acoustic grand piano.The experimental MRP system employs electromagnets suspended above each note's strings from a removable metal chassis that rests on the piano frame. Each electromagnet is controlled independently from a computer that senses which notes are played through optical sensors positioned at the back of the keyboard above the keys. By modulating the current fed to the appropriate electromagnets, sympathetic vibrations are established in the strings independently of the key hammers, creating infinite organ-like sustain, crescendos, or exciting individually selected harmonics. It can even be used to introduce pitch-bends and new timbres, but all controlled directly from the original piano keyboard by the pianist in real time.
A conventional piano produces sound by striking the strings with key-activated hammers. Once a note has been struck the pianist has no means of altering the sound before the key is released. The Magnetic Resonator Piano was designed to overcome this limitation by allowing the player to continuously shape each note in terms of its dynamics, pitch and timbre, while still retaining the richness and nuance of the traditional acoustic grand piano.In a radical change from a normal pianist's technique, if the keys are played so lightly that the hammers don't actually strike the strings, the electromagnets stimulate them instead, creating sustained, subtly evolving tones which have many of the acoustic qualities of the piano — including the sympathetic resonance between strings — but without the familiar percussion elements. Much more information on the MRP, as well as some impressive audio examples and videos, can be found here: www.eecs.qmul.ac.uk/~andrewm/mrp.html.