Category: Audio and Software

I did an hour about my research, and it’s on video. If you’re looking for the five-minute audio-only thrill, it’s in the post below.

 

The topic I jettisoned from my talk about MPE at last week’s conference was my thoughts about what makes a good Working Group, as I didn’t want to imply any criticism of the MIDI Manufacturers Association or its members.

As Chair of the MPE Working Group, I neither presume that I can pilot harder projects through technical and diplomatic adversity, nor will I tempt fate by attributing much talent to what might just be beginner’s luck. All I know is that, over the years, I have seen a couple of things.

Some time in the past, I sounded off about what is wrong with MIDI, and every now and then I still do, but always with a veneration for a standard that has managed to age so gracefully while seeing off so many rivals.

The HD Working Group has been meeting at least weekly for more than a decade now to define a successor to MIDI, and we’ve all been waiting, reading, and preparing. It’s not been an easy journey, and HD has recently stumbled at one of the last hurdles. The MIDI Manufacturers Association functions as a democracy of individual companies, and getting a new idea ratified means convincing important stakeholders that they will benefit from your changes and not be threatened by them.

This has always worked for small changes. But a wholesale replacement for MIDI must be non-disruptive, and a non-disruptive replacement is a paradox. After ten years of development, the HD spec is more complex and intimidating than MIDI, there’s no straightforward compatibility between the two, and it’s too new even to be clear how it’ll be used. MIDI has survived nearly 35 years being meddled with and patched up — MPE is just the latest way of doing this.

The HD spec would fix a lot of our problems automatically and for good, but it’s in a tricky place. Every passing year, the tweaks to the old spec gain some ground on the new, and the case for the new diminishes.

Either in its current or a modified form, I want HD to thrive. It seems that there are a few options open to the Working Group to keep it alive, all of which might work, in varying shades of palatability. Some of these are happening, and some aren’t:

• Keep working on the market model, convincing the company representatives who intend to vote ‘no’ that it’s in their interests to vote ‘yes’.
• Allow companies to start commercialising the draft specification without permission, on the basis that the first few compatible products will create a small ecosystem, prove a market for HD devices, and establish a commercial case without frightening incumbents.
• Encourage software framework developers to add implementations of HD to make it easier for third parties to assimilate.
• Approach a different standards body, such as the USB Implementers Forum or the ISO, to ratify the specification as a standard that is independent of MIDI, and bring it back to the MMA as a fait accompli. This might be less hard, but risks destroying the MMA by driving it into irrelevance (or making it look like this is the intention). Doing so could alienate much of the industry.
• Go back to the drawing board, and work out a roadmap where MIDI 1.0 can be turned into MIDI 2.0 by backwards-compatible steps, each of which will require a little work in exchange for some exploitable advantages. For example, MIDI 1.1 could mandate full-speed USB, assume bidirectionality by default, revise its note model to support MPE, respond to the System Exclusive Device Inquiry message, and abandon Polyphonic Aftertouch (the messages for which might then be reappropriated at some point in the future). All of these have clear commercial value, ease communication with computers, and pave the way for a ten-year plan of improvement. The new spec gets chucked away, but not the underlying vision, which is actually the important part. Who, after all, knows what the market will need by the time MIDI 2.0 sees the light of day?

The last one would, of course, be terrible. HD isn’t at this stage and I hope it doesn’t reach it. For what it’s worth, though, here is my conclusion from what little I’ve seen of Working Groups.

Creating a new specification is just like software design …

1. Have a short attention span, because you never know which way the industry will turn tomorrow.
2. Surgically fulfil specific, identifiable needs.
3. Always have an up-to-date implementation so that the spec makes sense.
4. Engineers have enough to learn. Resist the temptation to change established practices. Even if it hurts a bit, make the old spec fit new requirements.
5. Put something out within a year, or two things will happen: the world will forget you, and the sunk cost fallacy will sink its teeth in. In other words, hardly anyone will volunteer to junk more than a year’s work even when it’s the right thing to do.
6. Capture neat ideas in a roadmap, and leave space to put them in, but promise as little as possible. Non-essential features should be marshalled into a to-do list for a future spec. They may never be needed in practice, so don’t burden early adopters with them.

… Except for the bits that aren’t.

1. Keep the door open. Never work in secret or limit discussion. These days, being courteous and responsive to keyboard warriors is part of your marketing campaign. However —
2. Don’t let your guests take over. Good specifications need clear ownership, so limit scope and limit control.
3. In this industry, don’t sell your spec for money, as it will need all the help it can get. People have grown very accustomed to getting stuff for free.
4. Make it really easy. Great documentation isn’t enough these days: people expect TED talks, code libraries, unit tests, and preferably source code for a working demo.
5. Persistent and contrary people will drive you crazy along the way. It may not look like it, but many of the people that work you the hardest will turn out to be your greatest friends and advocates. Like you, they want this specification to meet their needs and succeed in the wider world, which is why they push back so hard. Do all you can to understand them, be unfailingly polite and positive, and go out of your way to accommodate them if that’s what it takes to keep them on-side.

As you’re possibly reading this because you’re bored of the Yuletide festivities and the wall-to-wall children’s TV, I’m taking advantage of your compromised boredom threshold, and tackling a subject that will take me about twice as many words as usual.

It’s Christmas, and Seaboard hasn’t shipped yet. Somewhat happily, the reason behind this isn’t our fault: a supplier made a catastrophic error of judgement that has cost us a month, and these things happen. Fortunately, it won’t kill us. In fact, it gives us time to do a bit more testing. But the road to delivery is seldom smooth, and the more you’re innovating, the more violently it undulates.

Experience improves foresight. What ROLI has ended up doing is what, upon reflection, we might have done all along. The Seaboard is currently a brilliant controller with a small ecosystem of third-party software synths that play really nicely with it, and that’s what our first customers will get. What we’ve had to postpone is our own synthesiser. Seaboard is supposed to make its own sounds: just switch it on, plug in headphones, and perform. Now we’ll be providing that later, as a free upgrade.

Our synthesiser runs (users: ‘will run’) on its own microprocessor inside Seaboard. There are another couple of chips that take care of getting the playing surface and Sound Dial to work, so making sounds is pretty much all that this microprocessor has to do.

Now, remember that we’re a startup, and what startups do best is schmoozing like mad and building a community, seeding goodwill, and making people stop in their tracks to smile and gaze in wonder at the smart execution of a brilliant idea. What we can’t do is pretend we’re Microsoft (or even Native Instruments). We’re terrible at working on several fronts at once, and even our talented brigade of red-eyed, caffeinated programmers cannot successfully produce a minimum viable product that is neither minimal nor technically viable.

This article isn’t going to be a reflection on what’s pushed us into this manoeuvre so late. That will come when things aren’t so sensitive. What it will describe is one of the larger and more interesting bumps on our road to delivery. ROLI used a lot of open source software in order to speed up the design of the synthesiser. This got us started quickly, enabling non-specialists to make huge contributions, but it turned out to be a mixed blessing in the long run. Perhaps using free software didn’t save us time or money after all.

Open source software is software that is written by somebody you didn’t pay, that is distributed for free under fairly generous conditions. There are many good reasons why people write open source software. Books have been written on this subject, so it’s beyond the scope of this article.

The two main lumps of open source software we used are Linux and JUCE. Linux is the world’s largest open source project: a complete operating system into which volunteers have poured cumulative lifetimes of work so that it can do pretty much anything. To many coders, it is a work of unsurpassable beauty. Its progenitor, Linus Torvalds, has not made a penny from selling it, but does very nicely out of lecture tours.

JUCE is a series of utilities that provide graphics, audio, and control capabilities in such a way that a program written in C++ can simultaneously run on Linux, Mac OS, or Microsoft Windows. Jules Storer put ten years of work into this and it’s remarkable. He makes money from it when people commercialise projects that contain JUCE.

Let’s consider JUCE first, as it’s simpler. Using JUCE, our synthesiser can run in a window on Mac OS, on Linux, or on Microsoft Windows, and also inside Seaboard. Four separate products are therefore one lump of code: we fix a bug or add a feature on one, and it’s fixed everywhere. Here’s the catch, though: managing a cross-platform project (as it’s called) is easier than managing four separate ones but, even with JUCE, a great deal harder than managing code for one platform. There are a lot of reasons for this:

1. It remains possible to write code that is fine on one system but stops with errors on another. In practice, every time the code changes, every platform has to be retested.
2. The chip we’re compiling onto inside the Seaboard is very, very different from the Intel Core chips inside Macs and PCs. It’s also our most important target. Every time we change code, the embedded build (the code that runs on Seaboard) requires special attention.
3. There are no graphics on Seaboard, so we’ve had to do some work to make sure that no vestige of the graphical stuff finds its way there.
4. Every time our files are changed or reorganised, each computer’s compiler instructions need to change too. JUCE provides a way of automating this, but we still need to hack it manually.

The second reason has given us the greatest headache. We’ve had to rewrite large tranches of the synthesiser to get it to perform acceptably as a standalone musical instrument, and have needed to have some external expertise to help us do this. We’ve also needed to automate builds for Seaboard. Our engineers work and audition their code on Macs. To allow them to keep working, an external computer automatically takes the code they’ve written, builds it for Seaboard, runs some elementary tests (does it crash?) and checks that these tests pass. The programmer can check that it still compiles, and download the latest build to test on a Seaboard once they’re ready. Time taken to get this build system working: about a week. Open-source software helped us to get it going, or it might have taken a month.

Jules has kindly made a no-graphics version of JUCE, and we’ve helped him in turn by putting considerable work into optimising JUCE for our little Cortex-A8 chip. This may seem like a small detail, but there were a lot of people waiting for it: Seaboard and Raspberry Pi share the Cortex-A8 as a microprocessor, so we’ve also optimised for the world’s best-selling project board. ROLI also opened the books on the special control protocol that Seaboard uses to communicate with its own synthesiser. Why MIDI is no good for this is a posting in itself.

That’s JUCE; now Linux. Linux contains millions of lines of code, comprising things such as the Unix command line and ‘make’ that are their own little environments; the bootloader that starts Linux requires learning yet another language. The kernel has its own tools to build correctly, once the compiler has been set up properly: it’s compilers within compilers within compilers.

Linux has been under development for over twenty years. That’s an unusually long journey in software terms, and consequently it takes hours and hours to gain proficiency in just using it, let alone extending it. Very few people are experts in many parts of Linux, but the advantages of using it are that it’s been widely tested, has ways of adding any feature you can imagine, and is comparatively very quick to get started. These are very big advantages. Writing an OS from scratch is hard, and the whole point of Linux is that nobody should ever have to again. The problems with Linux, though, are the inconsistency of documentation and the lack of an easy way in. A build of Linux that works on our little chip isn’t the same as the thing that runs on a PC. To get it working, even with people who knew what they were doing, took days. When we wanted to make the chip work with raw data inside the Seaboard, we had to write a driver that required knowledge of the innermost workings of Linux and the chip itself: a very arcane combination of skills that took a while to find.

So, for the first time in ages, I’m attempting to collaborate on something at which I am a novice, and have little chance of substantially improving my knowledge on my own. I’ve had to accept my ignorance and trust my team: those who built this structure know, and will always know, more than I do. Meanwhile, we’ve paid host to a horde of consultants specialising in:

• Setting up the right flavour of Linux with the right features;
• Getting the Linux kernel compiling on our embedded system;
• Getting JUCE compiling on our embedded system;
• Writing drivers to massage our raw incoming data and outgoing audio;
• Making the USB output work reliably;
• Optimising the C++ compiler, and then the synthesiser code, for the Cortex-A8.

This has all been in addition to Jules’s work for us. Most of the rest we’ve done ourselves. So, a considerable team of consultants has been hired, for a few days at a time. Many of these clever and expensive people have been needed just to help us with ostensibly ‘free’ software, and many were delayed or floored by having to grasp the complexity of our cross-platform code base.

What do we have to show for it? We have a reasonably good software synthesiser that runs on four platforms. However, it’s questionable whether this has contributed any more to the appeal of Seaboard than would writing two separate and far smaller programs: a dedicated synthesiser for the Cortex-A8 only, and a MIDI remote control in JUCE that would tell it what to do. There may yet be long-term strategic value in our cross-platform endeavour, but we cannot realise it until after we’ve got the work to an acceptable quality, and shipped it inside a Seaboard.

Linux running on Seaboard makes certain things (such as debugging and managing the on-board sample library) easier for our engineers. Currently, though, it provides no advantage to the user, and actually means that Seaboard takes longer to start up than it otherwise would.

In fact, we might have avoided the cross-platform chimera altogether by not running an operating system at all. Most stand-alone synthesisers don’t run one: the only benefits they bring are better debugging and a decent file system. But in paring down, we would have sacrificed or delayed the Mac and PC builds of our synthesiser, and made it harder for our programmers to test and iterate their work. This might have created other problems.

Lessons so far, only some of which I already knew:

1. The best software is no software. The fewer things your first product does, the better it will do them.
2. Remember Hofstadter’s Law: ‘It always takes longer than you think, even when you take into account Hofstadter’s Law.’ Engineers are optimists: they must be or they couldn’t turn up to work. There are always distractions, extra tools to build, bottlenecks and impasses in a project that cannot be fully anticipated. If a project is fundamentally innovative, multiply engineers’ resourcing estimates by about 4; otherwise, it depends on the engineer, but the average is about 2.
3. Basing your technology on free software is not a panacaea, but a technically subtle decision. It can end up costing more, or causing people to overpromise. The ramifications of a decision either way are potentially ruinous. Research licensing costs. Get the most experienced practitioner you can afford to inform the final decision based on expected implementation time and technical difficulty. Ignore all commercial pressures and trust their advice, or there’s no point in paying for it.
4. Use Agile, because it is designed to get the simplest software out as quickly as possible.
5. Employ a product manager to prevent stakeholders, be they testers, product specialists, or the CEO, from adding features once work has started and a milestone has been agreed.