The disadvantage of being self-employed is that life gets in the way sometimes, and the past week had to be spent in ways that don’t advance the plot. The advantage is that I’m able to let this happen without having to answer to a boss. Some weeks are slow, and last week was one of them.
I’m quietly getting on with putting a lot of the modulation routines together — rather tedious parameter wrangling, 8-bit maths with a load of variables for signalling and storing intermediate states, and unit testing across two platforms. It gives me little to show until the whole thing is ready. At some point we’ll have another demo, but not today. The progress-o-gram is barely flashing at all:
Whenever the ‘slow week’ flag is set, I get to pop the stack and talk about one of the things I said I’d discuss in a previous post. At the top of this stack: working with the source code.
For about the first five months of this project, from late spring to autumn last year, I’d assumed OSCar’s source code to be lost: it’s over forty years old, Chris is no longer around to tell us where to find it, and we didn’t know where he kept it.
I found one lead, when I happened across a video by Dave Spiers of Geforce who, apparently, had received an archive of OSCar source from Chris as he was designing the first impOSCar back in the early 2000s. If you watch the video to the end, though, you’ll see why I didn’t expect him necessarily to be good-humoured, chatty, and forthcoming to a stranger about his work at the moment.
The other lead was Bob Grieb at Tauntek, to whom I was steered by the OSCar Synth Facebook group, but he gave me the impression that he’d only disassembled the parts of code that he’d needed to improve, and wasn’t going to give it to me for free. At that time, Paul Whittington’s mind was on more pressing matters, so any kind of commercial negotiation was not going to be worth the time I’d have to wait.
My first enquiries came to nothing, and I’m not the kind of person to make a nuisance of myself. We had schematics and, because somebody copied the M2 EPROM and put it online, the object code is readily available. It’s only 8 kilobytes of Z80, so how hard can it be to reverse engineer?
Not really hard, it turns out, but really, really time consuming (probably 500 hours). As an idea of the starting point, somebody has pushed an automatically-generated OSCar disassembly onto GitHub here, presumably as an exercise for the reader. See how easy it is to drop your cursor anywhere in this file and understand what’s going on at that point.
Ghidra
First, I loaded the object code into Ghidra, and let that disassemble and analyse it. Ghidra’s an open-source Java-based tool that the US National Security Agency designed for reverse-engineering malware. They added Z80 to its list of competencies for, well, some reason: old military tech still uses it, I suppose. Ghidra quietly does the best job that any free, dumb tool could possibly hope to do. It makes reasonable (but fallible) guesses about which parts of the binary are data and which parts are code, and lets you edit those decisions with a few keystrokes, rename labels, and add your own comments. Somebody at the NSA bothered to read the chip’s datasheet too, so it understands how the Z80 divides its addressable spaces into memory and I/O, and where you’re going to find the interrupt service routines.
Best of all, it builds a call tree, so you can see which code links to which other code, in both directions.
When you start work like this, it’s like a Sudoku puzzle: you attack it in multiple places until you hit an inflection point where the code starts joining up, and suddenly you can start making sense of big, important parts of it. It turns out that there are a few reliably good starting places.
Starting Place 1: the bottom of the call tree
The green text starting XREF[11] in the image above was added by Ghidra to inform us that this little routine is called from 11 places in the code. There aren’t any CALL or JR instructions before the RET statement either, and no RAM is referenced: this code just does its job in eight instructions and returns. This makes it a great starting place for understanding what’s going on, and it’s the first routine I decoded.
Ghidra will represent this as C for you, but that feature is really rotten. Every implicit decision it’s made in getting to this code obfuscates its purpose.
void FUN_ram_1bca(ushort param_1,byte param_2)
{
bool bVar1;
char cVar2;
cVar2 = '\b';
param_1 = param_1 & 0xff00;
do {
bVar1 = CARRY2(param_1,param_1);
param_1 = param_1 * 2;
if (bVar1) {
param_1 = param_1 + param_2;
}
cVar2 = cVar2 + -1;
} while (cVar2 != '\0');
return;
}
How would I write this as C? Um …
return E * H;It doesn’t emulate the side effects, but you only have to check in 11 places to see if those get used.
I abandoned Ghidra after a month or so. In some ways, its treating of the source code as a sacred relic prevents accidental corruption by the user, but it makes adding comments or changing the way data is represented a pain in the arse.
Instead, I exported my early work-in-progress as a couple of source files (ROM code and RAM data labels), and threw together a DOS batch file that would run them through a good free Python assembler I’d found online, before comparing its output with the EPROM binary I’d started with.
Since then, I’ve been cleaning up the source exclusively in Notepad++, where I created a syntax highlighting scheme for Z80, and enjoy my new-found freedom. As long the batch file is run from time to time to prove that the code still assembles and matches the EPROM, I can comment and organise away and be confident that I’m not changing the code that it generates.
Illuminating the symbols in RAM, quite a lot of which are bit fields reflecting the synth’s internal status, turns out to be most of the work. Aside from that, it’s adding comments and explanations, and giving the ROM routines sensible names to clarify the organisation of the code. The routines for the new synth are written in C and unit-tested against an emulation.
FUN_ram_1bca is, as I have already revealed, the multiplier. The Z80 microprocessor pre-dates the inclusion of hardware multipliers in commodity silicon so, if you wanted to multiply two numbers together, you’d perform long multiplication in binary. Here’s my annotated Z80:
MULTIPLY:
; long-multiplication
; the product H*E is returned in HL.
; on exit, E is preserved; B and D are 0.
LD B,8
LD D,0
LD L,D
@loop:
; HL <<= 1
ADD HL,HL
JR NC,@no_carry
; HL += E
ADD HL,DE
@no_carry:
DJNZ @-loop
RETThat’s the same 8-instruction routine labelled and commented up. Compared with the single-instruction multiply that you’d find in every modern microcontroller, it runs at a glacial speed. It takes an average of 83 microseconds to multiply two 8-bit numbers. That’s about 5% of the throughput you’d need to write a digital volume control for a CD player. (Yes, I know you could optimise for that case, but you’d still be nowhere near.) OSCar will stop what it’s doing, disable interrupts, and have a little think whenever it’s asked to rebuild the custom waveforms: apart from anything else, it’s got to call this routine more than 6,000 times.
Starting Place 2: I/O routines
The other handy place to attack code is wherever there’s an IN or OUT instruction. These are used exclusively for accessing the hardware. By this stage, I’d spent several days with a good schematic, annotating it and making several pages of typed tables so that I knew what was dangling off the peripheral bus. The IN and OUT port numbers then gave away what the processor was busy discovering. After finding the multiplication routines (there are three: two others add some code to handle offset binary numbers), the very next thing I found was the button-reading routine, which begins 136 bytes into the memory. This sets up the hardware to present the buttons to the peripheral chip, reads them over two turns, reorders a few to make them easier to handle in later routines, and places them in RAM. Anything that reads these locations in memory is therefore looking for held buttons. By comparing the buttons they’re checking against the user manual, I obtained a picture of what those routines were handling.
The OSCar user manual became so indispensable that I spent a day OCRing it and made a searchable version along with a few extra tables that condensed its contents. We have fragments of the original document on our backups, but there are two scanned versions of the OSCar manual online, and we have a hard copy of our own. I cribbed from all of these. One scan, from an earlier firmware version, contains information about unimplemented or discarded features from the arpeggiator. I kept those details.
Starting Place 3: MIDI handlers
Chris did me a massive favour by putting MIDI compatibility into the synth. Once the IN and OUT instructions are found where it’s reading and writing to the UART, I could see it check for note messages, convert the pitch data into the bizarre internal note format that OSCar has (where the notes descend in pitch as the number increases), and set a couple of ‘notice me later’ flags. Again, this revealed the location of the voice management routines. The handlers for program change and System Exclusive messages revealed where the patch, waveform, and sequencer memory was located, and confirmed the location of the active program data. This then opened up even more of the synth.
Bad starting places
I covered this in my talk a little, but that isn’t published yet. You’d think that the main loop or interrupt service routines, with their parades of CALL instructions, would be good places to start. Or location 0 in memory. After all, that’s where the processor begins its task.
It turns out that these are quite dispiriting starting places. All the areas of memory that they run, test, and initialise, are unmapped and indistinguishable when you start out. Here, for example, is the main loop at the start of the journey:
@mainloop:
CALL FUN_ram_020e
CALL FUN_ram_09ac
CALL FUN_ram_1870
CALL FUN_ram_0dae
CALL FUN_ram_0e30
CALL FUN_ram_0aed
CALL FUN_ram_0a35
CALL FUN_ram_13c4
CALL FUN_ram_146d
CALL FUN_ram_04e7
CALL FUN_ram_1edb
CALL FUN_ram_087d
JR @-mainloopThe breadth of functionality that you don’t yet understand taunts you: an ever-present testimony to the incompleteness of the work.
Now this feels like cheating
In around September, my labour on the disassembly slowed down: I’d recovered about 60% of the source code with what turned out to be a good, if imperfect, degree of accuracy. (I guessed the purposes of a few bytes of RAM incorrectly when I got bored. This isn’t my first digital synth, and some routines were really boring.) I tried using Claude to help at one point, but Claude is rather more enthusiastic than competent at this class of task, and confidently labelled up routines in ways that were wrong most of the time. I’d deciphered almost everything except the arpeggiator, sequencer, and some of the deeper parts of the synth like modulation, voice management, and envelope generation. I knew exactly where to find them by now, but they were still unlabelled and unexplained.
Then Paul remembered, amid the flotsam he’d inherited from the Oxford Synthesiser Company, a stack of 5.25″ backup disks, marked as formatted on either MS/DOS or, for earlier ones, a couple of flavours of CP/M.
First I lent these to a very trusted friend who volunteers at the National Museum of Computing, who had a stack of hardware he could work with. He didn’t get that far with them. He’d noticed that some of the disks didn’t read on both sides, concluded it was a fault with his drive, and went away to ponder the situation in his own time.
Not wanting to be outdone I’d decided, by this stage, that my time was worth the £120 or so it would cost to buy an elderly but tested 5.25″ drive from eBay along with a Greaseweazle PCB. This acts as a specialist drive controller, and works with some command-line software to allow people to scan old disks at a very detailed level, make rhythmic whirring and clunking and buzzing noises that transport you back to a primary school computer room in late 1988, before, abruptly …
I converted these flux-level scrapes to text-editor readable disk images. All the MS/DOS disks and some of the CP/M disks played nicely, but CP/M is actually a multitude of different formats, some of which were too obscure for modern-day filesystem emulators. They could, though, be loaded as one big lump of data into an editor, and inspected manually.
At this stage, I got my first glimpse of some of the final-issue source code for OSCar, version M2A.
This actually differs from the final shipped M2 version in two instructions, but the changes aren’t earth-shattering and probably not worth bothering with: they speed up the interrupt polling a little, and ensure that the control voltages get updated a little more often.
I ran through all the disks. Some contained old schematic drawings for the Advanced Sound Generator in a format that I could not read, reverse-engineer properly, or find documented anywhere. Not really important in light of the fact we could recreate these from the actual device if we ever wanted to, but it would have been nice to view them. Claude had a go at extracting their contents (because why not) and didn’t manage to glean much from them either. Again, confident and wrong in equal measure.
There were a lot of old WordStar documents too: personal letters from Chris, letters to suppliers and would-be sponsors in English and French, replete with trade prices; customised factsheets sent by post to distributors and customers; snatches of the user manual; general assembly BOMs, and so on. These documents were in one of those unreadable flavours of CP/M, and stored peculiarly, with the contents of different files interleaved. This suggests that, rather than backups, they were disks in daily use, containing work in progress. It reads rather like a stream of consciousness of the way we used to conduct business: everything, including phone calls, had to be summarised, resolved, or confirmed in letters sent through the post. The contents may one day be useful to somebody, but it’s more likely that they’ll remain forever curiosities.
As it happens, the value of these files was tested recently when PWM was contacted to ask if we could help a customer replace the keybed wireform on their original OSCar that was beginning to corrode. I found a purchase order for the OSCar wireforms dated 19th August 1983, sent to Hawnt Electronics:
ITEM QTY PART No DESCRIPTION # EACH # LOT
---- --- ------- ----------- ------ ------
1 100 CABLE ASSEMBLY 17 WAY 1.80 180.00
2 x 7720-17 ON 7307-17
10 INCH CABLE
SEE ENCLOSED SAMPLE
2 100 CABLE ASSEMBLY 5 WAY 0.61 61.00
1 x 7720-05 ON 8996-05
7 INCH CABLE.FREE END STRIPPED
SEE ENCLOSED SAMPLE
3 200 6410-17 0.1" STRAIGHT PIN 0.41083 82.17
HEADER TO MATCH 17 WAY
CABLE ASSEMBLY
4 100 6410-05 0.1" STRAIGHT PIN 0.15012 15.01
HEADER TO MATCH 5 WAY
CABLE ASSEMBLY
5 100 4455-12AC 12 WAY RT. ANGLE 0.32905 32.90
SOCKET
6 100 4455-11AC 11 WAY RT. ANGLE 0.30412 30.41
SOCKETIt didn’t take much work to discover that Hawnt was bought by Deltron in 2001, who were absorbed into Avnet in 2010. At some point, the wireforms became obsolete and the catalogue numbers meaningless, so this stuff helped only by revealing the fact that we couldn’t help.
Some disks could not be read well because the heads on Chris’s drive must have been misaligned at one point. These are the ones that appeared to be single-sided at the National Museum of Computing. The disk imaging software showed something there, however unreliably. I decided against de-calibrating my drive in order to try and recover the data, because I spent enough time playing with tape machines at university to realise the danger posed by the rarity of calibration media.
Of course I was doing all this with the drive uncovered, so I watched as one disk sloughed off its oxide all over the head when I tried to read it. As I was doing a flux-level reading I let it continue: I figured there was a fair chance that the damage was happening after the head. Fortunately, we were lucky: it was a disk of old documents and we read them.
Anyway, Chris’s source was present on a few disks as a series of .MAC (‘MAchine Code’) files. These, and much of their history, were recovered. The first surprise is that he’d logically separated the source code into different files, whereas I’d just been knocking up a massive lump of code. The second was that he’d been working with with rather austere limitations. None of his variable names are longer than six characters. None of his lines exceeds 48 characters. Pretty much every line of code is commented, but in a pretty ugly way. A friend of mine who lived through this era and has written a lot of commercial code said ‘It looks like an electronic engineer trying to program’. To be fair to Chris, that’s exactly what he was, and I even have his CV from 1983 to prove it. He’d improved considerably by the time we did Mininova and Mantis, but his house style continued to carry a lot of the old habits.
I suppose it should have been obvious from the outset that the editing tools available in 1983 would have limited Chris’s descriptive powers, as well as his ability to file comments in the obvious place. Commentary about the organisation of variables stored in RAM, for example, is usually in a related area of the ROM source. He was working at a time before there was an established feel for best practice in coding, and when juggling files larger than a few tens of kilobytes would have been very unpleasant. My more senior friend rebuffs these excuses of mine, but that friend had the privilege of working with especially talented programmers at the time. Now we have an Internet full of good examples, any number of books about the subject and, if you’ve ever worked in a coding team, you will have had house styles and label and comment discipline drummed into you.
Here’s Chris’s version of the multiply routine:
; MULTIPLY ROUTINE. H IS MULTIPLIER
; E IS MULTIPLICAND
; HL BECOMES PRODUCT
MULT: LD B,8 ;BITS
LD D,0 ;CLEAR D
LD L,D ;CLEAR L
MUL1: ADD HL,HL ;L.SHIFT
JR NC,MUL2 ;SKIP ADD
ADD HL,DE ;ADD TO INTERMEDIATE RESULT
MUL2: DJNZ MUL1 ;8 BITS
RETIt’s a representative example. No lowercase letters in sight. Every line has a comment, and not all of them add useful exposition. There’s no concept of ‘local’ branch names, so loops and skips just use the main call name with a number added. There’s a bit of white space for the eyes, but very few internal subdivisions outside the main routines. Comments are seldom afforded lines of their own.
But the alternative to working with this is nothing. Having worked intensively with nothing, this is much better. My own recovered source code follows better practices, benefits from a bigger screen and better editors; these days we also have source control. Clarity is improved and intention is clearer because it’s written by one person trying to reach into the mind of another. Aside from making it work with the pyz80 assembler, I’ve done nothing to my working copy of Chris’s source code: it’d feel like retouching a Da Vinci with a Sharpie. So I’m still working entirely with my own version. If I’m missing details, uncertain, or something seems off, I now have an original to check against. This is still hard enough. The routines that were left were the least tractable, so I’m working with a hint book more than an answer book — but it’s been invaluable in clearing up red herrings and answering open questions.
The final part of this tale is that Paul eventually found, very deep in some data we inherited from Chris’s old computer, a file called oscar.zip, date-stamped 15th May 2002. It contains a backup of the complete source code, along with an MS/DOS batch file to assemble it. Those files, containing the M2 firmware, are date-stamped 27th February 1986. This must be the archive that Chris sent to GeForce. While it would have saved me a lot of time — an awful lot — if I’d happened across this data earlier, the hardest route tends to have its compensations.
Bonus content
Backups of pre-MIDI versions of code are intriguingly different from the final OSCar firmware. Because Chris had to free up a lot of memory for the MIDI features, the internal sine table used to be twice as long as it eventually became. This enabled it to be appreciably less distorted than the M2 version. I now have a choice between two authenticities when I get that feature going. The earlier version is probably definitive, the later a compromise to save memory. However, absolutely everybody with an OSCar is now driving with the later firmware, so that probably wins the argument.
The other useful feature in earlier versions of OSCar is the factory presets, which I wasn’t looking forward to trying to scrape from a wrinkled cassette. They are stored in ROM, so appear in the source code files in their unexpurgated glory. With names!
; 2
; VOICE CELLO
DEFB 0E3H ; FILTER DRIVE
DEFB 07DH ; OSC. BALANCE
DEFB 001H ; NOISE BALANCE
DEFB 09CH ; FILTER FREQUENCY
DEFB 097H ; FILTER SEPARATION
DEFB 0FEH ; SUSTAIN LEVEL 1
DEFB 03AH ; BEND AMOUNT
DEFB 018H ; PITCH AMOUNT
DEFB 000H ; FILTER AMOUNT
DEFB 0C6H ; PULSE WIDTH
DEFB 043H ; Q
DEFB 0FEH ; SUSTAIN LEVEL 2
DEFB 091H ; DETUNE
DEFB 08BH ; DIRECT FILTER MOD
DEFB 069H ; ENV.AMOUNT TO FILTER
DEFB 092H ; DIRECT PITCHMOD.
DEFB 060H ; GATE TIME
DEFB 09CH ; ATTACK 1
DEFB 05EH ; RELEASE 1
DEFB 001H ; DECAY 1
DEFB 0A8H ; ATTACK 2
DEFB 069H ; RELEASE 2
DEFB 001H ; DECAY 2
DEFB 001H ; GLIDE RATE
DEFB 0C0H ; LFO RATE
DEFB 080H ; INTRO TIME
DEFB 043H ; GLIDE TYPE,WAVE1
DEFB 052H ; WAVE2,OCT SHIFT
DEFB 000H ; LFO TYPE,FUNCTION
DEFB 000H ; TRIG TYPE,FILTER TYPE
DEFB 077H ; KPITCH,KTUNE
DEFB 027H ; OCT 1,ARPSTI believe this file, and possibly our cassette, to contain he only extant record of the original factory presets. The demos and files online that presume to demonstrate them are not showcasing these.
No such thing as a free lunch, though: some of these parameters could do with tidying up a little. They’re going to be reordered and mostly reduced to 7 bits. That’s their meaningful resolution, so it won’t change the sound at all. As Paul Wiffen confided to me last year, they are indeed mostly the same initial patch with a few small tweaks. We pay good money for sound designers these days and our standards are higher: these are going to require combing and curation for our own version of OSCar. But they’re authentic. They’ll sound like Thatcher’s Britain, and they’re ours again.
Post Scriptum: To publish or not to publish
My instinct is to pay this kind of work forward. The entire software industry builds its work on the generosity of engineers unglamorously toiling and sharing modules they’ve made in their own time, out of either leisure or necessity, without expectation of payment. Chris’s code looks so old-fashioned today because generations of professional programmers have been generous enough to let us appreciate well-written source code.
What’s stopping me is Behringer. They’ve done me before, they’ve done Chris before with the WASP, and they’ve set their sights on OSCar. I’d be a fool to make it easy for them, and it’s why I’ve used a rather trivial example of our code above, when a longer routine would have better illustrated my opinions on style.
Trust is a commons like any other: it can be plundered for private gain, and that will impoverish the ecosystem. Unless we proceed with care, IP theft and strategic lawfare will bring us low prices, but will lead to an industry that produces mountains of instant landfill, a hollow parody of variety, and products with neither true novelty nor intrinsic value. Good products take time and accommodation for failure. A pressure to depress wages, timeline, and material costs at the expense of practitioners and customers is already underway in fashion and food, and I’m buggered if I help to normalise it in my own industry.
The downside is that I sometimes have to stop myself from sharing nice things when I want to, which makes me a different kind of conspirator.
So, here’s the deal: I’ll publish my version of the OSCar source at some point. It represents hundreds of hours of work from somebody who’s been doing this for a while and who actually worked closely with Chris. I think it compares well with the quality of the code it describes, but it’s of absolutely no use to anyone rotting on my hard drive. Publishing it, even under a restrictive licence, would help other people to fix the bugs in the Z80 OSCar, replace the cassette routines with (perhaps) something more useful, and so on. Provided, of course, that anybody is actually willing to do so. Working in Z80 still isn’t much fun: my significant contribution to the legacy of OSCar will [I hope] be to drop a new processor into it and turn it into a modern instrument without changing its sound or damaging its essence.
But any kind of source code release will have to wait until our own synth has been out for a while. Watch this space a little longer.
Leave a Reply