6502 MIDI file player

[lordbubsy]

Wow, I didn’t see that it can be simplified even further to 1152/625.

Quote:

To implement the multiplications, observe that 1152 = $480 which has only two bits set (2^10 + 2^7), and 625 is $271 which means it can be written as (2^9 + 2^7 - 2^4 + 2^0). This is a big advantage of multiplying by constants

Thanks for showing me this “simplification”, I hope I can implement this into my program, but I’m still trying to figure out the math basics, which I didn’t really need until now. I’ll certainly need some time to wrap my head around it.

[lordbubsy]

Happy and healthy 2019 to all of you!

I finally created a universal 24x16 bit multiplication and a 40/40 bit division routine. They are not optimized for speed or size, but they are as readable as possible.

mul 24 by 16 bit no zp.asm

(1.82 KiB) Downloaded 79 times

div 40 by 40 bit no zp.asm

(2.29 KiB) Downloaded 74 times

I also tackled your (Chromatix’s) routine. I’m not entirely sure what it is with the “overflow” and “no_overflow” though. For the rest its about 99% clear to me now.

[dmsc]

Hi!

lordbubsy wrote:

Wow, I didn’t see that it can be simplified even further to 1152/625.

Quote:

To implement the multiplications, observe that 1152 = $480 which has only two bits set (2^10 + 2^7), and 625 is $271 which means it can be written as (2^9 + 2^7 - 2^4 + 2^0). This is a big advantage of multiplying by constants

Thanks for showing me this “simplification”, I hope I can implement this into my program, but I’m still trying to figure out the math basics, which I didn’t really need until now. I’ll certainly need some time to wrap my head around it.

Exact math almost never is needed, think that your real life clock is not as precise as your number.

Given that, you can always look for a simplification: 1152/625 ~= 59/32 , with an error of less than 0.03%. Now, your problem is reduced to multiply by 59, then divide by 32 (a simple shift-right five times, or shift-left three times and remove one byte).

And for more complex calculations, I would recommend using an FP package, the woz package is only 256 bytes and would give you code like (for multiplication by 1.8432):

Code: Select all

  ; Load 16 bit number to multiply, convert to floating point
  lda NUM1
  ldx NUM1+1
  stx M1
  sta M1+1
  jsr FLOAT 
  ; Load 1152/625 = 0x8075F6FD
  ldx #3
loop
  lda FACTOR,x
  sta X2,x
  dex
  bpl loop
  ; Multiply
  jsr FMUL
  ; Convert result into integer again
  jsr FIX
  ; Your number is in M1 & M1+1
  lda M1+1   ; Low part
  ldx M1       ; High part
 ........

FACTOR: !byte $80, $75, $F6, $FD

[lordbubsy]

Quote:

I would recommend using an FP package, the woz package is only 256

Thanks! I'll most likely need that in the future.

[lordbubsy]

I think my MIDI player works pretty well, all be it only for MIDI type 0 files. But that’s ok, because I can convert virtually any MIDI file to type 0. The only problem I have is when I play a MIDI file with a lot of information (delta time 0 and or running status), I get audible slowdowns.

To clarify:
My MIDI player operates in an interrupt routine. Each interrupt equals one MIDI tick. The time between interrupt is in the order of 5000us (derived from microseconds per quarter note and ticks per quarter note). Delta time is the time in MIDI ticks between MIDI events send to the MIDI device. Running status is a way to reduce the amount of bytes of the MIDI data. It reduces the data by omitting repeated MIDI commands by only sending the data.

Quote:

The MIDI spec allows for a MIDI message to be sent without its Status byte (ie, just its data bytes are sent) as long as the previous, transmitted message had the same Status. This is referred to as running status. Running status is simply a clever scheme to maximize the efficiency of MIDI transmission (by removing extraneous Status bytes). The basic philosophy of running status is that a device must always remember the last Status byte that it received (except for RealTime), and if it doesn't receive a Status byte when expected (on subsequent messages), it should assume that it's dealing with a running status situation.

http://midi.teragonaudio.com/tech/midispec/run.htm

I think that the MIDI file player probably stalls (audibly) in the following two situations:

1.
When there are multiple (a dozen) MIDI events with delta time 0. i.e. all those events have to be handled in one interrupt sequence.

2.
When there are multiple equal MIDI events in a row. They may or may not have a delta time of 0. i.e. running status.

In case 2, I’m not sure if those events with delta time > 0 will slow down playing, but probably only all events with delta time 0.


What I suspect is that each MIDI event with delta time 0 will be send in the next interrupt, i.e. one MIDI tick later. However, as far as I programmed the interrupt routine, that shouldn’t happen. If the events take more time than one or two interrupt sequences, it shouldn’t be noticeable. But it might be noticeable when each event would be send one interrupt later.

Here is the pseudo code of my interrupt routine:

Code: Select all

tc = tick counter
binvlq = delta time in binary
GetVarLength = get delta time
DecodeEvent = decode event and send it to the midi device

Initialisation:
tick counter = 0
GetVarLength
BeginIrq:
        if delta time == tick counter
-               DecodeEvent
                GetVarLength
                        if delta time == 0 goto -
                        if delta time <> 0 then tick counter = 0, goto EndIrq

        if delta time <> tick counter then tick counter = tick counter + 1, goto EndIrq

EndIrq:

I do not think that the CPU is too slow for playing those MIDI files, because the slowdowns are equal whether I run at 1MHz or 8MHz.

I’m at a loss here and I really don’t know what to try next.


Interrupt code:

Code: Select all

interrupt
                pha
                phx
                phy

                lda     isr
                and     #%00001000              ;check if interrupt is caused by C/T
                beq     endirq                  ;no? exit irq
                lda     rop12                   ;clear counter ready interrupt status bit

                lda     tc                      ;is binvlq = tc?
                cmp     binvlq
                bne     IncTc                   ;no, increment tc and end irq
                lda     tc+1
                cmp     binvlq+1
                bne     IncTc
                lda     tc+2
                cmp     binvlq+2
                bne     IncTc
                lda     tc+3
                cmp     binvlq+3
                bne     IncTc

ExEv            jsr     DecodeEvent             ;yes, decode a track event (midi/sysex/meta)
                jsr     GetVarLength            ;get variable length quantity

                lda     binvlq                  ;is binvlq zero?
                bne     ResetTickCounter        ;no, reset tc and end irq
                lda     binvlq+1
                bne     ResetTickCounter
                lda     binvlq+2
                bne     ResetTickCounter
                lda     binvlq+3
                bne     ResetTickCounter
                bra     ExEv

IncTc
                inc     tc                      ;increment tick counter tc
                bne     +
                inc     tc+1
+               bne     +
                inc     tc+2
+               bne     +
                inc     tc+3
+
                bra     EndIrq

ResetTickCounter
                stz     tc                      ;Reset Tick Counter tc  and end irq
                stz     tc+1
                stz     tc+2
                stz     tc+3

EndIrq
                ply
                plx
                pla
                rti

Decode Event code:

Code: Select all

DecodeEvent
                lda     (mfp)                   ;load byte from midi data (msb first)

                cmp     #$80                    ;if less than 128,
                bcc     +                       ;then running status
                bra     ++
+               lda     runstat                 ;load previous midi event
                jsr     midiout                 ;send it out
                and     #%11110000
                cmp     #$c0
                beq     de0
                cmp     #$d0
                beq     de0
                lda     (mfp)                   ;load data 1
                jsr     midiout
                jsr     IncreaseMidiFilePointer
de0             lda     (mfp)                   ;load data 2
                jsr     midiout
                jsr     IncreaseMidiFilePointer
                rts
++
                and     #$f0
                lsr
                lsr
                lsr     ; yes only three, because each jump address is 2 bytes
                tax
                lda     (mfp)
                jmp     (decodetable,x)         ;jump to separate commands

[Chromatix]

I think I know what's going on here. You say the behaviour doesn't depend on CPU clock speed, but there's another clock that you're not varying - the baud rate of the MIDI interface. Granted, that is set by the standard, but I think that's what you're running into.

How many MIDI messages can be sent during a single tick? Remember, it takes 10 baud cycles to send one byte in 8N1 format. I bet the symptoms start to appear exactly when the number of delta-0 messages exceeds twice that number - modulo the size of the UART's send FIFO.

What happens when the send FIFO is full? Your "midiout" routine probably busy-waits on the ready-flag of the UART. But interrupts are disabled meanwhile, and if a timer interrupt fires, it will be ignored until the entire ISR completes. If a *second* timer interrupt fires, the first one is lost forever, and your playback runs slow.

The solution is to put a bigger, software-managed buffer in front of your UART; 256 bytes is easy to arrange on a 6502 and probably big enough. Your "midiout" routine should add data to the tail of that buffer instead of stuffing it directly into the UART. Then, at the end of your timer ISR, make sure the UART is filled with as much data from the front of that buffer as it will hold. If the buffer is not empty at the end of that, you'll need to enable the UART's "send FIFO is empty" interrupt and service that; then, when the software buffer *does* empty, disable the UART's interrupt again.

A wrinkle is that now you need to distinguish between the FIFO and timer interrupts, and the 6502 doesn't give you any help in this since it only has one (conventional) interrupt line. Servicing the UART when it isn't actually empty isn't a big deal, but you don't want to increment the tick counter if the timer hasn't actually expired. So look for a timer register which unambiguously indicates whether its interrupt has fired, and skip over the MIDI tick handler to the UART handler if it hasn't.

[lordbubsy]

Quote:

I think I know what's going on here. You say the behavior doesn't depend on CPU clock speed, but there's another clock that you're not varying - the baud rate of the MIDI interface. Granted, that is set by the standard, but I think that's what you're running into.

I’m sure you might be right about that, but why play those MIDI files correctly on a PC, is that because they have proper IRQ based UART routines? I guess that’s the case.

Quote:

How many MIDI messages can be sent during a single tick? Remember, it takes 10 baud cycles to send one byte in 8N1 format.

Indeed the baud rate is 31.250Hz, which is 32us. Each event takes at most 3 bytes, which results into 30 baud cycles, which is 960us. One MIDI tick takes about 5000us, so 5 events could be send at once in an interrupt cycle. Probably less, because I’ve not taken into account the execution time of the rest of the code.

Quote:

Your "midiout" routine probably busy-waits on the ready-flag of the UART.

My UART routine is indeed pretty simple, I’m not using interrupts for sending the bytes. I’m also not using a hardware FIFO (I think) of the UART.

Quote:

If a *second* timer interrupt fires, the first one is lost forever, and your playback runs slow.

Would that be audibly noticeable? However, I’m guessing that I’m losing more than 2 interrupt cycles, and that would most likely be audible.

Quote:

The solution is to put a bigger, software-managed buffer in front of your UART; 256 bytes is easy to arrange on a 6502 and probably big enough. Your "midiout" routine should add data to the tail of that buffer instead of stuffing it directly into the UART. Then, at the end of your timer ISR, make sure the UART is filled with as much data from the front of that buffer as it will hold. If the buffer is not empty at the end of that, you'll need to enable the UART's "send FIFO is empty" interrupt and service that; then, when the software buffer *does* empty, disable the UART's interrupt again.

OK, that’s what I’m going to try. I’ll try to implement Garth’s “3.1 INTERRUPT SUPPORT FOR RS-232 RECEIVE”
http://www.6502.org/tutorials/interrupts.html#3.1
Of course adapting it to sending the MIDI data. It might take a while.

[Chromatix]

I believe your 28x9x-series UART has a 16-byte hardware FIFO for each send and receive channel. This would effectively absorb variations in your processing time, because it can hold a full 5 MIDI messages without waiting for them to be sent over the wire first. If you have one of the versions with 8-byte FIFOs, then it'll hold 3 messages (first byte goes to the TX shifter immediately and doesn't count against the limit) before you start busy-waiting.

So you could probably handle about 15 MIDI events in one tick before a subsequent timer interrupt would be lost, but you could only handle at most 5-6 events in the immediately following tick before the timer interrupt after that was lost. Every unused event in that quota recharges your peak 15-event handling capacity, as your ISR first catches up with real time and then your UART's FIFO drains. Conversely, if *every* tick has more than 5 events in it, you'll gradually fall behind and regularly lose some proportion of timer interrupts (but if your MIDI file is so dense that you can't physically send it over the channel fast enough, there's nothing you can do about it).

If you implement a circular buffer with 255-byte usable capacity, you'll be able to store exactly 83 events in it and a further 5 in the hardware FIFO, without even *beginning* to run behind the timer. Since it'll take about 16 ticks to completely drain that buffer, it should be enough. I advise aborting playback and displaying an error if you overflow the buffer, so that you know if you need to implement even more heroics - such as the improved timer below.

Quote:

…why play those MIDI files correctly on a PC, is that because they have proper IRQ based UART routines?

If it's using a softsynth or sending the data directly to the soundcard, then it's not having to stuff the data through a narrow channel along the way. This eliminates that particular timing pitfall entirely. But if it is sending over a physical MIDI cable, then it'll be implementing a decent send buffer as described above.

Mostly, PC-based software is only indirectly IRQ-driven, unless it lives in the kernel (as a UART driver would). Timer and other interrupts can cause userspace processes to be woken up. But that's a completely different level of software complexity which you don't need to think about on a 6502.

It'll also probably be using a more sophisticated timer routine which can recover from falling behind. This is partly because the timer-hardware driver is itself kernel software and merely provides an operating system service. We could implement a better timer too, just for the exercise - you'll still want the interrupt-serviced send buffer, otherwise you'll eat a lot of CPU time busy-waiting every time you get a "dense tick".

You'll need two status bytes in RAM for this. One becomes a mutex, preventing you from re-entering the MIDI ISR while it's still working on a previous tick's data. The other allows you to count how many timer interrupts have occurred without being serviced (yet) by the MIDI ISR. Then, just before entering the MIDI ISR proper, you set the mutex and clear the interrupt mask, then clear the mutex just before you leave the ISR. You also need to decrement and test the timer counter, and loop back to service the next tick if it's already become due. (If it's an empty tick, you might only need to decrement the delta counter, but you won't know that until you've finished processing the first tick.)

This is now in the realm of "concurrent programming", so the order in which you do things matters, and it also matters whether you use an atomic operation or temporarily set the interrupt mask flag when checking or updating a shared variable. But because you do the heavy lifting with interrupts unmasked, timer interrupts won't get ignored for so long that they get lost, and you can let the UART servicing routine drain the buffer while you're still filling it, and just busy-wait if the software buffer becomes full (which should be rare, perhaps only at the very beginning of a file with a lot of setup work).

[GARTHWILSON]

Yes, it sounds like a MIDI logjam. I hope I'm not way off here because I haven't been following this closely. Missing an interrupt here and there won't always be audible; but if you're syncing with something else, it'll gradually become noticeably behind. What I might do instead is a multitasking system (with no multitasking OS) like I tell about at http://wilsonminesco.com/multitask/#cycexec . The interrupt is simpler and the clock updates are never delayed because you don't spend much time in the ISR. Every time the task runs, it looks to see if you've arrived at the time for the next thing it's supposed to do. (This is #3 in the list at about the middle of the page.) It just takes the current time minus the next event's scheduled time. If the result is negative, you have not arrived yet, so you just exit the task without doing anything. But if the result of the subtraction is not negative (ie, it's either zero or positive), you have reached the time, and it's time to carry out the event. The time that you schedule for the next event is the time of the current event (not necessarily the current time) plus the appropriate offset. That offset could be anywhere from zero to some long delay between events. You could get so many microseconds behind, but the error will not accumulate, and heavy loads will get caught up at the rate allowed by MIDI (31.25kbps) and MIDI logjams don't have to be the end. In the multitasking, you can be doing several things at once, yet the timing will be correct, as least insofar as human perception allows in this case.

Hopefully that's not just as clear as mud.

There are several things in the code that could be shortened, for example

Code: Select all

                lda     binvlq                  ;is binvlq zero?
                bne     ResetTickCounter        ;no, reset tc and end irq
                lda     binvlq+1
                bne     ResetTickCounter
                lda     binvlq+2
                bne     ResetTickCounter
                lda     binvlq+3
                bne     ResetTickCounter

could be shortened to

Code: Select all

        LDA  binvlq
        ORA  binvlq+1
        ORA  binvlq+2
        ORA  binvlq+3
        BNE  ResetTickCounter

[lordbubsy]

@Chromatix
I’ve got the version SC26C92 with 8-byte FIFO in use, but I also got one with 16-byte FIFO. So those buffers are used automatically? I could do two quick checks, one is to replace the SC26C92 with the SC28L92. That should at least be noticeable. Secondly I could indeed check if the buffer overruns by your method described. However, I’m going for the following method:

Code: Select all

BeginIrq:

is interrupt caused by midi timer
        yes, goto HandleTimer
is interrupt caused by TxEMT? (uart transmit buffer is emty)
        yes, GOTO HandleUart
        no, EndIrq

HandleTimer:
        if delta time == tick counter
-               DecodeEvent (data goes to tail of software buffer)
                GetVarLength
                        if delta time == 0 goto -
                        if delta time <> 0 then tick counter = 0, goto EndTimerIrq

        if delta time <> tick counter then tick counter = tick counter + 1, goto EndTimerIrq

EndTimerIrq:
at the end of your timer ISR:
write data from the front of the software buffer to the UART’s FIFO until it’s full
	if the software buffer is not empty:
		enable TxEMT interrupt
goto EndIrq

HandleUart:
write data from the front of the software buffer to the UART’s FIFO until it’s full
	if the software buffer is empty:
		disable TxEMT interrupt

EndIrq:

Did I understand your suggestions correctly?


@GARTHWILSON
So in essence you are saying to keep the ISR short, and only update STATE variable(s) in the ISR. Outside of the ISR the tasks are executed (in a main loop), using those STATE variable(s) and update them when necessary.

If it’s possible, I’d prefer to execute the MIDI file player in its ISR and separately execute the UART output in its ISR, with use of the software buffer. Eventually there will also be a scan line interrupt, in which a MAIN LOOP will run, 60 times per second.

[GARTHWILSON]

lordbubsy wrote:

@GARTHWILSON
So in essence you are saying to keep the ISR short, and only update STATE variable(s) in the ISR.

The ISR increments the timer variable bytes, but not any state variables. The individual tasks keep track of their own states, and advance or reset their states when those tasks' own criteria or their various goals are met. The timer variable bytes are read by the various tasks like a whole office of people a century ago keeping their individual and independent schedules by all looking at the same clock on the wall. The clock is only concerned with supplying accurate time information, not what any of the many workers are doing with that information, or how many there are or what they're doing, etc..

There are of course many situations where it is desirable to have the ISR do more. One of those is where you need to eliminate jitter as much as possible, like when I've done audio sampling at tens of thousands of evenly spaced samples per second in the ISR; but in the case of MIDI, reducing the jitter to a maximum RMS of a few cycles is not as important as making sure you don't fall behind by ever-increasing amounts.

Garth, the interrupts junkie

[Chromatix]

Quote:

Did I understand your suggestions correctly?

That's more or less correct, though I don't think you need to duplicate the UART send code. Just enable the FIFO-empty interrupt (unconditionally, because you've just put data in the buffer) and either jump to the UART ISR with interrupts still masked, or RTI and let the hardware take care of it.

Quote:

If it’s possible, I’d prefer to execute the MIDI file player in its ISR and separately execute the UART output in its ISR, with use of the software buffer.

My "better timer" suggestion does that, probably with very little added code, and without the complexity of building even a simple multitasking OS. The usual way of handling timers in the latter is to keep a sorted list of deadlines, and set up the timer hardware to interrupt on the one due soonest. The ISR then resets the hardware for the next deadline, before handing control to the appropriate user routine. This inherently adds complexity and latency to each timed event, but it does let you get away with a finite amount of timer hardware.

Incorporating my "better timer" into your pseudocode:

Code: Select all

BeginIrq:

is interrupt caused by midi timer
        yes, goto HandleTimer
is interrupt caused by TxEMT? (uart transmit buffer is empty)
        yes, GOTO HandleUart
        no, EndIrq

HandleTimer:
        atomically increment missed-timers counter
        if midi mutex is set, then goto EndIrq
        set midi mutex
        clear interrupt mask (CLI)

MidiLoop:
        decrement tick counter (sets Z flag if it becomes zero)
        if tick counter not zero, then goto EndTimerIrq

        ; NB: delta time == tick counter
-       DecodeEvent (data goes to tail of software buffer, enables TxEMT interrupt if buffer was empty)

        GetVarLength
        if delta time == 0 goto -
        tick counter = delta time

EndTimerIrq:
        atomically decrement missed-timers counter (sets Z flag if it becomes zero)
        if Z flag set, goto MidiLoop

        set interrupt mask (SEI)
        clear midi mutex
        goto HandleUart

HandleUart:
write data from the front of the software buffer to the UART’s FIFO until it’s full
   if the software buffer is empty:
      disable TxEMT interrupt

EndIrq:

[GARTHWILSON]

Chromatix wrote:

My "better timer" suggestion does that, probably with very little added code, and without the complexity of building even a simple multitasking OS. The usual way of handling timers in the latter is to keep a sorted list of deadlines, and set up the timer hardware to interrupt on the one due soonest. The ISR then resets the hardware for the next deadline, before handing control to the appropriate user routine. This inherently adds complexity and latency to each timed event, but it does let you get away with a finite amount of timer hardware.

I don't know if this was referring to my cyclic executive multitasking suggestion; but to clarify here, the multitasking is with no multitasking OS at all—not even a simple one. The alarms method may be more fitting for certain jobs, even though it has more overhead. I describe this in the same article, at http://wilsonminesco.com/multitask/#alm .

[whartung]

But all this suggests that his current implementation is not saturating the MIDI line, whereas using the buffered FIFO will saturate the MIDI line. You can't go faster than the MIDI will accept.

If the CPU can fill the FIFO fast enough, then it should be able to saturate the MIDI line naturally. If he's simply "playing" data, vs generating it (using music synthesis or something), he should easily be saturating the MIDI line and be stuck waiting for the UART to clear more than anything else.

[Chromatix]

His problem is really with latency, not throughput. It currently takes so long to stuff the data for *some* ticks (not others) down the cable that timer interrupts are getting lost.

MIDI-over-cable doesn't carry timing information. MIDI files do. So he has to handle the timing in software.