It's still (just) possible to get hold of new floppy drives and disks, in the formerly ubiquitous 3.5" high-density format. Every so often someone remembers using floppy disks with a classic micro, and wonders whether they can also do so with a homebrew build. The answer is of course "yes, but there are better options" - but, being who I am, I decided to look into the problem a bit more deeply.
The "easy" option is to obtain a
standard PC-type drive (or two) and interface them through a
standard PC-type floppy drive controller. You then have to implement a filesystem and software driver conforming to the capabilities and restrictions of the standard format; frankly at this point you might as well just implement FAT12 and have done with it. The programming interface of the FDC is arcane and frankly bizarre in places, and requires you to either poll continuously for data, implement a pretty quick ISR and never mask it while accessing the disk, or add a DMA controller to your design.
Yuck.But then I looked at the
datasheet for the drive itself, and things started to actually make sense. (That's a "3-mode" version of the drive, which isn't precisely the same as what you get in a standard PC; for that, ignore the 1.6MB mode and look at the 1MB/2MB ones.) The drive actually has enough intelligence and drive electronics built in that you could reasonably build a controller by hand, without needing to know anything significant about analogue electronics. In particular, the drive takes care of converting magnetic flux reversals on the disk into active-low pulse trains, and vice versa, which I had thought would be the hardest part of implementing an FDC. What you *do* need to get right is the timing and encode/decode logic, but you get a fair amount of freedom in designing that if you ignore the MFM-based PC specs.
Stepping back a bit, most classic micros were satisfied with single-sided, 40-track, single-density 5.25" drives offering roughly 180KB of "formatted capacity". In principle you can replace any of those drives with a 3.5" drive and keep all the same formatting parameters. From there, if you step up to double-sided and 80 tracks, which any 3.5" disk supports, you get four times the capacity, still at "single density". But what does the latter mean?
In fact, single density just means you're using FM encoding with a 250KHz (4-microsecond) maximum flux-transition rate (referenced to the outer track at 300rpm). FM is a very primitive encoding, though one almost universal among classic micros - the Apple ][ being a notable exception - which simply lays down one pulse and one space for a zero, and two pulses for a one; this appears on the disk as a low frequency for zero, and a high frequency for one. You therefore need 8 microseconds to encode a bit on the disk, so at 300rpm (5 revs per second) you can theoretically fit 3125 bytes on a track. Much overhead is then spent on synchronisation and address marks, and gaps to allow writing individual sectors without accidentally overwriting a preceding or following one, so in practice you can only fit five 512-byte sectors per track, for 2560 bytes usable capacity, or eight 256-byte sectors for 2048 bytes usable capacity.
The Apple ][ uses a slightly more sophisticated encoding called 5-in-3 GCR and, like the C64's 1541 drive, reduces the number of sectors on the inner tracks of the disk (aka "zone recording") to keep the physical bit density within limits while employing higher density on the outer tracks. There's a lesson here worth following. Early Macs got a slightly upgraded version of this called 6-in-2 GCR, which allowed a double-sided double-density 3.5" disk to store 800KB, versus the PC's 720KB format on the same disks.
Double density works with exactly the same disks and drives as single density, and changes only the encoding to MFM, which requires at most one flux reversal per bit. With the same maximum flux-transition rate, this means MFM can store a bit in 4µs at 300rpm, so twice as much raw data per track as with FM. The encoding and decoding circuits are a little bit more complicated, but still reasonably practical for low-cost 1980s technology. The overheads actually work out such that you can put in nine 512-byte sectors per track, for 4608 bytes usable capacity out of the theoretical raw 6250 bytes. (This is still a rather poor 73.7% storage efficiency.)
High density disks use a better-quality disk coating with higher magnetic coercivity, allowing the maximum flux reversal rate to be doubled to 500KHz (2µs) at 300rpm. The PC's very straightforward adaptation of MFM to this gives 18 512-byte sectors per track. Multiplying by two sides and 80 tracks gives the familiar 1440KB per disk. But the theoretical raw capacity is now 12500 bytes per track, for a total of 2000000 bytes per disk, hence the drive manufacturer's description of "2MB mode". As for the "1.6MB mode", that comes from applying the same 500KHz MFM encoding to a disk spinning at 360rpm, where you get only 1/6th of a second and the flux reversals end up slightly further apart.
There was also a short-lived 2880KB "extended density" format, using special barium-coated disks and special drives with a different magnetic field orientation. I think those are now very difficult to obtain. The format again simply doubled the flux transition rate and applied MFM encoding, obtaining 36 sectors per track.
Now, why am I going into so much detail? Because MFM became obsolete as soon as the Compact Disc appeared. By employing a slight variation of the CD's encoding scheme, we can substantially increase the usable recording density on existing disks and, in the process, give ourselves a reasonable excuse for implementing our own FDC. All within spec.
MFM requires twice as many "symbol bits" as there are data bits, just as FM does. FM requires every "clock bit" to be a flux transition; MFM effectively deletes the "clock bit" if there's a neighbouring data bit with a flux transition. So the minimum distance between flux transitions is 1 symbol bit for FM, or 2 for MFM - or
three for the EFM (eight-to-fourteen modulation) used on CDs. So we can pack 50% more symbol bits onto a track with EFM as with MFM. Looking at the
maximum distance between flux transitions, which is important for accurate clock recovery, we get 2 symbols (twice the minimum) for FM, 4 symbols (again, twice the minimum) for MFM, and 11 symbols (nearly four times the minimum) for EFM. That indicates a more challenging clock-recovery task with EFM, but not beyond reason; it's hard to make a physically spinning disk change significantly in speed within 8µs, especially by accident.
CD's application of EFM actually requires 17 symbol bits per 8-bit data group, as 3 non-data-carrying "merging bits" are used to preserve the invariants of minimum and maximum transition spacings, as well as DC-balance between lands and pits which is apparently important for laser tracking. For a floppy disk, the DC-balance metric is not important, so we can use just 2 merging bits, thereby using exactly twice the number of symbol bits as data bits, just as with FM and MFM. With the 50% higher density of symbol bits, that means we can now put down 18750 raw bytes per track, for a theoretical disk capacity of 3000000 bytes. How many of those bytes are actually usable will depend on how much overhead (in terms of synchronisation and sector gaps) we accept, but it should definitely be possible to get 16KB per track, for a total of 2560KB per disk - almost twice the usable capacity of a standard PC format of the same disk, using the same drive! It may also be possible to include a smaller metadata sector alongside the large data sector(s).
Let's leave aside the question of how to convert EFM to binary and back, and simply assume we can write software to do that, feeding the controller directly with EFM patterns. Probably a CPLD could be made to do it too.
A more pressing question is how to correctly time the pulse trains, both on read and write, to produce the relatively complex and precise patterns that EFM requires. I've recently advocated the use of a 24MHz master clock to drive a DRAM controller and derive 8MHz or 12MHz CPU clocks, and it fortuitously turns out that a 24MHz master clock is also popular for FDC chips, from which a 1MHz, 500KHz, 250KHz or 300KHz symbol clock can be derived by simple frequency division - as can the 1.5MHz symbol clock we now need (divide by 16).
A frequency divider is really just a free-running counter with a reset upon reaching some matching value; we can use the same counter to produce EFM pulses by feeding it a sequence of delta counts, or to measure the time delta between pulses when reading a disk - and delta-time readings are naturally self-synchronising for clock recovery. An 8-bit counter/comparator should be sufficient since 11*16 fits comfortably. If we dump this delta-time data into a FIFO or a specialised bank of RAM, we can interface it to the host computer for decoding. Dumping it to a RAM buffer would eliminate the need to have the CPU grab it in realtime, and we can probably interface it as a giant circular FIFO to minimise the host address space consumption.
The same scheme, incidentally, should be capable of producing and decoding MFM format tracks, so we don't need to lose compatibility with PC-formatted disks after all.
A wrinkle here, which unfortunately the drive electronics don't handle for us, is that closely-spaced magnetic flux reversals have an unfortunate habit of migrating away from each other just after being written - this, broadly speaking, is because the effect of neighbouring magnetised regions overlaps to some extent, and the read head sees the sum of that overlap. Thus it's necessary to move closely-spaced flux reversals even more closely to each other as pre-compensation for that migration. The question of how closely-spaced and how much to compensate is an open one, as it's dependent on the disk, the drive, and even the track on the disk. PCs simply apply a flat 125ns pre-compensation (three 24MHz clocks) on minimum-spaced MFM pulses, but since EFM is more subtle, I suspect it'd be wise to record test disks using real hardware and then observe how the migration behaves in practice. The necessary pre-compensation can then be incorporated into the delta-times of the binary-to-EFM encoding when writing a real disk.