I am about 75% completed with an interface that enables an IDE hard drive or Compact Flash card to be accessed via an RS-232 or SPI port.

It uses an Atmel ATMega32 to provide the FAT16 drivers that will make it readable on a PC. It will support 16 open files at a time and use the standard DOS 8.3 file name conventions.

I need to finish verifying the FAT routines, write the SPI drivers, and add a Real-time Clock so that the files will be time-stamped. The added benefit is that the Clock will accessible to the host through added commands.

Commands supported (1st draft)

Format Drive
List Directory
make Directory
change directory
delete file/directory
delete all (files and directories)
open file for Read, write, or append
read file
write file
protect file/directory
unprotect file/directory
set file pointer
get file pointer
get file size
close file
close all files
rename file/directory/volume
return vol label
return free space
return drive info (make, model, ser#, size)
return date/time
set date/time
initialize system

If there is enough interest, I might be willing to lay it out on a small PCB and offer it as a kit.

Daryl

Yes, I'm interested.....

/Pontus

Count me in as well. Some external storage is all the SBC3 needs to be a perfectly usable standalone computer (audio would be nice too, but that's something for another day )

Tiny nitpick: you have a command to read the volume label, but no corresponding one to set it.

Since this will be a controller for removable media, it would be really convenient if there was a simple way to tell if the CF card was unplugged between commands (something simple that a timer interrupt could use to poll every couple of seconds). A command to force a disk change, to get everything back to a known state after bad errors, would be useful to have as well.

The rename file command actually will change a directory name and Volume label as well. (I should have included that).

I was thinking of adding an "initialize new media" command. All one needs to do is issue a close all command, replace the media, and then reinit.

As far as polling for media removal, I'll see what can be done.

Purhaps a verification of the volume label. To prevent corruption, this test should be performed before any disk access. However, this would really slow performance. If we assume the user is aware of the consequences of swapping media with files open, we can get away with periodic polling.

For SBC-3, I hope to come up with a DOS like front-end that would replace the system monitor. All programs could be loaded from CF and run. I can foresee EhBASIC, FORTH, and an assembler being very useful. Next could come TCP/IP support for dial-up Internet via a modem on the RS-232 port. Eventually, a direct Ethernet to SPI interface would be great. Whow there, let me finish this project first!

Daryl

I would use this algorithm, inspired by the semantics of how AmigaOS treated floppy disks:

Let's assume a module is mounted in the unit already. Assume further that no I/O to the module is active. Then, the storage driver can periodically poll the device by attempting to read block 0 (and discarding the results). The goal is to detect a time-out condition: if the operation times out, then we know the module was removed since the last poll. If files remain open, you can request the user to re-insert the module with the given volume label and formatting timestamp. (Be sure to check for both, since with only 11 characters in the volume label, it'll almost certainly run into duplicate volume names!) When insertion is detected, then you re-read the volume label to confirm it's the correct module.

Let's now assume that an I/O operation is in progress. There is no need to schedule a poll during this time, since we're already actively using the module. If a timeout occurs here, again, we can be quite confident that the module has been removed from the unit. Now, instead of politely asking the user to re-insert the module, you can use stronger language (particularly if writing): to quote from AmigaOS under these circumstances, "You MUST re-insert volume xyzzyx in drive zzz:!" Wait for insertion as before. However, prior to completing the write operation, kick off a filesystem validation operation to ensure the volume is in good condition first.

The only remaining condition is, regardless of pending I/O operations, when the unit is empty. In this case, periodic polls to the device are necessary to detect whether it's still empty. Reading from an arbitrary block (block 0 is fine) should do the trick; just discard the contents.

Finally, PLEASE PLEASE PLEASE auto-flush any cached storage blocks after a reasonable period of time. AmigaOS would automatically write-back its in-RAM disk buffers after five seconds of unit inactivity. This COMPLETELY eliminated the need for manual mounting or unmounting, as well as the need to issue a "shut down" command to the machine. It takes no appreciable overhead to implement this logic, and saves the user much grief. It also allows them to safely unmount the module in most cases.

(Note: since you're dealing with solid-state storage, you might get by with a one-second idle delay instead of five-second idle delay. That might be preferable.)

I completely agree, AmigaOS handled this situation much better than any other OS I've ever used.


Of course, the tricky thing here is that the filesystem is buried inside an IC that's on the far side of a serial line, making it a bit hard to put up a dialog box The simplest way I can think of solving this is to have an extra line in the ATMega, which is dedicated to handle this situation. If the CF card is pulled out in the middle of a write, the ATMega asserts the line and waits for the CF card to be reinserted. When the correct card is reinserted, the line is deasserted and the operation is resumed (when in this mode, the only valid command should be the one that resets the filesystem). This line can be connected to a VIA Cxx input, set to trigger an interrupt when asserted. The OS can then use this interrupt to inform the user of the problem, and keep monitoring the line to discover when the situation was resolved (or abort the write via a reset command, should the end user insist).


An important thing to note here is that the drive activity LED cannot be turned on by this read, because the constant flashing would be quite annoying and (more importantly) would undermine the LED's function as a warning indicator.

While we're discussing possible design changes, you should probably add commands to read and set file attributes (hidden, system, readonly, archive, directory) and get the file size; enough to be able to write an implementation of fstat(). Some way to help with an implementation of findfirst()/findnext() would be nice too. One way to do this could be to have the command that returns the directory listing take a pattern as a parameter, and have it return just the list of filenames that match (findfirst/findnext on the host would just return each of these in turn). A full "dir" implementation would be achieved by the host getting the full list of files (using "*.*" as a pattern) and iterating through each filename, getting the file attributes and file size from the ATMega and displaying the filename, attributes and size appropriately.

I was thinking that after you sent the command to the device, you'll get a response back. The response would indicate the nature of the error, and the SBC would act accordingly.

I would consider investing in this device if it supported the ability to read and write raw 512-octet blocks of storage too. Without at least those abilities, I don't even have the option of using an alternative filesystem, and positively no way of supporting Forth block storage. I'd like to use this with the SEAforth chip computer I'm trying to make. But, . . .

I thought of that, however there's a problem: if the user pulls out the CF card halfway though a write, only the device knows how far the write got before this happened. That means that if it returns an error code to signify this unfortunate event, it would first need to save its internal state so that a "resume last operation" command can pick up where everything left off after the card's reinserted. That's why I proposed just pausing everything and letting the host CPU know what's going on via a dedicated line.


Having raw sector access would be good, yes. It's also necessary to write the boot sector or MBR.

More details:

I did have a return file size command, I just overlooked putting on the list. I've update the list above.

The system does manage the volume and directory attributes. For this version, I am not managing any of the other file attribute flags. However, it would be good to use the read-only flag to prevent file writes and deletion. I'll see about adding that. I don't see the value added in trying to implement Hidden and System atributes to this platform.

Since I'm limited to the 2k of internal RAM on the ATMega32, each write command automatically flushes the buffer. Also, all FAT access is written immediately as well.

The read and write commands will have a parameter that tells how many bytes to read/write. In this version, its limited to 256. Again, immediately after a command is completed, the buffers are flushed.

My goal here was, and is, to provide a simple FAT16 compatible storage media. In using the ATM32 to handle the FAT16 format, the host's control over the media becomes restricted. In other words, the media cannot have two masters. Doing so would open the FAT16 compatibility to corruption. Keeping compatibility allows you easily move and manage your drive from a PC. If you are developing a large OS or application on a PC's development platform, you can easily copy the target files to the CF and move it to the SBC. Same goes for data files being created on the SBC and being able to move them to a larger platform for more detailed analysis. The FAT16 integrity must be maitained.

In order to have all the bells and whistles of a robust file system, you really need to put the HOST in control of the media. If things like block transfer and raw access are desired, then I suggest an option to turn off the FAT16 function, and give two simple commands to the HOST - sector read and sector write. Then the HOST can have total control over the media - the ATM32 becomes a low-level interface only. Media access outside of the HOST (PC for example) would require specialized software. I can fully support this "RAW" mode for those who want it.

I do agree a minimal check for media removal is needed. However, I still favor simplicity by making the user responsible for proper removal and installation of the media. Again, with RAW mode, you can provide strict control if desired. Instead of reading the volume label, I'll read the device serial number (returned by the Identify Drive ATA command). This should be a better way to ensure its the right media.

I'll be releasing the source code eventually and those who desire to may taylor it to their needs.


Daryl

Anyone here heard of the Vinculum VDrive2?

Whoa...why?

The Unix system calls give a better approach. Consider write() for example: given a file handle, a buffer address, and a length of the buffer to write, it will return an error code if something really went wrong, OR it will return the number of bytes known to have successfully been written. The latter MAY be smaller than the length parameter given.

This style of code is not uncommon, particularly in error-prone I/O situations, like networking:



A loop not unlike the above may be used at the OS level to affect the needed auto-retries desired. The only changes to the API I'd make are that you always return an error code of some kind AND the number of bytes written -- that eliminates the either-or error handling logic that Unix uses. It works, but it costs an extra procedure call; might as well save the overhead.



And to write CHKDSK as well.

Note also that FAT is singularly horrible for flash-based media, because FAT makes zero attempt to minimize sector rewrites. It is for this reason that I want the option to use an alternative filesystem, perhaps something like a true logging filesystem (as opposed to a journalling filesystem, which only journals meta-data, but leaves application data free to overwrite sectors in-place).

I have not, but it looks like it's horribly complicated to use. Glancing over the data sheet, it appears to be a device which attempts to perform too many functions at once.

No, but after reading the docs, it appears that only one file may be open at a time. That is rather limited for a general purpose computer. However, great for a data logger application or MP3 player.

Not only that, but if you write to a file it is truncated at the write pointer. I mentioned the device because, while there are issues, it's in sort of the right direction.