True, but if you're using a flash storage medium (CF card, USB stick) rather than a raw flash chip, wear-levelling is included.

Based on my experience, this doesn't appear to be the case.

This one will at least support 4 open files. The original version supported 8 I believe.

http://www.ghielectronics.com/details.php?id=1&sid=2

Daryl

Ah, but write() will make sure that the disk structures are consistent before returning the error code - you don't need to run ex2fsck after getting write errors. In the situation we're discussing, the "error" is interrupted access to the card, which could happen at any instant. I do agree that a perfectly valid (if kinda sucky) solution to the problem is to return an error code that lets the user know (in my best Mike Myers impersonation) "yer fooked!" Another possible way to deal with it would be to write out the data before updating the FAT, so that you have a sortof transaction model. I don't know if this is completely bulletproof though.

OK, maybe I'm just not communicating well here.

My point is to look at write() by way of analogy. Yes, of course, it ensures metadata is correct prior to returning an error. However, the implementation of write() itself could very well have just such a while()-loop talking to the driver of the module. In fact, the behavior of AmigaDOS's Write() call behaves exactly as though it did have this kind of loop in place. I'm positive trackdisk.device certainly doesn't.

Is this clearer? I cannot make it moreso, so if an understanding cannot be reached by now, I'm afraid that we're going to have to drop the discussion.

Fair enough. One thing you must absolutely do is to preserve the values of rest of the flags, so that files don't become "corrupted" when accessed using this device. For example, long filenames are implemented using special combinations of file flags, so if you set or clear the flags you don't use, you'll break long filenames. Come to think of it, when getting a file list you should not return "files" with those attributes set, because they're not real files - they're fragments of another file's long filename. From Wikipedia:

Whoa, chill out! This is just a friendly discussion, right?


OK, this is the scenario I'm thinking of:

1. Host sends "write to file" command with a 256 byte buffer full of data
2. Device reads the FAT, decides what cluster to use to append the data
3. Device writes out the user data
4. Device is in the middle of updating the FAT, when the user for whatever reason pulls out the CF card

What to do now? My proposal is to stop right there, signal what happened and wait for the card to be reinserted. Your proposal, as I understand it, is to return an error code so that the host can report the error to the user and (presumably - this could be where our communication mismatch lies) then try to complete the operation. Problem is, the host doesn't have enough info to resume the operation: it would need to know what the FAT must look like so it can retry. The error response could contain that, I guess, but it just seems an unwieldy way to handle things.

Yes, but, I'm at a total loss as to how clarify what I'm trying to say. I'd rather drop the discussion while it's still friendly, than elevate hostility trying to talk past each other.



It has all the info it needs to resume the operation. The very fact that the FAT was not updated is just cause for the host to retry the entire operation. Think about it: if CHKDSK runs, the errant FAT entries will be disposed of, and with any luck, the file will return to its pre-corrupted state (at least in terms of usability).

The reason it has this information is because the application's call to write() never returns until the user selects Cancel. Just as in AmigaOS, it'll continue, forever if necessary, popping up that pesky "You MUST replace volume XYZ in drive ABC" dialog.

If AmigaOS can do it, so can this thing.

The ATmega chip simply isn't going to have the resources to remember all the state needed to implement preservation of state on-chip, particularly since it allegedly supports 16 concurrently open files. I'd rather the host handle the retry logic, while the ATmega just concentrates on being a low-level DOS.

That's my opinion on the matter. Not trying to be feisty, but at this point, I really do feel like we're talking past each other and at cross-purposes.

Hmmm, I think I see where we're speaking at cross-purposes: I'm trying to prevent anything going wrong at all with the volume, whereas you're saying it's OK if things go awry, as long as no data is lost. Pragmatically, you're probably more correct than me since it's impossible to predict how data corruption will occur. Thank goodness there's a spare FAT in there (although that means it's even more important to have raw sector access so chkdsk can be written, unless one requires users to have a Windows or Linux box around to fix things if a CF card goes bad)
If the host is going to be in charge of handling retries, there needs to be an error code that specifies that there was a retryable write error - a generic "only 0 bytes were written" status won't be enough.


Right, my original aim was to implement that behavior inside the controller, the pesky dialog was the line that goes active when the card gets pulled out prematurely.

A few points to take from ths discussion.

My FAT16 driver will NOT support Long File Names. The support of Long File Names requires a license from Microsoft and I'm NOT going there. I did study the tricks used to create long file names and if memory serves, my system will just pass over the extra directory entries. I cannot promise that this will happen 100% of the time. And yes, new files created have the archive bit enabled and all other flag bits turned off. It also preserves existing attribute bits when accessing an existing file.

My file system creates a single FAT table when it formats a drive. From what I have read, the second FAT is rarely useful in restoring a corrupt drive and its space could be better used for file storage. It does supports two FAT tables on an existing file system (one formated on a PC for instance), and will update both tables.

I have decided to take a simple approach to media removal. The user needs to be responsible for proper installation/removal of the media at all times. A close all needs to be executed before the media is removed. An initialize media command needs to sent when a new media is installed. I plan to add an LED that will light up when its safe to remove the media.

I will have a jumper option that selects FAT16 mode or direct sector access mode on startup and during the initialize media command. Changing back and forth will require the jumper to be changed followed by the initialize drive command.

Dirt simple and error proof, as long as the user follows the rules. If the rules are broken, the system will not be responsible for any loss or corruption.

For those who want a stronger system, they can use the direct access mode and take care of the error checking on their host.

The other option is to directly connect the IDE/CF to a VIA (with a few extra gates), and bypass it all. The speed would be much faster and you would have total control. One CPLD could provide the necessary 16 bit to 8 bit translations. The only reason I chose the ATM32 was to offload the FAT16 junk from the HOST.

The RS-232 interface was convienient for testing purposes.

The SPI interface is to help keep the SBC-3 VIA's open for user devices and functions.

Once I have the actual interface completed (I'm waiting on the RTC parts to arrive), I'll post the details and each individual can decide if it fits their "mission."

Thanks for all of the discussion, it resulted in some good points for all to consider.

Daryl

I'm sorry if it sounded like I was suggesting you should implement them, that was not my intention at all. The reason I mentioned long filenames is that even though you don't support them, you should leave them alone if you find them. Sounds like you're doing that anyway


That makes perfect sense: most OSs do it that way, and if you return a specific error code when the media is prematurely ejected, the host can take care of more advanced error handling as suggested by kc5tja. One small point: the usual practice is for the LED to light up when it's bad to remove the media. Making it work the other way round will be confusing for end users.


I'm curious - why the jumper? It would be much more flexible if you make it so that every time the media is initialized, the init command dictates whether that "session" will be using raw access or filesystem access. That way you won't have to worry about a raw write doing things behind the back of the FAT driver, and one will still be able to write a version of fdisk.


Not only that, 32KB of ROM is not a whole lot of space, and I'd much prefer to use it for higher level functionality than filesystem access. Plus I come from the Commodore world, so intelligent drives just make sense to me


As long as we don't scare you off with our crazy suggestions

I wasn't sure if you wanted LFN or not. I just wanted to be sure everyone was informed that LFN support will not be supported. Sorry if my response was too harsh


OK. The LED functionality can go either way. Personally, I won't install it on my board. But I will program it to be on when the drive is in use.


My thinking was that the user would either plan to use FAT, or their own file system exclusively. But now that you mention it, some may want to use both (their own for programs, but use FAT for data transfer). I will drop the jumper and use a software approach. Good suggestion!


If you plan to use FAT, then yes, better to let the ATM32 do it. But if you plan to use your own, all that is needed in ROM is the read/write sector code and a small boot code that grabs the first "n" sectors. That could easily be placed in 32K. The loaded code would contain the bulk of the OS.



I won't get scared off. I will take in the suggestions I can, chose not to implement the one I feel are not within my "mission", and present the completed project when its done. Anyone interested can use it, modify it, or choose their own path. No harm, no foul.

When I first started building SBC-1, and through out my journey to SBC-3, I have asked for comments from this forum. I have tried to be open to suggestions, and incorporated those that made sense and were within my design budget (space, simplicity,cost, and functionality). I will continue to ask for suggestions as I value everyone's suggestions. This forum (more importantly the users who contribute to it) is a great resource. I have learned more from this site that I have from all the books I've read. You all are doing great things and each person brings their own expertise and experience to this group.

Daryl

I'm getting closer to completion if the FAT16 interface. Today, I installed the DS1302 Real-Time clock and wrote the handlers for it. I now have file/directory timestamping!

I found a bug in my set file pointer function. Once I fix that, I think I'm done with the root FAT16 support.

I still need to add support for "direct" mode. Also, support for the LED.

This will complete the RS-232 version.

In order to support the SPI version, I'll have to rearrange some of the IO and I will not have enough IO pins to support the LED. However, I may use the /IDE_RES pin to drive both the IDE device and an LED.

The SPI port will use the 4 standard SPI pins and 1 additional handshake output (just like the ATM8 did for my RS-232/PC keyboard).

Daryl

I'm interested as well.
Happily buy a kit off you.

Lets keep it simple for now and add to it later.

Ralph

_________________
Thanks and regards

Ralph

Here's an update. I had to split the design into two versions: one has an RS-232 interface and the other has an SPI interface. There are not enough pins to allow both without some major software acrobatics. There will also be two software versions: one for each interface.

Both versions will include FAT16 support and can be software switched to Direct mode.

I am finishing up the FAT16 command interface and have both boards designed. For those wanting to go cheap, this project can easily be done on perfboard using point-to-point wiring.

Once I finish the command interface and write the SPI interface, I'll do up the user manual and put it on my website. I'm not sure how much longer it will be... so stay tuned here for updates.

Have a safe and happy new year!

Daryl