Page 8 of 14
Re: Adventures in FAT32 with 65c02
Posted: Sat Dec 27, 2025 6:11 am
by BigDumbDinosaur
The standard principle is "read many, write once", so if a file is open for reading, it can be open again for reading as many times as you have resources, but only for reading. But any file that is open for writing blocks any further opening of any type. So, opening for update (read *and* write) blocks that file from being opened again as it is open for writing.
An alternative if exclusive access is needed during a write operation would be to implement a file locking method.
You make the action dependent on what the caller wants...In general, you should minimise how much of your calls fail, they should just pass a result back to the caller so that the caller can decide what to do.
Using UNIX as an example, the original open() kernel function returned with an error if the requested file couldn’t be opened, with an error code describing why. If the user then wanted to create a non-existent file, he/she would have to call the creat() kernel function to do so—which, of course, might also fail.
In later UNIX kernels, as well as with Linux, open() may be called with the optional O_CREAT flag as an argument so a non-existent file may be created as part of the open process—one-stop shopping, as it were. Note that despite that added feature, open() never assumes that it should create a non-existent file, which honors the “rule of least surprise.”
In a more expansive sense, low-level system calls of any type should do only the absolute minimum needed to carry out their designated functions. The more features added to a kernel call, the greater the chance that, at some point, the rule of least surprise will be broken and the programmer will end up chasing bugs caused by kernel behavior that is unwanted.
Re: Adventures in FAT32 with 65c02
Posted: Sat Dec 27, 2025 7:54 am
by barnacle
With regard to
open() I am firm that it will only open a file if it already exists. It won't ever create a file that doesn't already exist. That's what
f_create() is for. And I spell it with six letters, so apologies to Ken Thompson.
Of course, what is least surprising to me may not be least surprising to anyone else. For me it's surprising that C's
fopen() can open _and truncate_ a file if it already exists, and it's opened with "w"; yes, yes, yes, one should know one's tools, but if I want to start writing to an empty file, it's not a problem to delete and then create again.
Hence I have
f_open_r which opens an existing file and positions the file pointer at the start of it, and
f_open_w which opens an existing file and positions the file pointer at the end of it - so it appends. But 65c02's lack of a stack frame addressing mode makes passing parameters sometimes problematical, hence two variants. No reason why the other variants couldn't be done in a similar way.
By the way: May I take this opportunity to thank all who have contributed to this thread, guiding both myself and hopefully others? Cheers, guys!
Neil
Re: Adventures in FAT32 with 65c02
Posted: Sat Dec 27, 2025 11:02 am
by jgharston
Hence I have f_open_r which opens an existing file and positions the file pointer at the start of it, and f_open_w which opens an existing file and positions the file pointer at the end of it - so it appends. But 65c02's lack of a stack frame addressing mode makes passing parameters sometimes problematical, hence two variants. No reason why the other variants couldn't be done in a similar way.
Werl, on the BBC you have XY=>filename and A=action, 40=input, 80=output, C0=update, with extensions allowing options in the lower bits, eg 80+xx overwrite/don'toverwrite existing file, C0+xx create/don'tcreate if file absent. 8 bits of open options is quite enough.
Re: Adventures in FAT32 with 65c02
Posted: Sat Dec 27, 2025 11:14 am
by barnacle
Yeah, but the people that wrote the BBC OS were smart people, not like me.
Neil
Re: Adventures in FAT32 with 65c02
Posted: Sat Dec 27, 2025 1:03 pm
by BigDumbDinosaur
With regard to open()...It won't ever create a file that doesn't already exist. That's what f_create() is for. And I spell it with six letters, so apologies to Ken Thompson.
Supposedly, when Thompson was asked if there was anything he would do over in UNIX, he replied that he would spell
creat() with an
‘e’.
Of course, what is least surprising to me may not be least surprising to anyone else. For me it's surprising that C's fopen() can open _and truncate_ a file if it already exists, and it's opened with "w"...
...65c02's lack of a stack frame addressing mode makes passing parameters sometimes problematical, hence two variants. No reason why the other variants couldn't be done in a similar way.
By the way: May I take this opportunity to thank all who have contributed to this thread, guiding both myself and hopefully others? Cheers, guys!
Re: Adventures in FAT32 with 65c02
Posted: Sat Dec 27, 2025 2:15 pm
by fachat
Just a note. Some of the flags like O_CREATE have been added - as I understand it - to make operations atomic and avoid race conditions that may result in security issues for example.
André
Re: Adventures in FAT32 with 65c02
Posted: Sat Dec 27, 2025 2:36 pm
by barnacle
That's a fair point, but on a single-user single-threaded 65c02 system, the whole thing is a security disaster... as long as can interchange files with the linux box, I'll be happy.
Neil
Re: Adventures in FAT32 with 65c02
Posted: Sat Dec 27, 2025 11:41 pm
by BigDumbDinosaur
Just a note. Some of the flags like O_CREATE have been added - as I understand it - to make operations atomic and avoid race conditions that may result in security issues for example.
The combination of the O_CREATE and O_EXCL flags as parameters to an open() call guarantees exclusivity. An open() call with these flags will fail if the file exists, but will create the file if not present. As internal kernel functions are general immune to preemption (but, of course, may be interrupted), the result is an atomic operation.
Re: Adventures in FAT32 with 65c02
Posted: Fri Jan 02, 2026 3:14 pm
by barnacle
Well that's been a long week... I discovered that my
str_to_83 routine, (turns a string into an internal MS filename) had a bug with eight-character names. Took a while to fix.
Unfortunately, that _wasn't_ the problem I had...
I had a simple test that initialised the system, dir'd the existing files on the disk, then tried to create a new file. Which worked perfectly right up to the time I ran it
On the next boot, it was clear that it had in some way trashed the directory on disc.
It took a lot of testing to discover that everything being written appeared to be correct, but was not being written correctly. Turns out that rewriting
fc_write to accommodate multiple buffers broke it: it required the correct buffer to be defined before writing (doh) and I'd missed it. And of course, every time I break it, I need to reformat the disk...
Code: Select all
NeoDOS32 v0.0171
Press any key to boot:
BIOS.ASM
CF.ASM
FILE_SYS.ASM
MAIN.ASM
MAIN.HEX
MAIN.LST
U32MATHS.ASM
VARS.ASM
DIRECTOR
+
That's version 171 since xmas...
Neil
(edit: the files listed are added on the linux box after formatting except 'DIRECTOR' which is created locally (and will soon be a new directory))
(edit 2: Why so long? Well, there was a little diversion into MS time and date formats, and leap years and suchlike)
Re: Adventures in FAT32 with 65c02
Posted: Sat Jan 03, 2026 12:13 pm
by barnacle
Okay, I'm in a more productive mood today
I believe I have resolved the bugs outstanding in
f_create,
f_cat,
f_dir, and
f_del. Changing
cf_write and
cf_read to work with buffers other than transient caused a lot of subtle, sometimes spectacular, errors, most of which required the CF to be reformatted each time.
Here's an attempt to delete then recreate
"another.txt" with a directory listing between each attempt:
Code: Select all
NeoDOS32 v0.0180
Press any key to boot:
BIOS.ASM
CF.ASM
DOS.ASM
FILE_SYS.ASM
MAIN.ASM
MAIN.HEX
MAIN.LST
U32MATHS.ASM
VARS.ASM
ANOTHER.TXT
deleting 'another.txt'
BIOS.ASM
CF.ASM
DOS.ASM
FILE_SYS.ASM
MAIN.ASM
MAIN.HEX
MAIN.LST
U32MATHS.ASM
VARS.ASM
creating 'another.txt'
BIOS.ASM
CF.ASM
DOS.ASM
FILE_SYS.ASM
MAIN.ASM
MAIN.HEX
MAIN.LST
U32MATHS.ASM
VARS.ASM
ANOTHER.TXT
Finished
+
The directory entries for deleted versions of the file remain unchanged, so each attempt makes the directory a little fuller. The card can be read and written in the Linux file manager.
Neil
Re: Adventures in FAT32 with 65c02
Posted: Sat Jan 03, 2026 10:55 pm
by BigDumbDinosaur
Okay, I'm in a more productive mood today I believe I have resolved the bugs outstanding in f_create, f_cat, f_dir, and f_del. Changing cf_write and cf_read to work with buffers other than transient caused a lot of subtle, sometimes spectacular, errors, most of which required the CF to be reformatted each time.
So I do understand a little bit of your pain.
The directory entries for deleted versions of the file remain unchanged, so each attempt makes the directory a little fuller. The card can be read and written in the Linux file manager.
Re: Adventures in FAT32 with 65c02
Posted: Sun Jan 04, 2026 7:15 am
by barnacle
Not immediately; for my use case I can't see any way to fill half a gigabyte. But I do need to address the allocation anyway.
For the short term I think I'll follow MS's lead: use the on-disk indicator for the next free cluster (actually it's the last allocated block, AFAICS) and search the FAT forwards from there for an empty cluster. That works fine until you run out of disk space.
If you do get to the end of the cluster map, then you need to go back to the beginning of the root directory, find a deleted file (possibly recursively into any child directories), clear its directory entry, follow its FAT chain, erasing FAT entries as you go, so that you can look for them again while allocating. But it's messy: the only options for the directory entry are (1) filename begins with legal character = filename, in use, (2) filename begins with $E5 = file has been deleted, or (3) filename begins with $00 = there are no more directory entries. The sequencing for deleting things and reusing the space might have complications.
The old MS policy was not to reallocate deleted files' clusters immediately so altering the $E5 to an ascii character would magically restore the file from the dead, mostly. I'm not sure what the Linux file manager does; certainly the old filenames remain (with the $E5. While one obvious policy would be to zero the FAT entries for a deleted file as soon as its deleted, that would be unkind to Peter Norton, and would leave a potentially very fragmented filesystem. On the other hand, with no seek time, is fragmentation even an issue?
On a philosophical note: the basic 'user' file operations (dir, cat, mkdir, del, rmdir etc) can either be built into the BDOS rom, or, given the basic abillity to load and execute a file, can be independent files as part of the OS. None of them are particularly large, but I would like eventually to be able to max out the available RAM. Still thinking about that one... it does give a very easy way to change my mind later about how things work.
But first, I have to do some testing as to why f_dir sometimes starts by loading the last sector used, unless I provide a debug line to tell me where it's starting... then, it always works. Hmm.
Neil
Re: Adventures in FAT32 with 65c02
Posted: Sun Jan 04, 2026 1:45 pm
by barnacle
But first, I have to do some testing as to why f_dir sometimes starts by loading the last sector used, unless I provide a debug line to tell me where it's starting... then, it always works. Hmm.
Well, sorted that out at least: a
cf_wait is required before loading the lba into the CF registers. When I inserted debug outputs, they took sufficient time that the CF readied itself before I needed it.
Code: Select all
cf_read:
jsr cf_wait
jsr cf_set_lba ; set the desired sector
jsr cf_wait
lda # 0x20 ; read a sector command
sta CFREG7
cf_read_info: ; enter here if presetting info block ID: 0xec
; but only after having set lba
jsr cf_wait
lda #1 ; one sector please
sta CFREG2
jsr cf_wait
cf_r0:
lda CFREG7
and #%00001000 ; check the DRQ bit
beq cf_r0 ; loop if not set
ldx #2 ; one sector, two pages - 512 bytes
cf_r1:
ldy #0
cf_r2:
lda CFREG0
sta (sector_ptr),y
iny
bne cf_r2
inc sector_ptr + 1 ; page finished; inc pointer
dex
bne cf_r1 ; get next page if required
; at this point, the buffer should be filled with data
; FIXME - add error check
cf_rx:
rts
Onwards and upwards!
Neil
Re: Adventures in FAT32 with 65c02
Posted: Thu Jan 08, 2026 10:03 pm
by barnacle
There's an offset of two between the entry in the FAT and the associated cluster. The _position_ of the FAT entry is directly related to the 'cluster number' but the first two FAT entries are reserved. The first cluster is therefore indexed by entry
2.
This is confusing.
I think that I am converting from FAT entry to cluster number (and therefore the LBA address of the first sector of that cluster) too late in the process, at least conceptually. The next_cluster value in an entry, if there is one, is in sector entries (which rather annoyingly, MS refer to as 'cluster numbers')
This is what a section of the FAT looks like prior to seeking the next free cluster:
Code: Select all
creating a directory
7ED8
7E00 F8 FF FF 0F FF FF FF 0F F8 FF FF 0F 04 00 00 00 ................
7E10 05 00 00 00 FF FF FF 0F 07 00 00 00 FF FF FF 0F ................
7E20 09 00 00 00 0A 00 00 00 FF FF FF 0F 0C 00 00 00 ................
7E30 0D 00 00 00 0E 00 00 00 FF FF FF 0F 10 00 00 00 ................
7E40 FF FF FF 0F 12 00 00 00 FF FF FF 0F 14 00 00 00 ................
7E50 15 00 00 00 16 00 00 00 17 00 00 00 18 00 00 00 ................
7E60 19 00 00 00 1A 00 00 00 1B 00 00 00 1C 00 00 00 ................
7E70 1D 00 00 00 1E 00 00 00 1F 00 00 00 20 00 00 00 ............ ...
7E80 21 00 00 00 22 00 00 00 23 00 00 00 24 00 00 00 !..."...#...$...
7E90 25 00 00 00 26 00 00 00 27 00 00 00 28 00 00 00 %...&...'...(...
7EA0 29 00 00 00 2A 00 00 00 2B 00 00 00 2C 00 00 00 )...*...+...,...
7EB0 2D 00 00 00 2E 00 00 00 2F 00 00 00 30 00 00 00 -......./...0...
7EC0 FF FF FF 0F FF FF FF 0F 33 00 00 00 34 00 00 00 ........3...4...
7ED0 FF FF FF 0F 00 00 00 00 00 00 00 00 00 00 00 00 ................
7EE0 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
Immediately before it is the calculated offset to the next free cluster - actually, that's the last allocated cluster, but it's where you should start looking for free clusters -
7ED8. That should be
7ED0; it's wrong because I took the value from the disk and assumed the wrong sort of cluster...
So I think I need to do another rewrite. Rather than saying the cluster that lives at cluster_begin_lba is
0 I need to call it cluster
2 and change things accordingly.
Oh well, here we go again.
Neil
Re: Adventures in FAT32 with 65c02
Posted: Fri Jan 09, 2026 5:02 am
by BigDumbDinosaur
There's an offset of two between the entry in the FAT and the associated cluster. The _position_ of the FAT entry is directly related to the 'cluster number' but the first two FAT entries are reserved. The first cluster is therefore indexed by entry 2.
This is confusing.
So I think I need to do another rewrite. Rather than saying the cluster that lives at cluster_begin_lba is 0 I need to call it cluster 2 and change things accordingly.
Oh well, here we go again.