Page 12 of 14

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 6:42 am
by barnacle
Heh. My late mother, who was born in Yorkshire but moved to Scotland forty years ago, could famously spend a penny twice and still have change!

Neil

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 8:21 am
by BigDumbDinosaur
barnacle wrote:
And it gripes my economical Yorkshire soul to stick a megabyte in when I only need a few K :)
“Economical Yorkshire soul?”  Why would you use a five-syllable word when a one-syllable word that sounds like what birds say, and begins with the letters C and H would suffice?  :mrgreen:

For what it’s worth, I have a reputation around here (at home) that makes Ebeneezer Scrooge seem like a free-spending, inebriated sailor in comparison.  :shock:

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 8:28 am
by BigDumbDinosaur
drogon wrote:
And for those not in the UK, the definition of a Yorkshireman is along the lines of "a Scotsman with the sense of generosity removed" ... ;-)
My buddy Tom from my high school days was Scottish—his parents emigrated from Scotland in the 1930s and years later, still had that distinct “burr” when they talked.

Anyhow, Tom always claimed his old man was a cheapskate and had invented the limbo trying to get into a pay toilet.  :D

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 2:52 pm
by kakemoms
BigDumbDinosaur wrote:
EEPROMs are subject to wear-out failures.  If your parts are pulls, it’s anybody’s guess as to how many erase/program cycles they have endured.  Also, they may have been exposed to some ESD when they were being salvaged.

Incidentally, I have found that UV-erasable EPROMs wear out as well.  They will erase okay, but will end up with one or more stuck bits.  Over time, my large stock of EPROMs has shrunk as individual devices have failed verification after programming.  When that happens, I set the bad part aside with others and later subject all of them to a longer-than-normal erase cycle, followed by a test—almost always, they will fail again.  I have had some of these EPROMs for 20+ years, so I figure I’ve gotten my money’s worth out of them.
The aging of EPROMs is an interesting topic. Unlike PROMs that use fuses (metal or another conductive layer), EPROMs store data as charge on floating gates. That makes them ESD-sensitive and subject to gradual data loss over time. Permanent damage appears to be related to degradation of the gate oxide, which can eventually become conductive and cause complete failure. As a semiconductor engineer, I would expect such defects to reduce retention lifetime gradually rather than show up as sudden single-bit failures. There is some useful information on floating-gate degradation here.

As a side note, I have not been able to find any EPROM datasheet or technical note that specifies endurance (number of programming cycles). That is common for EEPROM, but apparently not for UV-erasable EPROMs. The reason for this is open to speculation.

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 3:48 pm
by barnacle
That was my understanding, too. What I'm seeing is that an eeprom requires multiple attempts to make the program 'stick' and execute on my test board.

In theory, my code does a verification stage, but since I no longer have it available, I can't check that. And the fact that it doesn't throw an error when there is no chip present suggests rather that it doesn't...

It's odd, though, that changing to another 'new' eeprom part makes things work again.

Perhaps I need to upgrade my programmer. It looks like the TL866/T48 is the way to go for something that works, but Linux might be problematical there - perhaps Wine? Or get designing again :)

Neil

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 5:54 pm
by GARTHWILSON
kakemoms wrote:
As a side note, I have not been able to find any EPROM datasheet or technical note that specifies endurance (number of programming cycles). That is common for EEPROM, but apparently not for UV-erasable EPROMs. The reason for this is open to speculation.
I vaguely remember the number 100 erase/program cycles from decades ago; so when I've been developing and keeping the eraser and programmer busy in a rotation, I've retired the EPROMs after perhaps a little over 100 cycles.  They were also only guaranteed to hold their data for ten years; but I've never seen a failure, either from time or from cycles.  My Datarase II eraser's manual said to see how many minutes it takes for the EPROM to start looking erased, and then double or quadruple (I don't remember) that amount of time.  The programmer I was using had a few choices for algorithm, and I think I was using one that worked for nearly everything, which might not have been quite as fast as "quick pulse," but would give pulses, 1ms each IIRC, continually checking, and then when the byte read back correctly, the programmer added twice that many pulses again, before moving on to the next location.  Then of course it would read the whole thing again at the end to make sure the programming of one address didn't change the content of another.  I've never had an EPROM failure.

Edit:  I said I haven't had any failures; but I will add that on things that were important, I've refreshed the EPROMs every dozen years or so.  I have one gizmo I got 40+ years ago that I only recently refreshed the EPROMs for the first time, and kept images on hard disc as backup.

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 6:03 pm
by barnacle
I've had one, in an ECU. I was able to copy another chip to an eeprom, and that's been working ten or more years. The original chip failed at around fifteen-twenty years, with regular usage.

Neil

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 6:12 pm
by BigDumbDinosaur
barnacle wrote:
Perhaps I need to upgrade my programmer.  It looks like the TL866/T48 is the way to go for something that works, but Linux might be problematical there - perhaps Wine?  Or get designing again :)
I have Xgecu’s T56 unit, which seems to be able to program just about anything.  For example, it works fine programming Atmel’s ATF750 GAL.  It also comes with some adapters for programming PLCC devices, such as the 39SF010 flash I will be using in my next POC unit.

From my perspective, it’s okay to use bricks made by someone else to build a new house.  Dunno about you, but my time has become very valuable—being an octogenarian, there isn’t much of it left.  When it comes to the decision between designing and building a piece of gear to support my hobby versus purchasing that piece of gear, I tend to lean toward the latter.  :D

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 14, 2026 6:24 pm
by BigDumbDinosaur
kakemoms wrote:
The aging of EPROMs is an interesting topic...As a semiconductor engineer, I would expect such defects to reduce retention lifetime gradually rather than show up as sudden single-bit failures.
I’ve also had some that were having retention problems...things would work for a while and then they wouldn’t, this starting to happen in the recent past.  Once I saw a pattern to it, I would pull the EPROM and run a verify against the original file from which it was programmed (I archive this stuff...just in case).  The check would always fail at some random address.

My knowledge of the internal aspects of semiconductor technology is light, but I understand enough to know how a failure comes about.

Interestingly, the PLA in the Commodore 64 was noted for having a retention problem as it aged.  When the PLA developed amnesia, the machine usually wouldn’t do anything when powered on.

Re: Adventures in FAT32 with 65c02

Posted: Mon Feb 23, 2026 8:51 am
by barnacle
After some diversion I believe I'm a little further into the 'interesting' subject of cluster allocation.

MS say fsi_next_cluster is the place to start looking; they don't explicitly comment in its value save to note it might not be accurate :shock: and nor do they give any algorithm to select a new cluster. As far as I have been able to observe, it actually contains the number of the last cluster allocated, which is useful.

(It occurs to me that I have not checked, but on a newly formatted but empty disk the only allocated cluster is the root directory's, so that should be '2' on a healthy disk. Other numbers are available in the case of the formatting deciding '2' is a bad sector.)

I have chosen to deallocate clusters at the time of deleting a file, but always allocating linearly starting with the first available cluster after fsi_next_cluster. That will most of the time be the next cluster, but you can't assume that; you have to look.

As the disk fills, the cluster location in the FAT increases... until you get to the end of the FAT. Then you have to start again at the beginning and hope that files have been deleted in the meantime, leaving holes in the FAT. This means that one can check for space in a single pass because the FATs are always adjacent:
  • Start at fsi_next_cluster
  • If the value of its record is not zero, increment to the next FAT record
  • If you've reached the end of the second FAT, there is no more space, oops!
  • If it is zero, save this record count as fsi_next_cluster
  • Otherwise, back to the top for the next record
Doing it this way means that most of the time, you will simply move to the next FAT record and find it available. If you reach the end of the first FAT, then you hope that earlier files have been deleted and their clusters deallocated. By carrying on to the end of the second FAT - a copy of the first - then you automatically check the older records which may have been cleared.

For sanity's sake I keep copies of fsi_next_cluster and fsi_free_count in memory, and rewrite them after each allocation. At the last test - because it's very easy to break the entire filing system if I get something wrong - everything appears fine except I somehow managed to lose fsi_next_cluster, which is a bit embarrassing, but I think I've fixed that. As always, keeping track of multiple temporary 32-bit variables, and which sector you have in transient, and whether you're dealing with clusters, sectors, or sector numbers, is a nightmare.

Testing continues.

Neil

Re: Adventures in FAT32 with 65c02

Posted: Mon Feb 23, 2026 4:15 pm
by barnacle
And after a lot of printf() style debugging, I have chased down the faulty calculation of the new sector and get it back as expected.

At least, on a nearly empty but large disk. I need to do more testing with a full disk with some earlier deletions.

Here are the two critical pointers on startup:

Code: Select all

Free cluster count $0001E553                                                    
Next free cluster $000000B0  
and the newly allocated cluster, properly marked as an end-of-cluster, at $7EC4

Code: Select all

7EB0  AD 00 00 00 AE 00 00 00 AF 00 00 00 B0 00 00 00 ................          
7EC0  FF FF FF 0F FF FF FF 0F 00 00 00 00 00 00 00 00 ................          
and finally the data to send back to the VIB sector, at $7FE8 and $7FEC

Code: Select all

7FE0  00 00 00 00 72 72 41 61 52 E5 01 00 B1 00 00 00 ....rrAaR.......          
7FF0  00 00 00 00 00 00 00 00 00 00 00 00 00 00 55 AA ..............U.
(those two are in the same buffer but not at the same time!)
The free cluster count has decremented, and the next free cluster is now at $000000B1. It's a long way to go to increment one uint32 and decrement another, but I suppose we have to suffer for our art!

Speaking of suffering:

Code: Select all

;-----------------------------------------------------------------------
; f_alloc
; Allocate a new cluster if one is available.
; Return new cluster number, or E_NO_SPACE if disc is full
; New cluster is returned in fsi_next_free
f_alloc:
	LYA fsi_free_count
	jsr u32_iszero
	bne f_all_01
		; disc is full
		lda #E_NO_SPACE
		jmp f_all_x
f_all_01:
	; free count is not zero, so seek next cluster
	; start looking at fsi_next_free
	LYA fsi_next_free
	jsr u32_tolba			; cluster is in lba
	jsr clus_to_fat_sector	; replace cluster # with the fat 1 sector
	LYA tmpq1
	jsr u32_fromlba			; tmpq1 holds address of fat 1 sector
							; but lba still holds it
	LYA transient
	jsr cf_read				; get the FAT sector into transient
	lda fsi_next_free		; pass the low byte of cluster
	and #$7f				; there are 128 records in a sector
f_all_05:
	tax						; save record count
	jsr clus_to_fat_rec		; fat_record points to record in transient
	ldy #1
	lda (fat_record)
	ora (fat_record),y
	iny
	ora (fat_record),y
	iny
	ora (fat_record),y		; is this a zero?
	beq f_all_10			; yes!
	; if non-zero, we have to try the next location
	inx
	txa
	bpl f_all_05			; 128 records per sample
		; else we need a new sector. By checking for zeros until either
		; we find one, or we get to the cluster start, we first check
		; for empty space at the end of the disk and if that fails, at
		; the beginning (finding the oldest deallocated cluster first).
		LYA tmpq1				; increment the fat sector
		jsr u32_inc
		jsr u32_tolba			; into lba
		LYA cluster_begin_lba
		jsr u32_cmp				; are we at the clusters yet?
		bne f_all_06			; no, carry on looking
			; else disc is full
			lda #E_NO_SPACE
			jmp f_all_x
f_all_06:
		LYA tmpq1
		jsr u32_tolba
		LYA transient
		jsr cf_read				; read this next FAT sector
		bra f_all_05			; and back for more
f_all_10:
	; if we get here, then we have found a zero.
	; first we calculate the new cluster number. We need to know which 
	; FAT we're in
	; tmpq1 contains the sector and fat_record the offset in transient
	PRINT "Found a zero "
	LYA fat_begin_lba
	jsr u32_tolba
	LYA sectors_per_fat
	jsr u32_add					; the start of the second FAT
	LYA tmpq1					; check against current FAT
	jsr u32_cmp					; are we less than that?
	php
	LYA tmpq1
	jsr u32_tolba				; put the FAT sector in lba
	plp							; retrieve the flags from the comparison
	bcs f_all_12
		; ah, we're in FAT 2
		LYA sectors_per_fat
		jsr u32_sub					; so move back to FAT1	
f_all_12:	
	; convert the current fat sector and fat_record to the cluster
	lda fat_record
	pha
	lda fat_record+1			; save sector_ptr; pointer_to_clus
	pha							; thumps it	
							
	jsr pointer_to_clus
	LYA fsi_next_free			; save the result as next free cluster
	jsr u32_fromlba
	; change the fat entry to end-of-cluster
	; we currently have the address of the FAT 1 sector in lba
	pla
	sta fat_record+1
	pla
	sta fat_record				; retrieve sector_ptr 
	ldy #0
	lda #$ff					; set record to end-of-chain
	sta (fat_record),y
	iny
	sta (fat_record),y
	iny
	sta (fat_record),y
	iny
	lda #$0f
	sta (fat_record),y			; record = $0fffffff
	; transient should now have the modified chain, so get it back in
	; the two FAT copies
	SHOWTRANS
	LYA transient
	;jsr cf_write				; FAT 1
	LYA sectors_per_fat
	jsr u32_add
	LYA transient
	;jsr cf_write				; FAT 2
	;but there's still stuff to do
	sec							; decrement fsi_free_count
	lda fsi_free_count
	sbc #1
	sta fsi_free_count
	lda fsi_free_count+1
	sbc #0
	sta fsi_free_count+1
	lda fsi_free_count+2
	sbc #0
	sta fsi_free_count+2
	lda fsi_free_count+3
	sbc #0
	sta fsi_free_count+3
	LYA fsi_sector
	jsr u32_tolba
	LYA transient
	jsr cf_read					; load FSI to transient
	LYA fsi_free_count
	jsr u32_tolba
	LYA free_count_ptr
	jsr u32_fromlba				; set updated free count
	LYA fsi_next_free
	jsr u32_tolba
	LYA next_free_ptr
	jsr u32_fromlba				; and updated next pointer
	LYA fsi_sector				; get the FSI sector back
	jsr u32_tolba
	SHOWTRANS
	LYA transient
	;jsr cf_write				; and write it to disk
	lda #E_NOERR				; ASSUME it worked

f_all_x:
	rts
I have not yet enabled actually writing the modified sector to the disk.

Neil

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 28, 2026 9:11 am
by barnacle
With the full disk writes now enabled, it appears that I am correctly assigning the correct metadata to sundry FATs and VIBs. So I can allocate a cluster and mark it as in use (though it's not attached to any file, that's still a correct but pointless activity since if not allocated at the time there's now way to know which, if any, clusters are allocated but unused without walking the entire directory structure).

Repeated execution selects a new cluster, decrements the cluster available count, and increments the next cluster address, as expected.

There's a minor change in the code in the previous post, and lots of debug stuff comment out for later removal (MISRA would have it that you should never have comment-out code in your source, but tough...)

Code: Select all

f_alloc:
	LYA fsi_free_count
	jsr u32_iszero
	bne f_all_01
		; disc is full
		lda #E_NO_SPACE
		jmp f_all_x
f_all_01:
	; free count is not zero, so seek next cluster
	; start looking at fsi_next_free
	LYA fsi_next_free
	jsr u32_tolba			; cluster is in lba

	jsr clus_to_fat_sector	; replace cluster # with the fat 1 sector
	LYA tmpq1
	jsr u32_fromlba			; tmpq1 holds address of fat 1 sector
							; but lba still holds it
	LYA transient
	jsr cf_read				; get the FAT sector into transient

	lda fsi_next_free		; pass the low byte of cluster
	and #$7f				; there are 128 records in a sector
f_all_05:
	tax						; save record count
	jsr clus_to_fat_rec		; fat_record points to record in transient
	ldy #1
	lda (fat_record)
	ora (fat_record),y
	iny
	ora (fat_record),y
	iny
	ora (fat_record),y		; is this a zero?
	beq f_all_10			; yes!
	; if non-zero, we have to try the next location
	inx
	txa
	bpl f_all_05			; 128 records per sample
		; else we need a new sector. By checking for zeros until either
		; we find one, or we get to the cluster start, we first check
		; for empty space at the end of the disk and if that fails, at
		; the beginning (finding the oldest deallocated cluster first).
		LYA tmpq1				; increment the fat sector
		jsr u32_inc
		jsr u32_tolba			; into lba
		LYA cluster_begin_lba
		jsr u32_cmp				; are we at the clusters yet?
		bne f_all_06			; no, carry on looking
			; else disc is full
			lda #E_NO_SPACE
			jmp f_all_x
f_all_06:
		LYA tmpq1
		jsr u32_tolba
		LYA transient
		jsr cf_read				; read this next FAT sector
		bra f_all_05			; and back for more
f_all_10:
	; if we get here, then we have found a zero.
	; first we calculate the new cluster number. We need to know which 
	; FAT we're in
	; tmpq1 contains the sector and fat_record the offset in transient
	;PRINT "Found a zero "
	LYA fat_begin_lba
	jsr u32_tolba
	LYA sectors_per_fat
	jsr u32_add					; the start of the second FAT
	LYA tmpq1					; check against current FAT
	jsr u32_cmp					; are we less than that?
	php
	LYA tmpq1
	jsr u32_tolba				; put the FAT sector in lba
	plp							; retrieve the flags from the comparison
	bcs f_all_12
		; ah, we're in FAT 2
		LYA sectors_per_fat
		jsr u32_sub					; so move back to FAT1	
f_all_12:	
	; convert the current fat sector and fat_record to the cluster
	lda fat_record
	pha
	lda fat_record+1			; save sector_ptr; pointer_to_clus
	pha							; thumps it	
							
	jsr pointer_to_clus
	LYA fsi_next_free			; save the result as next free cluster
	jsr u32_fromlba
	; change the fat entry to end-of-cluster
	; we currently have the address of the FAT 1 sector in lba
	pla
	sta fat_record+1
	pla
	sta fat_record				; retrieve sector_ptr 
	ldy #0
	lda #$ff					; set record to end-of-chain
	sta (fat_record),y
	iny
	sta (fat_record),y
	iny
	sta (fat_record),y
	iny
	lda #$0f
	sta (fat_record),y			; record = $0fffffff
	; transient should now have the modified chain, so get it back in
	; the two FAT copies
	LYA tmpq1				; <<<<<<<<<<<<<<<<<
	jsr u32_tolba				; <<<<<<<<<<<<<<<<<
	LYA transient
	jsr cf_write				; FAT 1
	LYA sectors_per_fat
	jsr u32_add
	LYA transient
	jsr cf_write				; FAT 2
	;but there's still more stuff to do
	sec							; decrement fsi_free_count
	lda fsi_free_count
	sbc #1
	sta fsi_free_count
	lda fsi_free_count+1
	sbc #0
	sta fsi_free_count+1
	lda fsi_free_count+2
	sbc #0
	sta fsi_free_count+2
	lda fsi_free_count+3
	sbc #0
	sta fsi_free_count+3
	LYA fsi_sector
	jsr u32_tolba
	LYA transient
	jsr cf_read					; load FSI to transient
	LYA fsi_free_count
	jsr u32_tolba
	LYA free_count_ptr
	jsr u32_fromlba				; set updated free count
	LYA fsi_next_free
	jsr u32_tolba
	LYA next_free_ptr
	jsr u32_fromlba				; and updated next pointer
	LYA fsi_sector				; get the FSI sector back
	jsr u32_tolba
	LYA transient
	jsr cf_write				; and write it to disk
	lda #E_NOERR				; ASSUME it worked
f_all_x:
	rts
Now it's time (again!) to look at creating a directory. That's quite complicated, but we have most the parts in place.

Neil

Re: Adventures in FAT32 with 65c02

Posted: Sat Feb 28, 2026 11:13 am
by barnacle
As an essential aside, I'm slowly also changing code so that it's all referenced to the current working directory, so that everything becomes part of a directory tree.

Earlier, I showed fs_find_first and fs_find_next which required the caller to provide a sector at the start of the active cluster; now it directly reads the cwd variable and organises the sector for itself:

Code: Select all

fs_find_first:
	LYA cwd
	jsr u32_tolba			; cluster of current working directory
	jsr clus_to_1st_sec
	LYA transient			; set the current directory
	jsr cf_read				; and load the first sector
	stz fs_dir_number		; start with record zero
	stz fs_dir_sector		; and the first sector
That, of course, required that the cwd variable be set properly, and that's controlled by an int32 in the VIB:

Code: Select all

	LYA rootclus_ptr
	jsr u32_tolba
	LYA root_clus
	jsr u32_fromlba				; save the root cluster
	LYA cwd
	jsr u32_fromlba				; and also to cwd
Most of that time it's going to be '2' but you never know.

* there are probably other places in the code that still need to change! No doubt they'll make themselves known...

Neil

Re: Adventures in FAT32 with 65c02

Posted: Sun Mar 01, 2026 9:48 pm
by barnacle
Creating a sub-directory
After fasting and sitting naked on a mountain-top engaging in deep meditation, our hero is feeling a little light-headed (and not a little chilly), but feels he may have achieved enlightenment on the subject of directories. He has decided he is too old to call them 'folders'.

Here's the general idea: file creation and deletion operate only in the current working directory, and so should directories. That is, creating a directory doesn't care what the CWD is; it will create the directory as a child of the CWD (as an aside, I'm going to have to write some code to move around in the directory tree).

The process is simply described:
  • Allocate a new cluster. If things fail after this point, then a cluster has been wasted, but the disk remains a valid disk though the cluster cannot be recovered without either reformatting the disk, or some very heavy and detailed disk analysis. If the cluster can't be allocated, of course, we're stuck; the disk is full. Otherwise, the new cluster number remains in fsi_next_free.
  • The newly allocated cluster must be cleared to zero. This can be done by filling transient with zeros, calculating the first sector of the cluster, and then writing that to the CF, incrementing sector LPA address for each sector_per_cluster.
  • Search for the filename (fs_find_file()) to ensure if doesn't already exist: if it does, exit with an error.
  • Otherwise, we have the requisite directory sector in transient and fs_dir_ptr is pointing to the start of a directory entry structure. So we can fill that in, and set the attribute byte to say its a directory. We set the size to zero, all the dates and times to now, and fill in the cluster number from fsi_next_free. That gives us a valid directory entry for the new directory, but we're not finished yet.
  • We have to create two entries in the new directory: '.' and '..' which have size zero, the current date and time, and cluster pointers to fsi_next_free (i.e. this directory) and cwd (i.e. the parent directory). There's a slight gotcha here: if the parent is the root directory, then the '..' entry should point to zero...
    Using the existing f_create() would require us to fill in the str_83 filename string; normally this is done from a string but this would break on either of these entries. Instead, I fill in the names directly in the first two entries of the new cluster, and take it from there.
When navigating through the tree, it can be handy (for a user prompt, perhaps) to use the name of the current working directory. At some depth in the tree, one can read the cwd variable to find the cluster of the directory; from there one can read its filename entry and display it. But you can't do that with the root directory: it doesn't have a parent. Using zero as the pointer indicates that you are at root, irrespective of which cluster the root directory happens to start at. Though to find that, you have to go to the cluster in root_clus, which was filled in during initialisation.

Code for this is written - first draft - but not yet tested. It's also missing that little detail about root directory pointers.

Neil

Re: Adventures in FAT32 with 65c02

Posted: Sun Mar 01, 2026 9:50 pm
by barnacle
(A thought on implementation: it would be _so_ much easier to work with complete clusters rather than individual sectors. Unfortunately, they can be up to 32kB which rather restricts the number of files one might have open on a simple memory model system. Oh well.)