Adventures in FAT32 with 65c02

[BigDumbDinosaur]

barnacle wrote:

(A thought on implementation: it would be _so_ much easier to work with complete clusters rather than individual sectors.  Unfortunately, they can be up to 32kB which rather restricts the number of files one might have open on a simple memory model system. Oh well.)

Suddenly RAM becomes as precious as water during a mission to Jupiter. 

Yep, it would be wonderful to load an entire cluster into core...if there was enough core.  I won’t say it can’t be done with the 65C02, but would likely be as difficult as enticing a rhino to perform dressage.  Even with the 65C816 and hundreds of Ks of RAM, it would be interesting to manage more than a few clusters at a time.  Also, the mass storage access times would be interesting.

Looks as though block-level access is your lot for now. 

[barnacle]

Yup. I'm planning two models: normal mode where file reads and writes happen - to the calling program - a byte at a time, and the hand-off between sectors and clusters is transparent, handled behind the scenes (ignore the man behind the curtain!).

The other approach is to use a predefined linearly allocated file, or more likely a second partition with _no_ file system as a flat 24-bit memory pool. Still a lot of thinking to do about that one, and to be of usable speed it would probably require a DMA system, but it would require only 32k sectors, and not a lot of arithmetic to convert a pointer to a sector.

Neil

[barnacle]

Well there's good news... before creating a directory

Code: Select all

AS65            60268                                                           
...                                                    
VARS.ASM        8960                                                            
PG6133.TXT      344217

and after creating it

Code: Select all

AS65            60268                                                           
...                                                      
VARS.ASM        8960                                                            
PG6133.TXT      344217
[DIRECTOR]        0

And there's bad news...

Looks like I got a pointer in the wrong place. Investigations continue.

Neil

[barnacle]

Oh rats! Just when I thought I'd got it sorted... more thinking to do, it seems. This should be an empty directory...

Code: Select all

barnacle@barnacle-Latitude-9430:/media/barnacle/19E5-4C34/DIRECT00$ dir
dir: reading directory '.': Input/output error
?╡\\\004⌡╛\035{.$¿å	  \031)É─\023äß,.\001\003\001  ~sxh(@â╡.\031W∩
0¼╠\ \024c\tΩ.└,f	  \f» gyº≈Γ.\n\022R	       v\002hφⁿl∩m.∩a┐
╜3o[T╧%\024		  I\n`≥\b`2ó.+z╜	       \002x\030f\004å8╕.╘\0179
áæ`ü\006\001@\021.\002√$  \036i⌠é\t╚Γ$.δ°φ	       ,\021δ.µ½╙
!;æ┐╒\037½_.6«≤		  k∩\n┤f░5\033.ê\035$	       εêCº\024\003¿Ç.\022╣w
>\004\002┬\003ææ╔.µ]║	  m■>╞─håx.ÿy╥		       ε\037╧▓╦«Lτ.▀╪t
≤'\033:█:æà.\fñ?	  \036Nj\033╗▒àÆ.«∩	       _µ*ä≥ü\b╙.\027"Æ
>â~┌û╗;r.╥═'		  ôñ?\021b\ πô.¬ûδ	       µVD\004@É\020.└¼ç
B\004\\\024V\022%@.ë█\r   ╒qäí\002ê[¿.\004┼╞	       \031\177\027π|«τz
c\bê$¿b╥ë.t\b\n		  qíë£≡µ¿a.\030;è	       τ∞·íτÖ∙╘.\022\tµ
&c\003p╦î(\035.`ª¡	  \002qî¡)÷t:.\027áB	       φ¼■~╟√╛è.éæ\r
barnacle@barnacle-Latitude-9430:/media/barnacle/19E5-4C34/DIRECT00$ 

Neil

[Martin_H]

barnacle wrote:

Oh rats! Just when I thought I'd got it sorted... more thinking to do, it seems. This should be an empty directory...

Well, at least the bug looks interesting. File systems are thorny beasts because subtle bugs and corruption are inevitable.

[BigDumbDinosaur]

barnacle wrote:

Oh rats! Just when I thought I'd got it sorted... more thinking to do, it seems. This should be an empty directory...

You sure that isn’t a directory listing from a Klingon file server? 

[barnacle]

Klingon files are not served. They force their way out, slaughtering all in their path.

I think it's working except for the bit that writes the new directory to disc; I think it's writing somewhere other than where it should.

Neil

[barnacle]

My head is really getting bent out of shape now; I cannot see what I am missing.

With a newly formatted disk, no files added, this is what I see from the various parameter blocks:

Code: Select all

NeoDOS32 v0.0411                                                             
VIB sector $00000800                                                         
FSI sector $00000801                                                         
Sectors in partition $000EA000                                               
Sectors per cluster $08                                                      
Free cluster count $0001D310                                                 
Next free cluster $00000002                                                  
FAT 1 sector $00000820                                                       
FAT 2 sector $00000BC8                                                       
Sectors per fat $000003A8                                                    
Start of clusters $00000F70                                                  
Root dir cluster $00000002 

Which is exactly right; the root directory lives at cluster 2 and so that's what's stored as the next free cluster. With everything empty, the start of the FAT table looks like this (in all these next images, ignore the addresses at the start of the line; that's the address of the transient buffer):

Code: Select all

7E00  F8 FF FF 0F FF FF FF 0F F8 FF FF 0F 00 00 00 00 ................          
7E10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................

Showing that (fake) clusters 0 and 1 are occupied by the formatting, and cluster 2 holds the root directory. That lives at $F70.
Cluster 2 at this point is empty, as required by the format rules; cluster 3 may contain previous data but is irrelevant as nothing in the FAT points to it.

Now I make a change. On the linux box, I copy a previously made directory into root (for simplicity: if a directory is created and renamed, it leaves a deleted file entry hanging around and it's not clear what it does with the old cluster. Also, a directory named in capitals and with fewer than eight characters does not create a long file name entry.) That gives me, as viewed from my code:

Code: Select all

Free cluster count $0001D30F                                                    
Next free cluster $00000003                                                     

- that is, the free cluster count has decremented and the next free cluster has incremented (as always, the 'next free cluster' always seems to mean 'last allocated cluster', so as it just allocated cluster 3...).
Now the start of the FAT looks like:

Code: Select all

7E00  F8 FF FF 0F FF FF FF 0F F8 FF FF 0F FF FF FF 0F ................          
7E10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................          

showing that cluster three is allocated.
Cluster two - the root directory - now includes a reference to the new directory:

Code: Select all

Cluster 2                                                                       
7E00  46 4F 4C 44 45 52 20 20 20 20 20 10 00 2D 12 49 FOLDER     ..-.I          
7E10  71 5C 71 5C 00 00 0C 49 71 5C 03 00 00 00 00 00 q\q\...Iq\......          

and the freshly allocated cluster 3 now has the dot and dotdot entries:

Code: Select all

Cluster 3                                                                       
7E00  2E 20 20 20 20 20 20 20 20 20 20 10 00 2D 12 49 .          ..-.I          
7E10  71 5C 71 5C 00 00 12 49 71 5C 03 00 00 00 00 00 q\q\...Iq\......          
7E20  2E 2E 20 20 20 20 20 20 20 20 20 10 00 2D 12 49 ..         ..-.I          
7E30  71 5C 71 5C 00 00 12 49 71 5C 00 00 00 00 00 00 q\q\...Iq\......          

Which is all well and good... but now, I start again with a newly formatted disk and attempt to create a directory in root. The initial state is as above; I won't repeat it. Cluster three contains the relics of the previous directory 'FOLDER', with the creating/access date of $5c71.

After creating the directory name in root, I get:

Code: Select all

7E00  44 49 52 45 43 54 30 30 20 20 20 10 00 00 D0 21 DIRECT00   ....!          
7E10  21 5C 21 5C 00 00 D0 21 21 5C 03 00 00 00 00 00 !\!\...!!\......          

(different name so I can see it changed, and a creation date of 5c21 (Jan 1 2026))
I then create the new directory structure (dot and dotdot entries) and write it to cluster three.
At this point nothing crashes...
When I look again at the sectors, I now see that the free cluster count and next free clusters are the same as when Linux moved the file,

Code: Select all

7E00  F8 FF FF 0F FF FF FF 0F F8 FF FF 0F FF FF FF 0F ................          
7E10  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................          

The FAT entries are correct (cluster three is now in use)

Code: Select all

Cluster 2                                                                       
7E00  44 49 52 45 43 54 30 30 20 20 20 10 00 00 D0 21 DIRECT00   ....!          
7E10  21 5C 21 5C 00 00 D0 21 21 5C 03 00 00 00 00 00 !\!\...!!\......          

Cluster 2 has the directory entry in root (datestamped 5C21)
And cluster 3 contains the dot and dotdot entries for the new folder (different time/date stamps than the linux version, as expected)

Code: Select all

Cluster 3                                                                       
7E00  2E 20 20 20 20 20 20 20 20 20 20 10 00 00 D7 21 .          ....!          
7E10  21 5C 21 5C 00 00 D7 21 21 5C 03 00 00 00 00 00 !\!\...!!\......          
7E20  2E 2E 20 20 20 20 20 20 20 20 20 10 00 00 D7 21 ..         ....!          
7E30  21 5C 21 5C 00 00 D7 21 21 5C 00 00 00 00 00 00 !\!\...!!\......   

So everything looks spot on. And yet, when I try and open the new file in linux...

Code: Select all

barnacle@barnacle-Latitude-9430:~$ cd /media/barnacle/AB17-FFB6/
barnacle@barnacle-Latitude-9430:/media/barnacle/AB17-FFB6$ dir
DIRECT00
barnacle@barnacle-Latitude-9430:/media/barnacle/AB17-FFB6$ cd DIRECT00/
barnacle@barnacle-Latitude-9430:/media/barnacle/AB17-FFB6/DIRECT00$ dir
dir: reading directory '.': Input/output error
$é╖!;£┐╒.\037½_     \002¿\025¡─\005æó.C\bê  iⁿó\t╚≥4δ.∙Φσ  [╔~}û┐÷╞.5f\022
3oKT╧$\024Ç.2P\001  \n°dp╩k{f.¿·\037	    ò!╕_┤ôδ⌐.k∩\n
A_E{Ö·╖\006.{╦\004  íëî°µ¿a8.;║2	    ß∙a█■ì]ì.@εî
barnacle@barnacle-Latitude-9430:/media/barnacle/AB17-FFB6/DIRECT00$ 

weird stuff happens. It looks as if it's reading some random part of the disk as if it were the directory cluster/sector, and I have exactly no idea why...

I can see the directory in /root. I can cd to that directory (and the tab extensions work to select it). But it ought to be reading directory '.' and it ain't. I'm assuming that the input/output error is something generic possibly relating to a pointer outside the partition; I am told that linux does check that. But it's still hinting that the contents of the directory are bent out of shape.

Ungood. Double plus ungood.

Neil

p.s. just checked: FAT1 and FAT2 are identical.

[BigDumbDinosaur]

barnacle wrote:

My head is really getting bent out of shape now; I cannot see what I am missing...

If I am correctly following you, you are formatting the CF card with FAT32 on your Linux box, creating a subdirectory named DIRECT00, and then attempting to read the root directory on your 6502 machine.  Is that correct?

We have to assume for now that when Linux formats the CF card to FAT32, it is doing it exactly to Micro$oft specs.  Disregarding the possibility of some sort of sneaky error resulting in a corrupted read from the CF card, the most like error cause is faulty arithmetic.  In particular, I’d be staring at the code to find a failure to correctly condition carry before doing multi-byte addition or subtraction—have had that bite me a number of times in the past.

Another possibility is an inadvertent endianess problem with a pointer.

This situation may be a good one for doing some “rubber-duck debugging.”  Perhaps you should “verbally” describe in general terms the exact sequence required to load and decode a directory, starting with how the LBAs for the blocks to be read from the CF card are determined, followed by the steps required to point to and fetch the bytes that are the directory.  Doing so might identify a procedural error.

BTW, the directory-read gibberish you posted looks different than the gibberish you had previously posted.  Sure smells like a wild pointer.

[barnacle]

BigDumbDinosaur wrote:

If I am correctly following you, you are formatting the CF card with FAT32 on your Linux box, creating a subdirectory named DIRECT00, and then attempting to read the root directory on your 6502 machine.  Is that correct?

Upside down, BDD. I'm trying to create DIRECT00 on a freshly formatted CF card using my 65c02 code, and comparing that with the cluster contents when I create a similar directory using linux. The linux version works; my version works perfectly right up to the moment I try and read it with linux.

They all look similar, but I too think there's a wild pointer floating around. Possibly what I _think_ of as cluster three is not actually cluster 'three'.

Neil

[barnacle]

Still confused... I have proven to my satisfaction that both procedures are using the same cluster # -> sector calculation.

• format partition (linux)
• copy directory into root (linux)
• copy file into new directory (linux)
• read file data (6502) - directory (cluster 3) shows dot, dotdot, and file entries (long file name, so there's a lot of them)
• format partition (linux)
• read file data (6502) - directory (cluster 3) shows dot, dotdot, and file entries - cluster is not cleared by formatting, just the cluster holding the root directory
• create directory in root (6502)
• read file data (6502) - directory (cluster 3) shows only dot and dotdot entries.

So both of us are using the same clusters to hold the directory data, and the cluster is correctly being cleared (by me) when creating the sub-directory's directory.

Here is the data in the root directory (cluster 2), Linux build, followed by the dot and dotdot entries in the new directory's directory (cluster 3)

Code: Select all

7E00  46 4F 4C 44 45 52 20 20 20 20 20 10 00 BB 32 47 FOLDER     ...2G           
7E10  72 5C 71 5C 00 00 37 47 72 5C 03 00 00 00 00 00 r\q\..7Gr\......           
	
7E00  2E 20 20 20 20 20 20 20 20 20 20 10 00 BB 32 47 .          ...2G           
7E10  72 5C 72 5C 00 00 32 47 72 5C 03 00 00 00 00 00 r\r\..2Gr\......           
7E20  2E 2E 20 20 20 20 20 20 20 20 20 10 00 BB 32 47 ..         ...2G           
7E30  72 5C 72 5C 00 00 32 47 72 5C 00 00 00 00 00 00 r\r\..2Gr\......           

And the same for the 6502 process.

Code: Select all

	
7E00  44 49 52 45 43 54 30 30 20 20 20 10 00 00 9B 11 DIRECT00   .....           
7E10  21 5C 21 5C 00 00 9B 11 21 5C 03 00 00 00 00 00 !\!\....!\......           

7E00  2E 20 20 20 20 20 20 20 20 20 20 10 00 00 A4 11 .          .....           
7E10  21 5C 21 5C 00 00 A4 11 21 5C 03 00 00 00 00 00 !\!\....!\......           
7E20  2E 2E 20 20 20 20 20 20 20 20 20 10 00 00 A4 11 ..         .....           
7E30  21 5C 21 5C 00 00 A4 11 21 5C 00 00 00 00 00 00 !\!\....!\......  

for info, the offsets within a directory entry are

Code: Select all

dir_name		equ 0		; eleven chars, space filled
dir_attrib		equ b		; one byte
dir_ntres		equ c		; one byte - reserved, set to zero
dir_timehun		equ d		; one byte - 0-199 hundredths of two secs
dir_timecrt		equ e		; two bytes - time file was created
dir_datecrt		equ 10		; two bytes - date file was created
dir_dateacc		equ 12		; two bytes - date last accessed
dir_frstclhi	equ 14		; two bytes - high word of first cluster
dir_timewrt		equ 16		; two bytes - time file was last written
dir_datewrt		equ 18		; two bytes - date time was last written
dir_frstcllo	equ 1a		; two bytes - low word of first cluster
dir_size		equ 1c		; four bytes

I'm wondering if the issue is anything to do with the creation time of the dot/dotdot entries not matching the time on the root directory entry? That's a few seconds earlier because I have a pile of debug/waitkey events during creation. Hmm.

Neil

[barnacle]

Nope... wasn't that. Though I did discover previous testing on the timer interrupt had left it running at 50 times normal speed...

Neil

[BigDumbDinosaur]

barnacle wrote:

BigDumbDinosaur wrote:

If I am correctly following you, you are formatting the CF card with FAT32 on your Linux box, creating a subdirectory named DIRECT00, and then attempting to read the root directory on your 6502 machine.  Is that correct?

Upside down, BDD. I'm trying to create DIRECT00 on a freshly formatted CF card using my 65c02 code, and comparing that with the cluster contents when I create a similar directory using linux. The linux version works; my version works perfectly right up to the moment I try and read it with linux.

They all look similar, but I too think there's a wild pointer floating around. Possibly what I _think_ of as cluster three is not actually cluster 'three'.

Gotcha.

Have you tried dumping some of the CF card’s raw content to a file after having creating the test directory in Linux and comparing that file to an identical dump after you create the test directory on your 6502 contraption?  Since the files’ content will be binary stuff, diff will likely choke on them, so you’d have to write a simple C program to do a byte-by-byte comparison of the two files.  Surely that would pinpoint the (apparent) structural problem in what your 6502 machine is generating.

[barnacle]

That's exactly my next thought, though I think I shall reduce the size of the partition to the minimum that will format as FAT32, to keep the size manageable.

Turns out that trying to use hexedit on multi-megabyte files is, um, a bit tedious.

Neil

[BigDumbDinosaur]

barnacle wrote:

That's exactly my next thought, though I think I shall reduce the size of the partition to the minimum that will format as FAT32, to keep the size manageable.

You can use dd with the count= option to dump only enough from the CF card to encompass the area of interest.  That way you can continue to partition to the full extent of the device.  If you shrink the partition, you will be changing test conditions.

Quote:

Turns out that trying to use hexedit on multi-megabyte files is, um, a bit tedious.

That’s kind of like saying it’s a long trip to get to Jupiter.