6502.org Forum  Projects  Code  Documents  Tools  Forum
It is currently Fri Nov 22, 2024 5:03 am

All times are UTC




Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: Tue Jan 07, 2020 7:48 pm 
Offline

Joined: Sun May 13, 2018 5:49 pm
Posts: 255
After helping scotws get Tali Forth 2 up to version 1.0, I decided my next adventure was to start using Forth. As it turns out, writing software in forth is quite a bit different than writing forth itself (in assembly), so it's been a good adventure so far. I know I enjoy reading some of the other folk's musings on their forth projects, even if I don't always say anything in their thread, so I figured I'd start a thread to document my progress. I'm currently working on FAT32 support for my SBC.

*caution* - I tend to be verbose. Skip to the bottom for the short version.

I added a CF card to my SBC and talk to it in 8-bit IDE mode. It connects directly to the data/address bus, and my address decoding GAL already had a spare select line as well as having separate active low *RD and *WR signals which I was planning to use with some x86 type chips (that I ended up not using). This was all that was required to get the hardware working. I used http://www.waveguide.se/?article=8-bit- ... -interface and the pinout for an IDE cable (I'm using a compact flash to IDE adapter) to get everything hooked up.

I've been implementing words for locating and reading files from a FAT32 partition (working from information found at https://www.pjrc.com/tech/8051/ide/fat32.html). I have it working enough that I can get a listing of the root directory (although my code should work with any directory if you know the starting cluster#) and can print a file to the screen.

I have since refactored my code twice and it's only just starting starting to look "forthy". I'm finally able to just "read" forth the same way I read C. I'm also starting to be able to see when/where words should be decomposed and I now seem to get the stack arguments in the right order for my new words (there is definitely a wrong order). When I started writing the code to locate a particular file name, I realized that it looked a lot like the directory listing I just wrote, so I was able to refactor that code to be reusable for both purposes.

My favorite thing so far is that, when having trouble with a particular word, I can just paste lines of its definition into forth and see what happens on the real hardware. .S and DUMP are very powerful to help with this. I can also try several solutions and pick the one I like best. In Brodie's "Thinking Forth", there's a quote attributed to Dr. Michael Starling that says "You don't know completely what you're doing till you've done it once. In my own experience, the best way to write an application is to write it twice. Throw away the first version and chalk it up to experience." I totally understand that now (although it takes me 3-4 tries due to lack of experience).

I have a hobby machine shop, and I find using Forth to be very similar. Sometimes I start a project not knowing what tools I will need, and when I get to where I need to use a tool I don't own, I will usually make it (which often takes longer than the original project I was working on). Sometimes I have to make the tool a couple of times (due to unforeseen (or sometimes ignored) circumstances or lack of skill), but then I have that tool for the next time I need it. Forth feels like it's the same way. I don't know what words I will be using or creating when I start a project, and it's a series of little puzzles along the way.

The short version:

I'm currently in the process of rewriting my FAT32 code to support multiple files open at the same time, so things are a little unstable at the moment. Once I get things a little more stable, I will post my code for others to look at/comment on/use. To get started, here is my code for reading sectors from the compact flash. I haven't implemented writing yet. I will likely split this up into smaller words once I know what is in common between reading and writing.

Code:
\ PROGRAMMER  : Sam Colwell
\ DATE        : 2020-01
\ DESCRIPTION : Compact Flash card support using 8-bit interface
\ LICENSE     : CC-BY 4.0

hex
7F40 constant cf.base ( Change this for your hardware )
cf.base constant cf.data
cf.base 1+ constant cf.feature ( Error Register when read )
cf.base 2 + constant cf.sectorcount ( Config for SetFeatures command )
cf.base 3 + constant cf.lba ( low byte )
\ A note about the LBA address - it's 28 bits, but we can't use a
\ double in Tali to store it because the two 16-bit halves are in the
\ wrong order in Tali.  We CAN use two regular 16-bit cells because
\ they are little endian like the compact flash card expects for the
\ LBA #, but will have to transfer them separately.  I will likely
\ use a double and then split it to load the LBA.
cf.base 6 + constant cf.drivehead
( bit 6 : set to 1 = LBA mode )
cf.base 7 + constant cf.status ( Command Register when written )
( bit 7 : 1 = BUSY )
( bit 6 : 1 = RDY [ready] )
( bit 3 : 1 = DRQ [data request] data needs to be transferred )

: cf.busywait begin cf.status c@ 80 and 0= until ;

: cf.init ( -- ) ( Initilize the CF card in 8-bit IDE mode )
   \ Reset the CF card.
   04 cf.status c!
   cf.busywait
   \ Set feature 0x01 (8-bit data transfers)
   01 cf.feature c!
   EF cf.status c!
;

: cf.read ( addr numsectors LBA.d -- )
   ( Read numsectors starting at LBA [double] to addr )
   \ Treat LBA double as two 16-bit cells.
   \ MSB is on top.  We can write this as a 16-bit cell to
   \ cf.lba+2, however we need to make sure we don't screw
   \ up the Drive and LBA bits (in the MSB)
   EFFF and ( make sure Drive is zero )
   E000 or  ( make sure LBA and reserved bits are 1 )
   cf.lba 2 + ! ( store high word of LBA )
   cf.lba !     ( store  low word of LBA )
   \ stack is now just ( addr numsectors )
   \ Tell the CF card how many sectors we want.
   dup cf.sectorcount c!
   
   \ Send the command to read (0x20)
   20 cf.status c!
   cf.busywait
   
   \ stack is still ( addr numsectors )
   \ Read all of the sectors.
   0 ?do
      \ Read one sector - 0x200 or 512 bytes
      200 0 do
         \ Read one byte.
         cf.data c@ over c!
         \ Increment the address.
         1+
      loop
      cf.busywait \ Wait for next sector to be available.
   loop
   drop ( the address )
;


Top
 Profile  
Reply with quote  
PostPosted: Wed Jan 08, 2020 2:10 pm 
Offline

Joined: Wed Jan 08, 2014 3:31 pm
Posts: 578
Neat project. I'm looking forward to how it turns out.

As far as Forth vs assembler. I sort of think of Forth as an assembler with one heck of a macro facility.


Top
 Profile  
Reply with quote  
PostPosted: Wed Jan 08, 2020 5:34 pm 
Offline

Joined: Sun May 13, 2018 5:49 pm
Posts: 255
Martin_H wrote:
As far as Forth vs assembler. I sort of think of Forth as an assembler with one heck of a macro facility.

In the case of Tali Forth 2, that's actually quite accurate as it supports native compiling.


Top
 Profile  
Reply with quote  
PostPosted: Wed Jan 08, 2020 5:43 pm 
Offline

Joined: Sun May 13, 2018 5:49 pm
Posts: 255
Here is my code for locating the FAT32 partition. It expects an MBR type boot sector and inspects the 4 primary partitions for one that could be FAT32 by inspecting the partition type (looking for $0B or $0C). If located, it prints a message and then loads the first sector of the FAT partition (called the Volume ID) and sets all of the variables needed for using this partition. This code assumes there is only one FAT32 partition and will use the last one it finds after checking all 4 possible primary partitions.

Because Tali2 uses NUXI format in memory for its doubles (thanks barrym for a name for that format), I had to make my own 32bit@ that can properly fetch doubles in little endian format. Once they are in memory, I keep everything in Tali2s native format so that Tali's built-in (and added) double words work on them.

Some of these words are kind of long, but they only run once and I don't see myself reusing the code so I did not factor them any further.

I do plan on upgrading the root directory variable to use a structure that has extra fields to keep track of where in the directory we are during a directory scan.

Code:
\ Continued from previous code listing...
\ fat variables
create sectorbuff 512 allot
2variable sector# \ The disk sector currently in the buffer

2variable partition_lba_begin
2variable fat_begin_lba
2variable cluster_begin_lba
variable sectors_per_cluster
2variable root_dir_first_cluster

\ file access variables
create filename 0B allot  \ Name of current file
2variable current_cluster \ The cluster we are on right now
2variable filesize        \ The size of the file
2variable fileoffset      \ The current offset in the file

\ Tali needs some extra double words.
: d=  ( d1 d2 -- f )
   rot = -rot = and ;
: d0=   ( d -- f ) 0= swap 0= and ;
: d0<>  ( d -- f ) or ;
: d2*   ( d -- f ) 2dup d+ ;
: d<       ( d1 d2 -- f )  ( flag is true iff d1 < d2 )
  rot 2dup =  if 2drop u<  else 2swap 2drop > then ;

   
\ Tali's 2@ doesn't do things in the order we need.
\ This version of @ is little endian.
: 32bit@ dup @ swap 2 + @ ;

\ A word for making offsets
: offset  ( u "name" -- ) ( addr -- addr )  create ,  does> @ + ;
\ Offsets for the MBR
1BE offset >parttable
4 offset >parttype
8 offset >partlba
16 constant /partentry
\ Offsets for the Volume ID
0D offset >sect/clust
0E offset >reservedsect
24 offset >sect/fat
2C offset >rootclust
\ Offsets for directory entries
00 offset >filename
0B offset >attrib
14 offset >clustHI
1A offset >clustLOW
1C offset >filesize

\ Words for initialization:

\ Given the address of a partition entry in the partition table
\ determine if it's fat32 or not.
: fat32part?  ( addr_partentry -- f )
   >parttype c@   dup 0B = swap 0C =   or ; allow-native

\ Read sector (a double) into sectorbuffer.
: sector>buffer  ( sector#.d -- )
   2dup sector# 2@ d= if 2drop exit ( already have this sector ) then
   2dup sector# 2!
   sectorbuff -rot 1 -rot cf.read ;

: fat.init.vars  ( -- )
   \ Load the FAT Volume ID from the sector in
   \ partion_lba_begin and intialize the variables
   \ used to access the file system.
   \ Read in the FAT Volume ID.
   partition_lba_begin 2@  sector>buffer
   \ Get sectors per cluster.
   sectorbuff >sect/clust c@  sectors_per_cluster !
   \ Get start of fat.
   sectorbuff >reservedsect @ ( reserved sectors ) 0 ( make a double)
   partition_lba_begin 2@ d+   2dup fat_begin_lba 2!
   \ Get start of clusters
   sectorbuff >sect/fat 32bit@ ( sectors per fat )
   d2* ( two copies of FAT ) d+   cluster_begin_lba 2!
   ( todo : subtract two "clusters" from cluster_begin_lba to make it 0-based )
   (        instead of starting at 2 and requiring offsetting clusters by 2.  )
   \ Get root dir first cluster.
   sectorbuff >rootclust 32bit@   root_dir_first_cluster 2!
;

: fat.init  ( -- )  ( Intialize the FAT values )
   0. sector>buffer \ get MBR
   \ Check the 4 partions for a FAT one.
   4 0 do
      sectorbuff >parttable \ Locate partition table
      i /partentry * + \ Pick a partition
      dup fat32part? if \ Check for fat32
         >partlba \ get starting LBA for this partition.
         32bit@   partition_lba_begin 2!
         ." Found FAT32 filesystem in partition " i .
         ."  at LBA " partition_lba_begin 2@ d.
      else
         drop
      then
   loop
   \ Make sure we found something.
   partition_lba_begin 2@ d0= if abort then
   \ Initialize all of the variables needed for operation.
   fat.init.vars
;


Edit: Fixed d<


Top
 Profile  
Reply with quote  
PostPosted: Thu Jan 09, 2020 7:35 pm 
Offline

Joined: Sun May 13, 2018 5:49 pm
Posts: 255
FAT32 has a notion of "clusters", which are a group of consecutive sectors. The sectors are 512 bytes and the clusters can have anywhere from 1-128 sectors (always a power of 2) in them. There is also an oddity in that clusters 0 and 1 don't exist on the disk (for historical reasons), so everything needs to be offset by 2 when working with clusters.

The FAT part of FAT32 is the File Allocation Table, which is a collection of linked lists of 32-bit cluster numbers. If you have reached the end of the last sector in cluster 7, for example, you can go to the FAT at index 7 to see which cluster comes next in that file. Files are not required to be contiguous, and a special value indicates the end of the linked list for the last cluster.

Because of this, I need some math operators for double words. Tali Forth 2 already has d+ and d-. All of multiplications and divisions will be by powers of 2, so I implemented DLSHIFT and DRSHIFT. These need access to the 65C02's carry bit, so I wrote them in assembler.

The following are the words I wrote to turn a cluster# into a sector# of data and also a sector# for looking up the next cluster# in the FAT. The .d in some of the stack comments indicates a double word.
Code:
\ Continued from previous code

: log2  ( u -- u )  ( determine log2 for exact powers of 2 )
   0 swap begin dup 1 > while 2/ swap 1+ swap repeat drop ;

\ Bring in the assembler for these words.
assembler-wordlist >order
: dlshift  ( d u -- d )  ( double shift left )
   0 ?do
      \ Tali uses NUXI format for doubles.
      [
      2 asl.zx 3 rol.zx \ Most significant cell
      0 rol.zx 1 rol.zx \ Least significant cell
      ]
   loop ;

: drshift  ( d u -- d )  ( double right left )
   0 ?do
      \ Tali uses NUXI format for doubles.
      [
      1 lsr.zx 0 ror.zx \ Least significant cell
      3 ror.zx 2 ror.zx \ Most significant cell
      ]
   loop ;
previous \ Remove the assembler wordlist.

: cluster>sector  ( cluster#.d -- sector#.d )
  \ Get starting sector for cluster
   2. d-
   \    sectors_per_cluster *
   \ We don't have 32-bit *, so fake it with left shifts.
   \ This works because sectors_per_cluster is always
   \ a power of 2.
   sectors_per_cluster @ log2 dlshift
   cluster_begin_lba 2@ d+ ;

: cluster>fatsector  ( cluster#.d -- sector#.d ) \ Get sector for fat.
   \ There are 128 FAT pointers in a sector.
   \ We don't have double division, so use right shifts.
   7 drshift
   \ Start at the fat beginning sector.
   fat_begin_lba 2@ d+ ;


Top
 Profile  
Reply with quote  
PostPosted: Fri Jan 10, 2020 8:50 am 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8505
Location: Midwestern USA
SamCoVT wrote:
FAT32 has a notion of "clusters", which are a group of consecutive sectors.

FAT is also very inefficient in its use of raw disk space, thanks to the notion of clusters. Modern filesystems with UNIX ancestry (that would be Linux's ext... series—ext2 and later, both based upon the Berkeley fast filesystem, itself a descendant of the UNIX S51K filesystem) allocate storage on a block-by-block basis (block = sector). This has some important implications:

  • Creation of many small files on a FAT filesystem may exhaust filesystem space, even though the sum of all the files' sizes is less than the filesystem capacity. If the cluster size is 32KB (once a very common size on hard disks) and a 1 byte file is created, that 1 byte file will effectively be 32KB in size. Since ext2 and its descendants consume filesystem space one block at a time, rather than as a cluster of blocks, the same 1 byte file will only consume 1KB.

  • FAT requires translation between cluster numbers and raw disk blocks, which is not only mathematically cumbersome—and error-prone, something which Peter Norton exploited to make a lot of money—but typically requires more disk activity to read or write a file, especially write.

  • FAT intimately binds a file's FAT entry(s) to its directory entry, which makes the filesystem prone to corruption, especially file cross-linking (a very common error with MS-DOS). ext2 and its descendants separate directory entries from inodes (which are the guts of a file and are roughly analogous to a FAT entry), which not only greatly minimizes the likelihood of cross-linking and other errors, it makes the use of hard links possible (a hard link is a alternate name for a file—a file can have multiple names).

  • FAT is strongly dependent on linked lists to manage storage. Doing so tends to increase fragmentation owing to the way linked lists are used in filesystem management. ext2 and its descendants use bitmaps, which not only are more compact (one disk block can point at 8192 data blocks) but are far simpler to manage. Hence the filesystem's performance is better during intermittent write operations.

I've never been able to understand why FAT in its various incarnations is still around. It's a gigantic kludge of what was originally a floppy disk filesystem. Even Microsoft has tried to distance itself from it in the NT kernel (although their NTFS has its own problems).

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Last edited by BigDumbDinosaur on Mon Feb 10, 2020 2:08 am, edited 2 times in total.

Top
 Profile  
Reply with quote  
PostPosted: Fri Jan 10, 2020 10:04 am 
Offline
User avatar

Joined: Thu Dec 11, 2008 1:28 pm
Posts: 10985
Location: England
Surely FAT is still around because it is simple and very widely supported. One gives up some properties in exchange for others - that's engineering.


Top
 Profile  
Reply with quote  
PostPosted: Sun Jan 12, 2020 3:23 pm 
Offline

Joined: Sun May 13, 2018 5:49 pm
Posts: 255
BigDumbDinosaur wrote:
FAT requires translation between cluster numbers and raw disk blocks, which is not only mathematically cumbersome—and error-prone, something which Peter Norton exploited to make a lot of money—but typically requires more disk activity to read or write a file, especially write.


This is quite true. I'm using a single sector buffer, so I have to calculate which sector# in the FAT has the next cluster number, read that sector into the buffer, get the next cluster#, turn that into a data sector# and finally read in the next sector. My first goal is just to implement reading, as I'll be doing the writing of the software on my PC. I can see that writing is indeed a bit more complicated.

The main reason I'm going with FAT32 is that it's the easiest common denominator between Windows and Linux, as I use both (Linux when I have a choice). I have a 2GB Compact Flash card, and with the dozen or so Forth files I'm creating/using the space inefficiencies are not a concern for me.


Top
 Profile  
Reply with quote  
PostPosted: Sun Jan 12, 2020 4:12 pm 
Offline

Joined: Sun May 13, 2018 5:49 pm
Posts: 255
Now that I have words to access the FAT and the data in the partition, it's time to start with directories. In a FAT32 filesystem, the directories are stored just like files, so some of the next words I'll be implementing can be used for files and for directories. To get started, I'll work with just the root directory (whose starting cluster we already got from the VolumeID at the end of the word fat.init.vars).

Unfortunately the size of the directory is not known, so I have to check for the end of the directory by looking for a NULL character as the first letter of a directory entry. I also have to be careful as the directory is not guaranteed to be contiguous, so I will have to use the FAT to continue to the next cluster if it's larger than a cluster. This is the complicated part BDD mentioned, and I've created the words nextsector (to move to the next sector) and nextcluster# (return the next cluster number) to handle this.

Because this is FAT32 with long file name support, the directories end up being rather large because multiple directory entries are used to hold the long file names (crammed into the bytes in such a way that the entry will be ignored by software that doesn't know about long file names). I have to make sure to ignore these entries and will only be supporting 8.3 filenames. These are marked with all 4 low bits in the ATTRIB byte being set. I also need to handle directory entires that used to be in use but the files have been deleted. These start with $E5 as the first letter of the filename. The word directory_invalid? handles this - it returns true for any directory entry I shouldn't process.

I'm writing this to ultimately implement the file wordset in the ANS2012 standard (because the rest of the Tali2 is ANS2012 - an older standard would probably be more appropriate here). It's a little interesting as the standard mentions that the underlying OS will be used to actually process the files, except in my case Tali Forth 2 is the operating system so I have to write it all. The ANS words have a notion of a fileid, and I'm just using the address of a structure that has all of the variables I need to process a file.

Code:
\ Continued from previous code
\ Some extra double processing words
: d0!  ( addr -- )  ( store double 0 at address )
   0. rot 2! ; allow-native
: d+!  ( d1 addr -- )  ( Add d1 to double at addr )
   dup >r 2@ d+ r> 2! ;
: d1+!  ( addr -- )  ( Add 1 to double at addr )
   1. rot d+! ;


( Offsets into file control structure.                    )
( A "fileid" is address of one of these structures and is )
( used for both files and directories.                    )

( reusing "00 offset >filename" < 11 bytes > )
( reusing "0B offset >attrib" for fam < 1 byte > )
0C offset >firstcluster   (   4 bytes )
10 offset >currentcluster (   4 bytes )
14 offset >currentsector  (   4 bytes )
18 offset >fileposition   (   4 bytes )
( reusing "1C offset >filesize" < 4 bytes > )
20 offset >linestart      (   4 bytes )
24 offset >linebuffaddr   (   2 bytes ) ( 0x26 bytes total )

1C constant /dirinfo      ( Bytes per dirinfo - only up   )
                          ( through >fileposition is used )
26 constant /fileinfo     ( Bytes per fileinfo structure )

: newfileid ( -- fileid )
   here /fileinfo allot ( Allocate memory )
   dup /fileinfo 0 fill ( Fill with zeroes ) ;

create working_dir /dirinfo allot ;

: finfo  ( fileid -- )  ( fileid info for debugging )
   cr ." FILEID: "          dup u.
   cr ." filename: "        dup >filename 0B type
   cr ." access mode: "     dup >attrib c@ u.
   cr ." first cluster: "   dup >firstcluster 2@ ud.
   cr ." current cluster: " dup >currentcluster 2@ ud.
   cr ." current sector: "  dup >currentsector 2@ ud.
   cr ." file position: "   dup >fileposition 2@ ud.
   cr ." file size: "       dup >filesize 2@ ud.
   cr ." linestart: "           >linestart 2@ ud. ;


\ Cluster handling words

: nextcluster#  ( fileid -- cluster#.d )
   ( Determine next cluster )
   ( Load the correct fat sector into the buffer. )
   dup >currentcluster 2@   cluster>fatsector sector>buffer
   \ index it and get the new cluster number.
   >currentcluster 2@
   drop ( we don't need the MSB )
   80 mod \ Index into sector
   4 * ( 32-bit clusters )
   sectorbuff + \ Locate the next pointer
   32bit@ \ Read the cluster pointer.
;

: nextsector ( fileid -- ) ( Adjust fileid for next sector)
   ( See if we can just move to the next sector. )
   ( Calculate #sectors from beginning of data area. )
   dup >currentsector 2@  1. d+  cluster_begin_lba 2@ d-
   drop ( we don't need the upper bits )
   ( See if we're still in the same cluster )
   sectors_per_cluster @ mod if
      ( fileid )
      >currentsector d1+! ( Increment current sector )
   else
      ( We need to move to the next cluster. )
      dup >r nextcluster#
      2dup r@ >currentcluster 2! ( Update current cluster )
      cluster>sector ( Turn it into a sector )
      ( Save this as the current sector )
      r> >currentsector 2!
   then
;


\ Words for handling directories

: directory_offset  ( fileid -- offset_in_sectorbuffer )
   >fileposition 2@
   drop ( MSBs not needed )
   200 ( sector size ) mod ;

: direntry_invalid?  ( addr_direntry -- f )
   dup c@ E5 = \ See if it begins with E5
   \ Offset to the ATTRIB and see if it has all 4 lower bits
   \ set.
   swap 0B + c@ 0f and 0f =
   \ If either of these things are true, it's an invalid entry.
   or ;

: dir_next ( fileid -- )
   \ Move along one entry
   dup >r
   >fileposition 2@ 20. ( 32-bytes per directory entry ) d+
   2dup   r@ >fileposition 2!
   \ See if a new sector is needed.
   drop ( don't need MSBs for this )
   200 ( sector size ) mod 0= if
      r> nextsector  else r> drop  then ;

: move_to_cluster  ( fileid clusternum.d -- )
   \ Update the cluster and sector in the fileid structure.
   rot >r ( Save fileid )
   2dup   r@ >currentcluster 2!   cluster>sector
   2dup   r> >currentsector 2!
   \ Load the sector into the buffer.
   sector>buffer ;

: init_directory  ( fileid -- )
   \ Set all of the variables for walking a directory.
   dup  dup >firstcluster 2@ move_to_cluster
   \ Start the the beginning of the directory.
   >fileposition d0! ; 

: dirend?  ( addr -- f )  c@ 0= ;

: walkdirectory  ( xt fileid -- addr | 0)
   \ Run routine (given by xt) on every starting address of a
   \ directory entry in directory described by fileid.  The routine
   \ should be ( direntry_addr -- continueflag ) where continueflag
   \ is 1 if directory processing should continue. The return value
   \ of walkdirectory will either be the address of the directory
   \ entry it stopped on (in the sector buffer) or 0 if it reached
   \ the end.
   >r ( Save the fileid )
   begin
      \ Get the offset of the directory entry into the
      \ sectorbuffer.
      r@ directory_offset
      \ Turn it into an address in the sectorbuffer.
      sectorbuff +
      \ See if we have reached the end of the directory.
      dup dirend? if
         drop ( direntry ) r> 2drop ( xt and fileid)
         0 ( false ) exit then
      \ See if it's an unused entry.
      dup direntry_invalid? if
         \ Just put 1 on the stack to tell the routine to keep
         \ going.
         drop 1
      else
         \ It's a valid entry.  Call the processing routine on it.
         over execute
      then
      \ There should be a flag telling us if we should proceed to
      \ the next directory entry.
   while
         \ Proceed to the next entry
         r@ dir_next
   repeat
   \ If we made it here, we were told to stop.
   drop ( the xt )
   \ Leave the address of the directory entry on the stack.
   sectorbuff  r> directory_offset  +
;

\ ls is a useful example of using walkdirectory.

: print-filename ( direntry_addr -- true )
   \ Print a filename and size.
   \ Always return true to continue on to the next file.
   dup >filename cr 0B type  space  >filesize 32bit@ ud.
   true ( Keep going ) ;

: ls ( -- ) ( List the working directory)
   working_dir init_directory
   ['] print-filename working_dir walkdirectory drop ;

ls gets a directory listing. While I currently only have the root directory working, ls has been written to work with any directory if working_dir is adjusted to that directory - a "cd" word will need to be implemented in the future for that.

I also updated fat.vars.init (which now needs to come after working_dir is declared) to set the current directory to be the root directory. The tail end of fat.init.vars (the part processing the root directory) now looks like:
Code:
    \ Get root dir first cluster.
   sectorbuff >rootclust 32bit@
   \ Save it so we always have a copy.
   2dup root_dir_first_cluster 2!
   \ And start the working directory there.
   working_dir >firstcluster 2!
   \ Set the name of the root directory.
   s" /          "  working_dir >filename swap  move ;


And now, we can actually do something almost useful... we can get a directory listing! This prints the filenames and the file sizes (it looks like my current base is hex). Note that there is no "." in the filenames - that's not actually stored in the directory entry. The filename is stored as 11 contiguous characters with the first 8 being the filename and the last 3 being the extension - spaces are used to pad any unused characters. I'll need to write some words to translate between the presentation and the storage format.
Code:
cf.init fat.init Found FAT32 filesystem in partition 0  at LBA 800  ok
ls
LCD     FS  29C
REDIRECTFS  3D2
SCREENS FS  A13
BLOCK   BLK 3C00  ok


Top
 Profile  
Reply with quote  
PostPosted: Sun Jan 12, 2020 7:22 pm 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8505
Location: Midwestern USA
SamCoVT wrote:
It's a little interesting as the standard mentions that the underlying OS will be used to actually process the files, except in my case Tali Forth 2 is the operating system so I have to write it all.

Perhaps it would be more efficient and less cumbersome to treat Tali Forth as an application, which it technically is—essentially, Forth is a type of language interpreter—and have a separate kernel for managing low-level disk access and the filesystem. That separation would not only ease the task of developing your FAT implementation, it would make it adaptable to non-Forth environments.

Also, no matter how cleverly written, filesystem management executed in Forth will not perform at the level that could be achieved in assembly language. Ritchie and Thompson recognized this chasm in the early stages of developing UNIX, and despite Ritchie's C compiler emitting efficient machine code, much of the low-level disk stuff was written in assembly language to get the last bit of performance.

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
PostPosted: Sun Jan 12, 2020 9:35 pm 
Online
User avatar

Joined: Fri Aug 30, 2002 1:09 am
Posts: 8543
Location: Southern California
BigDumbDinosaur wrote:
Perhaps it would be more efficient and less cumbersome to treat Tali Forth as an application, which it technically is—essentially, Forth is a type of language interpreter—and have a separate kernel for managing low-level disk access and the filesystem.

AIUI, Tali Forth is subroutine-threaded code (STC), which means it's assembly language that calls the various routines if they're long enough, or straight-lines the assembly-language code for things that are too short to justify the JSR-RTS overhead. It also has an onboard assembler, making it easy to do do whatever you want in assembly language if maximum performance or down-to-the-metal control is required. (STC is not necessary for having an assembler onboard though; I have an assembler in my ITC Forth as well.) Most non-STC Forths (mostly meaning direct or indirect threaded code) are compiled as a list of addresses of routines though, not tokens that must be looked up for interpretation. The only one I know of that is interpreted in that sense is token-threaded code Forth which makes for super compact code (as the 255 most common instructions are one byte each) but adds overhead to look up the addresses of the routines represented by the tokens.

I would like to see some examples of Tali Forth compilation. They're probably on the github pages, but I can't find them. That's one of the reasons I hate github. I can never find what I want. I don't know if it's something about github, or if it's just that people don't make the stuff clear.

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?


Top
 Profile  
Reply with quote  
PostPosted: Sun Jan 12, 2020 10:26 pm 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8505
Location: Midwestern USA
GARTHWILSON wrote:
AIUI, Tali Forth is subroutine-threaded code (STC), which means it's assembly language that calls the various routines if they're long enough, or straight-lines the assembly-language code for things that are too short to justify the JSR-RTS overhead.

In other words, it's an application. The Thoroughbred Dictionary-IV BASIC interpreter I use on Linux for business software is also machine code (compiled ANSI C), but is still an application.

Quote:
It also has an onboard assembler, making it easy to do do whatever you want in assembly language if maximum performance or down-to-the-metal control is required.

However, in-line assembly language doesn't change the fact that Forth is an application and hence carries a certain amount of overhead common to all applications.

Quote:
Most non-STC Forths (mostly meaning direct or indirect threaded code) are compiled as a list of addresses of routines though, not tokens that must be looked up for interpretation.

That may be, however, interpretation has to occur because Forth words are (mostly) English and hence must be translated into the equivalent machine instructions needed to execute whatever the word does. In that sense, Forth is no different than BASIC, except for better performance and more extensibility (although Thoroughbred's CALL instruction allows "remote" subroutines to be created that can do things not possible using only core language statements).

Quote:
I would like to see some examples of Tali Forth compilation. They're probably on the github pages, but I can't find them. That's one of the reasons I hate github. I can never find what I want. I don't know if it's something about github, or if it's just that people don't make the stuff clear.

I'm no fan of github as well. Too much hoop-jumping required to 'git' to whatever it is you want. If someone says they've posted their project on github I move on to something else.

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
PostPosted: Mon Jan 13, 2020 1:06 am 
Online
User avatar

Joined: Fri Aug 30, 2002 1:09 am
Posts: 8543
Location: Southern California
BigDumbDinosaur wrote:
That may be, however, interpretation has to occur because Forth words are (mostly) English and hence must be translated into the equivalent machine instructions needed to execute whatever the word does.
That happens at compile time though, not run time, just as happens when assembly language is assembled and mnemonics are looked up, labels are turned into addresses, branch distances are calculated, expressions are evaluated and only the results are kept, comments discarded, etc.. Early Forths were usually the operating system as well. They didn't run on top of something else.

Quote:
I'm no fan of github as well. Too much hoop-jumping required to 'git' to whatever it is you want. If someone says they've posted their project on github I move on to something else.
Me too, except this time I did look.

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?


Top
 Profile  
Reply with quote  
PostPosted: Mon Jan 13, 2020 2:03 am 
Offline

Joined: Sun May 13, 2018 5:49 pm
Posts: 255
BigDumbDinosaur wrote:
SamCoVT wrote:
It's a little interesting as the standard mentions that the underlying OS will be used to actually process the files, except in my case Tali Forth 2 is the operating system so I have to write it all.

Perhaps it would be more efficient and less cumbersome to treat Tali Forth as an application, which it technically is—essentially, Forth is a type of language interpreter—and have a separate kernel for managing low-level disk access and the filesystem. That separation would not only ease the task of developing your FAT implementation, it would make it adaptable to non-Forth environments.


In this case, I'm not too concerned with speed. I've been using an I2C flash memory and mostly just pasting over a 19200 baud serial link, and what I have working so far is way faster than either of those. I'm more interested in being able to have several "libraries" as files that I can easily bring in when I want to use them, and not being limited to blocks (although block file support is coming). This is also my first real attempt to use Forth to solve a real problem, so it's more about learning Forth and less about speed or optimizing.

As Garth noted, Tali is an STC Forth and I can totally just replace any slow words (or even parts of a word) with straight assembly, but I think this is going to be "fast enough" (eg. it will take 2-3 seconds to process a large file and that speed is fine with me). I'll leave the optimization to those who enjoy that kind of thing - some people find that kind of stuff fast-inating.

I am posting my code to github, along with my libraries for working with various hardware that I've attached to my SBC. You can search for SamCoVT on github to find my projects there, but I just barely started my ForthSoftware project so I am guilty of Garth's complaint about things not being well described. The file.fs is the file I am working my way through now, and the fat32_tali.fs is the previous version that I got basically working but needed to rewrite to make it follow the ANS2012 standard. Ultimately there will be support for multiple files open at the same time and even one forth file including another. Following the standard gives a somewhat clear path towards that goal.


Top
 Profile  
Reply with quote  
PostPosted: Mon Jan 13, 2020 3:43 am 
Offline

Joined: Fri Apr 15, 2016 1:03 am
Posts: 140
Here are a couple small samples of Tali generated code.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 23 posts ]  Go to page 1, 2  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 4 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: