Fleet Forth design considerations

[JimBoyd]

barrym95838 wrote:

Here's my clumsy stab at it:

Code: Select all

: PAUSE  ( -- F )
   ?KEY DUP IF
      DUP 3 <> IF
         DROP KEY THEN
      3 = THEN
;

Am I in the right ballpark?

Yes, but I like the factoring where both THEN's are one after the other followed by the semicolon. I'll show why.

Quote:

P.S. eFORTH has the stack comment for ?KEY as ( -- c T | F )

In Fleet Forth it's ( -- c | 0 ) because that is what the C64 kernal supports.
PAUSE is the name of Fleet Forth's task switcher (which is set to a no-op when not multitasking), so I'm just going to call this word DONE? , which is what it is called in Fleet Forth.

Code: Select all

: DONE?  ( -- F )
   ?KEY ?DUP
   IF
      3 <>
      IF  KEY 3 =  ELSE  TRUE  THEN
   ELSE
      FALSE
   THEN ;

Notice the ELSE clause at the end of the word?

Code: Select all

ELSE
   FALSE
THEN ;

If ?DUP is replaced with DUP , that clause is no longer needed.

Code: Select all

: DONE?  ( -- F )
   ?KEY DUP
   IF
      3 <>
      IF  KEY 3 =  ELSE  TRUE  THEN
   THEN ;

In my ITC Forth, this saves 3 cells.
The next improvement is a little harder to see.
3 <> is equivalent to 3 = 0=
At first glance, this doesn't seem to save memory. if ?DUP is inserted between = and 0= the final ELSE clause is no longer needed.

Code: Select all

: DONE?  ( -- F )
   ?KEY DUP
   IF
      3 = ?DUP 0=
      IF  KEY 3 =  THEN
   THEN ;

A savings of another cell.
Fleet Forth also has the primitives ?EXIT and 0EXIT .
Both consume the top stack item.
?EXIT exits the word if the top stack item is TRUE (any non-zero value).
0EXIT exits the word if the top stack item is zero.
Using these two words, some more memory savings can be had, as well as a performance boost. Both IF's branch to the exit at the end of the word. They can both be replaced with 0EXIT and the THEN's removed.

Code: Select all

: DONE?  ( -- F )
   ?KEY DUP
   0EXIT
      3 = ?DUP 0=
      0EXIT  KEY 3 = ;

A little reformatting.

Code: Select all

: DONE?  ( -- F )
   ?KEY DUP 0EXIT
   3 = ?DUP 0= 0EXIT
   KEY 3 = ;

Finally, 0= 0EXIT can be replaced with ?EXIT

Code: Select all

: DONE?  ( -- F )
   ?KEY DUP 0EXIT
   3 = ?DUP ?EXIT
   KEY 3 = ;

[barrym95838]

My way of thinking FORTH is brutally incomplete, but I did think of ?EXIT briefly. It's kind of like a conditionally executed RTS in the middle of a subroutine, efficient but a bit lacking in what I perceive to be "structured" technique. Come to think of it, I often break those techniques when I'm doing my own thing, so ...

My (probably flawed) reasoning on the subject of where to place my IFs and ELSEs and THENs is that I feel weird needlessly inserting line breaks between the data or code and the word immediately responsible for consuming or executing it. In my example, the 3 <> is immediately consumed by the following IF, and the DROP KEY is within the execution scope of the following THEN ... kind of an "object verb verb" "noun noun verb" thing, like Yoda tends to use in his sentences. As my understanding hopefully improves, I may start to see things more the way you and Garth format your code, but many miles to go still have I ...

[GARTHWILSON]

Nothin' wrong with ?EXIT, as there's nothing pending on either stack like there would be with DO...LOOP (where you can remove the exit address, loop limit, and loop index with UNLOOP). Any of the structures will have stuff on the stack during compilation, but not necessarily at run time.

Putting the condition and flow-control words this way makes if very clear, and there's no visual searching:

Code: Select all

: FOOBAR    ( n1 -- n2 f )
   <do stuff>
   <condition?>
   IF    ┌─────────────────┐
         │ <do this stuff> │
         └─────────────────┘
   ELSE  ┌─────────────────┐
         │ <do this stuff> │
         └─────────────────┘
   THEN
   <do more stuff>    ;

But if you want the IF on the same line with the condition so the line is more stack-autonomous, you can re-arrange it this way:

Code: Select all

: FOOBAR    ( n1 -- n2 f )
   <do stuff>
   <condition?>          IF
   ┌─────────────────┐
   │ <do this stuff> │
   └─────────────────┘   ELSE
   ┌─────────────────┐
   │ <do this stuff> │
   └─────────────────┘   THEN
   <do more stuff>     ;

The latter works out really well if you have a lot of nested IF's, and no visual searching is necessary. (Color syntax highlighting is not only unnecessary when you do this, but totally undesirable, IMO.) Then you can put them one above the other instead of indenting and indenting and indenting way over to ridiculous extremes. You might end with a line of

Code: Select all

   THEN  THEN  THEN  THEN  THEN

Some may turn up their nose at this and say if you're having to do that, then you're taking the wrong approach; but then their "fix" usually ends up overfactoring or doing something else that's just as undesirable.

[GARTHWILSON]

I seem to remember the humorous book "Starting Forth" had the following logic example as part of a getting-dressed word, to show Forth's differing approach.

Are you lacking socks?
IF so, socks-install
THEN shoes-install

(The "shoes install" object-noun word order is natural for native Korean speakers.) The THEN is not part of the IF, but instead meas "After you've taken care of that, do this." It makes perfect sense this way, even though it's different from other programming languages which would take the approach,

process dress_feet:
IF your are lacking socks, THEN put on socks. ENDIF
Now put on shoes.

[JimBoyd]

GARTHWILSON wrote:

The first sample follows what so many Forthers are dogmatic about, that definitions should be limited to two lines. This kind of dogma hurts readability. It tends to result in over-factoring too, taking more memory for all the unnecessary headers for factors that are only used once, more time to run because of the extra nesting, and it can be hard sometimes to come up with a descriptive name for the factor that is any shorter than the code it replaces! There was a regular contributor to Forth Dimensions magazine who defended this strongly, and said screens should be viewed as 3x5 cards. He kept pushing this as a plus, yet I just saw it as inadequate room to make things clear and add adequate comments. Then in a private mail exchange (which today we would do by email, but back then it was on paper), he sent me his code for something we were discussing, and he had a definition that was over 100 lines long! I didn't call out his hypocrisy.

I tend to view screens as a consequence of the limitations of the technology of the day, although blocks on the C64 can also be used as a form of virtual memory. On my Linux box, I use plain text files for source.
If I have a definition on the C64 that will not fit on a single block, I just let it span however many I need. -NUMBER takes two blocks but I load something like that with THRU rather than LOAD , so it's not a problem.
I think some of the over zealous factoring came about because some Forth programmers tried to minimize what they perceived as wasted space in a source screen. I hope this example makes it evident that the "wasted space" in a source screen is not so valuable a resource as the understanding that comes from well formatted source.
I made a copy of the source for -NUMBER and reformatted it to pack it into as few lines as possible. The only rules where:
1) Three space indentation for every line in a definition after the name.
2) two spaces around each control flow word unless it is the first or last word on a line.
To say it is highly unreadable is an understatement. When I deleted the white space, I may have accidentally deleted a word or two. From the compacted form it is not easy to tell.

Code: Select all

: -NUMBER  ( ADR -- D FLAG )  RB DUP 1+ C@ ASCII # - DUP 3 U<
   IF  BASE.TABLE + C@ BASE ! 1+ DUP  THEN  DROP DPL ON  0 0 ROT
   DUP 1+ C@ ASCII - = DUP>R - DUP>R  BEGIN  CONVERT DUP C@
   VALID?  WHILE  DUP 1+ C@ VALID? 0=  WHILE  DPL OFF  REPEAT
   THEN  DUP C@ BL <> SWAP 2- DPL @ 0< - R> = OR R> 0EXIT >R
   DNEGATE R> ;

If this was somebody's ideal, I can see why they would want to aggressively factor it (over factor it). Personally, this form looks like software that went through a virtual trash compactor.
In this case, there is no need to refactor this word, just don't format the source like this.
Even if there are any useful factors in the definition of -NUMBER , it is not easy to find them in this mess.

[JimBoyd]

I was just reading "Real Time Forth" by Dr. Tim Hendtlass and becoming reacquainted with some of F-PC's words. I had forgotten that F-PC has a word called NUMBER? which is similar to Fleet Forth's -NUMBER . The only apparent difference is NUMBER? returns a TRUE flag if successful. I use the word 'apparent' because without a copy of F-PC (which I had at one time, but it's been a while) I do not know what F-PC's NUMBER? rejects as an invalid number. Fleet Forth's -NUMBER requires a numeric string to:
1) Have at least one valid digit in the current base (or the temporary base specified by a leading # , $ , or % )
2) Have no more than one occurrence of valid punctuation in a row.
3) Have a trailing blank. When interpreting, this is automatically supplied by WORD .
For example, the following are valid numbers:
1.2
1.
.1
.1.
.1.2.3.4.
-1.
$-FF
$-DEAD.BEEF
the following are not:
1..2
.
..
...
.1..2
..1
123...
-.
-
Since there is already a word that does what -NUMBER does, but with a different return flag and a slightly different name, I changed -NUMBER to NUMBER? and changed the return flag.
-NUMBER had the following test to determine if the result was valid:

Code: Select all

   DUP C@ BL <> SWAP
   2-  DPL @ 0< - R> = OR

FALSE = valid
Thanks to De Morgan’s laws, I know I can change this test to reverse the flag without just tacking a 0= on the end to flip the flag.

Code: Select all

   DUP C@ BL = SWAP
   2-  DPL @ 0< - R> <> AND

TRUE = valid
I also had to change a few (very few) things in the kernel and saved two bytes overall.

[JimBoyd]

GARTHWILSON wrote:

As long as things don't conflict, it's nice when you can accommodate multiple standards. The only drawback is that it takes more memory.

I wasn't trying to accommodate multiple standards. I thought it would be useful if I could specify a number as hexadecimal, decimal, or binary regardless of the current number base. The Ansi Forth Standard specified that behaviour so I appropriated that part of it.
It's still early days, but for 44 extra bytes in the kernel, I think it will be worth it.

[JimBoyd]

Fleet Forth has a word, ?MEM , to check if the amount of memory to ALLOT is available. At one point I defined ?MEM like this:

Code: Select all

: ?MEM  ( N -- N )
   MRU OVER PAD + U< 0EXIT
   ABORT" NO MEMORY" ;

0EXIT was used to exit the word immediately, if there was no error, to avoid the overhead of (ABORT") .
This was the source for (ABORT")

Code: Select all

: (ABORT")  ( F -- )
   IF
      WHERE CR R@ S?
      ABORT
   THEN
   R> COUNT + >R ;

Including docolon, that is seven passes through NEXT just to check for an error and keep going. Rather than using code like the above code for ?MEM to avoid that overhead in some places, I decided to make (ABORT") faster. Here is the source for the new (ABORT") .

Code: Select all

CODE (ABORT")  ( F -- )
   0 ,X LDA,  1 ,X ORA,
   0= IF,
      IP )Y LDA,  SEC,
      IP ADC,  IP STA,
      CS IF,  IP 1+ INC,  THEN,
      POP JMP,
   THEN,
   >FORTH
   WHERE CR R@ S?
   ABORT ;
   -2 ALLOT

The new version of (ABORT") is only nine bytes larger than the older version.

[GARTHWILSON]

UNUSED is pretty standard for your ?MEM, and is even in ANS Forth. It can be defined as:

Code: Select all

: UNUSED        ( -- #bytes )   \                                   ANS_CORE_EXT
   LIMIT  HERE  -       ;       \ Leaves room for a max-length comment line from
                                \ TIB , HLD , and a max-length string in PAD .

I did it as a primitive in my '816 Forth, this way:

Code: Select all

        HEADER "UNUSED", NOT_IMMEDIATE   ; ( -- bytes )
UNUSED: PRIMITIVE           ; Tells how much RAM is left to expand the dictionary
        LDA     #LIMITdata  ; I omitted CLC since 1 byte either way won't matter.
        SBC     DPdata
        JMP     PUSH

[JimBoyd]

?MEM is different. It does not return the amount of bytes free. Given a number of bytes, it performs the test to see if there is enough memory available. It takes the number of bytes and if that number is available, it returns that same number. This is why, in a previous post, I called it "transparent".
For example, here is the source for Fleet Forth's ALLOT .

Code: Select all

: ALLOT  ( N -- )
   ?MEM  DP +! ;

ALLOT was previously defined without memory checking:

Code: Select all

: ALLOT  ( N -- )
   DP +! ;

This seems more convenient to me than picking a "reasonable" amount of memory that needs to be available in CREATE or the need to check memory if I need to ALLOT some. If there is not enough memory, HERE does not change and the system aborts.
So far ?MEM is only used in two other places:
" places a string at PAD while interpreting or compiles (") and inlines the string while compiling.
FBUF in the editor.
The new and improved version of (ABORT") wasn't just to remove 0EXIT from ?MEM , but to make all uses of (ABORT") faster when there is no error to report.

[GARTHWILSON]

My UNUSED is also used in , (comma). My ALLOT uses it too, but in looking at it again now, I think I could simplify it.

[JimBoyd]

GARTHWILSON wrote:

My UNUSED is also used in , (comma). My ALLOT uses it too, but in looking at it again now, I think I could simplify it.

Wouldn't your , (comma) use ALLOT ?

Code: Select all

: ,  ( N -- )  HERE 2 ALLOT ! ;
: C,  ( C -- )  HERE 1 ALLOT C! ;

It's funny. I intended my post to be about (ABORT") with ?MEM as an example.

[GARTHWILSON]

, uses 2 DP +! instead of ALLOT. Yeah, 2 ALLOT would be more efficient.

[JimBoyd]

I've talked about Fleet Forth's words BLOCK and BUFFER in other posts and I thought it would be good to go into them in a bit more depth here.
First is a quick look at R/W , Fleet Forth's block read/write word. Depending on the block number requested, R/W will either access a disk drive (for lower numbered blocks) or access RAM (for higher numbered blocks). R/W is not a deferred word, but the word to access the disk, DR/W , and the word to access RAM, RR/W , are.
R/W , and therefore DR/W and RR/W , take the address of a buffer, a block number, the read/write flag, and a count. This differs from Blazin' Forth. Blazin' Forth's R/W only takes three parameters, the address of a buffer, a block number, and the read/write flag. Fleet Forth's R/W was written to require a count to support a general version of INDEX . INDEX is a handy utility word that, given a starting block number and an ending block number, displays the first line for each block in that range. Ideally, INDEX only reads in one line as well (64 bytes). Fleet Forth's INDEX does so it only needs 64 bytes of memory at PAD. Since it uses R/W directly, it will work with different drive access methods, for different drive types, just like BLOCK and BUFFER .

Next is a look at the buffer table. The inspiration for the table came from Blazin' Forth. Fleet Forth, like Blazin' Forth, can support a variable number of blocks, configurable on the fly. There should never be less than one buffer, unless there will be no block access. There can be as many buffers as will fit in memory. The table consists of one entry per buffer plus one, the phantom entry. The entries are arranged in order from the phantom entry then the most recently used to the least recently used.
There are two system values used to manage the table, MRU and LRU. MRU points to the phantom entry. LRU points to the entry for the least recently used block. LRU is also headerless, since it is only used in the Forth kernel. Each entry consists of six cells: the block number, the buffer address, and an update flag. R/W does not update the order of the entries in the table. BLOCK and BUFFER handle that. The following shows four blocks with the resulting 5 table entries.

Code: Select all

MRU:   5 | $C800 | FLAG
       5 | $C800 | FLAG
      42 | $C000 | FLAG
     137 | $C400 | FLAG
LRU:   1 | $CC00 | FLAG

A quick reminder that Fleet Forth is an ITC Forth for the Commodore 64 and when a CODE word is executed, the Y-register is set to zero. BLOCK starts out as a CODE word.

Code: Select all

CODE BLOCK  ( BLK -- ADR )
   DEY,
   BLK/BUF STY,
   ' MRU >BODY    LDA,  N    STA,
   ' MRU >BODY 1+ LDA,  N 1+ STA,
   6 # LDY,  N )Y LDA,  0 ,X CMP,
   0= IF,
      INY,  N )Y LDA,  1 ,X CMP,
      0= IF,
         INY,  N )Y LDA,  PHA,
         INY,  N )Y LDA,
         PUT JMP,
      THEN,
   THEN,

The first thing BLOCK does is decrement the Y-register and store it's value for (possible) later use. BUFFER is a bodiless CODE word with a code field that points one byte into the body of BLOCK.
BLOCK then checks to see if the requested block is the most recently used. If so, it replaces the block number with the address of the buffer used for that block. Up to this point BLOCK behaves like a CODE word and jumps back to NEXT by way of PUT and we're done.

If the requested block is not the most recently used, the jump to PUT is branched over.

Code: Select all

   >FORTH
   #BUF 1+ 2
   ?DO
      DUP I >BT @ =
      IF
         DROP I UNLOOP
         AHEAD CS>A
      THEN
   LOOP

>FORTH causes a transition to high level code. The ?DO LOOP checks if the requested block is in the table. If it is, the index into the table is saved on the stack (not to be confused with Forth's word INDEX ), the loop parameters are discarded and AHEAD branches out of the loop to elsewhere in BLOCK .

Code: Select all

      LRU 2+ 2+ @
      IF
         LRU 2@ 0 B/BUF R/W
         LRU 2+ 2+ OFF
      THEN

If the requested block is not in the table, the least recently used block is written to disk (or ram) if its update flag is true. The update flag is not cleared until after the block is written in case the write fails.

Code: Select all

      BLK/BUF C@
      IF
         LRU ON
         LRU 2+ @ OVER 1 B/BUF R/W
      THEN
      LRU ! #BUF

The value of the Y-register that was saved earlier is fetched to see if this code was called as BLOCK or BUFFER . If this is BLOCK , if the read fails, the contents of the buffer will not be valid (most likely) so the entry for the block number of the least recently used block is set to TRUE (-1) to indicate an unused buffer. The new block is read in to the least recently used buffer.
Both BLOCK and BUFFER then store the requested block number in the block number entry for the least recently used block/buffer and leave the index to the entry for the least recently used block/buffer on the stack.

Code: Select all

   A>CS THEN
   DUP >BT MRU 6 CMOVE
   MRU 1 >BT ROT 6 * CMOVE>
   MRU 2+ @ ;

The branch out of the ?DO LOOP branches to this section of code. At this point one of two index values will be on the data stack.
If the requested block number was found in the table the index to the table entry with the desired block number is on the data stack.
If it was not found in the table the index of the least recently used block/buffer is on the data stack.
This table entry is copied to the phantom entry. The phantom entry and all entries above the one with its index on the stack are now slid down by one entry.
If a block that was not in the table, such as 123, was requested, this is what it would look like:

Code: Select all

MRU: 123 | $CC00 | 0
       5 | $C800 | FLAG
      42 | $C000 | FLAG
     137 | $C400 | FLAG
LRU: 123 | $CC00 | 0

Code: Select all

MRU: 123 | $CC00 | 0
     123 | $CC00 | 0
       5 | $C800 | FLAG
      42 | $C000 | FLAG
LRU: 137 | $C400 | FLAG

If however a block that was in the table, such as 42, was requested, this is the result:

Code: Select all

MRU:  42 | $C000 | FLAG
       5 | $C800 | FLAG
      42 | $C000 | FLAG
     137 | $C400 | FLAG
LRU:   1 | $CC00 | FLAG

Code: Select all

MRU:  42 | $C000 | FLAG
      42 | $C000 | FLAG
       5 | $C800 | FLAG
     137 | $C400 | FLAG
LRU:   1 | $CC00 | FLAG

[IamRob]

Thanks for delving into BLOCK and BUFFER for Fleet Forth.

I was in the process of deciphering what they do and how they work in ProForth. ProForth is a Forth for the Prodos OS on an Apple II. ProForth also has 2 buffers laid out as blocks. A ProForth block is 1024 bytes which happens to also be 16 lines of 64 bytes. The same as Fleet Forth's line limit.

Because of the great similarities between ProForth and FleetForth, I have taken a great interest in a lot of your improvements for FleetForth, and hope to adapt many of them to ProForth.

Just a couple of quick questions.

How does FleetForth set a device # to know which device to read from or write to before using BUFFER and BLOCK?

Under ProDOS, the BLOCK parameter list contains a device #. So far this has been hard coded into ProForth. I would like to change that so I can load any text file from any slot/drive combination. Just seeing what else is out there for word primitives that may already exist. DR0 and DR1 were only meant to signify drives and were originally programmed for floppies. Under ProForth, they are not used at all but are still included in word definitions. What does the C64 have in the way of device types. I believe there was an add-on SD card for the C64 port? That would mean you would need a word under FleetForth to identify which Port to save to other than the floppy drive, would you not?

Is FleetForth reading text files as a sequential text file (one line at a time) or as a chunk of 1024 byte screen files? Proforth is set up to load 16 screens all at the same time and the text file is a fixed 16384 ($4000) = 16 screens of 1024 bytes. Then the screens are accessed with the LIST word from 0-15 and a screen is loaded using the LOAD word.

I hate this method and want to get away from it. Would simply like to use OPEN "textfile" and LOAD to load it.