Fleet Forth design considerations

[GARTHWILSON]

As I look over your code above (without examining any details), I think it's similar to what I did. type for example examines variable OUTDEV (output device) and follows with a CASE structure to determine which routines to use to do the outputting. OUTDEV's contents are constants placed there by PRINTER, CONSOLE, etc., and one that can be custom is AUXDEV, in which case type will do AUXtype PERFORM, where AUXtype is a variable holding the address of the version of type that may have been written on the spur of the moment for some kind of custom output. I had already been thinking of having several AUXDEV's like AUXDEV1, AUXDEV2, etc. so you wouldn't have to keep swapping them out if the program needed to be bouncing around among several (even having multiple screens comes to mind); but I haven't used it enough to do anything about it yet. Then earlier today, even before I saw your post, I was thinking I should allow for outputting to more than one device at a time, like if you want a printer trace of everything sent to the display. I've thought of doing this for input too, but you'd have to be careful so the input streams from two simultaneously transmitting sources don't get shuffled. I suppose ANS Forth has a way to handle this, but I have not looked into it. Too many other projects ahead of it.

[JimBoyd]

These days I use VICE to simulate a Commodore 64 and it sends output meant for the printer to a PRINT.DUMP file. Back about twenty five years ago when I first wrote my own Forth for the C64, I didn't think of logging the output of an interactive session to the printer ( probably because of the noise ). If I'd had more time back then, I might have hit upon the idea to open a sequential file to one of the disk drives ( I had two ), preferably one I was not using for blocks. The logger word could then append the text to the sequential file instead of dumping it to a printer ( much quieter ).
I also didn't have to worry about corrupting a disk of Forth Blocks because the Forth I wrote back then used Commodore's relative files as opposed to directly accessing sectors on the disk. I didn't use relative files this time around because it seems to me that Blazin' Forth, which used direct access, was more popular than 64Forth, which used relative files. That and the version of VICE I have doesn't seem to work to well with large relative files.
Once I release Fleet Forth, when I get it and the documentation finished, if someone wants to use relative files for blocks, the word DR/W , the block read write word for disk access is a deferred word. R/W the word which calls DR/W , and DR/W itself, takes the following parameters:
Address of buffer, block number, read write flag ( 0 is write ), and the amount of bytes to read or write. The last parameter is useful for INDEX. As long as any new word for DR/W honors the count, INDEX does not have to be rewritten for each new form of block access.

[JimBoyd]

JimBoyd wrote:

I've removed the multitasker support ( such as it was and what there was of it) from the vectored I/O words CONSOLE , PRINTER , and LOGGER. Fleet Forth still supports multitasking, but there are better ways than maintaining an I/O state variable to do what I wanted.

The I/O redirection word used can be determined by examining the first cell of TYPE.

[JimBoyd]

JimBoyd wrote:

I was just rereading the thread on Extra stacks.
I named the words to transfer data between the data stack and the auxiliary stack with an 'A' rather than an 'H' because I didn't write the Auxiliary stack to be a high precision math stack, but rather a helping hand for manipulating high level and assembly control flow data. That and I wasn't thinking about that thread at the time I implemented the Aux stack.
As for implementing a split stack, that might save a few bytes in the stack transfer words, but it would cost more than that in the definition of .AS , the Aux stack counterpart to .S .

There shouldn't be a conflict if math words were written which use the Auxiliary stack to hold the highest cells ( or double cells) of the numbers being manipulated. Because I'm reclaiming what is otherwise unused memory in the Commodore 64 that is below the dictionary, and below screen memory, my Aux stack is limited to 43 cells, although I think that should be plenty.

[JimBoyd]

I've developed Fleet Forth about as far as I can. There are other things I wanted to try, but lately it has been getting difficult for me to stay focused on what is essentially a hobby. The documentation is very incomplete. I will try to get more of it done as time permits.

[JimBoyd]

Fleet Forth does include the complete source code.

[JimBoyd]

I've updated the Fleet Forth zip file. In addition to an acknowledgement file, there is a file on copying Forth blocks from one disk to another and maybe a few additions or corrections. I'm sorry this is taking so long. It is really hard to focus on this right now.

[JimBoyd]

I've had to update the zip file yet again. In the last one I inadvertently included a disk image with code from possibly Forth Dimensions as well as some code from Blazin' Forth which I was checking for compatibility with Fleet Forth. Here is the new zip file.
[Edit: Moved the download to the head post]

[JimBoyd]

I'm still working on the documentation for Fleet Forth, but in the meantime I've made some improvements. One of the improvements was to the metacompiler. Previously, I handled the metacompiler version of control flow in the target by defining VALUE like words to hold the addresses of the primitives to be metacompiled. Any words which had to compile another word ( control flow, ." , ABORT" , etc. ) looked something like this, for example:

Code: Select all

: ."  ( -- )
   '(.") ,-T
   ,"-T ; IMMEDIATE
: IF  ( -- ADR CS )
   '?BRANCH ,-T
   >MARK ; IMMEDIATE
: AHEAD  ( -- ADR CS )
   'BRANCH ,-T
   >MARK ; IMMEDIATE

Which meant I had something like the following for each word compiled by an immediate word (such as the control flow primitives) in the kernel source:

Code: Select all

: (.")  ( -- )
   R> COUNT 2DUP + >R TYPE ;
' (.") TIS '(.")

Even worse, when trying to do something clever, such as having the code field of a primitive point somewhere other than its body ( a 'code word' without a body):

Code: Select all

CODE BRANCH  ( -- )
   -2 ALLOT XBRANCH 2+ , END-CODE
' BRANCH TIS 'BRANCH
CODE ?BRANCH
   -2 ALLOT ?BRANCH.BODY , END-CODE
' ?BRANCH TIS '?BRANCH

the following lines

Code: Select all

' BRANCH TIS 'BRANCH

' ?BRANCH TIS '?BRANCH

tend to be a little distracting, as well as making it a little harder to test a new version of such a primitive on the host system before committing to a new kernel build.

My solution was MCOMPILE , so that the control flow words ( and others) now looked like this:

Code: Select all

: ."  ( -- )
   MCOMPILE (.")
   ,"-T ; IMMEDIATE
: IF  ( -- ADR CS )
   MCOMPILE ?BRANCH
   >MARK ; IMMEDIATE
: AHEAD  ( -- ADR CS )
   MCOMPILE BRANCH
   >MARK ; IMMEDIATE

Now the definitions for the primitives used by the control flow words ( and others) no longer need that annoying line:

Code: Select all

' SOME.PRIMITIVE TIS 'SOME.PRIMITIVE

When the metacompiler is built, the versions of the primitives found in the host Forth vocabulary are compiled into the definition of the control flow words.

Code: Select all

SEE AHEAD
AHEAD IMMEDIATE
 6D17 6072 MCOMPILE
 6D19  C4C    BRANCH   \ host BRANCH
 6D1B 6CA8 >MARK       \ metacompiler version of >MARK
 6D1D  966 EXIT
8 
 OK
SEE IF
IF IMMEDIATE
 6D05 6072 MCOMPILE
 6D07  C58    ?BRANCH  \ host ?BRANCH
 6D09 6CA8 >MARK       \ metacompiler version of >MARK
 6D0B  966 EXIT
8 
 OK

Which is not a problem because MCOMPILE copies the name to HERE and uses it as the search string to find the address of the primitive in the target FORTH vocabulary. M, ( meta comma) then compiles the correct address in virtual memory.
Here is the source for M, and MCOMPILE :

Code: Select all

: M,  ( CFA -- )
   DUP ?TW             \ Is it a target word?
   >BODY
   1 OVER 2+ 2+ +!     \ Record some statistics
   @                   \ fetch virtual address
   ,-T ;               \ and compile to virtual HERE
: MCOMPILE  ( -- )
   ?COMP
   R> DUP 2+ >R @            \ Get the code field
   >NAME COUNT 1F AND >HERE  \ Back up to name and copy to HERE
   DUP COUNT >LOWER          \ clear high bit of each byte
   FIND 0= ?HUH M, ;         \ Search and metacompile

I've used the backslash for comments because the comments were added after the fact.

[JimBoyd]

JimBoyd wrote:

These days I use VICE to simulate a Commodore 64 and it sends output meant for the printer to a PRINT.DUMP file.

I'm not sure if I mentioned it, but I also use the latest version of Fleet Forth on the Commodore 64 simulator as my development environment for Fleet Forth and its tools.

[JimBoyd]

From the first version of Fleet Forth onward I've implemented what I think is the best EMPTY. I say this because I've seen what is a really bad implementation of EMPTY ( ahem, Blazin' Forth ).
When Fleet Forth cold starts after being loaded from disk, some of the user variables, up to and including DP the dictionary pointer, are initialized with default values from what some would call the 'boot area'. The first three user variable locations are reserved for multitasking.

Code: Select all

// FORTH SOURCE - BOOT AREA
HEX
LABEL BOOT
//  SET CURRENT COLORS FOR TARGET
HOST D020    C@ 0F AND TARGET C,
HOST D021    C@ 0F AND TARGET C,
HOST  286    C@ 0F AND TARGET C,
LABEL USER.DATA
// SYSTEM USER AREA BOOTUP VALUES
USER.AREA ,           // ENTRY
  0 ,                 // WAKE?
  0 ,                 // TOS
1FF ,                 // RP0
 7E ,                 // SP0
 1E ,                 // SPLIM
// AUXILIARY STACK
AUX.BASE AUX.SIZE + , // AP0
AUX.BASE ,            // APLIM
HERE TIS 'THERE
  0 ,                 // DP

This initial value for DP is used by EMPTY. This value CAN change. FORGET will set it to the minimum of its current address and HERE so this 'empty point' is never greater than the top of the dictionary. Saving the system as a new Forth will change the address of this location to the current value of DP.
EMPTY fetches this address from the boot area and sets FENCE equal to it then branches into FORGET to let FORGET do the heavy lifting of trimming back the VOC-LINK and vocabularies. 'THERE is a metacompiler label holding the address of this location.

Code: Select all

: EMPTY  ( -- )
   FORTH DEFINITIONS
   [ 'THERE ] LITERAL @
   DUP FENCE !
// BRANCH INTO FORGET
   BRANCH [ (FORGET) , ]
   ; -2 ALLOT

: FORGET  ( -- )
   SINGLE
   NAME  CURRENT @ DUP CONTEXT !
   VFIND 0= ?HUH  >LINK
   LABEL (FORGET)        \ EMPTY branches to this location
   DUP>R FENCE @ U<  R@
   LIT
   LABEL KERNEL-FENCE
   [ 0 , ]  // WILL BE PATCHED
   // REST OF FORGET

The above is from the source the metacompiler uses to build a new kernel.
The new kernel is built in virtual memory ( the C64's REU ) and LABELs are compiled in host memory ( that's how I can have labels in the middle of a definition ).

[JimBoyd]

I made a correction to Fleet Forth's EMPTY . FORGET turns off multitasking in case some of the tasks are forgotten. EMPTY did not turn off multitasking ( though it should have). I've moved SINGLE, the word that turns multitasking off, in FORGET to the point where EMPTY branches into the body of FORGET.

Code: Select all

: EMPTY  ( -- )
   FORTH DEFINITIONS
   [ 'THERE ] LITERAL @
   DUP FENCE !
// BRANCH INTO FORGET
   BRANCH [ (FORGET) , ]
   ; -2 ALLOT

: FORGET  ( -- )
   NAME  CURRENT @ DUP CONTEXT !
   VFIND 0= ?HUH  >LINK
   LABEL (FORGET)        \ EMPTY branches to this location
   SINGLE
   DUP>R FENCE @ U<  R@
   LIT
   LABEL KERNEL-FENCE
   [ 0 , ]  // WILL BE PATCHED
   // REST OF FORGET

[JimBoyd]

Normally, when a deferred word's vector ( the word that a deferred word executes ) is forgotten, that deferred word points to a word that no longer exists. In Fleet Forth, there is a table of all the deferred words in the kernel as well as their default vectors. when FORGET is executed ( or EMPTY or COLD ), the table is checked from beginning to end to see if any of the deferred words had a vector that was forgotten. If a deferred word has a vector that was forgotten, the deferred word is reset to its default vector.
This works great for deferred words in the Forth kernel, but it's no help for deferred words added to the system. I'm thinking about adding a variable, DEFER-LINK , to the kernel. Each deferred word defined outside the kernel would have an extra cell which would be part of a chain much like vocabularies are with VOC-LINK. Some new functionality would be added to FORGET. It would trim the DEFER-LINK, like it does with VOC-LINK, and each deferred word in the link would be checked to see if its vector is defined after what will be the new HERE. Any such deferred words would be reset to the word -SET.
I'm trying to decide if the protection against a possible crash from forgetting a deferred word's vector is worth an extra two bytes per deferred word and some extra code in the kernel. Any advice?

[JimBoyd]

Since I'm using VICE to simulate a Commodore 64, I copied everything to a new folder and built a new kernel and system with 'forget protection' for deferred words. Now I can compare the two versions of Fleet Forth. Each deferred word ( not the ones in the kernel ) is now two bytes bigger. The kernel is 52 bytes bigger: 46 more bytes in FORGET, 4 more bytes in DEFER, 4 bytes for the headerless variable DEFER-LINK, and 2 bytes saved by removing the deferred word RR/W from the table of kernel deferred words and making it the first deferred word added to the DEFER-LINK chain.
The table for kernel deferred words uses 4 bytes per deferred word: 2 bytes for the address of the body of the deferred word and 2 bytes for the address of the deferred word's default vector.
(RR/W) is not defined in the kernel since someone without a ram expander would not want the overhead of that word. The deferred word RR/W is in the kernel because it is needed by R/W, the block read/write word. Since (RR/W) is not defined in the kernel, the default vector of RR/W is the word -SET , therefore removing it from the table and making it the first of the deferred words in the DEFER-LINK chain does not change what happens when RR/W's vector is forgotten.
Now, except for all but one of the kernel deferred words, when a deferred word's vector is forgotten, the deferred word's vector is reset to -SET , the vector it had when it was created. -SET aborts with a message that the deferred word is 'NOT SET'. If the deferred word is interpreted, the actual message will be 'EXECUTE NOT SET'. If compiled into another word, the message will be '<name of deferred word> NOT SET'.
Here is the new DEFER.

Code: Select all

HEX
: -SET  ( -- )
   WHERE
   R@ 2- @ >NAME CR RON ID.
   ." _ NOT SET" ABORT ; -2 ALLOT
: DEFER  ( -- )
   CREATE ['] -SET ,
   DEFER-LINK ADD
   ;CODE
   2 # LDY,
   W )Y LDA,  PHA,  INY,  W )Y LDA,
   0 # LDY,
   W 1+ STA,  PLA,  W STA,
   W 1- JMP,
END-CODE

That underscore character in the definition of -SET is because the C64 character would not print ( or print.dump, in the case of VICE ). It turns reverse video off.

And here is the new FORGET.

Code: Select all

: FORGET  ( -- )
   NAME  CURRENT @ DUP CONTEXT !
   VFIND 0= ?HUH  >LINK
   LABEL (FORGET)
   SINGLE
   DUP>R FENCE @ U<  R@
   LIT
   LABEL KERNEL-FENCE
   [ 0 , ]  // WILL BE PATCHED
   U< OR
   ABORT" PROTECTED"
   VOC-LINK R@ TRIM
   VOC-LINK @
   BEGIN
      DUP 2- 2- R@ TRIM
      @ ?DUP 0=
   UNTIL
   DEFER-LINK R@ TRIM
   DEFER-LINK @
   BEGIN
      DUP 2- @ R@ U< 0=
      IF
         ['] -SET OVER 2- !
      THEN
      @ ?DUP 0=
   UNTIL
   R> DUP DP ! [ 'THERE ] LITERAL @
   UMIN [ 'THERE ] LITERAL !
// RESET ANY DEFERRED WORD WITH
// CFA ABOVE HERE
   HERE [ END.FORGET ] LITERAL
// BRANCH INTO IORESET
   BRANCH [ (IORESET) , ]
   ; -2 ALLOT

And the 'helper' words.

Code: Select all

: ADD  ( ADR -- )
   HERE OVER @ ,
   SWAP! ;

: TRIM  ( ADR LIMIT -- )
   OVER
   BEGIN
      @ 2DUP SWAP U<
   UNTIL
   NIP
   SWAP! ;

@Garth: When I saw SWAP! I thought that was cool so I added it to my kernel ( both of the current ones ).

[JimBoyd]

whartung wrote:

You want to share any details about your meta compiler? How it works, what challenges you ran in to, things like that?

I'm sorry if my explanations were less detailed than you would like. Since I originally wrote a metacompiler about a quarter of a century ago, writing this one has not felt like blazing new territory for me with the strong impressions of obstacles overcome. It has been more like following a faint trail down a side alley of memory lane.
If there are specifics that you would like to know more about, don't hesitate to ask. Given something specific to focus on, I'll do my best to piece together what I had to do to arrive at my current solution.