In solving this problem I came up with two different solutions which resulted in two versions of the metacompiler. They were the same except for what feature was added.
For reference I will refer to the version of the metacompiler without either of these solutions as the base metacompiler. It is this base upon which version A and version B are built.
First, some words in the base metacompiler were cleaned up.
The original source for PNAME and VHEADER .
: PNAME ( -- )
1 VALLOT 1 PNAMES +!
COLS ?CR ." PADDED: R"
HERE S? CR ;
: VHEADER ( >IN -- >IN )
VNAME TFIND NIP
ABORT" REDEFINITION IN TARGET"
HEAD @
IF
DUP >IN !
HERE C@ THERE + 4 +
SPLIT DROP 0=
IF PNAME THEN
WLINK VADD
VNAME C@ THERE $80 VTOGGLE
VALLOT THERE $80 VTOGGLE
1 VALLOT
EXIT
THEN
THERE 1+ SPLIT DROP 0=
IF PNAME THEN
1 HEADLESS +! ;
HOST
: PNAME ( ADR -- )
1+ SPLIT DROP ?EXIT \ If the low byte of the address is not $FF then exit.
1 VALLOT 1 PADDED +! \ Pad the virtual header and increment the padded count.
COLS ?CR ." PADDED: R" \ If not at left column perform carriage return
HERE S? CR ; \ before displaying the name of the padded word.
: VHEADER ( -- )
>IN @ VNAME TFIND NIP \ Save >IN. If the parsed name already exists
ABORT" REDEFINITION IN TARGET" \ in the target vocabulary then abort.
HEAD @ \ If HEAD is on
IF \ create header.
>IN ! \ Restore >IN to parse the name again.
HERE C@ THERE + 2+ 1+ PNAME \ Avoid the indirect jump bug.
WLINK VADD \ Create the link field.
VNAME C@ THERE $80 VTOGGLE \ Create the name field with high bit set
VALLOT THERE $80 VTOGGLE \ for count and last character.
1 VALLOT \
EXIT
THEN \ If HEAD is off
DROP 1 HEADLESS +! \ increment the count of headerless words
THERE PNAME ; \ and avoid the indirect jump bug.
PNAME now takes an address and pads the header, if necessary, to avoid the code field straddling a page boundary. VHEADER now no longer requires the value of >IN on the data stack and it returns nothing.
I will present the first solution, metacompiler A.
Here is the base metacompiler's vocabulary structure.
FORTH
META
SHADOW
FORTH
ASSEMBLER
RUN
EDITOR
ASSEMBLER
The variable ALIASES is added to keep a count of how many words have an alias.
The new word ALIAS creates a header in virtual memory and a target word (a handle) in the RUN vocabulary. ALIAS also switches off header creation in virtual memory for the next word with NH .
: ALIAS ( ++ )
ORDER@ [META] RUN DEFINITIONS \ Save the search order and
HEADER NH \ create header in RUN vocabulary.
ORDER! \ Restore the search order.
TRUE HEADLESS +! 1 ALIASES +! ; \ Correct the headless count and
\ increment the count of aliases.
CODE MCOMPILE ( -- )
>FORTH \ Shift to high level Forth.
?COMP \ Abort if not compiling.
R> DUP 2+ >R @ \ Get the CFA of next word in Forth thread containing MCOMPILE
>NAME COUNT $1F AND >HERE CLIP \ and use the name field to create a search string at HERE .
TFIND ?TARGET M, ; \ Search the TARGET FORTH vocabulary and abort if not found.
\ If found, compile to virtual memory with M, .
CODE MCOMPILE ( -- )
>FORTH \ Shift to high level Forth.
?COMP \ Abort if not compiling.
R> DUP 2+ >R @ \ Get the CFA of next word in Forth thread containing MCOMPILE
>NAME COUNT $1F AND >HERE CLIP \ and use the name field to create a search string at HERE .
TFIND TRUE <> \ If not found or if immediate
IF
DROP HERE [META] \ drop the result and use the string at HERE
[ ' RUN >BODY ] LITERAL \ to search the RUN vocabulary.
VFIND TRUE <> \ If still not found
ABORT" RUN-TIME NOT FOUND" \ abort.
THEN
M, ; \ If found, compile to virtual memory.
The modified MCOMPILE for metacompiler A still creates a search string from the name field of the next word in it's definition.
When TEST is executed, MCOMPILE will create the search string "(TEST)" at HERE .
The TARGET FORTH vocabulary is searched by TFIND .If the sought word is not found or if it is immediate, the result is dropped and the RUN vocabulary is searched. Once more, the result has to be for a word which is not immediate. VFIND does not search parent vocabularies.
This version of the metacompiler allows changing the names of run-time words with ALIAS ; however, there is one downside.
Using DO and it's run-time as an example.
ALIAS DO
CODE (DO) ( LIMIT START -- )
...
...
END-CODE
There are now two handles on the host system for the same target word in virtual memory, this is not the problem. If the name of the run-time for DO on the host system does not match either one, a new alias is needed.
ALIAS DO \ Name of run-time for new kernel. Creates handle DO in RUN vocabulary and header DO in virtual memory.
ALIAS D \ Name of run-time in host system. Creates handle D in RUN vocabulary. header creation in virtual memory suppressed.
CODE (DO) \ Name of run-time used in kernel source. Creates handle (DO) in TARGET FORTH vocabulary. header creation in virtual memory suppressed.
...
...
END-CODE
This creates three handles for the same word in virtual memory, one in the TARGET FORTH vocabulary and two in the RUN vocabulary.
There is only one header created in virtual memory for this word since ALIAS suppresses header creation in the following word.
I will now present the second solution, metacompiler B.
No new metacompiler vocabularies are added. This metacompiler's ALIAS does not create a target word. It creates a header in virtual memory. It does use NH to switch off header creation in virtual memory for the next word.
CODE MCOMPILE ( -- )
>FORTH \ Shift to high level Forth.
?COMP \ Abort if not compiling.
R> DUP 2+ >R @ \ Get the CFA of next word in Forth thread containing MCOMPILE
>NAME COUNT $1F AND >HERE CLIP \ and use the name field to create a search string at HERE .
TFIND TRUE <> \ If not found or if immediate
ABORT" RUN-TIME NOT FOUND" \ abort.
M, ; \ If found, compile to virtual memory.
With these modifications it is possible to create a kernel where each of the DO LOOP run-time words has the same name as it's compiling word. The next addition to metacompiler B will allow it to work when built on the new system.
The word :: (double colon) is an alias for the host colon. This will allow accessing colon's functionality while defining words in the META vocabulary, where colon was redefined.
These words defined in the META vocabulary are stubs. They don't get compiled or metacompiled. MCOMPILE uses their names to find the corresponding word in the TARGET FORTH vocabulary.
Recall that the new semicolon will behave just like the host semicolon when metacompiling is off.
I could have defined these metacompiler DO LOOP compiling words with COMPILE since COMPILE gets patched when metacompiling.
This version of the metacompiler works well; however, there is a slight downside. If I want to rename the run-time for ." so it has the same name as ." , I will need to define a stub for the run-time and a metacompiler version of ." . The same is true for any run-time I wish to rename.
Metacompiler C, the, hopefully, last solution to this problem.
The only differences with metacompiler B: :: (double colon) is not defined. MCOMPILE is different.
: CFA>STR ( CFA -- HERE )
>NAME COUNT $1F AND >HERE CLIP ; \ Use CFA to generate a search string
\ from a word's name field.
CODE MCOMPILE ( -- )
>FORTH
?COMP R> DUP 2+ >R @ CFA>STR \ Get CFA of next word in definition
\ to generate search string.
TFIND TRUE <> \ If the word is not found or is
IF \ immediate
DROP R@ 2- 2- BODY> CFA>STR \ Back up to CFA of the word containing
ASCII ) OVER COUNT + C! \ MCOMPILE to generate a search string
DUP DUP 1+ BL MOVE \ and modify string to be enclosed in
ASCII ( OVER 1+ C! \ parenthesis.
2 OVER +! TFIND TRUE <> \ Search the TARGET FORTH vocabulary
ABORT" RUN-TIME NOT FOUND" \ and abort if not found.
THEN
M, ; \ If found, compile to virtual memory.
CFA>STR takes the CFA of a word and generates a string at HERE which is the same as the word's name. MCOMPILE still generates a search string based on the name of the word to be metacompiled by MCOMPILE . If the search is unsuccessful or the word found is immediate, MCOMPILE will generate a search string from the name of the word which contains METACOMPILE . This search string will be the compiling word's name enclosed in parenthesis.
Using DO as an example.
Suppose the run-time for DO on the host system is still named (DO) . MCOMPILE generates the string "(DO)" and searches for this string in the TARGET FORTH vocabulary. It will be found and the address for (DO) in virtual memory will be compiled to virtual memory.
Suppose the run-time for DO on the host has a different name. Let's suppose it is something absurd like DO.RUNTIME . MCOMPILE tries to find DO.RUNTIME in the TARGET FORTH vocabulary and fails. MCOMPILE then generates a string from the name field of the compiling word, in this case DO . The string will have parenthesis added.
Here is a sample test.
FOOBAR does not exist in the TARGET FORTH vocabulary. In this example, (TESTWORD) does exist in the TARGET FORTH vocabulary. MCOMPILE tries to find the target word FOOBAR in the TARGET FORTH vocabulary. When that fails, it tries to find (TESTWORD) in the TARGET FORTH vocabulary and succeeds.
Run-time words with the same name in the host system and the source used by the metacompiler work fine.
Examples are ?BRANCH and BRANCH used by most of the control flow words.
Run-time words with different names in the host system and the source used by the metacompiler will work if the name of the run-time in the source is a parenthesized version of the compiling word's name.
Examples are (DO) compiled by DO and (LOOP) compiled by LOOP . These run-time words can be given a different name on the target being built in virtual memory by using the word ALIAS .
At present, I don't see a downside to this version of the metacompiler
I recently came up with three variations of Fleet Forth's metacompiler. The third variation seemed the best way to go so it is Fleet Forth's metacompiler; indeed, if I'd thought of it first, I would not have bothered with the other two.
As I previously mentioned, the metacompiler patches the word COMPILE to MCOMPILE . This is so I don't have to define a metacompiler version of every word which compiles ?BRANCH or BRANCH , or words which contain such words. I also don't have to define a metacompiler version of some of the other compiling words.
Although the target being built in virtual memory can have redefinitions, a target word (a handle for it's counterpart in virtual memory) in the TARGET FORTH vocabulary can not be a redefinition of another target word.
To allow redefinitions in the target being built, the word ALIAS is used.
There is one run-time word in the kernel which could not be given an alias with the way the latest MCOMPILE was written. One thing to keep in mind, what I mean by this is that any word can be given an alias and the kernel will be metacompiled successfully; however, the Forth system built on the new kernel might not be able to host the metacompiler.
The following deals with using the metacompiler on the new system (I really don't want to maintain two versions of ITC Fleet Forth's kernel).
The run-time word (") , PAREN-QUOTE, is compiled by the word " , QUOTE. The source for QUOTE.
: " ( -- )
STATE @
IF COMPILE (") ," EXIT THEN
ASCII " CHAR ?MEM PAD TUCK C!
TUCK COUNT MOVE ; IMMEDIATE
If compiling, QUOTE compiles PAREN-QUOTE, it's run-time word, and compiles an inline string with ," , COMMA-QUOTE.
If interpreting, the text string in the text stream is copied to PAD and the address of PAD is left on the data stack.
PAREN-QUOTE could not be renamed to QUOTE.
When QUOTE is executed while metacompiling, the name for QUOTE would not be found in the TARGET FORTH vocabulary. I didn't mean a non immediate version couldn't be found, although MCOMPILE checks for that, no version would be found. QUOTE is not even defined in the Forth kernel, it is in the system loader. MCOMPILE would not be able to back up to the name field of QUOTE to generate a parenthesized version of it's name because COMPILE is not the first word in the body of QUOTE.
The latest improvement to MCOMPILE uses AFIND . AFIND is a word which takes an address and returns that address and the closest link field address below and the closest link field address above the given address.
ALIAS DO
CODE (DO) ( LIMIT START -- )
...
END-CODE
...
: DO ( -- >SYSL <SYSL )
COMPILE (DO)
>MARK 2+ <MARK 2+ ; IMMEDIATE
The run time of DO is also named DO because of ALIAS .
When DO is executed, MCOMPILE would get the code field of the run-time DO and back up to it's name field. It would generate a search string based on that name and attempt to find a word with the name DO in the TARGET FORTH vocabulary. Failing to do so, it would get the address within the word containing it, DO , and back up to it's name field. MCOMPILE would generate a parenthesized version of the name and use it , (DO) , for the search string.
This is the latest source for MCOMPILE and two helper words.
: LFA>STR ( LFA -- HERE )
L>NAME COUNT $1F AND >HERE CLIP ;
: ?RT ( FLAG -- )
ABORT" RUN-TIME NOT FOUND" ;
CODE MCOMPILE ( -- )
>FORTH
?COMP R> DUP 2+ >R @ // GET ADR OF NEXT WORD
AFIND DROP TUCK LINK> = DUP // DOES IT HAVE A HEADER?
IF DROP LFA>STR TFIND THEN // IS IT IN TARGET FORTH
TRUE <> // VOCABULARY?
IF // NOT IN VOC OR HEADERLESS
R@ AFIND 2NIP DROP LFA>STR // GET ADR OF WORD
ASCII ) OVER COUNT + C! // CONTAINING MCOMPILE
DUP DUP 1+ BL MOVE // USE NFA TO GENERATE
ASCII ( OVER 1+ C! // SEARCH STRING AND
2 OVER +! TFIND TRUE <> ?RT // PARENTHESIZE
THEN
M, ; // IF FOUND, COMPILE TO VM
LFA>STR takes a word's link field address, converts it to the name field address and copies the word's name to HERE . ?RT aborts if there is a true value on the data stack.
This version of MCOMPILE will work even if the run-time in question has no header on the system the metacompiler is built upon. MCOMPILE gets the address of the next word in the definition containing MCOMPILE , or COMPILE since COMPILE is patched while metacompiling. AFIND is used to find the closest link field address below this word. This LFA is converted to the CFA and compared with the CFA of the run-time word. If they differ the run-time is headerless.
If it is not headerless MCOMPILE uses it's name for the search string to search the TARGET FORTH vocabulary with TFIND , target find.
If the run-time is not found or it is headerless MCOMPILE uses AFIND to get the name field address of the word which contains MCOMPILE and generate a search string which is a parenthesized version of that name.
With this version of MCOMPILE it does not matter if a run-time word is headerless in the system upon which the metacompiler is built. MCOMPILE will generate a string which is a parenthesized version of the compiling word's name.
It also does not matter if COMPILE or MCOMPILE is not the first word in the compiling words body. Once again AFIND is used to find the word's name field address.
The metacompiler has to define it's own version of >ASSEM , even though COMPILE is patched, because it has to set CONTEXT to the metacompiler's ASSEMBLER vocabulary.
Notice the dummy word (>ASSEM) . This stub is defined because the name (>ASSEM) is no longer found in the new system. The run-time for >ASSEM also has the name >ASSEM .
Also notice the metacompiler's version of >ASSEM has MCOMPILE in it's definition.
With the new version of MCOMPILE it is possible to redefine the metacompiler's version of >ASSEM
In the high level portion of the decompilation, the first number on a line of decompilation is the address in the decompiled word. The second number is the CFA of the word in the definition of the decompiled word.
7751 7921 WHERE \ Defined after ABORT" runtime.
7753 6778 CR
7755 4789 R@
7757 7169 S?
7759 8742 ABORT \ Defined after ABORT" runtime.
With my Forth's older version of INTERPRET , the ABORT" runtime was needed by INTERPRET to ABORT" if a string which was not found in the dictionary could not be converted to a number. INTERPRET was needed by QUIT which was needed by ABORT which was needed by the ABORT" runtime.
Looking at this fragment and noting that WHERE and ABORT are defined after the ABORT" runtime might lead one to believe this Forth was assembled. This is not the case. It was metacompiled. Here is the source for the ABORT" primitive.
Each label holds the value of THERE , the target system's HERE , when it is created. The word NOOP is used as a placeholder.
The following shows the resolution of the address at label DO.ABORT
The resolution for DO.WHERE is similar.
That is how to handle forward references in the source used by the metacompiler without resorting to creating a deferred word in the target for something which should not change.
My Forth, an ITC Forth, has a few words with no body and a code field which points into the body of a primitive. The definition of these words follows this pattern:
CODE ON ( ADR -- )
DEY TYA DEX DEX
' SWAP! @ 4 + 0= NOT BRAN
END-CODE
CODE OFF ( ADR -- )
' ON @ 1+ LATEST NAME> !
END-CODE
This metacompiler technique does not work if the word is to be headerless. Although there is a word with a header on the host to serve as a handle, a headerless word has no header in the target, which is being built in virtual memory; therefore, the metacompiler versions of LATEST and NAME> do not work.
Since the metacompiler version of ' (tick) does not require the presence of a header in the target, the following technique works.
The creation of Fleet Forth's metacompiler's assembler vocabulary lexicon takes advantage of a property of Fleet Forth vocabularies. Although Fleet Forth's vocabularies have a branching structure just as FIG Forth vocabularies, there is no false header embedded in a Fleet Forth vocabulary. A Fleet Forth vocabulary has a body with three cells. The first cell points to the link field of the latest word defined in that vocabulary, the second cell points to the parent vocabulary and the third cell is part of the chain of all vocabularies pointed to by VOC-LINK .
Normally, the first word in a Fleet Forth vocabulary has a link field with the value Zero. I say "normally" because the metacompiler assembler violates this rule. The first word defined in the metacompiler vocabulary has a link field which points to the word NOT in Fleet Forth's main assembler vocabulary. This allows the words defined in the Fleet Forth main assembler vocabulary between NOT and the first word in that vocabulary to be searched when the metacompiler assembler vocabulary is searched. This was done to save memory since these words are, or would have been, defined exactly the same in the metacompiler assembler vocabulary. Since Fleet Forth vocabularies have a parent link rather than a false header (the parent link in the main Forth vocabulary is, obviously, zero), the search order when searching starts with the metacompiler assembler is still: metacompiler assembler, target Forth, shadow vocabulary, meta vocabulary and then the main Forth vocabulary.
TARGET ASSEMBLER ORDER
CONTEXT: ASSEMBLER
FORTH
SHADOW
META
FORTH
CURRENT: FORTH
SHADOW
META
FORTH
OK
I will keep the metacompiler assembler vocabulary like this on Fleet Forth; however, it is my desire that others will implement a metacompiler on their own Forths. To that end, I suggest the following change for all Forths which do not have VOCABULARY defined similarly to Fleet Forth's.
Remove the following from the source for the metacompiler vocabulary lexicon.
TARGET VOCABULARY ASSEMBLER
HOST
: [ASSEMBLER] [TARGET] ASSEMBLER ;
IMMEDIATE
TARGET ASSEMBLER DEFINITIONS
\ OPCODE OFFSET TABLE AND MODES
CREATE TABLE
$0008 , $0810 , $1404 , $2C14 ,
$1C0C , $0818 , $1000 , $0400 ,
$1414 , $0C2C , $1C1C ,
VARIABLE MODE
: AM ( N -- )
CREATE ( N -- )
C,
DOES> ( -- )
C@ MODE ! ;
0 AM .A 1 AM X) 2 AM )Y
3 AM # 7 AM ) 8 AM MEM
9 AM ,X $A AM ,Y
\ Z
: Z ( OA? ADR -- OA? BOP BMAP )
COUNT SWAP @
MODE @ 7 > 0EXIT
2PICK $100 U< 0EXIT
1 MODE @ 2- 2-
LSHIFT OVER AND 0EXIT
-4 MODE +! ;
\ ?EXEC BO BRANCHES NOT
FORTH DEFINITIONS
: ?EXEC ( -- ) STATE @
ABORT" FOR ASSEMBLING" ;
ASSEMBLER DEFINITIONS
: BO ( DEST SRC -- BRANCH.OFFSET )
1+ - DUP 0< OVER ABS + $7F >
ABORT" BRANCH RANGE EXCEEDED" ;
$50 CONSTANT VS \ OVERFLOW
$90 CONSTANT CS \ CARRY SET
$D0 CONSTANT 0= \ EQUAL TO ZERO
$10 CONSTANT 0< \ LESS THAN ZERO
$90 CONSTANT >= \ NOT LESS THAN
\ >= ONLY CORRECT AFTER SUB, OR CMP
: NOT ( CC1 -- CC2 )
BL XOR ; \ REVERSE TEST
It is my hope that others will explore this fascinating method for building new Forth kernels. Metacompiling is my preferred method for building a Forth kernel because it allows me to more easily improve the kernel.
When I change the definition for a kernel word, I have a fully functional kernel I can test it with ( partly because my Forth allows redefinitions). Since I work directly with the source used by the metacompiler, once testing is done and the new version of the kernel word works as expected, the source for said word is metacompiler ready. If I was using an assembler to build my Forth kernel and I wanted to improve one of the kernel words, I would have to write that word in a form suitable for my Forth system. Once the kernel word was working, I would have to translate the source to a form suitable for the assembler used to build the kernel and hope the translation process didn't introduce any errors from typos or other mistakes.
Yes, this is a hobbyist metacompiler. In my opinion it is better than a commercial "black box" metacompiler for two reasons, it is transparent and it leaves the Forth system it is built on exposed. If the metacompiler misbehaves, it is possible to troubleshoot it. If it lacks a feature, it is possible to add it, within reason.
The word to switch metacompiling on, META.ON , patches some of the kernel words; therefore, metacompiling must be switched off with META.OFF before the metacompiler lexicon, and its associated vocabularies, is forgotten.
META.OFF is included in the definition of FINISH and FINISH is included in the definition of FINISH-FORTH ; however, it is possible to work with the system with metacompiling on to get a better feel for what the metacompiler is doing or work on enhancements to the metacompiler. It can be all to easy to neglect switching metacompiling off with META.OFF before removing the metacompiler lexicon by forgetting the first of the metacompiler words.
One simple redefinition eliminates this potential hazard.
The redefinition of FORGET should be defined immediately after META.OFF and defined in the host FORTH vocabulary ( the original FORTH vocabulary) just like META.OFF .
Those who prefer to reset the dictionary to its current "empty" point with the Forth word EMPTY should also include a redefinition of EMPTY in the metacompiler source along with the redefinition of FORGET .
I realise that the redefinitions return to high level Forth which is no longer a part of the dictionary to execute EXIT at the end of the redefinition; however, Fleet Forth's FORGET does not wipe the memory between the new HERE and the original. Nor does it execute any words which use memory at HERE or PAD . FORGET just resets the dictionary pointer after resetting the pointers in each VOCABULARY . Since EMTPY branches into the body of FORGET (to handle the "heavy lifting"), it also does not affect memory above the new HERE address. The code for these words still exist in memory until something alters that memory AFTER the new FORGET or EMPTY finish executing and return control.
I've mentioned in the thread "Fleet Forth design considerations" that Fleet Forth's CREATE will add a pad byte before any fields of a word are compiled if the code field or any of the following four cells will straddle a page boundary. This allows Fleet Forth to have multi code field words with up to five code fields, even though it is for a Commodore 64 and the 6510 processor with its indirect jump bug.
To add multi code field VALUE words to the kernel, it was necessary to modify the metacompiler so the first few cells after the code field also avoid straddling a page boundary.
One word in the metacompiler source needed changed. The word PNAME is passed the address where the code field will land. If the code field will straddle a page boundary, PNAME allots a byte, increments the variable PADDED and displays the name of the word which will be padded.
Where the source shows "{reverse}" the Commodore and R keys are pressed. This places the Commodore in reverse video mode.
The new version of PNAME which keeps the code field and the next four cells from straddling a page boundary:
My Forth system has three VALUE words in the kernel. DO.VALUE and TO.VALUE are built without headers or code fields. They have labels. The system DO.VARIABLE also has the label DO.VARIABLE . The three VALUE words are built like this.
4 VALUE #BUF 4 ALLOT
3 , 0 ,
HSUBR DO.VALUE ( -- N )
6 # LDY
' CONSTANT >BODY 8 + JMP
END-CODE
HSUBR TO.VALUE ( N -- )
4 # LDY
0 ,X LDA W )Y STA INY
1 ,X LDA W )Y STA
POP JMP
END-CODE
DO.VALUE ' #BUF !
TO.VALUE ' #BUF 2+ !
DO.VARIABLE ' #BUF 4 + !
4 ' #BUF 6 + !
Defining VALUE words in this way in the kernel seemed a bit clumsy. The following changes were made to the Fleet Forth kernel source to make it easier.
An empty block was added just after the load block and the load block was modified.
The original load block
and the new.
// LOAD BLOCK FOR FLEET FORTH
DECIMAL
1 FH #164 FH THRU
HEX
FINISH-FORTH
START was removed, which means the very next block which is loaded is loaded with metacompiling off.
Note: Fleet Forth VARIABLEs and 2VARIABLEs are created with their parameter fields set to all zeros.
// METACOMPILER VALUE
HOST
2VARIABLE VALUE.CF1
2VARIABLE VALUE.CF2
2VARIABLE VALUE.CF3
META DEFINITIONS FORTH
: VALUE ( N -- )
[HOST]
>IN @ HEADER >IN !
VALUE.CF1 CF,
VALUE.CF2 CF,
VALUE.CF3 CF,
DUP V,
>SHADOW CONSTANT ;
START is added to the top of the very next block to initiate metacompiling.
This definition of VALUE is a metacompiler word used to build the VALUE words in the target kernel being built in virtual memory.
Another change was made to the metacompiler. The word UNCHAIN was factored out of PATCH-CF
// DO.VALUE TO.VALUE
3 , 0 ,
HSUBR DO.VALUE ( -- N )
VALUE.CF1 UNCHAIN
6 # LDY
' CONSTANT >BODY 8 + JMP
END-CODE
HSUBR TO.VALUE ( N -- )
VALUE.CF2 UNCHAIN
4 # LDY
0 ,X LDA W )Y STA INY
1 ,X LDA W )Y STA
POP JMP
END-CODE
UNCHAIN is also used in the last block which has the definition of the kernel's CREATE . CREATE itself is unaltered. The third code field of a VALUE word points to DO.VARIABLE
.
.
.
;CODE
LABEL DO.VARIABLE
VALUE.CF3 UNCHAIN
CLC
W LDA 2 # ADC
W 1+ LDY CS IF INY THEN
AYPUSH JMP END-CODE
With these changes, the '4 ALLOT' after the definitions of the three VALUE words in the kernel was removed and the need to manually set each code field and parameter field was removed.
This was no longer needed:
C@ " USER.AREA" [SHADOW]
['] SHADOW >BODY VFIND
HOST FORTH
0= ABORT" NO USER AREA"
>BODY @ + ;
For 'THERE I first changed it to EMPTY.POINT . The new name is meant to convey the idea that the memory location it references holds the address the dictionary will be pruned back to by EMPTY . I use the word 'prune' because EMPTY performs a proper pruning of the dictionary by setting FENCE equal to the empty point and branching into FORGET with that address on the data stack.
'THERE 0= ABORT" 'THERE IS ZERO"
USER.AREA 0= ABORT" NO USER AREA"
so the overall complexity may be approximately the same.
The idea is <MUSER> searches the dictionary for the word USER.AREA and fetches the data in its parameter field. Similarly, PATCH-FORTH searches the dictionary for EMPTY.POINT and fetches the data from its parameter field.
The nice thing about this approach is it eliminates those clumsy looking sections in the Fleet Forth source.
I've already mentioned elsewhere that using labels within a definition is not a problem. The label is defined in the host dictionary while the target is built in virtual memory. This is why labels can be defined as needed.
When testing a kernel word on the host system, labels can not be created in the middle of a word. Here is a version of LABEL which does not create a new word. It resides in the host Forth vocabulary and requires a label to be a VALUE which has already been defined.
Both versions of LABEL exist in the metacompiler. The first version is defined in the META vocabulary and is found when metacompiling. The second is defined in the host FORTH vocabulary and is found when loading a kernel word into the host for testing.
The metacompiler no longer has two versions of LABEL . The HOST version was removed.
A new version of LABEL can be loaded either with the metacompiler or independent of it for testing words in Fleet Forth's kernel source.
This new non metacompiler version of LABEL does not require the existence of predefined VALUEs, it is built on the shadow words described in the thread on shadow words in Forth.
places the start of the shadowlands five kilobytes from the block buffer table.
In this usage, PRUNE isn't even needed as the words being tested will also be deleted when testing is done; although, it doesn't hurt to have it.
I recently had an idea for state smart DEFINE words. Here is the source for the metacompiler's state smart DEFINE and LABEL , which is based on DEFINE .
META DEFINITIONS
: DEFINE ( W -- )
>SHADOW
CREATE IMMEDIATE
,
DOES>
@ STATE @
IF
[COMPILE] LITERAL
THEN ; IMMEDIATE
: LABEL ( -- )
HERE THERE [COMPILE] DEFINE
HERE SWAP - CDP +! ; IMMEDIATE
HOST
The word >SHADOW has nothing to do with shadow words. The metacompiler lexicon adds four new vocabularies to Fleet Forth. One vocabulary is SHADOW . The metacompiler builds its target in virtual memory; therefore, nothing in the target is executable. Some variables, such as STATE and SPAN , need interpreted while assembling code words. The SHADOW vocabulary has constants with the same names as the target variables to return the correct addresses in the target. There are also shadow constants for the constants in the target image.
With state smart versions of these words, I was able to remove MACRO from the metacompiler source. As useful as macros can be, they were only used for the evaluation of numerical strings.
Not only was the source for the metacompiler simplified ( even if only slightly), there is now one less word needed for the test suite to test changes to a word, or group of words, in Fleet Forth's kernel before building a new kernel.
I mentioned that Fleet Forth's LATEST is a system value set by CREATE . What this means is LATEST is created by the metacompiler as a CONSTANT ; therefore, a constant with the name LATEST is defined in the SHADOW vocabulary at that time. This made it impossible to use LATEST while meta interpreting because the constant defined in the SHADOW vocabulary would be found before any version of LATEST in the META vocabulary, at least until I rewrote the metacompiler yet again and changed the order in which the metacompiler vocabularies are created.
The SHADOW vocabulary is created first.
The META vocabulary is created next.
Then the target FORTH vocabulary.
Finally the metacompiler's ASSEMBLER vocabulary.
Now when meta compiling or meta interpreting, the search order starts with the metacompiler's ASSEMBLER if target assembling, to the target FORTH vocabulary, then the META vocabulary, the SHADOW vocabulary, followed by the host FORTH vocabulary.
