Page 3 of 4
Re: Fleet Forth metacompiler redesign
Posted: Thu Feb 23, 2023 3:42 am
by JimBoyd
If either HEAD or HEADS are ON then the word being defined in virtual memory will have a header. HEAD is normally used to switch headers off for one word at a time. HEADS is normally zero to allow some headerless words and true to forbid headerless words.
NH is used just before a word is defined and is a shortcut for the phrase HEAD OFF .
This behavior is not necessary.
Headerless words can be prevented by redefining NH as a no-op just before metacompiling.
Once this is done and the new target is saved, the dictionary can be pruned back with EMPTY . This will expose the original NH and make the system ready for metacompilation again by removing all the TARGET and SHADOW words except the vocabularies defined in those vocabularies.
The same kernel can then be rebuilt with headerless words enabled to compare the size difference.
Given this realization, HEADS would appear to be unnecessary; however, I have a better use for it.
First, I redefine VHEADER and HEADER .
The line just before IF in the source for VHEADER is changed from this:
to this:
Now the value of the variable HEAD will determine if a header is to be built.
The line to set HEAD in HEADER is changed from this:
to this:
HEADS is now the default flag for header creation. NH switches HEAD off to prevent a word from having a header. It is then set to the value of HEADS .
A few more words will offer some interesting possibilities. NH and friends.
Code: Select all
: CH ( -- ) // CREATE HEADER
HEAD ON ;
: NH ( -- ) // NO HEADER
HEAD OFF ;
: HEADERS ( -- )
HEADS ON CH ;
: NOHEADERS ( -- )
HEADS OFF NH ;
CH will force the creation of a header for the following word in the source just as NH will force the absence of a header for the following word in the source.
HEADERS will make creating headers the default. This is overridden for individual words with NH .
NOHEADERS will make not creating headers the default. This is overridden for individual words with CH .
START is also modified.
The following:
is replaced with this:
There are a few changes to META-STATUS
Code: Select all
: META-STATUS
SETWIDTH
CR ." METACOMPILING " RON META?
IF ." ON" ELSE ." OFF" THEN
ROFF
CR ." STATE: " STATE @ U.W
CR ." WLINK: " WLINK @ U.W
CR ." HEADERS: " HEADS @ U.W
CR ." HEADLESS: " HEADLESS @ U.W
CR ." ORIGIN: " (ORIGIN) @ U.W
CR ." THERE: " TDP @ U.W
CR ." 'THERE: " 'THERE U.W
CR ." USER AREA: " USER.AREA U.W
CR ." PADDED: " PNAMES @ U.W
ORDER ; IMMEDIATE
By placing NOHEADERS at the beginning of the target source for each disk of source, an entire Forth kernel can be made headerless. CH can be used to cause a few select words to have headers.
Does this bring to mind any interesting possibilities?
[Edit: fixed a typo]
Re: Fleet Forth metacompiler redesign
Posted: Thu Feb 23, 2023 10:00 am
by GARTHWILSON
The following is not specifically regarding a metacompiler, rather a comment on your words to direct whether or not headers are compiled:
In my '816 Forth kernel assembly-language source, I used the assembler variables HEADERS? and OMIT_HEADERS. HEADERS? gets turned on and off for local use, whereas OMIT_HEADERS is to be turned on only if you want to do the whole thing without headers (which, besides saving memory, would mean that the target computer cannot do its own compilation or assembly), or at least large sections that may have multiple places saying NO_HEADERS...<some code here>...HEADERS. This was mainly so if you want a totally headerless version, you don't have to comment-out all the invocations of the HEADERS macro which turns on the HEADERS? assembler variable.
Later I've thought I should change this to a stack-based thing (again, in the assembler, not the target computer) so you could nest more levels and still have it remember, when it gets to the end of a section, whether it was supposed to go back to laying down headers or not, rather than just automatically turning headers generation back on. The variables are used by the HEADER (no 'S' or '?' on the end) macro which normally automates the creation of each header but does nothing if it's not supposed to be creating a header at the time. I don't remember at the moment what the situation was that told me I should do this; but a problem situation I'm imagining now is where an INCLude file turns the creation of headers off and on but has no memory of what the situation should be when it finishes and returns assembly to the file that called it. Another is where you might have one or more words you want in a NO_HEADERS...HEADERS section, and if you decide to move it to a different part of the file, or even to an INCLude file, you won't have to see if you'll need to modify it.
I'm slowly laying out a board for the 65816 computer that will use this Forth. Up to now I've only run this Forth on my 65802 which is an '816 which drops into an '02 socket, meaning it gives almost all the advantages of the '816 minus the ability to address anything outside bank 0 (ie, the first 64KB). When the true '816 is up, I'll get back on the '816 Forth and possibly implement this headers-on/off stack to keep track of whether or not headers should be created at any given point in the code. Most of what I need to do for it is just finish and test the material to address data in other banks.
Re: Fleet Forth metacompiler redesign
Posted: Tue Feb 28, 2023 12:37 am
by JimBoyd
I've added two words to fetch and store the header building state.
Code: Select all
: HEADER@ ( -- F )
HEADS @ ;
: HEADER! ( F -- )
DUP HEADS ! HEAD ! ;
Since these words are not intended to be used in the middle of a word's definition, there shouldn't be anything getting in the way on the metacompiler's data stack. It's far more likely that things getting in the way would be on the auxiliary data stack, at least the way I tend to write code like Fleet Forth's kernel.
Code: Select all
CODE ?EXIT ( F -- )
INX INX
$FE ,X LDA $FF ,X ORA
0= IF CS>A
0= NOT IF CS>A END-CODE
CODE 0EXIT ( F -- )
INX INX
$FE ,X LDA $FF ,X ORA
0= NOT IF CS>A
0= IF CS>A END-CODE
CODE ?LEAVE ( F -- )
INX INX
$FE ,X LDA $FF ,X ORA
0= IF CS>A
0= NOT IF CS>A END-CODE
CODE (LOOP)
PLA TAY INY
0= IF // BRANCHING OUT OF WORD
SEC PLA 0 # ADC
VS IF // BRANCHING OUT OF WORD
VS NOT IF
A>CS A>CS THEN CS-SWAP
LABEL LEAVE.BODY
PLA PLA
THEN
PLA PLA
A>CS A>CS THEN
A>CS A>CS THEN
LABEL EXIT.BODY
PLA IP STA PLA IP 1+ STA
THEN THEN THEN CS-SWAP CS>A CS>A
LABEL NEXT
1 # LDY
IP )Y LDA W 1+ STA DEY
IP )Y LDA W STA CLC
IP LDA 2 # ADC IP STA
CS NOT IF
W 1- JMP
THEN
IP 1+ INC
W 1- JMP END-CODE
Here is an example of using HEADER@ and HEADER! . Fleet Forth has a word, SR/W , to read and write individual disk sectors. Here are the headerless words used by SR/W .
Code: Select all
NH
: ",#NUM" ( N -- )
ASCII , DEMIT
0 <# #S #> DTYPE ;
NH
: DOUT ( ADR CNT CHAN -- )
>SLF# CHKOUT IOERR DTYPE ;
NH
: CBP ( -- ) // CLEAR BP
" B-P:3,0" COUNT 15 DOUT
CLRCHN ?D ;
NH
: BCMD ( S T DR ADR -- )
RB DECIMAL
COUNT 15 DOUT
",#NUM" ",#NUM" ",#NUM"
CLRCHN ;
This does the same thing.
Code: Select all
HEADER@ NOHEADERS
: ",#NUM" ( N -- )
ASCII , DEMIT
0 <# #S #> DTYPE ;
: DOUT ( ADR CNT CHAN -- )
>SLF# CHKOUT IOERR DTYPE ;
: CBP ( -- ) // CLEAR BP
" B-P:3,0" COUNT 15 DOUT
CLRCHN ?D ;
: BCMD ( S T DR ADR -- )
RB DECIMAL
COUNT 15 DOUT
",#NUM" ",#NUM" ",#NUM"
CLRCHN ;
HEADER!
By placing NOHEADERS at the beginning of the target source for each disk of source, an entire Forth kernel can be made headerless. CH can be used to cause a few select words to have headers.
START makes header building the default.
RESUME does not affect heading building.
Likewise, if the target source were in an ordinary text file and there were include files, just including the include files would not affect header building. Each include file could change whether headers are built with the words I've mentioned.
Redefining HEADERS and NOHEADERS to reclaim four bytes.
Code: Select all
: CH ( -- ) // CREATE HEADER
HEAD ON ;
: NH ( -- ) // NO HEADER
HEAD OFF ;
: HEADER@ ( -- F )
HEADS @ ;
: HEADER! ( F -- )
DUP HEADS ! HEAD ! ;
: HEADERS ( -- )
TRUE HEADER! ;
: NOHEADERS ( -- )
FALSE HEADER! ;
Re: Fleet Forth metacompiler redesign
Posted: Thu Mar 09, 2023 3:10 am
by JimBoyd
[BEGIN] and [UNTIL] may not be needed on systems with EVALUATE .
Some source in ram block 1.
Code: Select all
10
" CR DUP . 1- ?DUP 0= >IN !"
COUNT EVALUATE
CR .S
and the session log.
Code: Select all
1 RAM LOAD
10
9
8
7
6
5
4
3
2
1
EMPTY OK
When interpreting, " returns the address of a counted string. The Ansi Forth word S" returns the address of a string and its count.
Even with this taken into consideration, this example doesn't work with one of the versions of Gforth on my computers without further modification.
Code: Select all
10
S" CR DUP . 1- ?DUP 0= SOURCE NIP AND >IN !"
EVALUATE
CR .S
My Forth treats the value of >IN as unsigned. Any value of >IN greater or equal to the size of the text stream causes WORD to return a size of zero without causing an error.
My Forth's WORD uses a the word 'STREAM to obtain the address of the current location in the text stream and its remaining length.
Code: Select all
: 'STREAM ( -- ADR N )
BLK @ ?DUP
IF
BLOCK B/BUF
ELSE
TIB #TIB @
THEN
>IN @
OVER UMIN /STRING ;
OVER UMIN clips the value fetched from >IN so it doesn't exceed the size of the text stream. UMIN returns the unsigned minimum of two numbers.
/STRING is a word from Ansi Forth which I found to be quite useful.
Code: Select all
/STRING “slash-string” STRING
( c-addr1 u1 n – – c-addr2 u2 )
Adjust the character string at c-addr1 by n characters. The
resulting character string, specified by c-addr2 u2, begins
at c-addr1 plus n characters and is u1 minus n characters long.
Re: Fleet Forth metacompiler redesign
Posted: Thu Mar 09, 2023 3:20 am
by GARTHWILSON
very interesting idea (in a sense looping in a string that's being interpreted) !
Re: Fleet Forth metacompiler redesign
Posted: Thu Mar 09, 2023 3:51 am
by JimBoyd
There are a few places in the source for Fleet Forth where I use EVALUATE .
The following fills in the table holding the default vector for all the deferred words defined in Fleet Forth's kernel. This is done after the vectors for the deferred words have been set.
Code: Select all
START.I&F
" TARGET DUP @ @ OVER 2+ ! 4 +
END.FORGET OVER 2+ U<
HOST >IN !"
COUNT EVALUATE TARGET
DROP
I also use EVALUATEd strings to automate defining parameters to use while building the kernel. For example, in the source for the kernel there is the following:
That number is the largest number of blocks a dual disk version of the 1581 drive could hold, if such a device existed. The EVALUATEd strings which follow round that number up to the closest power of 2 and set other parameters used to define words such as R/W , RAM and DR+ .
Code: Select all
// BLOCK SUBSYSTEM PARAMETERS
// THESE PARAMETERS ARE DETERMINED
// BY THE VALUE DR.OFFSET
DR.OFFSET $100 UMAX $8000
" HOST 1 RSHIFT 2DUP SWAP U< >IN !"
COUNT EVALUATE TARGET
NIP 7 RSHIFT DEFINE DR.OFS.HI
0 DR.OFS.HI
" HOST 1 UNDER+ 2/ ?DUP 0= >IN !"
COUNT EVALUATE TARGET 7 + 0
HEX <# #S ASCII $ HOLD #> DECIMAL
MACRO DR.OFS.PWR
#DRIVES DR.OFS.PWR LSHIFT 0
HEX <# #S ASCII $ HOLD #> DECIMAL
MACRO RAM.OFFSET
Re: Fleet Forth metacompiler redesign
Posted: Sun Mar 19, 2023 10:47 pm
by JimBoyd
The source for R/W , RAM and DR+ . Note: RR/W and DR/W are DEFERred words.
Code: Select all
CODE R/W ( ADR BLK# R/WF CNT -- )
BEGIN
SEC 5 ,X LDA DR.OFS.HI # SBC
CS WHILE
5 ,X STA INY #DRIVES # CPY
0= UNTIL
' RR/W SPLIT SWAP
# LDA # LDY
(EXECUTE) 0= NOT BRAN
THEN
DRB STY
' DR/W SPLIT SWAP
# LDA # LDY
(EXECUTE) 0= NOT BRAN END-CODE
: RAM ( BLK#1 -- BLK#2 )
RAM.OFFSET + ;
: DR+ ( BLK#1 #DR -- BLK#2 )
7 AND DR.OFS.PWR LSHIFT + ;
and what gets assembled and compiled.
Code: Select all
R/W
3931 SEC
3932 5 ,X LDA
3934 8 # SBC
3936 3951 BCC
3938 5 ,X STA
3940 INY
3941 8 # CPY
3943 3931 ^^ BNE
3945 184 # LDA
3947 11 # LDY
3949 3920 ^^ BNE ' EXECUTE >BODY 6 +
3951 2686 STY ' DRB >BODY
3954 173 # LDA
3956 11 # LDY
3958 3920 ^^ BNE ' EXECUTE >BODY 6 +
29
RAM
9988 2497 LIT 16384
9992 4944 +
9994 2469 EXIT
8
DR+
12004 3308 CLIT 7
12007 4556 AND
12009 3308 CLIT 11
12012 5268 LSHIFT
12014 4944 +
12016 2469 EXIT
14
The use of RAM and DR+ is mentioned here.
R/W is Fleet Forth's BLOCK reading and writing word.
Re: Fleet Forth metacompiler redesign
Posted: Thu Mar 23, 2023 2:25 am
by JimBoyd
The redesign of Fleet Forth's metacompiler is complete. I'm fairly certain I presented all the source and covered everything.
Any questions?
Re: Fleet Forth metacompiler redesign
Posted: Thu Mar 23, 2023 2:30 am
by GARTHWILSON
Just use it a lot to wring out any bugs, and write a good manual for it.
Re: Fleet Forth metacompiler redesign
Posted: Tue Mar 28, 2023 1:52 am
by JimBoyd
The new metacompiler's first big test was metacompiling an existing version of Fleet Forth. Since I'm using VICE instead of a real Commodore 64, the disks are actually disk images and the disk image with the new kernel was identical to the one created with the original metacompiler. I also have SEEALL to test kernels I build. Successfully loading the system loader and the utilities is a test. SEEALL showing the expected disassembly/decompilation is another.
Code: Select all
: (SEE) ( CFA -- )
DUP CR .NAME
SETWIDTH <SEE> ;
: SEE ( -- ) ' (SEE) ;
: SEEALL ( -- )
WITH-WORDS NAME> (SEE)
COLS ?CR ;
I've simplified some of the more obscure code for the metacompiler by adding two new words. These new words use the coroutine word CO
Code: Select all
: >META ( -- )
ORDER@
META DEFINITIONS
CO ORDER! ;
: >SHADOW ( -- )
ORDER@
[SHADOW] SHADOW DEFINITIONS
CO ORDER! ;
The word >META saves the values of CONTEXT and CURRENT to the auxiliary stack then changes the search and compile vocabularies to META . CO causes the rest of >META to run after >META's caller exits. That last part of >META restores the CONTEXT and CURRENT vocabularies.
>SHADOW does the same thing for the SHADOW vocabulary.
This source
Code: Select all
: DEFINE ( W -- )
ORDER@
[SHADOW] SHADOW DEFINITIONS
CONSTANT ORDER! ;
META DEFINITIONS
: MACRO ( ADR CNT -- )
ORDER@
[SHADOW] SHADOW DEFINITIONS
[HOST] MACRO ORDER! ;
HOST
: <MCONSTANT> ( N ADR -- )
>IN @ HEADER >IN !
CF, DUP V,
ORDER@
[SHADOW] SHADOW DEFINITIONS [HOST]
CONSTANT
ORDER! ;
: <M2CONSTANT> ( D ADR -- )
>IN @ HEADER >IN !
CF, 2DUP V, V,
ORDER@
[SHADOW] SHADOW [HOST]
DEFINITIONS 2CONSTANT
ORDER! ;
: <MCREATE> ( ADR -- )
>IN @ HEADER >IN ! CF,
ORDER@
[SHADOW] SHADOW DEFINITIONS
THERE CONSTANT
ORDER! ;
: <<MUSER>> ( C -- )
CREATE
C,
[ HERE 2+ >A ]
DOES>
C@ USER.AREA + ;
: <MUSER> ( C ADR -- )
>IN @ HEADER >IN ! CF,
DUP VC,
ORDER@
[SHADOW] SHADOW [HOST]
DEFINITIONS
<<MUSER>>
ORDER! ;
A> CONSTANT MUSER-CF
is simplified to this
Code: Select all
: DEFINE ( W -- )
>SHADOW CONSTANT ;
META DEFINITIONS
: MACRO ( ADR CNT -- )
>SHADOW MACRO ;
HOST
: <MCONSTANT> ( N ADR -- )
>IN @ HEADER >IN !
CF, DUP V,
>SHADOW CONSTANT ;
: <M2CONSTANT> ( D ADR -- )
>IN @ HEADER >IN !
CF, 2DUP V, V,
>SHADOW 2CONSTANT ;
: <MCREATE> ( ADR -- )
>IN @ HEADER >IN ! CF,
>SHADOW THERE CONSTANT ;
0 VALUE MUSER-CF
: <MUSER> ( C ADR -- )
>IN @ HEADER >IN ! CF,
DUP VC,
>SHADOW
CREATE
C,
[ HERE 2+ TO MUSER-CF ]
DOES>
C@ USER.AREA + ;
Likewise, this source
Code: Select all
: DEFINER ( CFA -- )
['] ORDER! >BODY >R
ORDER@ META DEFINITIONS
CREATE
, 0 , 0 , 0 ,
[ HERE 2+ >A ]
DOES>
DUP 2+ SWAP @ EXECUTE ;
A> CONSTANT DEFINER-WORD
: DEF-RESET ( -- )
['] ORDER! >BODY >R
ORDER@ META WITH-WORDS
NAME> DUP @ DEFINER-WORD <>
IF DROP EXIT THEN
>BODY 2+ 6 ERASE ;
: VERIFY ( -- )
['] ORDER! >BODY >R
ORDER@ META WITH-WORDS
DUP NAME> DUP @ DEFINER-WORD <>
IF 2DROP EXIT THEN
>BODY 2+ @
IF DROP EXIT THEN
CR RON ID.
." . NOT DEFINED IN TARGET" ;
is simplified to this
Code: Select all
: DEFINER ( CFA -- )
>META
CREATE
, 0 , 0 , 0 ,
[ HERE 2+ >A ]
DOES>
DUP 2+ SWAP @ EXECUTE ;
A> CONSTANT DEFINER-WORD
: DEF-RESET ( -- )
>META WITH-WORDS
NAME> DUP @ DEFINER-WORD <>
IF DROP EXIT THEN
>BODY 2+ 6 ERASE ;
: VERIFY ( -- )
>META WITH-WORDS
DUP NAME> DUP @ DEFINER-WORD <>
IF 2DROP EXIT THEN
>BODY 2+ @
IF DROP EXIT THEN
CR RON ID.
." . NOT DEFINED IN TARGET" ;
The metacompiler built with these changes successfully metacompiled Fleet Forth's kernel.
Re: Fleet Forth metacompiler redesign
Posted: Tue Mar 28, 2023 8:48 am
by BigEd
That's quite the milestone - congratulations!
Re: Fleet Forth metacompiler redesign
Posted: Thu Apr 06, 2023 2:03 am
by JimBoyd
I mentioned that the metacompiler supports LABELs because the target is built in virtual memory. The version of LABEL which is used to test compile target source on the host uses VALUE's which are defined before the test code. The HOST version of LABEL was defined like this:
Code: Select all
: LABEL ( -- )
CONTEXT @
FORTH // HOST FORTH VOCABULARY
HERE ' DUP @
[ ' #BUF @ ] LITERAL <>
ABORT" NOT A LABEL"
>BODY ! CONTEXT ! ; IMMEDIATE
I was testing a modification to Fleet Forth. Some of the words branch to NEXT . This version of LABEL wasn't going to work since I needed some of the test words to branch to the address for NEXT in one of the test words. I needed a VALUE defined in the host assembler to successfully test the new code.
This is the new definition of the HOST version of LABEL .
Code: Select all
: LABEL ( -- )
HERE ' DUP @
[ ' #BUF @ ] LITERAL <>
ABORT" NOT A LABEL"
>BODY ! ; IMMEDIATE
To test the modification to Fleet Forth on the HOST , I needed to be able to alter the address of NEXT in the HOST ASSEMBLER yet be able to EMPTY the dictionary back to its pre test state and have NEXT assume its former contents.
Code: Select all
ASSEMBLER DEFINITIONS
NEXT VALUE NEXT
FORTH DEFINITIONS
Once a VALUE is defined for each label in the test code, the test code (including NEXT and the word which passes through it) is loaded.
Testing then commences; although, the first test was loading the source to see if all the intended branches to NEXT were within range. Fleet Forth's assembler and the metacompiler's assembler both abort with an error message if the intended branch is out of range.
BO for branch offset.
Code: Select all
: BO ( DEST SOURCE -- BRANCH.OFFSET )
1+ - DUP 0< OVER ABS + 7F >
ABORT" BRANCH RANGE EXCEEDED" ;
Once testing is done EMPTY is used to prune back the dictionary and forget the VALUE NEXT which exposes the original CONSTANT NEXT .
Re: Fleet Forth metacompiler redesign
Posted: Sun Apr 16, 2023 8:51 pm
by JimBoyd
Another test run called for another modification to one of the metacompiler words.
Fleet Forth has SUBR to create a word which returns its own PFA like a VARIABLE . Unlike a variable, no storage is allotted, the assembler is invoked. For example, (>FORTH) returns the address of the code used to transition from a code word to high level Forth.
Code: Select all
SUBR (>FORTH) ( -- )
CLC
PLA 1 # ADC TAY
PLA 0 # ADC
LABEL (CO)
N STA
IP 1+ LDA PHA
IP LDA PHA
N LDA IP STY
' EXIT @ 4 + JMP
END-CODE
>FORTH assembles a JSR to (>FORTH) as well as setting STATE to compiling and setting CONTEXT equal to CURRENT .
Code: Select all
: >FORTH ( -- )
?EXEC
(>FORTH) JSR
[ FORTH ]
CURRENT @ CONTEXT !
] ; IMMEDIATE
A SUBR or subroutine in Fleet Forth is a word which returns the address of a routine which is exactly that, a subroutine.
Code: Select all
SUBR ROUTINE1
<SOME ASSEMBLY>
RTS END-CODE
CODE TESTWORD
ROUTINE1 JSR
<SOME MORE ASSEMBLY>
NEXT JMP END-CODE
The metacompiler's SUBR works as expected.The metacompiler also supports HSUBRs, subroutines without headers. Since there is no header, there is no need for a code field. HSUBR creates a CONSTANT on the host containing the address of the routine in virtual memory.
The problem arose when testing target source on the host. When testing code on the host, HSUBR would just call the host assembler's SUBR . This normally works. When it didn't work was when there was a branch in range on the target because there was no header or code field to get in the way, but the test on the host didn't assemble because the branch was out of range. The header and code field got in the way. The new version of HSUBR works the same as the old when metacompiling. When testing on the host, it requires the name which follows in the text stream to be a predefined VALUE just like the host version of LABEL . Now there is no header or code field to get in the way of code during testing on the host.
Code: Select all
0 VALUE DOUBLE.COMPARE \ These definitions are only needed
0 VALUE POP3.TRUE \ when testing on the host
0 VALUE POP3.FALSE \ not when metacompiling
HSUBR DOUBLE.COMPARE
6 ,X LDA 2 ,X CMP
7 ,X LDA 3 ,X SBC
4 ,X LDA 0 ,X SBC
5 ,X LDA 1 ,X SBC
RTS END-CODE
CODE DU< ( UD1 UD2 -- F )
DOUBLE.COMPARE JSR
CS NOT IF
LABEL POP3.TRUE
DEY
LABEL POP3.FALSE
THEN
6 ,X STY 7 ,X STY INX INX
POPTWO JMP END-CODE
CODE D< ( D1 D2 -- F )
DOUBLE.COMPARE JSR
VS IF $80 # EOR THEN
POP3.TRUE 0< BRAN
POP3.FALSE 0< NOT BRAN
Source for the new HSUBR .
Code: Select all
: HSUBR ( -- )
META?
IF
THERE CONSTANT ASSEMBLE EXIT
THEN
[COMPILE] LABEL [ ASSEMBLER ]
W TRUE ASSEMBLER MEM ;
I found out about this problem when testing a modification for source close to NEXT . The modification should have worked. I tested it on the host before committing to building a new kernel with the metacompiler. When I did, I got the dreaded "BRANCH RANGE EXCEEDED" error.
I did not get that error when testing with the new version of HSUBR .
Re: Fleet Forth metacompiler redesign
Posted: Mon Jul 03, 2023 11:39 pm
by JimBoyd
As I've already mentioned, for each type of CREATE DOES> or CREATE ;CODE word supported by the metacompiler, all the code fields for a given class of word are linked in a chain until the parent word of that type is defined for the target. CODE words are the only exception. I also mentioned the problem I ran into with trying to use CREATE rather than CODE for code words with no body. There is another problem which only occurs when trying to minimize the amount of virtual memory needed to build the Forth kernel. Some Commodore 64s may not have a Ram Expansion Unit. In this case, it is possible to write a version of (RR/W) to access the ram underneath the kernal ROM and I/O.
A reminder, this is how the current build of Fleet Forth maps blocks on the computer to blocks on devices.
Code: Select all
Blocks: Device: Range seen by device:
0 - $7FF 8 0 - $7FF
$800 - $FFF 9 0 - $7FF
.
.
.
$3800 - $3FFF 15 0 - $7FF
$4000 - $FFFF REU 0 - $BFFF
Each disk drive "sees" the blocks in the range 0 to 2047 and none of them have the capacity for 2048 blocks. Most have considerably less.
The target built by the metacompiler loads at location $801 so the first $801 bytes of virtual memory are not used to build it. On a Commodore 64 without a Ram Expansion Unit, virtual memory could be set to start two blocks below the start of blocks used from ram by setting VOFFSET like this:
This will reduce by two the number of blocks needed for virtual memory; however, the first $800 bytes of virtual memory are mapped to blocks $3FFE - $3FFF and are no longer accessible.
There is a side effect with the version of chains which uses two cells, one for the latest link and one for the previous link. Initialization causes a read from memory location zero in virtual memory. I could have written a test to avoid this, but it was easier to just remove the unnecessary feature form the metacompiler.
MEND-CHAIN and ADD-CHAIN were removed. The following words were rewritten.
Code: Select all
: CF, ( ADR -- )
DUP @
IF @ V, EXIT THEN
2+ VADD ;
: DEFINER ( CFA -- )
>META
CREATE
, 0 , 0 ,
// CFA OF META WORD TO EXECUTE
// FLAG -- ZERO OR TARGET CFA
// CHAIN OF WORDS
[ HERE 2+ >A ]
DOES>
DUP 2+ SWAP @ EXECUTE ;
A> CONSTANT DEFINER-WORD
: PATCH-CF ( -- )
LATEST COUNT $1F AND >HERE CLIP
['] META >BODY VFIND
0= ABORT" DEFINER MISSING"
DUP @ DEFINER-WORD <>
ABORT" NOT A DEFINER WORD"
>BODY 2+ THERE OVER !
2+ @
BEGIN
?DUP 0EXIT
DUP V@ THERE ROT V!
AGAIN -;
: DEF-RESET ( -- )
>META WITH-WORDS
NAME> DUP @ DEFINER-WORD <>
IF DROP EXIT THEN
>BODY 2+ 4 ERASE ;
The code field patching portion of PATCH-CF is slightly smaller. I also improved VTRAVERSE .
Code: Select all
: VTRAVERSE ( VADR DIR -- VADR2 )
BEGIN
DUP UNDER+ OVER VC@ $7F >
UNTIL
DROP ;
I have already tested these changes to the metacompiler by building a Forth kernel and comparing it to the latest build. They were identical.
Re: Fleet Forth metacompiler redesign
Posted: Tue Sep 12, 2023 6:36 pm
by JimBoyd
The DO LOOP run-time words are not specifically mentioned in the Forth-83 Standard and could have different names. A DO LOOP run-time word could even have the same name as the word which compiles it; however, the metacompiler does not allow redefinitions in the TARGET vocabulary.
To summarize the previous posts, while metacompiling is on:
When a target word is found which is to be interpreted, because STATE is off or the word is immediate, the search continues with the parent vocabulary.
When a NON target word is found which is to be compiled, the search continues with the parent vocabulary.
Only non-immediate target words can be compiled without the use of [COMPILE] and only non-target words can be interpreted.
Consider simply changing the name of (DO) to DO .
Code: Select all
CODE DO ( LIMIT START -- )
...
...
END-CODE
There could be DO LOOPs in the source before the compiling word DO is defined. Indeed, DO and the other DO LOOP compiling words might not even be defined in the source for the Forth kernel. They could just as easily be defined in the system loader which is loaded by the new kernel. Rather than encountering the immediate word DO and continuing the search in a parent vocabulary, the non-immediate run-time DO would be compiled. The DO LOOPs would not be compiled correctly.
In spite of this, there is a way to give the run-time for DO the same name. A helper word needs added to the metacompiler.
Code: Select all
ALIAS DO
CODE (DO) ( LIMIT START -- )
...
...
END-CODE
The TARGET FORTH vocabulary on the host will now have the name (DO) but the target in virtual memory will have the name DO .
This will work; however, there is one gotcha. When the system is built on this new kernel and the metacompiler lexicon built on top of that, the metacompiler on the new system will not be able to compile DO LOOPs.
In solving this problem I came up with two different solutions which resulted in two versions of the metacompiler. They were the same except for what feature was added.
Both versions of the metacompiler worked. I tested each one by metacompiling the new kernel then building the system and metacompiler on top of that. The new metacompiler on the new system was then used to metacompile a kernel again. The kernels were identical. All four of them.
Access to the DO LOOP run-time words was still needed to build the high level portion of SEE . The run-time words are defined before the word FORTH-83 and the DO LOOP compiling words are defined after FORTH-83 . Fleet Forth's FIND searches the CONTEXT vocabulary (and parents) then, if necessary, searches the CURRENT vocabulary.
Code: Select all
: FIND ( ADR -- ADR2 F )
CONTEXT @ (FIND)
?DUP ?EXIT
CURRENT @ VFIND ;
To access the DO LOOP run-time words a false vocabulary was created and CONTEXT was pointed to it.
Code: Select all
' FORTH-83 >LINK 2 ! 4 OFF \ Create false vocabulary starting at FORTH-83
2 CONTEXT ! \ and point CONTEXT there.
AMONG (?BRAN)
] BRANCH ?BRANCH
DO ?DO LOOP +LOOP
[ 0 ,
CONTEXT was reset after the word (?BRAN) was created.
SEE works fine on the new system.
Code: Select all
OK
: TEST
5 0
DO CR I . LOOP ; OK
TEST
0
1
2
3
4 OK
SEE TEST
TEST
23855 3332 CLIT 5
23858 3317 0
23860 2313 DO 23874
23864 6753 CR
23866 2417 I
23868 7492 .
23870 2195 LOOP 23864
23874 2472 EXIT
21
OK
SEE DO
DO IMMEDIATE
11718 8406 COMPILE
11720 2313 DO
11722 11474 >MARK
11724 4843 2+
11726 11494 <MARK
11728 4843 2+
11730 2472 EXIT
14
OK
2313 (SEE)
DO
2315 INY
2316 251 )Y LDA IP
2318 PHA
2319 DEY
2320 251 )Y LDA IP
2322 PHA
2323 CLC
2324 3 ,X LDA
2326 128 # ADC
2328 PHA
2329 3 ,X STA
2331 2 ,X LDA
2333 PHA
2334 SEC
2335 0 ,X LDA
2337 2 ,X SBC
2339 TAY
2340 1 ,X LDA
2342 3 ,X SBC
2344 PHA
2345 TYA
2346 PHA
2347 INX
2348 INX
2349 INX
2350 INX
2351 2275 ^^ BPL ' ?BRANCH >BODY 8 +
2353 2250 ^^ BMI ' ! >BODY 20 +
40
OK
Although it is a little odd to SEE DO and notice that DO compiles DO .