6502.org • View topic - Fleet Forth metacompiler redesign

View unanswered posts | View active topics

Board index » 6502.org Users Forum » Forth

All times are UTC

Fleet Forth metacompiler redesign

Page 3 of 4

[ 51 posts ]

Go to page Previous 1, 2, 3, 4 Next

Previous topic | Next topic

Author

Message

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Thu Feb 23, 2023 3:42 am

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

JimBoyd wrote:

If either HEAD or HEADS are ON then the word being defined in virtual memory will have a header. HEAD is normally used to switch headers off for one word at a time. HEADS is normally zero to allow some headerless words and true to forbid headerless words.
NH is used just before a word is defined and is a shortcut for the phrase HEAD OFF .

This behavior is not necessary.
Headerless words can be prevented by redefining NH as a no-op just before metacompiling.

Code:

: NH ( -- ) ;

Once this is done and the new target is saved, the dictionary can be pruned back with EMPTY . This will expose the original NH and make the system ready for metacompilation again by removing all the TARGET and SHADOW words except the vocabularies defined in those vocabularies.
The same kernel can then be rebuilt with headerless words enabled to compare the size difference.

Given this realization, HEADS would appear to be unnecessary; however, I have a better use for it.
First, I redefine VHEADER and HEADER .
The line just before IF in the source for VHEADER is changed from this:

Code:

HEAD @ HEADS @ OR

to this:

Code:

HEAD @

Now the value of the variable HEAD will determine if a header is to be built.

The line to set HEAD in HEADER is changed from this:

Code:

HEAD ON

to this:

Code:

HEADS @ HEAD !

HEADS is now the default flag for header creation. NH switches HEAD off to prevent a word from having a header. It is then set to the value of HEADS .
A few more words will offer some interesting possibilities. NH and friends.

Code:

: CH  ( -- )  // CREATE HEADER
   HEAD ON ;
: NH  ( -- )  // NO HEADER
   HEAD OFF ;
: HEADERS  ( -- )
   HEADS ON CH ;
: NOHEADERS  ( -- )
   HEADS OFF NH ;

CH will force the creation of a header for the following word in the source just as NH will force the absence of a header for the following word in the source.
HEADERS will make creating headers the default. This is overridden for individual words with NH .
NOHEADERS will make not creating headers the default. This is overridden for individual words with CH .

START is also modified.
The following:

Code:

HEAD ON HEADS OFF

is replaced with this:

Code:

HEADERS

There are a few changes to META-STATUS

Code:

: META-STATUS
   SETWIDTH
   CR ." METACOMPILING "  RON META?
   IF  ." ON"  ELSE  ." OFF"  THEN
   ROFF
   CR ."     STATE: " STATE @ U.W
   CR ."     WLINK: " WLINK @ U.W
   CR ."   HEADERS: " HEADS @ U.W
   CR ."  HEADLESS: " HEADLESS @ U.W
   CR ."    ORIGIN: " (ORIGIN) @ U.W
   CR ."     THERE: " TDP @ U.W
   CR ."    'THERE: " 'THERE U.W
   CR ." USER AREA: " USER.AREA U.W
   CR ."    PADDED: " PNAMES @ U.W
   ORDER ; IMMEDIATE

By placing NOHEADERS at the beginning of the target source for each disk of source, an entire Forth kernel can be made headerless. CH can be used to cause a few select words to have headers.
Does this bring to mind any interesting possibilities?

[Edit: fixed a typo]

Last edited by JimBoyd on Tue Feb 28, 2023 2:01 am, edited 1 time in total.

Top

GARTHWILSON

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Thu Feb 23, 2023 10:00 am

Joined: Fri Aug 30, 2002 1:09 am
Posts: 8543
Location: Southern California

The following is not specifically regarding a metacompiler, rather a comment on your words to direct whether or not headers are compiled:

In my '816 Forth kernel assembly-language source, I used the assembler variables HEADERS? and OMIT_HEADERS. HEADERS? gets turned on and off for local use, whereas OMIT_HEADERS is to be turned on only if you want to do the whole thing without headers (which, besides saving memory, would mean that the target computer cannot do its own compilation or assembly), or at least large sections that may have multiple places saying NO_HEADERS...<some code here>...HEADERS. This was mainly so if you want a totally headerless version, you don't have to comment-out all the invocations of the HEADERS macro which turns on the HEADERS? assembler variable.

Later I've thought I should change this to a stack-based thing (again, in the assembler, not the target computer) so you could nest more levels and still have it remember, when it gets to the end of a section, whether it was supposed to go back to laying down headers or not, rather than just automatically turning headers generation back on. The variables are used by the HEADER (no 'S' or '?' on the end) macro which normally automates the creation of each header but does nothing if it's not supposed to be creating a header at the time. I don't remember at the moment what the situation was that told me I should do this; but a problem situation I'm imagining now is where an INCLude file turns the creation of headers off and on but has no memory of what the situation should be when it finishes and returns assembly to the file that called it. Another is where you might have one or more words you want in a NO_HEADERS...HEADERS section, and if you decide to move it to a different part of the file, or even to an INCLude file, you won't have to see if you'll need to modify it.

I'm slowly laying out a board for the 65816 computer that will use this Forth. Up to now I've only run this Forth on my 65802 which is an '816 which drops into an '02 socket, meaning it gives almost all the advantages of the '816 minus the ability to address anything outside bank 0 (ie, the first 64KB). When the true '816 is up, I'll get back on the '816 Forth and possibly implement this headers-on/off stack to keep track of whether or not headers should be created at any given point in the code. Most of what I need to do for it is just finish and test the material to address data in other banks.

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Tue Feb 28, 2023 12:37 am

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

I've added two words to fetch and store the header building state.

Code:

: HEADER@  ( -- F )
   HEADS @ ;
: HEADER!  ( F -- )
   DUP HEADS ! HEAD ! ;

Since these words are not intended to be used in the middle of a word's definition, there shouldn't be anything getting in the way on the metacompiler's data stack. It's far more likely that things getting in the way would be on the auxiliary data stack, at least the way I tend to write code like Fleet Forth's kernel.

Code:

CODE ?EXIT  ( F -- )
   INX  INX
   $FE ,X LDA  $FF ,X ORA
   0= IF CS>A
   0= NOT IF CS>A  END-CODE
CODE 0EXIT  ( F -- )
   INX  INX
   $FE ,X LDA  $FF ,X ORA
   0= NOT IF CS>A
   0= IF CS>A  END-CODE
CODE ?LEAVE  ( F -- )
   INX  INX
   $FE ,X LDA  $FF ,X ORA
   0= IF CS>A
   0= NOT IF CS>A  END-CODE
CODE (LOOP)
   PLA  TAY  INY
   0= IF  // BRANCHING OUT OF WORD
   SEC  PLA  0 # ADC
   VS IF  // BRANCHING OUT OF WORD
   VS NOT IF
      A>CS  A>CS  THEN  CS-SWAP
      LABEL  LEAVE.BODY
      PLA  PLA
   THEN
   PLA  PLA
   A>CS A>CS THEN
   A>CS A>CS THEN
   LABEL EXIT.BODY
   PLA  IP STA   PLA  IP 1+ STA
   THEN THEN THEN  CS-SWAP CS>A CS>A
   LABEL NEXT
   1 # LDY
   IP )Y LDA  W 1+ STA  DEY
   IP )Y LDA  W    STA  CLC
   IP LDA  2 # ADC  IP STA
   CS NOT IF
      W 1- JMP
   THEN
   IP 1+ INC
   W 1- JMP  END-CODE

Here is an example of using HEADER@ and HEADER! . Fleet Forth has a word, SR/W , to read and write individual disk sectors. Here are the headerless words used by SR/W .

Code:

NH
: ",#NUM"  ( N -- )
   ASCII , DEMIT
   0 <# #S #> DTYPE ;
NH
: DOUT  ( ADR CNT CHAN -- )
   >SLF# CHKOUT IOERR  DTYPE ;
NH
: CBP  ( -- )  // CLEAR BP
   " B-P:3,0" COUNT 15 DOUT
   CLRCHN ?D ;
NH
: BCMD  ( S T DR ADR -- )
   RB DECIMAL
   COUNT 15 DOUT
   ",#NUM" ",#NUM" ",#NUM"
   CLRCHN ;

This does the same thing.

Code:

HEADER@  NOHEADERS
: ",#NUM"  ( N -- )
   ASCII , DEMIT
   0 <# #S #> DTYPE ;
: DOUT  ( ADR CNT CHAN -- )
   >SLF# CHKOUT IOERR  DTYPE ;
: CBP  ( -- )  // CLEAR BP
   " B-P:3,0" COUNT 15 DOUT
   CLRCHN ?D ;
: BCMD  ( S T DR ADR -- )
   RB DECIMAL
   COUNT 15 DOUT
   ",#NUM" ",#NUM" ",#NUM"
   CLRCHN ;
HEADER!

JimBoyd wrote:

By placing NOHEADERS at the beginning of the target source for each disk of source, an entire Forth kernel can be made headerless. CH can be used to cause a few select words to have headers.

Although that will work, it's not necessary. Since RESUME does not affect the building of headers, NOHEADERS would only need to be at the beginning of the target source after START on the first source disk.
START makes header building the default.
RESUME does not affect heading building.

Likewise, if the target source were in an ordinary text file and there were include files, just including the include files would not affect header building. Each include file could change whether headers are built with the words I've mentioned.

Redefining HEADERS and NOHEADERS to reclaim four bytes.

Code:

: CH  ( -- )  // CREATE HEADER
   HEAD ON ;
: NH  ( -- )  // NO HEADER
   HEAD OFF ;
: HEADER@  ( -- F )
   HEADS @ ;
: HEADER!  ( F -- )
   DUP HEADS ! HEAD ! ;
: HEADERS  ( -- )
   TRUE HEADER! ;
: NOHEADERS  ( -- )
   FALSE HEADER! ;

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Thu Mar 09, 2023 3:10 am

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

[BEGIN] and [UNTIL] may not be needed on systems with EVALUATE .
Some source in ram block 1.

Code:

10
" CR DUP . 1- ?DUP 0= >IN !"
COUNT EVALUATE
CR .S

and the session log.

Code:

RAM LOAD 










EMPTY  OK

When interpreting, " returns the address of a counted string. The Ansi Forth word S" returns the address of a string and its count.
Even with this taken into consideration, this example doesn't work with one of the versions of Gforth on my computers without further modification.

Code:

10
S" CR DUP . 1- ?DUP 0= SOURCE NIP AND >IN !"
EVALUATE
CR .S

My Forth treats the value of >IN as unsigned. Any value of >IN greater or equal to the size of the text stream causes WORD to return a size of zero without causing an error.
My Forth's WORD uses a the word 'STREAM to obtain the address of the current location in the text stream and its remaining length.

Code:

: 'STREAM  ( -- ADR N )
   BLK @ ?DUP
   IF
      BLOCK B/BUF
   ELSE
      TIB #TIB @
   THEN
   >IN @
   OVER UMIN /STRING ;

OVER UMIN clips the value fetched from >IN so it doesn't exceed the size of the text stream. UMIN returns the unsigned minimum of two numbers.
/STRING is a word from Ansi Forth which I found to be quite useful.

Code:

/STRING   “slash-string”   STRING
   ( c-addr1 u1 n – – c-addr2 u2 )
   Adjust the character string at c-addr1 by n characters. The
   resulting character string, specified by c-addr2 u2, begins
   at c-addr1 plus n characters and is u1 minus n characters long.

Top

GARTHWILSON

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Thu Mar 09, 2023 3:20 am

Joined: Fri Aug 30, 2002 1:09 am
Posts: 8543
Location: Southern California

very interesting idea (in a sense looping in a string that's being interpreted) !

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Thu Mar 09, 2023 3:51 am

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

There are a few places in the source for Fleet Forth where I use EVALUATE .
The following fills in the table holding the default vector for all the deferred words defined in Fleet Forth's kernel. This is done after the vectors for the deferred words have been set.

Code:

START.I&F
" TARGET  DUP @ @ OVER 2+ ! 4 +
  END.FORGET OVER 2+ U<
  HOST >IN !"
COUNT EVALUATE  TARGET
DROP

I also use EVALUATEd strings to automate defining parameters to use while building the kernel. For example, in the source for the kernel there is the following:

Code:

$63E DEFINE DR.OFFSET

That number is the largest number of blocks a dual disk version of the 1581 drive could hold, if such a device existed. The EVALUATEd strings which follow round that number up to the closest power of 2 and set other parameters used to define words such as R/W , RAM and DR+ .

Code:

// BLOCK SUBSYSTEM PARAMETERS
// THESE PARAMETERS ARE DETERMINED
// BY THE VALUE DR.OFFSET
DR.OFFSET $100 UMAX $8000
" HOST 1 RSHIFT 2DUP SWAP U< >IN !"
COUNT EVALUATE  TARGET
NIP 7 RSHIFT DEFINE DR.OFS.HI
0 DR.OFS.HI
" HOST 1 UNDER+ 2/ ?DUP 0= >IN !"
COUNT EVALUATE  TARGET  7 + 0
HEX <# #S ASCII $ HOLD #> DECIMAL
MACRO DR.OFS.PWR
#DRIVES DR.OFS.PWR LSHIFT 0
HEX <# #S ASCII $ HOLD #> DECIMAL
MACRO RAM.OFFSET

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Sun Mar 19, 2023 10:47 pm

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

The source for R/W , RAM and DR+ . Note: RR/W and DR/W are DEFERred words.

Code:

CODE R/W  ( ADR BLK# R/WF CNT -- )
   BEGIN
      SEC  5 ,X LDA  DR.OFS.HI # SBC
   CS WHILE
      5 ,X STA  INY  #DRIVES # CPY
   0= UNTIL
      ' RR/W SPLIT SWAP
      # LDA  # LDY
      (EXECUTE) 0= NOT BRAN
   THEN
   DRB STY
   ' DR/W SPLIT SWAP
   # LDA  # LDY
   (EXECUTE) 0= NOT BRAN  END-CODE

: RAM  ( BLK#1 -- BLK#2 )
   RAM.OFFSET + ;

: DR+  ( BLK#1 #DR -- BLK#2 )
   7 AND  DR.OFS.PWR LSHIFT  + ;

and what gets assembled and compiled.

Code:

R/W
  3931          SEC
  3932     5 ,X LDA
  3934     8  # SBC
  3936  3951    BCC
  3938     5 ,X STA
  3940          INY
  3941     8  # CPY
  3943  3931 ^^ BNE
  3945   184  # LDA
  3947    11  # LDY
  3949  3920 ^^ BNE ' EXECUTE >BODY 6 +
  3951  2686    STY ' DRB >BODY
  3954   173  # LDA
  3956    11  # LDY
  3958  3920 ^^ BNE ' EXECUTE >BODY 6 +
29 

RAM
  9988  2497 LIT 16384 
  9992  4944 +
  9994  2469 EXIT
8 

DR+
 12004  3308 CLIT 7 
 12007  4556 AND
 12009  3308 CLIT 11 
 12012  5268 LSHIFT
 12014  4944 +
 12016  2469 EXIT
14 

The use of RAM and DR+ is mentioned here.
R/W is Fleet Forth's BLOCK reading and writing word.

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Thu Mar 23, 2023 2:25 am

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

The redesign of Fleet Forth's metacompiler is complete. I'm fairly certain I presented all the source and covered everything.
Any questions?

Top

GARTHWILSON

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Thu Mar 23, 2023 2:30 am

Joined: Fri Aug 30, 2002 1:09 am
Posts: 8543
Location: Southern California

Just use it a lot to wring out any bugs, and write a good manual for it.

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Tue Mar 28, 2023 1:52 am

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

The new metacompiler's first big test was metacompiling an existing version of Fleet Forth. Since I'm using VICE instead of a real Commodore 64, the disks are actually disk images and the disk image with the new kernel was identical to the one created with the original metacompiler. I also have SEEALL to test kernels I build. Successfully loading the system loader and the utilities is a test. SEEALL showing the expected disassembly/decompilation is another.

Code:

: (SEE)  ( CFA -- )
   DUP CR .NAME
   SETWIDTH <SEE> ;
: SEE  ( -- )  ' (SEE) ;
: SEEALL  ( -- )
   WITH-WORDS  NAME> (SEE)
   COLS ?CR ;

I've simplified some of the more obscure code for the metacompiler by adding two new words. These new words use the coroutine word CO

Code:

: >META  ( -- )
   ORDER@
   META DEFINITIONS
   CO ORDER! ;
: >SHADOW  ( -- )
   ORDER@
   [SHADOW] SHADOW DEFINITIONS
   CO ORDER! ;

The word >META saves the values of CONTEXT and CURRENT to the auxiliary stack then changes the search and compile vocabularies to META . CO causes the rest of >META to run after >META's caller exits. That last part of >META restores the CONTEXT and CURRENT vocabularies.
>SHADOW does the same thing for the SHADOW vocabulary.
This source

Code:

: DEFINE  ( W -- )
   ORDER@
   [SHADOW] SHADOW DEFINITIONS
   CONSTANT  ORDER! ;

META DEFINITIONS
: MACRO  ( ADR CNT -- )
   ORDER@
   [SHADOW] SHADOW DEFINITIONS
   [HOST] MACRO ORDER! ;
HOST

: <MCONSTANT>  ( N ADR -- )
   >IN @  HEADER  >IN !
   CF,  DUP V,
   ORDER@
   [SHADOW] SHADOW DEFINITIONS [HOST]
   CONSTANT
   ORDER! ;

: <M2CONSTANT>  ( D ADR -- )
   >IN @  HEADER  >IN !
   CF,  2DUP V, V,
   ORDER@
   [SHADOW] SHADOW [HOST]
   DEFINITIONS  2CONSTANT
   ORDER! ;

: <MCREATE>  ( ADR -- )
   >IN @ HEADER >IN !  CF,
   ORDER@
   [SHADOW] SHADOW DEFINITIONS
   THERE CONSTANT
   ORDER! ;

: <<MUSER>>  ( C -- )
   CREATE
      C,
      [ HERE 2+ >A ]
   DOES>
      C@ USER.AREA + ;
: <MUSER>  ( C ADR -- )
   >IN @  HEADER  >IN !  CF,
   DUP VC,
   ORDER@
   [SHADOW] SHADOW [HOST]
   DEFINITIONS
   <<MUSER>>
   ORDER! ;
A> CONSTANT MUSER-CF

is simplified to this

Code:

: DEFINE  ( W -- )
   >SHADOW CONSTANT ;

META DEFINITIONS
: MACRO  ( ADR CNT -- )
   >SHADOW MACRO ;
HOST

: <MCONSTANT>  ( N ADR -- )
   >IN @  HEADER  >IN !
   CF,  DUP V,
   >SHADOW CONSTANT ;

: <M2CONSTANT>  ( D ADR -- )
   >IN @  HEADER  >IN !
   CF,  2DUP V, V,
   >SHADOW 2CONSTANT ;

: <MCREATE>  ( ADR -- )
   >IN @ HEADER >IN !  CF,
   >SHADOW THERE CONSTANT ;

0 VALUE MUSER-CF
: <MUSER>  ( C ADR -- )
   >IN @  HEADER  >IN !  CF,
   DUP VC,
   >SHADOW
   CREATE
      C,
      [ HERE 2+ TO MUSER-CF ]
   DOES>
      C@ USER.AREA + ;

Likewise, this source

Code:

: DEFINER  ( CFA -- )
   ['] ORDER! >BODY >R
   ORDER@  META DEFINITIONS
   CREATE
      ,  0 ,  0 , 0 ,
   [ HERE 2+ >A ]
   DOES>
      DUP 2+ SWAP @ EXECUTE ;
   A> CONSTANT DEFINER-WORD

: DEF-RESET  ( -- )
   ['] ORDER! >BODY >R
   ORDER@ META WITH-WORDS
   NAME> DUP @ DEFINER-WORD <>
   IF  DROP EXIT  THEN
   >BODY 2+ 6 ERASE ;

: VERIFY  ( -- )
   ['] ORDER! >BODY >R
   ORDER@ META WITH-WORDS
   DUP NAME> DUP @ DEFINER-WORD <>
   IF  2DROP EXIT  THEN
   >BODY 2+ @
   IF  DROP EXIT  THEN
   CR RON ID.
   ." . NOT DEFINED IN TARGET" ;

is simplified to this

Code:

: DEFINER  ( CFA -- )
   >META
   CREATE
      ,  0 ,  0 , 0 ,
   [ HERE 2+ >A ]
   DOES>
      DUP 2+ SWAP @ EXECUTE ;
   A> CONSTANT DEFINER-WORD

: DEF-RESET  ( -- )
   >META WITH-WORDS
   NAME> DUP @ DEFINER-WORD <>
   IF  DROP EXIT  THEN
   >BODY 2+ 6 ERASE ;

: VERIFY  ( -- )
   >META WITH-WORDS
   DUP NAME> DUP @ DEFINER-WORD <>
   IF  2DROP EXIT  THEN
   >BODY 2+ @
   IF  DROP EXIT  THEN
   CR RON ID.
   ." . NOT DEFINED IN TARGET" ;

The metacompiler built with these changes successfully metacompiled Fleet Forth's kernel.

Top

BigEd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Tue Mar 28, 2023 8:48 am

Joined: Thu Dec 11, 2008 1:28 pm
Posts: 10985
Location: England

That's quite the milestone - congratulations!

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Thu Apr 06, 2023 2:03 am

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

I mentioned that the metacompiler supports LABELs because the target is built in virtual memory. The version of LABEL which is used to test compile target source on the host uses VALUE's which are defined before the test code. The HOST version of LABEL was defined like this:

Code:

: LABEL  ( -- )
  CONTEXT @
  FORTH // HOST FORTH VOCABULARY
  HERE ' DUP @
  [ ' #BUF @ ] LITERAL <>
  ABORT" NOT A LABEL"
  >BODY !  CONTEXT ! ; IMMEDIATE

I was testing a modification to Fleet Forth. Some of the words branch to NEXT . This version of LABEL wasn't going to work since I needed some of the test words to branch to the address for NEXT in one of the test words. I needed a VALUE defined in the host assembler to successfully test the new code.
This is the new definition of the HOST version of LABEL .

Code:

: LABEL  ( -- )
  HERE ' DUP @
  [ ' #BUF @ ] LITERAL <>
  ABORT" NOT A LABEL"
  >BODY ! ; IMMEDIATE

To test the modification to Fleet Forth on the HOST , I needed to be able to alter the address of NEXT in the HOST ASSEMBLER yet be able to EMPTY the dictionary back to its pre test state and have NEXT assume its former contents.

Code:

ASSEMBLER DEFINITIONS
NEXT VALUE NEXT
FORTH DEFINITIONS

Once a VALUE is defined for each label in the test code, the test code (including NEXT and the word which passes through it) is loaded.
Testing then commences; although, the first test was loading the source to see if all the intended branches to NEXT were within range. Fleet Forth's assembler and the metacompiler's assembler both abort with an error message if the intended branch is out of range.
BO for branch offset.

Code:

: BO  ( DEST SOURCE -- BRANCH.OFFSET )
   1+ - DUP 0< OVER ABS +  7F >
   ABORT" BRANCH RANGE EXCEEDED" ;

Once testing is done EMPTY is used to prune back the dictionary and forget the VALUE NEXT which exposes the original CONSTANT NEXT .

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Sun Apr 16, 2023 8:51 pm

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

Another test run called for another modification to one of the metacompiler words.
Fleet Forth has SUBR to create a word which returns its own PFA like a VARIABLE . Unlike a variable, no storage is allotted, the assembler is invoked. For example, (>FORTH) returns the address of the code used to transition from a code word to high level Forth.

Code:

SUBR (>FORTH)  ( -- )
   CLC
   PLA  1 # ADC  TAY
   PLA  0 # ADC
   LABEL (CO)
   N STA
   IP 1+ LDA  PHA
   IP    LDA  PHA
   N LDA  IP STY
   ' EXIT @ 4 + JMP
   END-CODE

>FORTH assembles a JSR to (>FORTH) as well as setting STATE to compiling and setting CONTEXT equal to CURRENT .

Code:

: >FORTH  ( -- )
   ?EXEC
   (>FORTH) JSR
   [ FORTH ]
   CURRENT @ CONTEXT !
   ] ; IMMEDIATE

A SUBR or subroutine in Fleet Forth is a word which returns the address of a routine which is exactly that, a subroutine.

Code:

SUBR ROUTINE1
   <SOME ASSEMBLY>
   RTS END-CODE

CODE TESTWORD
   ROUTINE1 JSR
   <SOME MORE ASSEMBLY>
   NEXT JMP  END-CODE

The metacompiler's SUBR works as expected.The metacompiler also supports HSUBRs, subroutines without headers. Since there is no header, there is no need for a code field. HSUBR creates a CONSTANT on the host containing the address of the routine in virtual memory.
The problem arose when testing target source on the host. When testing code on the host, HSUBR would just call the host assembler's SUBR . This normally works. When it didn't work was when there was a branch in range on the target because there was no header or code field to get in the way, but the test on the host didn't assemble because the branch was out of range. The header and code field got in the way. The new version of HSUBR works the same as the old when metacompiling. When testing on the host, it requires the name which follows in the text stream to be a predefined VALUE just like the host version of LABEL . Now there is no header or code field to get in the way of code during testing on the host.

Code:

0 VALUE DOUBLE.COMPARE  \ These definitions are only needed
0 VALUE POP3.TRUE       \ when testing on the host
0 VALUE POP3.FALSE      \ not when metacompiling

HSUBR DOUBLE.COMPARE
   6 ,X LDA  2 ,X CMP
   7 ,X LDA  3 ,X SBC
   4 ,X LDA  0 ,X SBC
   5 ,X LDA  1 ,X SBC
   RTS  END-CODE
CODE DU<  ( UD1 UD2 -- F )
   DOUBLE.COMPARE JSR
   CS NOT IF
      LABEL POP3.TRUE
      DEY
      LABEL POP3.FALSE
   THEN
   6 ,X STY  7 ,X STY  INX  INX
   POPTWO JMP  END-CODE

CODE D<  ( D1 D2 -- F )
   DOUBLE.COMPARE JSR
   VS IF  $80 # EOR  THEN
   POP3.TRUE 0< BRAN
   POP3.FALSE 0< NOT BRAN

Source for the new HSUBR .

Code:

: HSUBR  ( -- )
   META?
   IF
      THERE CONSTANT ASSEMBLE EXIT
   THEN
   [COMPILE] LABEL [ ASSEMBLER ]
   W TRUE ASSEMBLER MEM ;

I found out about this problem when testing a modification for source close to NEXT . The modification should have worked. I tested it on the host before committing to building a new kernel with the metacompiler. When I did, I got the dreaded "BRANCH RANGE EXCEEDED" error.
I did not get that error when testing with the new version of HSUBR .

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Mon Jul 03, 2023 11:39 pm

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

As I've already mentioned, for each type of CREATE DOES> or CREATE ;CODE word supported by the metacompiler, all the code fields for a given class of word are linked in a chain until the parent word of that type is defined for the target. CODE words are the only exception. I also mentioned the problem I ran into with trying to use CREATE rather than CODE for code words with no body. There is another problem which only occurs when trying to minimize the amount of virtual memory needed to build the Forth kernel. Some Commodore 64s may not have a Ram Expansion Unit. In this case, it is possible to write a version of (RR/W) to access the ram underneath the kernal ROM and I/O.
A reminder, this is how the current build of Fleet Forth maps blocks on the computer to blocks on devices.

Code:

Blocks:    Device:  Range seen by device:
    0 -  $7FF   8      0 - $7FF
 $800 -  $FFF   9      0 - $7FF
                .
                .
                .
$3800 - $3FFF  15      0 - $7FF
$4000 - $FFFF  REU     0 - $BFFF

Each disk drive "sees" the blocks in the range 0 to 2047 and none of them have the capacity for 2048 blocks. Most have considerably less.

The target built by the metacompiler loads at location $801 so the first $801 bytes of virtual memory are not used to build it. On a Commodore 64 without a Ram Expansion Unit, virtual memory could be set to start two blocks below the start of blocks used from ram by setting VOFFSET like this:

Code:

-2 RAM TO VOFFSET

This will reduce by two the number of blocks needed for virtual memory; however, the first $800 bytes of virtual memory are mapped to blocks $3FFE - $3FFF and are no longer accessible.
There is a side effect with the version of chains which uses two cells, one for the latest link and one for the previous link. Initialization causes a read from memory location zero in virtual memory. I could have written a test to avoid this, but it was easier to just remove the unnecessary feature form the metacompiler.
MEND-CHAIN and ADD-CHAIN were removed. The following words were rewritten.

Code:

: CF,  ( ADR -- )
   DUP @
   IF  @ V,  EXIT  THEN
   2+ VADD ;

: DEFINER  ( CFA -- )
   >META
   CREATE
      ,  0 ,  0 ,
      // CFA OF META WORD TO EXECUTE
      // FLAG -- ZERO OR TARGET CFA
      // CHAIN OF WORDS
   [ HERE 2+ >A ]
   DOES>
      DUP 2+ SWAP @ EXECUTE ;
   A> CONSTANT DEFINER-WORD

: PATCH-CF  ( -- )
   LATEST COUNT $1F AND >HERE CLIP
   ['] META >BODY VFIND
   0= ABORT" DEFINER MISSING"
   DUP @ DEFINER-WORD <>
   ABORT" NOT A DEFINER WORD"
   >BODY 2+ THERE OVER !
   2+ @
   BEGIN
      ?DUP  0EXIT
      DUP  V@  THERE ROT V!
   AGAIN -;

: DEF-RESET  ( -- )
   >META WITH-WORDS
   NAME> DUP @ DEFINER-WORD <>
   IF  DROP EXIT  THEN
   >BODY 2+ 4 ERASE ;

The code field patching portion of PATCH-CF is slightly smaller. I also improved VTRAVERSE .

Code:

: VTRAVERSE  ( VADR DIR -- VADR2 )
   BEGIN
      DUP UNDER+ OVER VC@ $7F >
   UNTIL
   DROP ;

I have already tested these changes to the metacompiler by building a Forth kernel and comparing it to the latest build. They were identical.

Top

JimBoyd

Post subject: Re: Fleet Forth metacompiler redesign

Posted: Tue Sep 12, 2023 6:36 pm

Joined: Fri May 05, 2017 9:27 pm
Posts: 895

The DO LOOP run-time words are not specifically mentioned in the Forth-83 Standard and could have different names. A DO LOOP run-time word could even have the same name as the word which compiles it; however, the metacompiler does not allow redefinitions in the TARGET vocabulary.
To summarize the previous posts, while metacompiling is on:
When a target word is found which is to be interpreted, because STATE is off or the word is immediate, the search continues with the parent vocabulary.
When a NON target word is found which is to be compiled, the search continues with the parent vocabulary.
Only non-immediate target words can be compiled without the use of [COMPILE] and only non-target words can be interpreted.

Consider simply changing the name of (DO) to DO .

Code:

CODE DO  ( LIMIT START -- )
   ...
   ...
   END-CODE

There could be DO LOOPs in the source before the compiling word DO is defined. Indeed, DO and the other DO LOOP compiling words might not even be defined in the source for the Forth kernel. They could just as easily be defined in the system loader which is loaded by the new kernel. Rather than encountering the immediate word DO and continuing the search in a parent vocabulary, the non-immediate run-time DO would be compiled. The DO LOOPs would not be compiled correctly.
In spite of this, there is a way to give the run-time for DO the same name. A helper word needs added to the metacompiler.

Code:

ALIAS DO
CODE (DO)  ( LIMIT START -- )
   ...
   ...
   END-CODE

The TARGET FORTH vocabulary on the host will now have the name (DO) but the target in virtual memory will have the name DO .
This will work; however, there is one gotcha. When the system is built on this new kernel and the metacompiler lexicon built on top of that, the metacompiler on the new system will not be able to compile DO LOOPs.
In solving this problem I came up with two different solutions which resulted in two versions of the metacompiler. They were the same except for what feature was added.
Both versions of the metacompiler worked. I tested each one by metacompiling the new kernel then building the system and metacompiler on top of that. The new metacompiler on the new system was then used to metacompile a kernel again. The kernels were identical. All four of them.

Access to the DO LOOP run-time words was still needed to build the high level portion of SEE . The run-time words are defined before the word FORTH-83 and the DO LOOP compiling words are defined after FORTH-83 . Fleet Forth's FIND searches the CONTEXT vocabulary (and parents) then, if necessary, searches the CURRENT vocabulary.

Code:

: FIND  ( ADR -- ADR2 F )
   CONTEXT @ (FIND)
   ?DUP ?EXIT
   CURRENT @ VFIND ;

To access the DO LOOP run-time words a false vocabulary was created and CONTEXT was pointed to it.

Code:

' FORTH-83 >LINK 2 ! 4 OFF \ Create false vocabulary starting at FORTH-83
2 CONTEXT !                \ and point CONTEXT there.
AMONG (?BRAN)
   ] BRANCH ?BRANCH
   DO ?DO LOOP +LOOP
   [ 0 ,

CONTEXT was reset after the word (?BRAN) was created.

SEE works fine on the new system.

Code:

 OK
: TEST 
   5 0 
   DO  CR I .  LOOP ;  OK
TEST 
0 
1 
2 
3 
4  OK
SEE TEST 
TEST
 23855  3332 CLIT 5 
 23858  3317 0
 23860  2313 DO 23874 
 23864  6753 CR
 23866  2417 I
 23868  7492 .
 23870  2195 LOOP 23864 
 23874  2472 EXIT
21 
 OK
SEE DO 
DO IMMEDIATE
 11718  8406 COMPILE
 11720  2313 DO
 11722 11474 >MARK
 11724  4843 2+
 11726 11494 <MARK
 11728  4843 2+
 11730  2472 EXIT
14 
 OK
2313 (SEE) 
DO
  2315          INY
  2316   251 )Y LDA IP
  2318          PHA
  2319          DEY
  2320   251 )Y LDA IP
  2322          PHA
  2323          CLC
  2324     3 ,X LDA
  2326   128  # ADC
  2328          PHA
  2329     3 ,X STA
  2331     2 ,X LDA
  2333          PHA
  2334          SEC
  2335     0 ,X LDA
  2337     2 ,X SBC
  2339          TAY
  2340     1 ,X LDA
  2342     3 ,X SBC
  2344          PHA
  2345          TYA
  2346          PHA
  2347          INX
  2348          INX
  2349          INX
  2350          INX
  2351  2275 ^^ BPL ' ?BRANCH >BODY 8 +
  2353  2250 ^^ BMI ' ! >BODY 20 +
40 
 OK

Although it is a little odd to SEE DO and notice that DO compiles DO .

Top

Page 3 of 4

[ 51 posts ]

Go to page Previous 1, 2, 3, 4 Next

Board index » 6502.org Users Forum » Forth

All times are UTC

Who is online

Users browsing this forum: No registered users and 33 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum