dawnFORTH: Yet another crude Forth for the 65C02.

[electricdawn]

IMPORTANT:
Please check Change History at the bottom of this post for the latest update!
---

Hi, everybody.

I'm new here to this forum. I share your love for old 8 bit CPUs and hope to contribute something worthwhile.
I am a retired computer service technician. Nothing glamorous, but it payed the bills. I started with electronics in the mid 70's and my lifelong obsession with computers was kindled in the late seventies when I programmed some punch cards for some mini-computer at my town's university (probably a PDP-8). I owned a lot of different computers in the 80's (and got myself quite in debt, which my dad had to bail me out of). I learned the usual programming language suspects: Basic, Pascal, C, but also 6502, 8080/Z80 and 8086 assembler. I always loved messing around in assembler, as it feels so much closer to the bare metal than any other programming language (C maybe an exception).

But when the 90's came and went, I pretty much stopped programming at home and used the computer more or less exclusively for gaming (another obsession which had started, of course, in the 80's). I only programmed "professionally" for about two to three years in my career. Other than that I was just another computer gamer (I never played on consoles. I don't like controllers).

Fast forward to last year. I was already semi-retired and had too much time on my hands when I stumbled over Ben Eater's channel on Youtube. I binge-watched all of his videos and my interest in programming the 6502 bare metal was re-kindled! After watching another video on Aaron Lanterman's channel, where his student had programmed a Forth interpreter/compiler on one of those WDC 65xx boards, I started thinking: "If this guy can do it, why can't I?".

I caved and ordered me a W65C02SXB board from Mouser.

I started to dig in and learned how to get this thing going. And, me massively overestimating my skill level, I started hacking away at that Forth interpreter/compiler that I had only a very vague idea of on what it was even supposed to be.

I only had very limited knowledge of compiler building. I had more or less zero knowledge of Forth.

And here is the result. This is the fourth iteration. It works. It's not complete. It's probably messy. It's not very optimized. Documentation might be lacking in places. But it works. It uses only two routines from the WDC monitor (character read and write), so it should be easily adaptable to any 65C02 computer out there. It's not going to work on the classic 6502 since I use some 65C02 exclusive commands (PHY and such).

I'm posting this here to get some reactions from you. Whether it's encouragement, constructive criticism or that you actually installed this thing somewhere and got it to work. Saying hi is also welcome.

I have no fancy Github page, so I will put future versions here in this thread.

I used a Mac mini M2 Pro as a cross development machine. I programmed dawnFORTH using the Vim/MacVim editor. The source was assembled using the vasm cross assembler. I wrote two little scripts to assist me with assembling and uploading the code to the board. The main uploading script was graciously provided to me by David Gray from WDC, he gave me his permission to upload it, so you can find at the bottom of this post.

Please let me know what you think. See you on the forums.

vasm webpage: http://sun.hasenbraten.de/vasm/
To make the correct vasm that works with my code I use:

Code: Select all

make CPU=6502 SYNTAX=std

The correct command line to create an object file that I can upload to my W65C02SXB board is:

Code: Select all

vasm6502_std -c02 -Fbin -foenix-pgz -exec=_ENDPROG -o dawn.o dawn.s

To upload dawnFORTH into the ROM of my W65C02SXB board:

Code: Select all

upload.py dawn.o -d /dev/tty.usbserial* -m write -k

More detailed explanations are in the README.txt.

vim is part of macOS, but the GUI version MacVim is available here: https://macvim.org

Edit: I forgot to mention one "speciality" of dawnFORTH: It has variable CELL sizes. Default is two bytes, standard 16 bit integer. But you can blow it up to 16(!) bytes if you want to. No re-compile necessary. You can do that by writing the new cell size into CELL. Please let me know what you think of that.

Edit2: Another thing I forgot is that the line editor and EMIT are multibyte character capable. I'm German, so I need my äöüÄÖܧß...

Change History:
20251228-1359 CET+1
- Fixed... annoyance in "ACCEPT", where ACCEPT would use the input buffer pointer instead of a temporary one, leading to the input buffer pointer (thus SOURCE) being overwritten and subsequent input getting store into whatever buffer address you provided to ACCEPT.
- Fixed bug in "int_header" that would produce a random code address when called by ":NONAME".
- Fixed bug in "int_insertXT" that would produce an incorrect XT when paired with ":NONAME".
- Fixed bug in ">" that would produce incorrect flags with certain values.
- Fixed bug in "ALIGNED" that would incorrectly align an already aligned address, producing "a-addr CELL+".
- Changed "SAVE-INPUT" so it will compress multiple characters into one cell (depending on cell size).
- Added "NAME>STRING".
- Added "U/MOD" and "U/".

20251223-1217 CET+1
- SLITERAL now adds a zero-byte to the end of the string, so EVALUATE will work. This is not reflected in the length value and should be transparent to the user.
- EVALUATE now correctly saves the entire input specification and restores it on exit. Correctly passes the EVALUATE test suite on "forth-standard.org".
- Fixed "lean" to only delete comments. Thank you, Sam.
- Expanded and clarified README.txt.

20251221-2137 CET+1
- Bug fix. Fixed a bug where PARSE, PARSE-WORD would continue to read characters past the end of a string that wasn't zero-delimited.

20251221-0944 CET+1
- POINT UPGRADE. We're at version 0.6 now.
- Added the EXCEPTION word set.
- Realized that I was stupid and removed pushing HERE on the data stack from ALLOT and VARIABLE. Which resulted in a lot of hand wringing and rewriting of code all over the place.
- Optimized multiple functions in "stack.s" and "functions.s" and probably elsewhere.
- Removed "AHEAD", "int_resolveForward" and "int_resolveBackward" from the internal word set. They're now in "system.fth".
- Added "DP@", "DP!", "RP@" and "RP!" to the internal word set.
- Fixed several bugs (as usual).

20251206-1232 CET+1
- I added multi-byte capability to HOLD (and fixed a bug that resulted thereof).
- Added saving and restoring the data stack when using the (currently) one-level undo (accessed by pressing cursor-up on your ANSI terminal).
- Numerous small changes to stuff not worth mentioning.

20251126-0940 CET+1
- Uploaded the Python upload script graciously provided by and with permission of David Gray, WDC.
- Expanded README.txt to include the Python upload script.
- Added more CORE words in "system.fth".

20251125-1035 CET+1
- Worked on and expanded README.txt. Not finished, but has much more information now.
- Cleaned up dawnFORTH folder and resulting zip file. It's much smaller now. No changes to source code.

20251124-1802 CET+1
- Fixed problem where range testing the data stack pointer would corrupt processor flags. Division stopped working due to that.

[BigEd]

Welcome! Good to hear your story, and thanks for sharing your project.

[electricdawn]

Thank you, BigEd.

Here is an example of the variable cell size capability of dawnFORTH:

Code: Select all

: fib  compiled
    0 1  compiled
    rot 0 ?do  compiled
        over dup . + swap  compiled
    loop  compiled
    drop u.  compiled
;  ok
 ok
24 fib 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368  ok
4 cell c!  ok
47 fib 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 1346269 2178309 3524578 5702887 9227465 14930352 24157817 39088169 63245986 102334155 165580141 267914296 433494437 701408733 1134903170 1836311903 2971215073  ok
max-int . 2147483647  ok
6 cell c!  ok
.s 
<0>  ok
70 fib 0 1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 1597 2584 4181 6765 10946 17711 28657 46368 75025 121393 196418 317811 514229 832040 1346269 2178309 3524578 5702887 9227465 14930352 24157817 39088169 63245986 102334155 165580141 267914296 433494437 701408733 1134903170 1836311903 2971215073 4807526976 7778742049 12586269025 20365011074 32951280099 53316291173 86267571272 139583862445 225851433717 365435296162 591286729879 956722026041 1548008755920 2504730781961 4052739537881 6557470319842 10610209857723 17167680177565 27777890035288 44945570212853 72723460248141 117669030460994 190392490709135

[GARTHWILSON]

electricdawn1 wrote:

I have no fancy Github page, so I will put future versions here in this thread.

That's perfectly fine; and in fact, we have members who've run projects here, 40+ pages.  You can of course edit your past posts too, and if you're familiar with how phpBB forums work, you probably already know how to do code sections, bullet points, larger headings, hypertext links, etc..  There are a few things that are not possible, but it's mostly pretty good.  Github has its problems anyway, causing some of us to shy away from it.  My son got me set up with my own website (linked below) which is mostly 6502 and is now rather voluminous, and I could probably get you some help getting your own site set up too if you're interested.  It's really easy and quick for him, but he's not the best at how to communicate it.  As far as writing web pages go, I just use a plain text editor and write the html tags and so on, without any web-design software.  You can start with a template for the headers and stuff, then you only need to know a small handful of tags, for example <u>...</u> to underline something; and a cheat sheet helps too of course.

About your dawnFORTH attachment:  There are an awful lot of files there; so can you give us a guide to what's what.

We look forward to your involvement and progress.  My own story about how I got into this stuff is the first post in the "Introductions" sticky topic in the "General Discussions" portion of the forum.

[BigEd]

Definitely a fan of the adjustable CELL size - adjustable at run time - marvellous.

[electricdawn]

Thank you for the warm welcome, both Garth and BigEd!

About the "awful" lot of files in my attachment: I will make a README file with a list of what is what. For now, look in "dawn.s". It is the main file and explains all the other ones. (I probably should have named it something more intuitive, not sure what... )

Edit: I updated the zipped file with a small (for now) README.txt and the new "temp.s". I don't know yet on how to delete the old file...
Edit2: Ok, got it. Now there's only the new zipped file left. Please re-download.

[Dr Jefyll]

Welcome, electricdawn!

Definitely some original ideas here! I look forward to hearing more from someone who has "only a very vague idea" of what a Forth interpreter/compiler is supposed to be!

-- Jeff

[SamCoVT]

electricdawn wrote:

And, me massively overestimating my skill level, I started hacking away at that Forth interpreter/compiler that I had only a very vague idea of on what it was even supposed to be.

This is probably the very best way to get started in writing your own Forth! It's less daunting when you have no idea how much work it is and only a few pesky details to wrangle at a time as you come to understand what needs to be done. You're welcome/encouraged to tell us about your discoveries and findings, even for simple things. It's always nice to see how someone else approached a problem and whether they were satisfied with the results.

I do like your adjustable cell size.

I'd be interested to hear about the hashes I see in your dictionary.

I think there's some room for optimization in your code, but it looks (from reading your readme) like you're not focusing on that right now, and I think that's a good idea. Get things functional and correct first and then worry about optimizing later (if at all - some things don't really need to be super-optimized).

I'm the current maintainer of Tali Forth 2, which is also an ANS-2012 Forth (the original author is Scot Stevenson from Germany). You're welcome to look at the source for TaliForth2, and the code there is public domain so you're welcome to yoink anything you like, although your stack works a bit differently so it won't be copy/paste. Some of the things in the 2012 standard are a little silly/overkill for a 65C02 computer (I'm thinking of wordlists, for example), so a lot of folks use an earlier standard instead, but the core set of words is fine.

Is your Forth an Subroutine Threaded Code (STC) Forth? It looks like your compiling compiles JSRs (which is how STC Forths usually work) but sometimes JMP. I'd love to hear about that as well.

[electricdawn]

Thank you as well, Jeff and Sam.

a. I use hashes to quickly find a word in the dictionary, without having to string compare each words name. The dictionary is a simple linked list, sorted by an ascending hash. Although a lot of words in Forth have short names, equally a lot have longer names, so I thought that comparing just two 16bit values is faster than comparing maybe five or even eight bytes. Once I have a matching hash, I will compare the names of the Forth words directly.

I must say that I didn't measure if it really is that much faster, as search times are more or less instant (and I never tried it without a hash, go figure).

b. Thank you, I might have a look at Tali Forth 2 later on, but right now I don't want to get too much distracted.

c. Yes, I just thread one subroutine call after the other, with a sprinkling of directly inserted code (mostly for conditional jumps). I originally thought it would be useful to be able to jump to a subroutine directly instead of calling it, but I currently have no word that actually uses this. I might just get rid of it.

If I would want to convert this to a 16-bit processor (like the 65C816) and if I would use a fixed cell size, I'd probably want to go for more directly inserted code instead of just subroutine calls. But for an 8-bit processor like the 65C02 it is much easier to just call subs. Additionally, when I started this, I had even less of an idea on how to implement this, so calling subroutines it was!

[electricdawn]

Another update to dawnFORTH:
- I added multi-byte capability to HOLD (and fixed a bug that resulted thereof).
- Added restoring the data stack when using the (currently) one-level undo (accessed by pressing cursor-up on your ANSI terminal).
- Numerous small changes to stuff not worth mentioning.
- Removed the "jump directly" capability. It's really not needed right now.

The data stack will only be restored to a maximum amount of less than 256 bytes, which should usually be enough. Don't get close to that limit, as it will still start overwriting stuff. I will fix this soon(tm).

Please let me know if you have questions, suggestions, wishes or if you just want to say hi. Thanks for trying out dawnFORTH!

[electricdawn]

Gather round, kids. Gran'pa is gonna tell a story...

Well, this is a fairly big update, which wasn't originally intended. But I stumbled over a lot of things (like, why did I push HERE when ALLOT-ing memory? I guess, I was trained on "alloc" from C.) that needed cleaning things up... which in turn made me rewrite (and optimize) a LOT of other code. I'm also continually testing words for compatibility with Forth 2012 and find new bugs that need to be fixed.

Adding the EXCEPTION word-set was very interesting. I, of course, had no idea how these words really worked and started with writing CATCH in Forth (only taking a furtive glance at the implementation on forth-standard.org). I somehow got CATCH to work without having a THROW yet... well, it worked when no error occurred, but I couldn't get it to work when one happened, as there was nothing really there to clean up when it happened.

I then realized that THROW wasn't just simply storing the error code somewhere and then calling a vectored error-handler. No. THROW pretty much IS the error handler and responsible for cleaning up after an error happened. Which is, IMHO, bass-awkward. You don't CATCH a ball before someone can THROW it, but that is what CATCH and THROW do in FORTH. At least that's how I understood and implemented it.

As a result, THROW is now an internal word and is called by all functions that can produce an error instead of my previous generic error handler. THROW will clean up the stacks and call a vectored error handler (the vector of which can be changed by CATCH). CATCH is still written in FORTH as THROW can happily operate without it. It just calls the generic error handler instead.

After that adventure I started cleaning up some old code (sometimes rewriting it completely) to make it smaller and optimized (most affected was RPICK). While compliant testing my words, I realized that I had made a bad assumption. You know, by ASSuming something I had made an ass out of myself.

I had assumed that ALLOT (and consequently VARIABLE) pushed the previous HERE, just like "alloc" in C. Or "New" in Pascal.

That had been very naïve of me.

Once I realized that this wasn't the case, I had to rewrite a LOT of code where I allocated memory, and also had to rewrite VARIABLE (CONSTANT saves its value inline, no CREATE necessary). A valuable lesson indeed to never blindly assume things.

Anyway, all of this was worthy of a point increase, so we're now at version 0.6. One of the new words is called "int_split", which is currently written in FORTH and is called mainly by "int_resolveForward" and "int_resolveBackward". It splits a 16-bit value into its high- and low-byte. I will most definitely turn that into an internal function for speed reasons, but it's ok for now.

Thank you for reading through this lengthy post. Please let me know if you have questions. I will happily try to answer them. Or if you have suggestions I'll be glad to assess them as well. Have a merry Christmas (or whatever feast you enjoy) and see you next year!

Edit: CREATE, of course, DOES return the value of HERE at time of invocation... Changed text to reflect that.

[electricdawn]

BUG FIX: Fixed a bug that would cause PARSE, PARSE-WORD to read characters past the end of a string that wasn't zero-delimited.

Also added a script "lean" that reads "full.fth" and strips extraneous white space as well as comments and prints it to console. You can redirect the output into "system.fth" or any other text file.

lean:

Code: Select all

#! /bin/bash
awk '/'.'/ {
		gsub("\\( .*\\)", "", $0);
		gsub("\\\\ .*", "", $0);
		gsub("[[:blank:]]+", " ", $0);
		gsub("^[[:blank:]]", "", $0);
		if(length) print($0)
}' <(cat full.fth)

Edit: Fixed "lean", so it only looks for sequences of "(<space>" and "\<space>". Thank you, SamCoVT

[SamCoVT]

electricdawn wrote:

Code: Select all

		gsub("\\(.*\\)", "", $0);
		gsub("\\\\.*", "", $0);

Note that these will gobble things they are not supposed to, such as

Code: Select all

\This is not a comment
(Neither is this)
s\" This is a real string\n"

Adding a blank to the regex after the \ or ( should fix it. While there aren't many standard words that use these characters, it is pretty common to name runtimes (a short routine that gets copied into a word being compiled) in parenthesis, such as the runtime for "DO" might be named "(DO)". That's a built-in word, but programmers writing their own control and data structures often use that convention.

If you are testing to ANS2012, you are welcome to check out the test suite I put together for Tali Forth 2, based on the test setup from John Hayes S1I and the tests available from ANS. It uses the Forth to test itself (available here). The main test apparatus is in tester.fs, so that is fed in first (if running in a simulator, you can likely just copy/paste it into the Forth interpreter). Then you can feed in the text for whatever tests you want. There are some python scripts there to automate this using either the py65 simulator or the c65 (comes with Tali) simulator. The latter is way faster and what I use for my regular testing.

Just remove the MARKER line (if it exists near the top of the test file) and the matching word at the end if you don't have the word MARKER. These are used to reclaim memory as the entire tests use more than 32K if you run them all and my hardware only has 32K of RAM.

If you'd like assistance adapting this test suite to your Forth, let me know. Having the test suite has been invaluable to me as it lets me know when I break something (sometimes not even the thing I was editing!).

[electricdawn]

Thank you, Sam. "lean" is a quick hack and I really didn't put too much thought into it as it seems. I will follow your advice and add a space.

About the test suite... Well, my current approach of using

Code: Select all

parse-name drop 1- find

where "calcHASH" (subroutine for calculating a hash) and "srchHASH" (subroutine for searching a hash in the dictionary) are not going by the length byte in the counted string but are actually looking for a delimiting zero-byte is completely falling apart in the light of

Code: Select all

s" 123 drop" evaluate

where S" is NOT adding a delimiting zero-byte at the end of the string and thus EVALUATE will read past the end of the string.

So... I need to fix this first... then I will come back to your offer. And I will. Thank you. This has been bugging me for the last two days. I will fix this. I just don't know how and keep my code alive as well as keep compliance with the standard. Can I add a delimiting zero-byte to strings created with S" and still be compliant? Additionally, my S" creates a 16-Bit length "byte" to cover for larger text. Do I even need this? Is my approach still valid? Is 42 still the answer? Right now, my FORTH world is in shambles...

Help.

Have a merry Christmas.

[SamCoVT]

Forth, generally, doesn't use null-terminated strings like C does. The standard specified two types of strings (neither of which is null terminated):
A "counted string" is stored in memory with the length in the first byte and the characters following. These can be passed by only their address, and they often have a limit of 255 for the length. These are mostly around for historical reasons and you don't see them as often in ANS2012 as you do in earlier forths (except for FIND - I think that's the only word that requires a counted string)

"Character strings", on the other hand, are passed by giving the address AND the length on the stack. When you see ( c-addr u ) or sometimes just ( addr u ) in a stack comment, that's most likely a character string. There is a word COUNT that turns a counted string into a character string (by placing the length onto the stack and incrementing the address so it points to the first character).

Regarding ANS2012 compliance:
You are allowed to limit strings to 255 characters, but not less than that.

You are allowed to add a null to the end of your string data - just don't include the null character in the length of the string, but you certainly can ALLOT space for one on the end of the string and place a null character there. This might be the easiest option if all of your other routines are expecting null terminated strings, but it would be better if your routines worked with non-null terminated strings as well. Other people's code might build strings in memory (using "C,", for example) and leave an ( addr u ) on the stack expecting it to be treated as a string.