Compiling TaliForth2 on Windows

[SamCoVT]

Druzyek wrote:

One thing that threw me was that the Undefined word error only shows part of the word. If you enter "bar" for example and it's not defined it shows ">r< Undefined word" instead of >bar<. I thought maybe the pointer to the input buffer was getting scrambled somehow switching between the banks.

This is normal behavior for Tali, and I can already tell you are in HEX. If a word can't be found, Tali tries to turn it into a number. In this case, BA are valid hex digits, so 0xBA gets put on the stack, but it stops when the "r" shows up. The BA were already "used" from the input, so remaining letters are treated as the next word.

Quote:

ram_end had to be moved from $4000-1 to $C000-1 since otherwise moving the cp to $4000 would give an out of memory error. cp_end is now also $C000-1. hist_buff changed from ram_end-$03ff to $4000-$03ff so it won't point to the banked memory (I recognized the problem when I got lots of colored dots on the screen ) Hmm, how exactly does hist_buff work? Do I need all 8 buffers?

Those memory locations make sense, but the history buffer will likely be an issue for you. The history buffer is used with CTRL-P and CTRL-N to recall the previous and next line of input. It store 8 lines as older counted strings (first byte is length) with 128 bytes reserved for each line, using 1K of RAM. For working interactively, it's sometimes nice. If you're working on a PC and pasting over to Tali, then you probably don't need it. We can either: move the history buffer, or remove it entirely. Let me know which way you would like to go.

As a side note, this is all really good practical experience for me, as one of the things Scot (the author of Tali2) and I had discussed was making the build configurable so that certain components could be omitted or shuffled around. This is a good exercise in seeing what needs to be done to make that happen (and mostly it hasn't been that difficult). Our original expected audience was folks building 32K RAM/32K ROM type single board computers, which will hopefully explain some of the design decisions that were made (like just stuffing the input buffer at the top of RAM)

Your initial code looks good. If you are comfortable with this method, it is probably a good least-effort solution.
I will throw one more thought out, and that is that it might be possible (but would be a bit more work) to have the header be in bank 0 where the word would always be found, but when called the word would automatically switch to the bank with it's code before jumping to the code to complete execution. All of your words would be available (if you have space in the first 16K for all of their headers) and they would switch to their bank before running. You technically could even make them "put it back the way it was" as they return. The down side would be that these words would always do that and it would run a little more slowly, even when you didn't need to be switching the banks around.

[SamCoVT]

Druzyek wrote:

I think mod may be wrong. Here is some test output:

Code: Select all

13 5 mod . 3  ok
-13 5 mod . -3  ok
13 -5 mod . 3  ok
-13 -5 mod . -3  ok

It should be this (confirmed in Gforth):

Code: Select all

13 5 mod . 3  ok
-13 5 mod . 2  ok
13 -5 mod . -2  ok
-13 -5 mod . -3  ok

-x/y = -(x/y) but -x mod y != -(x mod y)

Unfortunately, I believe you have discovered one of the "the standard did not standardize" issues in Forth. There's a little bit of discussion on that at viewtopic.php?f=9&t=4466 and you can also check out http://forth-standard.org/standard/core/MOD for the 2012 ANS standard. The issue occurs when the signs of the two values are different. One method is "symmetrical division", which is what Tali uses, and the other is "floored division", which is what you are expecting and what gforth is doing. To make things more interesting, it depends on what platform gforth is compiled for, so on some systems gforth will also do symmetrical division. See section 3.2.2.1 at http://forth-standard.org/standard/usage#usage:div (supposed to be a direct link, but I always have to scroll up a little bit) for examples of the two types of division.

If you want mod to use floored division, you can totally just redefine it using the version shown on the ANS standard page that uses the FM/MOD type division instead of the SM/REM type of division (Tali has both types built in - it's just that it uses symmetrical division for the words "/" and "MOD").

Code: Select all

: MOD  >R S>D R> FM/MOD DROP ;

This code makes the first number a double-cell value so it can use FM/MOD (F for Floored, M for mixed double and single cell value sizes) and it does both division and calculates the remainder, and then we drop the division result. You may also want to check if / has the behavior you want, and it also can be redefined as (from the ANS page for "/"):

Code: Select all

: /  >R S>D R> FM/MOD SWAP DROP ;

This is the same as above, only it drops the remainder instead. If you need both the result and remainder of the division, you might want to use FM/MOD directly in your code.

I believe the reason this wasn't standardized one way or the other is that some CPUs natively do it one way or the other (which might explain why gforth on ARM might be different than gforth on x86).

[Druzyek]

Quote:

If a word can't be found, Tali tries to turn it into a number. In this case, BA are valid hex digits, so 0xBA gets put on the stack, but it stops when the "r" shows up. The BA were already "used" from the input, so remaining letters are treated as the next word.

Ok, that makes sense. Sometimes it's hard to know if the problem is in the original Tali, my emulator, my Forth code, or the changes we are making Speaking of that, this code produces "AB $%C" but shouldn't it be "AB C" (3 spaces between B and C which the forum software eats)?

Code: Select all

65 emit 66 emit 3 spaces 67 emit

Also, the list of native words has the word for THRU listed as "list" although LIST is also "list." Is that right?

Quote:

It store 8 lines as older counted strings (first byte is length) with 128 bytes reserved for each line, using 1K of RAM. For working interactively, it's sometimes nice. If you're working on a PC and pasting over to Tali, then you probably don't need it. We can either: move the history buffer, or remove it entirely. Let me know which way you would like to go.

I don't think I'll really use it. Pasting everything in works well, and I only type stuff in myself to do a little testing. The C version takes about 27k and it doesn't have a 14k runtime, so I think I'll need as much RAM as I can get.

Quote:

As a side note, this is all really good practical experience for me, as one of the things Scot (the author of Tali2) and I had discussed was making the build configurable so that certain components could be omitted or shuffled around.

That's a great idea! I was hoping that's what I'd get with an STC Forth. The really neat thing would be some kind of linking stage where it figures out exactly what calls it makes to the kernal and only includes that part of the kernal in the final product so the binary is standalone. That might be too much to ask on a self hosted system though, especially without bank switching which could take just about any form on other machines.

Quote:

I will throw one more thought out, and that is that it might be possible (but would be a bit more work) to have the header be in bank 0 where the word would always be found, but when called the word would automatically switch to the bank with it's code before jumping to the code to complete execution.

That's a neat idea. I think for a big system that kind of automatic bank switching would be necessary for sanity's sake but this is small enough that it should be ok switching the bank manually.

Quote:

Unfortunately, I believe you have discovered one of the "the standard did not standardize" issues in Forth.

Interesting. I ran into this on one of my calculator projects without knowing there was any debate. The way it is now should work for this, so I'll leave it as it is.

[SamCoVT]

Druzyek wrote:

Sometimes it's hard to know if the problem is in the original Tali, my emulator, my Forth code, or the changes we are making Speaking of that, this code produces "AB $%C" but shouldn't it be "AB C" (3 spaces between B and C which the forum software eats)?

Code: Select all

65 emit 66 emit 3 spaces 67 emit

Yes, it should print AB (3 spaces here) C. Running your "smallerized" version in the py65mon simulator gives that result:

Code: Select all

65 emit 66 emit 3 spaces 67 emit AB   C ok

If you have py65 installed, you can just run "make sim" and it will assemble with the py65mon platform file and then run the result in the py65mon simulator. That might be useful to determine if it's odd Tali behavior or odd simulator behavior. When you are done, "bye" gets you out of Tali and to the py65mon prompt (a period) where you can type "q" to quit that as well.

Druzyek wrote:

Also, the list of native words has the word for THRU listed as "list" although LIST is also "list." Is that right?

Nope, definitely not right - I've put in an issue report on it, and it looks like it's just an error in the comment for the word THRU (that wordlist is automatically generated from special headers in the comments above each word in native_words.asm). Good eyes to spot that.
Speaking of those words, those are for the block support, which I removed in your version. There is a WORDLIST.md in your branch that should have only the words that actually exist in your version of Tali.

Druzyek wrote:

SamCoVT wrote:

It store 8 lines as older counted strings (first byte is length) with 128 bytes reserved for each line, using 1K of RAM. For working interactively, it's sometimes nice. If you're working on a PC and pasting over to Tali, then you probably don't need it. We can either: move the history buffer, or remove it entirely. Let me know which way you would like to go.

I don't think I'll really use it. Pasting everything in works well, and I only type stuff in myself to do a little testing. The C version takes about 27k and it doesn't have a 14k runtime, so I think I'll need as much RAM as I can get.

That seems reasonable. I think only ACCEPT and COLD end up needing to be modified to remove all references to it. I'll look into that and see what's involved. For now, you can move the "hist_buff" label in your platform file. It's currently "ram_end-$03ff" but you can set it to anywhere that 1K of access from that point won't disturb anything. That will keep it from giving you bonus pixels on your screen.

[cjs]

SamCoVT wrote:

I also had to change all references of "python3" to "python" as 3.8.1 did not appear to install a "python3.exe".

Oh, don't do that! You're right that Windows doesn't install a python3.exe, but python on Linux is always Python 2, and this will not change soon. (More on this below.)

Linux generally has no more than two system installs of Python present, Python 2 as python and Python 3 as python3. Windows, however, may have multiple installs present; these register themselves in the Windows registry during install for use by the Python launcher.

Thus, the correct way to run Python on Windows is via the Python Launcher for Windows, py. While python will run a random version of Python (usually the first one you installed), py -3 will run the latest version of Python 3 that's available (or fail, even if Python 2 versions are present), which is what you want. But if you're running a script with a Unix shebang (e.g., #!/usr/bin/env python3) you can just py script.py and py will find the right version for you.

As for figuring out if you're on Windows or not, so you know whether to use python3 or py, I know you can check if MSYSTEM is set (to anything) to see if you're running in one of the MSYS2 environments; I presume CygWin has something similar. But I'm not really too familiar with all the details of the various ways of getting Linuxy on Windows. (When I use Windows I just use the MSYS2 stuff, including Bash, that ships with Git for Windows.) But I'd be interested in knowing how you end up doing the check so that I can update my notes.

BigEd wrote:

(As of very recently, python2 is no longer maintained, so it's become reasonable for 'python' to mean 'python3' - which is not to say that everything everywhere will still work. We've had a long time to make the adjustment, but many of us haven't.)

Actually, that adjustment isn't going to happen for some time, if ever, because though Python 2 is no longer maintained there are still a lot of older Linux systems out there that define python to mean Python 2 (and have Python 2 software on them). These are going to take a long time to go away; I know of plenty of systems still running CentOS 5. Those might finally go away soon, but consider that if it took someone ten years to upgrade a server from CentOS 5, it may well be 2028 before they upgrade that Ubuntu 18.04 machine they just installed.

And really, it's just not worth going through a breaking API change (defining #!/usr/bin/env python to run 3 instead of 2) when we already have working and widespread solutions for both Linux (keep it as it is) and Windows (use py).

[Druzyek]

Quote:

Yes, it should print AB (3 spaces here) C

Looks like the loop in spaces only loads A once with the space character and expects it to still be there when the character printing routine returns, which is not the case for mine since it does pointer arithmetic to draw the character. It works correctly now.

Quote:

I think only ACCEPT and COLD end up needing to be modified to remove all references to it. I'll look into that and see what's involved.

Cool and thanks!

[BigEd]

(I'm afraid I don't have any useful comment about the python2 deprecation - of course I accept that older linux distributions will be around for a long while, as will python2 scripts, and I don't know quite what I'd do if I were still running engineering systems. It is indeed a breaking change, but it's been coming for well over a decade. I always used to use modules to ensure that each workflow had exactly the right tooling on the path, but that's not something we see in a domestic environment. It might be that 'python3' will always be the preferred interpreter for a python3 script.)

[SamCoVT]

I realized, after looking at the code for spaces, that your output routine must have modified the A register. I checked both the manual and the platform file, but it seems that neither of them explicitly states that the char should be left in A (and indeed they probably should BOTH state that). I'll put in an issue for updating the documentation. Fortunately, I saw your message saying you had figured it out, but it would be nice to be very clear about that for future Tali2 users.

I've pushed the removal of the history buffer into your "example/smalltali2", so you should be able to pull that into your working copy. As a bonus, the code size shrunk a bit as well.

I ran the core, string, and double tests (most of the other test sets require words that have been removed now, such as wordlists or block support) to make sure I didn't break anything, and the only tests that failed were the ones that use ENVIRONMENT? (which I had removed earlier because it uses a lot of strings). As an interesting side note to our previous discussion, one of the ENVIRONMENT? queries that failed is "FLOORED" which can be used to determine if "/" and "MOD" on a particular Forth use floored or symmetrical division. All of the other tests passed, so I think you should be in good shape to start developing your application. I'm sure folks in this forum would love to hear about your progress - a lot of them are lurkers and may not weigh in, but it's nice to see what other folks are doing with Forth.

(To CJS and BigEd regarding the original python issue, I did end up modifying the Makefile to use py on windows, but I find it quite annoying that "python3 scriptname.py" used to work on ALL platforms to launch a script with python3 (regardless of whether python2 was installed or not and without caring which version "python" might run), and now it doesn't. It's also interesting that it's just the most recent windows install that doesn't include a python3.exe. A couple of minor version numbers ago, it *DID* include a python3.exe executable in the windows installer, and everything was happily cross platform - perhaps they oopsied when they created the latest windows release?
Regardless, I have patched things up enough that things are working. Hopefully "py" will be brought to the other platforms so I can have one command to rule them all again.)

[Druzyek]

Quote:

I've pushed the removal of the history buffer into your "example/smalltali2", so you should be able to pull that into your working copy. As a bonus, the code size shrunk a bit as well.

Great. That is working now, and as expected, the history memory is not touched in the new version.

Quote:

I'm sure folks in this forum would love to hear about your progress - a lot of them are lurkers and may not weigh in, but it's nice to see what other folks are doing with Forth.

I'm gonna try to finish the other versions and post them all at once. The Forth version especially will probably have a lot of room for improvement since I'm a Forth beginner.

The next request would be a way to recognize errors immediately. As it is now, all the text gets pasted in instantly from a text file and any errors scroll immediately off the screen. Would it be possible to implement a separate output for error messages like stdout and stderr in C? It seems like modifying or replacing ABORT might do it. Another way might be to add a flag in zero page that causes an error message to go into an infinite loop when on but that can be switched off later after the file has been pasted in so the prompt still works interactively even when you cause errors. What do you think?

[SamCoVT]

Druzyek wrote:

The next request would be a way to recognize errors immediately. As it is now, all the text gets pasted in instantly from a text file and any errors scroll immediately off the screen. Would it be possible to implement a separate output for error messages like stdout and stderr in C? It seems like modifying or replacing ABORT might do it. Another way might be to add a flag in zero page that causes an error message to go into an infinite loop when on but that can be switched off later after the file has been pasted in so the prompt still works interactively even when you cause errors. What do you think?

The great part about Forth is that you can actually go in and change the things that you think should work differently. I'll be interested to see what you come up with for a solution to this problem, as I also end up pasting very large inputs at times, and it would be nice to have it stop at the first error instead of having to scroll back and find it. I often want to throw away all the rest of the input, as a word that didn't compile will mess up most of the code that follows, and I would need to modify ACCEPT to do that. A flag in zero page, like you suggested might work well for my situation, and I could look for a control code (perhaps CTRL-Q) to allow regular input again. You will find (although I commented it out) where CTRL-P and CTRL-N were handled in ACCEPT for the input history. Do be careful as some CTRL combinations are handled by the terminal and not passed to the application (at least on Linux, anyway), such as CTRL-L (clears the screen).

ABORT (which you will find really just falls into QUIT) might be one place to make changes. It is called (jumped to, really, as it resets the stack anyway) after the error message has been printed. If you want the location where the error message is actually printed, you will need to look at "error:" in taliforth.asm. I believe this routine is used for all of the errors that would end up calling ABORT. You'll also want to look in "xt_number:" (just before the "_all_converted:" label) in native_words.asm if you want the name of an unknown word.

If Forth can't find a word, it tries to turn it into a number, so the spot where it realizes that the text at the input is not a number (and therefore an unknown word) is actually in the NUMBER routine.

You have the luxury of writing your own emulator, so you could make a "stderr" that goes somewhere else without actually needing to add hardware. You could just make an extra address that you write to for output, and then write some forth words to use it.

Keep us posted on whatever you end up with for a solution.

[dwight]

Symetric math is often better for polar math. Floored math is better for single lines or XY rectangular vectors. Symetric math makes zero an unique point. Floored math makes zero an offset. This can be vary important when doing multiple precision math on things like XY tables. Using math that rounds towards zero can really mess things up because it makes zero a unique point. You can't smoothly cross zero. As soon as you try to include offsets, thing go bad quickly.
It is easier to understand which is which by doing /mod instead of just mod. Both still need to follow the rule Dividend=Divisor*Quotient+Remainder. I've learned that I prefer floored math for most calculations but most processors round toward zero.
As an example. If one makes a table of sine and cosine with symetric math and incrementally calculates the next point around a circle, it won't close. If you use floored math, the circle will be a little off center but will close. This happens even for floating point but most don't notice the tiny error until many revolutions have been completed.
Dwight

[scotws]

After a couple of irksome months involving everything from minor surgery to the IRS, life is finally back to normal, or at least as normal as it will ever be, and I can get back to the really important things in life .

First of all, thanks for all the work everybody! I'm amazed you got Tali so small. I agree that this experience shows that we should try to figure out how to make Tali modular - there was an attempt to make it easy to leave out (say) the ed clone, but we probably need to create a better system. There is an issue open for "where do we go from here" at https://github.com/scotws/TaliForth2/issues/232 that I'd suggest we use discuss low-level details; roughly, I agree as well that this might be a good time to switch to another assembler that allows conditional assembly, which Ophis does not. If anybody has any preferences which assembler to use, I'm open to suggestions.

As for the actual Windows topic -- this is an honest question, I had sort of assumed that since the Windows Linux Subsystem (https://docs.microsoft.com/en-us/window ... tall-win10) was introduced most people would just use that for development instead of native Windows? I haven't really had a chance to fool around with it yet (the Windows computer is used for gaming, and with two teenagers in the house, I sort of have to fight to use it), but I have a bunch of developer friends who say they really love it. One way or the other, we should definitely make it easy to run under Windows and if possible macOS.

Thanks again!

[Druzyek]

SamCoVT wrote:

The great part about Forth is that you can actually go in and change the things that you think should work differently. I'll be interested to see what you come up with for a solution to this problem, as I also end up pasting very large inputs at times, and it would be nice to have it stop at the first error instead of having to scroll back and find it.

I just added a little code to "error:" that inserts a BRK if the flag is set that I put in zero page to turn off screen output while the file is loading (printing characters pixel by pixel with 6502 code is slow). All output is directed to a debug window anyway, so I can see what word caused the error when it hits the break.

Quote:

You will find (although I commented it out) where CTRL-P and CTRL-N were handled in ACCEPT for the input history. Do be careful as some CTRL combinations are handled by the terminal and not passed to the application (at least on Linux, anyway), such as CTRL-L (clears the screen).

My emulator just reads raw keystrokes from the browser window with JavaScript so no terminal getting in the way or intercepting CTRL combinations

Most of the data for the game is pasted into the Forth version now and it overflowed the remaining RAM alarmingly fast. There are about 250 constants for things like initialization and screen coordinates of objects. It looks like each one uses 13 bytes of ram plus the length of the constant name, so 1F constant COLOR_CRYSTAL_YELLOW1 takes 34 bytes. Altogether this is around 7k just for constants. What would be an alternative? I thought about this:
1. Drastically shorten constant names, which makes everything less readable.
2. Get rid of most constants and use numbers with comments for the numbers.
3. Insert #define statements from C in place of constant and run the file through a C preprocessor.
4. Dedicate a 16k bank just for constants and make a prefix word like "const" that switches the wordlist to the other bank, fetches the constant, then switches the bank back to screen memory. This would probably be a big performance hit but might work if you switch once and push all constants your word will need before switching back to minimize banking.
5. Put the constants in a 16k bank but change the code so that it somehow inserts a literal any time the constant is referenced. After all code is loaded, the 16k bank is no longer needed. Not sure exactly how to do this in Forth but will give it a try.

scotws, glad you're back! Tali Forth 2 is really neat and I'm enjoying it a lot. Thanks for all your work.

[GARTHWILSON]

Quote:

COLOR_CRYSTAL_YELLOW1 takes 34 bytes. Altogether this is around 7k just for constants. What would be an alternative? I thought about this:
1. Drastically shorten constant names, which makes everything less readable.

In electronics, "crystal" is often shortened to XTAL and everyone knows what it is. Same with colors, BLK, BRN, RED, ORG, YEL, GRN, BLU, VIO, GRA, and WHT, and other colors can probably be made just as clear, like PNK, CYA, etc. (not that anything forces them to be three letters; the point is that they can be shortened a lot).

[Druzyek]

Looks like there are a couple ways to solve the problem of constants eating up RAM. Someone in the #forth IRC channel on freenode suggested this:

Code: Select all

: constant create immediate , does> @ state @ if postpone literal then ;

With this you can define all the constants in one bank then as long as the bank is enabled only the literal value will be inserted into any words that use the constant. After all the words referencing the constants are defined in main RAM, the bank with the constants can be disabled since the words in main RAM don't need them any more. This also works for enumerations where a begin_enum word sets a counter to 0 and enum creates a constant in the second bank based on the counter.

With all the graphics data loaded for the game and about 7k just for the constants, this takes up 11.6k in main RAM and 11.7k in the second bank, which doesn't leave much room for the game code. The next thing to try was filtering the source through a Python script every time it is run to shorten the names of any words that are not numbers or forth words. This saves a lot of space. Here's an example

Input:

Code: Select all

5 constant StartingValue
begin_enum
   enum UP
   enum DOWN
variable LongVariableName
variable Direction
: InitVariables
   StartingValue LongVariableName !
   UP Direction ! ;
: IncrementLongVariable LongVariableName @ 2 + LongVariableName ! ;
InitVariables
IncrementLongVariable

Output:

Code: Select all

5 constant x0
x1
   x2 x3
   x2 x4
variable x5
variable x6
: x7
   x0 x5!
   x3 x6! ;
: x8 x5 @ 2 + x5 ! ;
x7
x8

Of course this makes everything unintelligible but that doesn't matter as long as you're just running code. There is a list of words in the script that are left with their original names so that they are still accessible for interactive debugging if needed. This gets the size from 11.6k in RAM and 11.7k in bank 2 to 10.4k in RAM and 7.9k in bank 2.

The next step is to get rid of the constants and enums completely and replace them with numbers, so that they don't take up any room at all in memory. The script looks for any constants defined with "const" (so the the original "constant" still works) and removes the line from the source then replaces any references to that constant with it's numeric value. Enums work the same. This brings memory usage down to 10k in main RAM and 3.2k in bank 2.

Input:

Code: Select all

5 const StartingValue
begin_enum
   enum UP
   enum DOWN
variable LongVariableName
variable Direction
: InitVariables
   StartingValue LongVariableName !
   UP Direction ! ;
: IncrementLongVariable LongVariableName @ 2 + LongVariableName ! ;
InitVariables
IncrementLongVariable

Output:

Code: Select all

variable x0
variable x1
: x2
   5 x0 !
   0 x1! ;
: x3 x0 @ 2 + x0 ! ;
x2
x3

A debug file is output with all the shortened word names and the constants that were replaced. It also has a list of forth words that were never referenced so that they can be removed from the kernal if you need more space.