What is Forth?

[GARTHWILSON]

25 years old (slightly before ANS Forth) but still relevant:  This page has loads of good info on Forth's history, efficiency, versatility, and accomplishments:  https://www.forth.com/resources/forth-p ... -language/

There are too many features to repeat here, but I'll mention these:

• task-switching times of different OSs on the 68010, with PolyForth's being many times as fast as the others (under the "2.4.3.6 Multiprogramming" heading)

  

  The text says, "In theory this non-preemptive algorithm is vulnerable to a task monopolizing the CPU with logically or computationally intensive activity, but in practice real-time systems are so dominated by I/O that this is rarely a problem.  Where CPU-intensive operations do occur, PAUSE is used to 'tune' performance."
• under the "2.2.2 Application requirements" heading, some of the principal application areas in which Forth achieved early success:
  • 32-terminal access to a 300MB data base in 1974, doing 100,000 transactions in its first week, with response times under one second, on a Data General Nova, for Vernon Graphics, Inc., a service bureau to Pacific Telephone, in 1974;
  • this system was subsequently marketed for business applications in banking and hospital management
  • FORTH, Inc. developed a series of image processing applications for the Naval Weapons Research Center, NASA’s Goddard Space Flight Center, the Royal Greenwich Observatory in England, and others.  The approach taken included many features now associated with object-oriented programming.
  • instrumentation and control at observatories
  and under the "3.3.1 Environment and applications" heading,
  • roughly 500 networked processors used for an extensive facility management system at the King Khaled International Airport at Riyadh, Saudi Arabia
  • In 1990, Federal Express won the prestigious Malcolm Baldridge quality award for its package-tracking system, whose data entry is performed by Forth-based hand-held devices carried by Federal’s 50,000 couriers and agents world-wide.
  • many satellite and space shuttle experiments
  • The November, 1990 Columbia shuttle flight carried four astronomy payloads, of which three were programmed in Forth.
  • The January, 1992, Spacelab flight featured a Microgravity Vestibular Investigation (MVI) experiment using a polyFORTH system for on-board control and analysis.
  • Probably the most prolific single purveyor of embedded Forths is Sun Microsystems, whose SPARC workstations all use a programmable Forth-based monitor called Open Boot.

6502 content:  Under "3.1 The Forth Interest Group," it tells of Bill Ragsdale becoming aware of the value of Forth and wanting it to use in his successful bay-area security-system manufacturing company, and his writing a 6502 Forth kernel, exploiting page zero and stack-implicit addressing architectural features in the 6502.  He and six others formed the Forth Interest Group, and later added kernels for other processors.

The page has lots more good reading on Forth as well.

[whartung]

It's fun to read about Moore and ColorForth and his views of simplicity and sparsity. Reading some of the late Jeff Fox's papers about it. What code of theirs that I've seen is beyond inscrutable. It's very dense, and it's just gorged with magic numbers with no context. The simple example of ColorForth IDE reading word, its basically ports and INPUT instructions. Perhaps if you know its an IDE routine, and you know the actual I/O ports IDE uses, it's a bit more clear.

I wonder how often these guys went back to their code after 6 months of working on something else.

I'm in the process in my spare time of trying to port F83 to 6502. F83 is a PC implementation, but it's complete, has a meta compiler, etc. Ostensibly I simply need to convert the code words (and I'll use my Fig implementation for inspiration there), and then figure out how to get it to dump out a big hex dump instead of a COM file.

That is a very large source base. And trying to make heads and tails of what is where, especially with he meta compiler -- boy I tell ya, it's not for the meek.

F83 is really well documented, they had shadow screens for most of the code to explain what it does. But it, too, take serious study to follow along.

Since I use it to modify itself, I find I have to reset a lot. Sometimes I'm able to exit to DOS and reload, but more than not, if you get a loop bad, I have to restart the VM.

It's definitely challenging to work back in the day of a 80x25 screen with no scroll back.

I managed to create text files of all the source, with the shadow screen side by side. Those are great for references.

But, this is well written, well documented code, and it's work to get through it and figure stuff out for sure.

What I should do is load the entire system in to a single file. Switching between files is a bit of a pain, and if you use the VIEW command to see the source of a word, it swaps files underneath and you have to switch back to the file you're working on.

I know, this is not indicative of how things are done today, for sure. But it's how it was done back in the day. Both F83 and Pygmy Forth were pretty advanced. F83 even has the tasking system, which should port easily to the 6502. Just still have to get in the groove of it. We are certainly spoiled and jaded today.

[BigEd]

Your post reminds me a bit of APL... APL can be very very dense, and with single-letter variables, and the idea is that you read a program the way a mathematician might read a proof - very slowly and carefully, and with great concentration, and with every character being crucial. At the opposite extreme, perhaps, is COBOL, with long meaningful names and lots and lots of lines in the program. You might skim a COBOL program the way you'd skim a newspaper.

So, maybe the Forth style you're describing is somewhat like a dense APL style, where the reader is expected to study and internalise the logic, moving very slowly. That's more demanding, which means fewer readers are going to manage it, which means it's unsuitable for large corporate teams - but for a tiny team or a one-person show, so long as those people can maintain their mathematical proof-reading performance, it might be a win.

[whartung]

I think, for me, the fundamental issue is the lack of visibility of data in flight.

Simply, at the beginning (barring a comment), you don't have a view of what the stack should look like.

Next, without intimate knowledge of the individual words, you don't know what the stack looks like in during processing. With normal code, you see the values bouncing back and forth across work registers and memory as the data is passed from routine to routine.

Finally, the terse nature of the commands doesn't help for casual reading either. So, its several levels of keeping the stack in your head, trying to intuit what each words does by its name, then actually knowing what each word does to the stack.

There's a lot of unfamiliarity going on, so, especially early on, the cognitive load per character is really high.

With experience, it will go away, but for now, it's a slow start.

I do think I will combine the system in to a single file. This will entail combining the 3-4 source files in to a single file, and then rebuilding the system from that. That will make VIEW much less expensive (I'd sure like a command history, boy I'd buy that for a dollar! ). It'll also just give yet more exposure to the overall system layout.

Heck, I'm still relearning the editor when I come back after a day or two.

[GARTHWILSON]

The way I avoid that is to put in the comments what's relevant on the stack at the end of each line.  Here's an example of the form a few lines from one of my words used in feeding the graphic LCD, to find and modify individual pixels in the map:

Code: Select all

     DUP -3 SHIFT (   8 / )     \ Get LCD page# (range 0-7).                            ^ LCDcol#  LCDrow#  page#
          7 SHIFT ( $80 * )     \ Get how far to go into VID_MEM for right page begin.  ^ LCDcol#  LCDrow#  pageBaseAdr
     ROT      +                 \ Get byte number in VID_MEM.                           ^ LCDrow#  Byt#inArray

The ^ tells what's on the stack at the end of the line.  Even in rather complex routines, if I want to come back to it later to change something, I can dive into the middle where I think I want to make the change, and figure it out quickly, rather than start at the beginning and try to keep a lot in my head at once in something that's no longer fresh in my mind.

[whartung]

Fox and Moore seem to follow the "Denser the better" idiomatic Forth Screen model of development. One screen I saw that was supposed to be an example of Moore's work was essentially packed full. Basically, imagine the screen filled with colon definitions that are simply word wrapped, with no other new lines. He really seemed to advocate this style of work.

[scotws]

whartung wrote:

There's a lot of unfamiliarity going on, so, especially early on, the cognitive load per character is really high.

Yes, Forth has a brutal learning curve. I spent the first weeks yelling What?! at the screen so much I felt like the Tenth Doctor. Some things that helped me:

Swallow your pride and put a stack comment after every single word, at least at the beginning. You'll feel stupid, yes, but you can walk through the code step-by-step. Once you know the code works, you can go back and condense the comments. But don't. You'll thank yourself when you come back in a year.

Use a modern Forth. Old Forths try to do all kinds of stuff at once - exhibit A is FIND which returns ( addr 0 | xt 1 | xt -1 ) depending if it found the word, and if the word is immediate, and I can never remember what addr is. ANSI Forth (and Gforth, which seems to have become the real standard) follow the philosophy of doing one thing: FIND-NAME is ( addr u -- nt | 0 ) (or xt instead of nt, depending on your Forth). REFILL just returns a flag if it worked or not. PARSE-NAME replaces WORD and just does ( "name" -- addr u ) while WORD ends up being BL WORD COUNT anyway and then does something weird with an extra space character at the end of the input.

Write shorter words. This is still one of my biggest problems, my words still tend to be multi-line. It seems wasteful, but remember that a good Forth compiler will turn everything into very small assembler snippets.

Rewrite again and again. I mostly use Python, where there is supposed to be one obvious way of doing things, and there usually is. Forth rewards going back over your code and refractoring it. That is where you'll have sudden insights about how CREATE DOES> would be really cool or how you really should use DEFER (I love DEFER). This means that your Forth code will follow Hemingway's famous line about normal writing: "The first draft of everything is ****."

I know what you mean about not knowing what the words are for. It took me ages to realize that COUNT is also used to walk through strings, not only to convert old style to new style. Maybe somebody should start a "bestiary" of Forth words with examples and such?

[barrym95838]

scotws wrote:

"The first draft of everything is s**t."

... unless you are a one-in-a-million genius, then the odds get a bit better. I have ample proof that I am not a member of that elite group, but I know that they are out there.

https://www.youtube.com/watch?v=uq2eD-POpDI

Mike B.

[GARTHWILSON]

I would not tolerate the way whartung is saying Chuck Moore wrote.  He might be an absolute genius; but that writing style (or lack of it) sure won't help someone else who has to come along later when it's time to modify something.  I did have to work briefly in about 1990 with screen files, and I absolutely hated it, especially the lack of space for comments, and the problems with inserting lines or sections when a screen is pretty full and you have to scoot all the subsequent screens down to make room, changing all the associated references to screen numbers.

I also think the common recommendation for short words results in overfactoring, where it becomes harder to come up with an adequate, short, descriptive name for a factor.  I do rely a lot on visual factoring though, and well-commented stack-autonomous lines.

scotws wrote:

Yes, Forth has a brutal learning curve. I spent the first weeks yelling What?! at the screen so much I felt like the Tenth Doctor.

I found it quite easy and natural myself, going through "Starting Forth."  I had already been programming with my HP-41cx RPN calculator which might have helped a lot, but I liked the fact that there was no punctuation, no parentheses, and almost no syntax rules.  Now I'm on about page 95 of [Edit: later finished, with the same conclusion] the K&R standard book on C which Samuel Falvo recommended, and I cannot believe how such an incredibly idiotic language (C) got accepted by the industry!  I assume it took a good salesman taking advantage of perfect timing and other factors that just happened to line up.  Samuel's words were, "C is a very simple language to learn.  Just be careful, since C is also a terribly difficult language to master."  I can't agree with it being simple to learn.  Difficult to master, definitely.  I have a million notes in the book.  I can only take a couple of pages at a time, as it's so exasperating.  IMO, C is an absolute disaster; and the C code in the book (which is supposed to be a model) looks to me like when you see the helicopter views of houses in Tornado Alley that have been turned into a pile of toothpicks by a tornado.

[White Flame]

Very loosely referencing the cathedral & bazaar, I've called that ultra-dense and ultra-customized style "building a little cathedral". Problem is, if something foundational needs to change, you often need to take the wrecking ball to it and rebuild a slightly different cathedral.

[BigEd]

I think the rebuilding idea is indeed built in - scotws mentions refactoring. It's not necessarily a bad idea to have a page or two of very dense code which is sometimes completely rewritten. But it doesn't scale to multiple authors or maintainers.

Just for amusement, here's some C code written by an APL guru:

Code: Select all

C*ppd(I a)
{
  R QS(a) ? XS(a)->n :
    (QN(a)?(APLpick(n0,ns,n2)):QP(a)?(APLpick(p0,ps,p2)):xfs_desc)[U(a)];
}
I aplus_pi(C *s){HTN z=chtgi(APLpick(p0ht,psht,p2ht),s);if(z)R z->a;
 if(*s=='_'&&(ISlower(*(s+1))||'_'==*(s+1)))R xslu(s);R 0;}
Z C*fn(C *s,I n){for(;--n;)s=1+nx(s);R s;}
Z C*ss(C *q,C *s){I n=strlen((DEV_STRARG)s);for(;strncmp(q,s,n);)++q;R q;}
Z C*sb(C *q){I i=0;C c;for(;c=*q++,i||c!='{';)i+=(c=='(')-(c==')');R q;}

Don't look too closely or you might lose your mind in there. We're used to understanding ten lines of code in ten seconds. In this case, probably ten hours would be enough.

[Tor]

I always used HP calculators and I'm very familiar with RPN. That hasn't helped me one bit with Forth though, the more I look at it (and I have, for years, if not decades) the worse it gets. As it is now, I don't understand anything of what has been posted lately in the Forth section here.
As for C though, I had a brief start with BASIC in the dark ages, then quickly got disgusted, and found structured programming. I wrote Fortran 77, and Pascal, mostly. Later C came around, but I can't remember any learning curve worth mentioning for C. I started with an existing piece of code, as I recall, and Henry Spencer's 10 Commandments for C Programmers (to learn about pitfalls). Taking a step back, C looks just like any other structured programming language. Thus the non-issue of moving from Fortran and Pascal to C.

Despite my initial attraction to Forth, I'm now more and more in agreement with those who have said that the way to write Forth is to find a problem which fits Forth, and ignore every other kind of problem. That may be very black and white, but it doesn't feel too far from the truth either.

[barrym95838]

Tor wrote:

... Later C came around, but I can't remember any learning curve worth mentioning for C. I started with an existing piece of code, as I recall, and Henry Spencer's 10 Commandments for C Programmers (to learn about pitfalls). Taking a step back, C looks just like any other structured programming language. Thus the non-issue of moving from Fortran and Pascal to C.

Despite my initial attraction to Forth, I'm now more and more in agreement with those who have said that the way to write Forth is to find a problem which fits Forth, and ignore every other kind of problem. That may be very black and white, but it doesn't feel too far from the truth either.

I had a similar experience moving from BASIC to C ... it just "flowed", with minimal fuss. If I need to hack something out, my "GOTO"s are still BASIC and C. I want to learn Forth because I feel like I might be missing out on something really neat, but there seems to be a barrier in my thought processes that is making it difficult. I'm certainly not ready to give up yet.

Mike B.

[Tor]

Yeah. I won't give up just yet either.. thanks Mike. Will fiddle a bit more with gforth I think. It's been a while now.

[scotws]

One insight that helped me was realizing what Forth was originally made for and what it is used for in Real Life (TM) until this day: Situations when your hardware is really not adequate for the job you have to do - when you have to "do more with less". You're NASA and you're sending a probe into the cold and radiation-bathed hellscape around Saturn, so you're hardware has to be robust and dirt simple. You're trying to generate a Mandelbrot set on an 8-bit processor. Stuff where you have to punch far above your weight.

Forth manages this by brutally offloading stuff to the user. It exposes the stacks instead of hiding them like the Java VM. It uses RPN for math because infix needs special rules ("multiplication comes before addition") and requires special characters like ( and ). You have to remember things like cell sizes and word alignment issues, create your own data structures, balance said stacks. It doesn't hold your hand and check your types or even protect your memory.

Put differently, Forth places an enormously high "cognitive load" (for lack of a better term) on the user. This is the price for being able to get every last byte and cycle out of a system without having to drop down to assembler and not having to create a super-efficient, complex compiler. What runs on my Devil's Canyon PC in 64-bit Gforth will (er, should) run on little 6502 8-bit Tali.

Pretty much every other language out there tries to reduce the cognitive load, because this lets the user think of other stuff and makes programming accessible to more people. The Zen of Python states (https://www.python.org/dev/peps/pep-0020/) Simple is better than complex, and its "executable pseudocode" is arguably the best of example of a low cognitive load -- it can't get much easier than

Code: Select all

list = [1, 2, 3]
for element in list:
    twice = element * 2

Even if you don't know jack about computers, you sorta, kinda see what is happening here. Forth looks like the cat walked over your keyboard with the caps lock on.

What this means in practice (I tell myself) is that it's no use waiting for some flash of insight with Forth that will make everything suddenly come together and turn easy. It's hard, and it's going to stay hard, because that's the price you're paying for doing "more with less". The fun part, if you enjoy these things, is that you're allowed, even encouraged, to (try to) be clever, because if you're desperate enough to be using Forth for your project, that's probably what it is going to take. With every other language telling you to "keep it simple, stupid", that's kind of a nice change once in a while.