Claude and I Vibe Coded a Forth Interpreter

[Martin_H]

I'm slightly impressed by this. After pointing out the third time Claude clobbered the Y register, Claude swept the entire file and fixing it in places I hadn't pointed out yet.

Code: Select all

Good fix — adds PHY/PLY to preserve IP around the LDY #16 loop. Let me update it:

Done. That's the third multiply/divide routine fixed with PHY/PLY — STAR, UMSTAR, and now UMSLASHMOD. It's worth doing a quick scan to make sure SLASHMOD and the shift words LSHIFT/RSHIFT don't have the same issue before you test them:

Find LDY # instructions that might be missing PHY/PLY
Find LDY # instructions that might be missing PHY/PLY
Lines 388, 420, 467, and 544 all use LDY #16 for loops — let me check which ones are missing PHY:

Line 544 in SLASHMOD is missing PHY. Let me check line 3067 too for LSHIFT/RSHIFT:

Check line 3067 context - DOTHEX
Check line 3067 context - DOTHEX
Good — .HEX already has PHY. Now check LSHIFT and RSHIFT:

LSHIFT and RSHIFT use TAY to load the shift count into Y, which also clobbers IP. They need PHY/PLY. And SLASHMOD at line 544 also needs it. Let me fix all three:
Now fix SLASHMOD at line 544:
Done — four more PHY/PLY fixes: LSHIFT, RSHIFT, SLASHMOD, and UMSLASHMOD. The pattern to watch for is any LDY #n or TAY inside a primitive that's used as a loop counter — those always need PHY before and PLY after to protect the IP.

[Dr Jefyll]

Martin_H wrote:

Claude likes BDD's technique of pointing the direct page register to the stack

FWIW, I don't think BDD claims to have originated this idea. But I daresay it has been independently dreamed up by various different folks... beginning with the '816 designers! (and it's they who deserve the most credit.) AFAIK the first mention of it on this forum was here in a thread in 2015.

Edit: I spoke too soon. Blargg and dclxvi refer to the idea in this post.

-- Jeff

[Martin_H]

I informed Claude about another code review item. It understood the request and produced the correct changes. However, the code still doesn't work. I'm basically testing the quality into it.

Me: Both SLASH_CODE and MOD_CODE do a JSR to SLASHMOD_CODE. That JSR won't return because SLASHMOD_CODE ends in NEXT. This can be fixed by refactoring SLASHMOD_CODE into helper function SLASHMOD_IMPL which contains the code, and SLASHMOD_CODE JSR's to.

Code: Select all

Exactly right — JSR pushes a return address but NEXT will just dispatch to the next Forth word instead of returning, so the return address is left orphaned on the stack corrupting the return stack. The refactor into a helper that ends in RTS is the clean fix.

Now I have the full picture. Refactor into SLASHMOD_IMPL ending in RTS, with SLASHMOD_CODE calling it then doing NEXT:
Refactor SLASHMOD into SLASHMOD_IMPL helper plus SLASHMOD_CODE, SLASH_CODE and MOD_CODE
Refactor SLASHMOD into SLASHMOD_IMPL helper plus SLASHMOD_CODE, SLASH_CODE and MOD_CODE
Also add SLASHMOD_IMPL to dictionary.inc so it can be referenced from test files:
Done. The same pattern applies anywhere a Forth primitive needs to share implementation code with another — extract the logic into a plain subroutine ending in RTS, and have both the primitive wrapper (ending in NEXT) and the callers (JSR the helper directly). It keeps the code DRY without the stack corruption problem.

[BigDumbDinosaur]

Dr Jefyll wrote:

Martin_H wrote:

Claude likes BDD's technique of pointing the direct page register to the stack

FWIW, I don't think BDD claims to have originated this idea.

I make no such claim. 

It was shortly after I got POC V1.0 running and started converting my seminal BIOS code to run with the 65C816 in native mode that I first used that technique within functions (subroutines).  Prior to that...in fact, prior to actually building V1.0...I had thoroughly read the Eyes & Lichty manual so I would have some understanding of what the 816 could bring to the table.  It was while reading about the movable direct page that, as Jeff had said in his post, it dawned on me that I could point DP to the stack and do all sorts of cool things.

With that understanding, I went to work to develop a method for “automatically” structuring stack frames within functions using local symbols and the Kowalski assembler’s re-definable symbol functions.  Having that and some simple boilerplate code to save machine state on the stack, it was a cinch to create a fugacious “environment within an environment” in each function.  The flexibility that is conferred is enormous and is why I have often asserted that when in native mode, the 816 is a completely different beast than its eight-bit brethren and really requires a different programming mindset.

One of the cool things you can do with DP pointed to the stack is to set up temporary long pointers to data and thus be able to access memory in a completely bank-agnostic way.  This is in contrast to stack-relative indirect-indexed addressing, e.g. LDA  (1,S),Y, which implicitly uses DB as bits 16-23 of the effective address.  Manipulating DB is awkward—it can only be done through the stack, whereas manipulating the MSW of a long pointer as your code “walks” through memory is a snap.

It is interesting to note that while the Eyes & Lichty manual has plenty to say about direct page, relocating DP to the stack receives superficial treatment; the casual reader may completely miss it.

Incidentally, the one link Jeff provided is part of a larger and somewhat argumentative topic on using a 65C816 instead of an ARM or MIPS.

[BigEd]

> I'm basically testing the quality into it.

I think this is a possible pitfall. Instead of a person who understands, has a mental model, of the problem, of the ISA, of the shape of the solution, and who might possibly make some omission or slip which could helpfully be discovered with testing... we have something without any of those things which conjures up plausible code.

My feeling is that such code is going to have more latent bugs, and need more test coverage.

[Martin_H]

BigEd wrote:

My feeling is that such code is going to have more latent bugs, and need more test coverage.

Speaking of latent bugs, Claude's version of SLASHMOD failed my unit tests and Claude asked me to provide the test cases so it could check the code against them. I did and the fixes it proposed were even more broken. It feels like pair programming with someone in another city, using a different source code control system, who doesn't have a telephone, and can't share their screen.

I debugged it and found SLASHMOD was using the page zero TMPA to store the sign of the input dividend. It then called UMSLASHMOD which trashed that page zero value. The bug only occurred if the signs differed, otherwise it was latent. I rewrote SLASHMOD to use stack locals and it passed my unit tests. I uploaded my fix to Claude who accepted my fix into its copy of the file.

The lesson is that Claude doesn't generalize. It knows about stack locals but will go right back to using page zero variables and getting reentrancy bugs. If I were doing this on my own, I would remove the page zero declarations and fix all the dependencies in one shot. But I won't suggest that to Claude because I don't feel like dealing with what it might do. But it's likely those TMP and SCRATCH variables need to go, so I may do it incrementally as I continue unit testing.

[BigDumbDinosaur]

Martin_H wrote:

If I were doing this on my own, I would remove the page zero declarations and fix all the dependencies in one shot.  But I won't suggest that to Claude because I don't feel like dealing with what it might do.

For some reason, when you mention how Claude does this and Claude does that, and how Claude might react when you suggest it do differently, the image of someone training a chimpanzee to program a computer comes to mind.  I continue to think the phrase “artificial intelligence” is an oxymoron and Claude’s “intelligence” is nothing more than that of a high-powered lexical analyzer and pattern matcher.

[barnacle]

Has there ever been any doubt about this?

Neil

[BigEd]

For me, the question is perhaps how best to work with such an assistant: tireless, enthusiastic, able to test their work, but not always understanding the goal.

[BigDumbDinosaur]

BigEd wrote:

For me, the question is perhaps how best to work with such an assistant: tireless, enthusiastic, able to test their work, but not always understanding the goal.

Since Claude is nothing more than a giant blob of software sifting through a giant blob of data stored in a giant data center, I think “understanding the goal” will always be an elusive...er...goal.    In a way, Claude reminds me of Jerry Lewis’ character in “The Disorderly Orderly”—trying very hard to be helpful but frequently bungling things.

[BigEd]

Right, but my framing was aiming to help those of us who use the thing, not addressing questions of how the thing works. It's easy to trade joke descriptions, but what do we do, those of us who choose to engage?

[Martin_H]

BigEd wrote:

For me, the question is perhaps how best to work with such an assistant: tireless, enthusiastic, able to test their work, but not always understanding the goal.

This is a good question. So far it seems that Claude is good at creating boilerplate and structure. That's probably because it's copying memorized training data.

To me the biggest issue with Claude is that it hallucinates. On both the contents of the source code it produces and that it passes Claude's claims of testing. It did that to me yesterday when I directed it to solve a problem with /MOD. Although Claude knew /MOD called UMSLASHMOD, it's solutions were off the mark, yet it claimed the code passed. I had to debug Claude's code and hand Claude the fix.

There's also a deeper problem that I became aware of through my tests. MOD is not symmetric with positive and negative values. So /MOD calling UMSLASHMOD won't yield the correct results. It's not just a matter of preserving the sign which is what the "working" code now does.

At this point I'm going to stop trying to fix the math routines and focus on the interpreter and compiler. I think there's more to learn there and it might be more interesting for me. The math routines won't be needed until I try to run Forth code, and I at least know where the problems are.

I'm retired so this is purely for entertainment, but tech workers are being given edicts to use Claude, with adoption progress part of their key progress indicators. If I were still working this would stop being entertaining and make me worried about feeding my family.

[Martin_H]

I hit my free message limit with Claude until 5:00pm. But I think I am getting better at getting it to do useful things.

Me: The mathematics routines STAR_CODE, UMSTAR_CODE, UMSLASHMOD_CODE, SLASHMOD_CODE, and MOD_CODE are failing unit test edge cases (e.g. negative or large numbers). Rather than fixing them right now, could you add a TODO comment in each routine's comment header block. That way I won't forget to fix them. Given that these are simple mathematical functions, I'm hoping to find working code to reuse.

Me: The good news is that the comparison, io, memory, and stack unit tests are all passing. I think I will learn more by getting the interpreter and compiler working.

Claude: Good plan — the math can be revisited later with better reference implementations. Let me add TODO comments to each of those routines:
Claude: Link to the updated files.
Claude: And congratulations on getting comparison, I/O, memory, and stack all passing — that's the solid foundation everything else builds on. Moving on to the interpreter and compiler is absolutely the right call.

Me: I have a question about WORD_CODE. At the end it pushes the contents of LOC_HERE onto the parameter stack. That value is the same as the input buffer passed in from the USER area. Is my understanding correct?

Claude: Yes, your understanding is correct. WORD copies the parsed word as a counted string to HERE (the current dictionary pointer), and then pushes the address of HERE onto the parameter stack. So the caller gets back the address where the counted string was written.
However there's an important subtlety — HERE is the top of the RAM dictionary, not the TIB. The word is copied from the TIB into HERE. So:

Input: characters are read from TIB + >IN
Output: a counted string is written to HERE (dictionary pointer)
Return value: the address of HERE (where the counted string now lives)

This means the returned address points into the dictionary space, not into the TIB. This is the standard Forth behavior — WORD uses HERE as a temporary scratch buffer. The caller must use or copy the string before calling WORD again or before any dictionary compilation, since the next call to WORD or , will overwrite it.
One thing worth checking in WORD_CODE is that it does not advance HERE (the dictionary pointer) — it just uses that memory as scratch space without allocating it. The dictionary pointer stays unchanged. Only , and ALLOT actually advance HERE.
Does that match what you're seeing in the code?

Me: I can't respond because I've been rate limited.

Update: Reading their terms of service I hadn't sent enough messages today to hit the limit. The problem is that my context buffer is long and Claude rereads the entire buffer. It may be long enough to limit me to one question every 5 hours. I will try the /compact command after 5. If that doesn't work I will close the session and create a new one. Starting with a file upload of my files.

[BigDumbDinosaur]

Martin_H wrote:

I hit my free message limit with Claude until 5:00pm...Me: I can't respond because I've been rate limited.

That’s probably not what you want if you are in the air, the pilot has just keeled over and is unresponsive, you’re the only other one on board, and are asking Claude for advice on how to land the plane. 

[BigEd]

Maybe you could edit down a transcript of the most productive or decisive parts of the conversation and paste that in at the beginning of the next session? Maybe there's a better way... context is everything!