I'd like to talk about creative uses of self modifying code. To start things off, I present some self modifying code from my Forth's kernel:

This line in SCAN

modifies the branch for the UNTIL, loop.
These lines in SKIP

modifies the branch in the same loop.
As chitselb said, Waddya think, sirs?

Glad you got it working, but there's room for improvement, I'd say. There are one or two spots where the code is a little scary (IMO).

To be fair, the sort of tricks you're using are a bit tough to pull off when the assembler is geared toward structured conditionals (such as IF - THEN and BEGIN - UNTIL). You end up being forced to invent workarounds to defeat the assembler's error-checking (which supports the elegant, nested conditionals). I'm not judging -- I've done stuff like this. But some of the ways of doing it turned out to be less ugly than others. That's my $.02 worth in terms of general comment. Now, specifics:

I don't fully understand this part. Is it Bill Ragsdale's assembler you're using? 0= is both a Forth word and an assembler word. What is the value you intend to load in A? Is it part of a PFA or a CFA, perhaps?

What's clear is that, at run time, the STA (up top) resolves a reference that's further down in the code. You could consider doing it the other way 'round, and doing it at compile time. IOW have code up top that leaves an address on stack or elsewhere, and the resolution would happen further down (writing to the address that was noted earlier). Because at that point the forward reference is resolved you can eliminate the dangerously error-prone 24 + that's used in the code! (It seems odd and unnecessary that you're resolving the reference at runtime, but maybe you have a reason that I'm missing.)

[ Edit: alright, 0= # LDA, puts an opcode in A, is that it? (Hence the run-time operation.) Some additional commenting wouldn't hurt; this illustrates the point noted below. ]

Self-modifying code is a powerful technique, and it can be done fairly cleanly. The "pro"s can be pretty dramatic, but there are also "con"s to consider, and that's an evaluation that is best done on a case by case basis, IMO.

In most case you'll just modify operands -- not opcodes -- and thus the technique needn't be mind-bendingly complex.

I suspect SMC became stigmatized largely because it requires very careful commenting in the source code, and that effort wasn't always forthcoming. That's one main drawback. The others are:

• SMC isn't reentrant
• SMC needs to execute from RAM.
• SMC might fail due to cache issues (and 6500 family processors obviously are exempt from this concern ).

-- Jeff
[2nd edit: clarity brevity courtesy]

_________________
In 1988 my 65C02 got six new registers and 44 new full-speed instructions!
https://laughtonelectronics.com/Arcana/ ... mmary.html

I'm always interested in SMC, as I think there's a lot of additional capability there that most of us have been neglecting. It does, as Jeff says, require really clear commenting. (Speaking of commenting, Jim, where did the // come from? The only standard ones I know of for Forth are the \ to tell Forth to ignore the rest of the line, and the ( to tell it to ignore everything up to the following ) on the same line.) I hadn't thought of using SMC to modify a branch (since I have not really spent time thinking/working out all the things that could be done with SMC). That's pretty cool. However, I, too, would certainly want the compiler to figure out the offset where to store whatever it is that's being modified.

I'm hoping my work will give me more time next year to take on some of these projects. It goes in waves, and this year has been busier; but I'm on the home stretch for my current work project. [Edit, 6/17/19: SMC article posted, at http://wilsonminesco.com/SelfModCode/ .]

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?

Alright, this is pretty creative. I hope it's not too disappointing if I continue to focus on niggly details.

Here's one way to get rid of the hard-coded offset. At compile time we note the address where a dummy instruction operand has been placed; then later the genuine operand gets installed. The address is temporarily held in a variable, which regrettably isn't a very Forth-y way of doing things. We could keep the address on stack, but the stack already has error-checking items getting pushed and popped, and the gymnastics -- SWAPs and whatnot -- to keep everything straight can get crazy pretty fast (and of course they do nothing to improve readability).



I've only shown one NASTYSTASH variable, but NASTYSTASH2 and NASTYSTASH3 would be handy to have, as I see SKIP also refers to some hard-coded offsets.

You may prefer to give the variables different names than the ones I've chosen. And luckily they get recycled. As part of the assembler, they have a one-time memory cost (not an incremental cost every time they're used). I apologize if I've drifted off-topic. This post has more to do with working around Ragsdale's structured conditionals than with self-modifying code per se.

_________________
In 1988 my 65C02 got six new registers and 44 new full-speed instructions!
https://laughtonelectronics.com/Arcana/ ... mmary.html

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?

Alas, the Commodore 64 lacks a backslash key ( \ ), so I followed the example from Blazin' Forth and defined double slash ( // ) for a comment to the end of the line.
The Forth-83 standard also specifies .( to display the following text up to ) .

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?

Yes, it puts an opcode in A. The assembler uses Bill Ragsdale's syntax, but the assembler is one I wrote.

I noticed that the only difference between SCAN and SKIP was one byte! the type of branch used at the bottom of the loop. By using SMC I was able to save 40 bytes.
As for dangerous, this is from the source for my Forth kernel. It's built with a metacompiler that I wrote which runs on the Commodore 64. When I first got the idea for this, I tested it by compiling on the host and using SEE to make sure I got the address correct in both SCAN and SKIP.

What makes using the hand calculated offsets worse is that I added labels to my metacompiler before I tried this.
LABEL creates a constant in the host dictionary ( so it doesn't disturb the target code ). The new constant has the address of THERE, the target's HERE in virtual memory ( the Commodore REU ).
I probably should have defined SCAN like this:

And SKIP should have been:

SKIP still has hand calculated offsets, but they are small.
Note: I haven't tested this yet.

Any code definition which uses the scratchpad area N isn't reentrant.
If I was writing this for a ROM-able Forth, I'd have to rewrite a lot more than some SMC.
If I was writing for a different processor family, there is a great deal I'd have to rewrite. I wouldn't think of trying something like this on an intel processor.

Not that I'm aware of and it's PETASCII so some of the characters are graphics characters.
On the Commodore 64, the characters look like upper case. using the shift key with one of the keys A-Z produces graphics characters.
shift 'm' does kind of look like a backslash, but if the shift and Commodore keys are pressed what displayed as upper case now looks lower case and the graphics characters now look like upper case characters. Having every comment to the end of the line begin with an upper case 'M' would be odd.

My example of how I should have coded SKIP uses much smaller offsets. I don't see a problem with this as long as the offset is around half a dozen bytes or less ( skipping over maybe two instructions ).

Using labels in the metacompiler has no cost to the target system.

Although my assembler uses Ragsdale's assembler syntax, my conditionals are somewhat different. I tried to make them as much like the high level Forth conditionals as possible for increased flexibility.

Notice the definitions of ELIF, ELSE, WHILE, and REPEAT,
BRAN, is used like this:

Since the high level conditionals and the assembler versions use exactly two items on the stack for control flow ( an address and a security code ), I also have the following which work with assembler or high level conditionals:

It is my hope that flexible conditionals and a straight forward way to manipulate the control flow data will make writing self modifying code easier and clearer

What about an SMC array? One name with a few places to store addresses.


[Edit: fixed incorrect comment for >= in the assembler]

Yup -- understood. ( And W is another case where RAM is used in a non-reentrant way.)

Btw some of my points (like those you quoted here) aren't especially directed at you, in case you're wondering. There's still some FUD out there regarding Self Modifying Code, so whenever the topic comes up I try to shine a light on what the real issues are.

Interesting that you wrote an assembler using Bill Ragsdale's syntax. The code in Bill's version is interesting to say the least! I remember there was one particular word -- UPMODE -- which seemed to demonstrate rather a lot (arguably too much) cleverness!

-- Jeff

ps -- a link: 6502 Assembler in 96 lines of Forth (July 1980)

_________________
In 1988 my 65C02 got six new registers and 44 new full-speed instructions!
https://laughtonelectronics.com/Arcana/ ... mmary.html

I've following this discussion with some interest. I can't say that I have a strong opinion one way or another. Generally, I will fall on the side of avoiding SMC. Many of the reasons given here reinforce the idea that SMC should be avoided.

That being said, there has been a development, not necessarily recent, that appears to resurrect / rely on the use of SMC in every day programming. That development is the increasing reliance on lambda functions, closures, and other similar dynamically created code fragments / functions / procedures or whatever you'd like to call them. I see these dynamically created functions as SMC in the same sense as the approaches discussed in this thread.

Comments?

_________________
Michael A.

Yup -- downsides and upsides. And the tradeoff should be evaluated on a case by case basis, since one situation can differ from another.

( In FIG Forth for 6502 I think there's only one occurrence of Self Modifying Code, and that's in the inner interpreter. The NEXT sequence is essentially a double-indirect jump. The 65xx instruction set doesn't directly support that, so instead a JMP (ind) is used, and the two-byte operand that follows the $6C opcode gets modified before the JMP (ind) is executed.

Along similar lines, I've read that some early computers lacked indexed address mode, so the programmers did indexing simply by coding the access to use absolute address mode. Then at runtime other code would compute the desired effective address and write that EA to the operand field of the absolute-mode instruction before it executed. It's typical of how SMC can offer escape from between a rock and a hard place. )

That's a good insight, I'd say. Not that I'm an authority on dynamically created functions. I'd be glad to hear more about this.

-- Jeff

_________________
In 1988 my 65C02 got six new registers and 44 new full-speed instructions!
https://laughtonelectronics.com/Arcana/ ... mmary.html

This is completely different, simply by the fact that while you may have some mechanism of create dynamic functions, each instance is unique. You're not changing the functions themselves, you're replacing them.

A, perhaps, more relevant tact is code transformation such as in Java, notably when the code is loaded.

It is common in Java to have class loaders that augment the classes that they are loading. A typical use case it to manifest behaviors via techniques like "Aspect Oriented Programming". Simple examples is adding things like logging around implementations, or collecting statistics.

Most of these will change the byte code of the loaded classes at runtime.

Mind, once loaded, they leave them alone. It's more a part of the "loading and linking" phase than central to the execution of the code.

One could argue that systems like Forth, or any other "image" based system (Smalltalks, Lisps, even BASICs), are contingent on SMC. We just don't consider them such because of the limited, and formal approaches around them.

In Lisp and Smalltalk it's a Lisp and Smalltalk program that's actually changing the running image. In BASIC, it's a bit different as it's simpler to semantically separate the BASIC runtime from the running BASIC program (since the runtime is NOT BASIC).

Lisp, Smalltalk, and Javascript all have some mechanism akin to "eval" that will convert a string in to running code. Historically, the use of such a construct is frowned upon (as it's a bit of a large hammer), but it is available.

At a lunch time discussion with a colleague, we came to the conclusion that the function code was pre-compiled, i.e. static. The operating data of the function was somehow dynamically constructed and at run-time linked to the relevant instruction stream. Can I assume that your definition of SMC is such that since the "functional" behavior is static, then the resulting dynamically constructed function is not SMC in the same sense as possible in FORTH where the instruction stream can be physically changed?

I've avoided these type of functions. It looks like now is a good time to explore how a modern C++ compiler implements these creatures. As I was posing the question above, I lost sight of how the instruction stream might be tied to the data that defines its desired behavior.

_________________
Michael A.