Very new to assembler and 6502. I've been choosing various simple algorithms to implement in order to learn. I keep assuming that I can use RTS to return from a sub routine after branching to it. But from what I'm reading, I can only do that after using JSR. So I'm wondering if there's a common way to branch to a subroutine so that I can use RTS? I'm not just interested in how to do this, but if its good programming practice or not. In other words, if I run into this situation, is it a sign I should be rethinking my flow control?

Here's an example. I'm currently working on implementing a multiplication add/shift method. Here's a code snippet of where I test for whether or not I need to add before shifting:


Advice is appreciated, thanks!

Two points:

1.
Read the actual write up about "RTS", the CPU does not actually "remember" the flow of the program as if it is a sentient being, it just follows the flow of the instruction execution.

Technically speaking yes "RTS" is used to return from a subroutine call.. but it's all a matter of perspective......

....and here is the but...
The "RTS" instruction does this by pulling data from the stack....., so you can actually use "RTS" (and other return instructions RTI )to do all sorts of code subversion, by manipulating the contents of the stack or indeed the stack pointer & other registers...

2.
What you are thinking about, is using branches to get to a section of code terminated by an "RTS", in this case, the RTS would return you to whatever address was on your stack.....
so if you have not prepared the stack content beforehand by using a "JSR" well.... who knows where you might end up....

so lets take an example





The first call using the "JSR" in the code "program_entry", will return back to label "return1:" at completion of the subroutine
However, the "branch" to the subroutine would execute the subroutine code, BUT then it would return BACK to Return2:
and the value #02 would NEVER be put into the Accumulator register...(also making reading the code confusing)

Now this perfectly valid code, but jumping out of functions like this is fraught with all sorts of problems...., not least of which, if changes are made to the function "mysub"
Then potentially it could have all-sorts of consequences...

Since from the program perspective the function " mysub" is only a function if called via a JSR, but becomes NOT a pure subroutine if branched to or jumped to.

Ideally code should "flow" so that it branches as little as possible(fall through), branches add delay & possible confusion.

Welcome razorfishsl! Yes, jedSavage, like razorfishsl said, JSR, RTS, and RTI can be used to pull various not-so-obvious tricks. The processor doesn't care where you've been. When it sees RTS, it just pulls the top two bytes of the page-1 hardware stack (typically put there by JSR but not necessarily) and treats them as an address, adds 1 to that address, and jumps to the resulting address.

You might benefit from my 6502 stacks treatise. For now, just see the first two sections, about stack basics and subroutine return addresses and nesting. Other tricks alluded to come in section 11.

_________________
http://WilsonMinesCo.com/ lots of 6502 resources
The "second front page" is http://wilsonminesco.com/links.html .
What's an additional VIA among friends, anyhow?

On the 6502, only the relative branch conditions may be conditionally executed, and they have no stack effect that can be utilized to allow RTS to return to the following instruction if the branches are taken. Refactoring your code could be an option, or you could just do something like:
... which uses the opposite condition to avoid the action in question.

_________________
Got a kilobyte lying fallow in your 65xx's memory map? Sprinkle some VTL02C on it and see how it grows on you!

Mike B. (about me) (learning how to github)

yep.. you might have missed a "trick" when writing "protection" routines for the commodore C64 & disk drive (since the disk drive was programmable).....
you could pull in a "dummy stack" to overwrite the existing stack, to execute routines.. so that you can hide execution,... bit tricky to pull off since you need to know where the Stack pointer is..
but if you were really good.. you could hide addresses from those "replay cartridges", since they would destroy some stack space on NMI's etc....... as a function of the CPU.

It's been over 36 years.. but it comes back to you....

Thanks all for the comments and links!

Ah, return-oriented programming. Still relevant in the computer security arena.

Welcome! Let me try to answer at a simple level.


Good question, and well put. You want to understand, and to understand best practice.

Aside from advanced techniques, there are three ways for your program flow to be something other than a straight line - just one instruction after another. The branch is used when there are two ways to continue, and the subroutine is used when you want to do something else temporarily and then come back automatically. The jump is used when you want to go elsewhere unconditionally.

In some CPUs there are more facilities, but that's all we have. And, the branch is a small fast instruction which can only reach a limited distance. So you might have

but if your code has grown so that the branch destination is no longer nearby, you need to jump, because that's not distance limited:


Now we see this pattern, it's the same pattern for a conditional call to a subroutine. For a normal call we'd have

and for a conditional call we'd have


In both cases you'll note that the branch which skips over the bit in the middle has to have the opposite sense to what you'd use if you had conditional calling, or far branches.

Hope this helps!

(And welcome to you too, razorfishsl)