What exactly happens if you do an RTS without doing a JSR or something similar first? Does the stack pointer wrap to 0x00?

The reason I ask is because I'm currently making a 6502 emulator, and my testsuite is based on actual 6502 assembly code. It works by first assembling, then executing, and finally perform asserts on the mpu's state.
At this moment (with tests for every opcode except the ones that manipulates the stack) I do this by stepping as many times as there's lines in the assembly file.

Pseudocode:


This works great for the tests I have right now, but it won't work for more complex assembly (using labels and JSR/RTS, etc). What I find most logical is having a rule that an RTS while stack poiner is at 0xFF stops the loop; but of course, that won't work if people rely on some other behaviour (I can't seem to google an answer on this one).

Another idea I've had is having a 'test mode' state in the MPU, where it treats an arbitrary and unused opcode as 'the end of test'.

Other suggestions are welcome.

Yes. Architecturally, the S register is no different than the X or Y registers, so as DEX or DEY will wrap X and Y, so too will the decrementer logic inside the 6502 wrap the S register too.



I'm finding an ever-increasing reliance on such tests -- this is a good thing!



The original NMOS 6502s would "jam" when executing opcode $02. That might be a good candidate.

Usually you'll do an LDX #$FF, TXS as part of your reset routine to initialize the stack pointer. You don't have to start at $FF, but if you use page 1 for anything other than the stack, you'll want to make sure the stack stays where it should so it won't step on your other variables or whatever. The stack can be used for a lot of things besides subroutine return addresses, one of them being that you can calculate an address (minus 1) to jump to, put it on the stack, and later when you're ready, do an RTS without having an associated JSR.

Yep, I tested this on an NMOS 6502 and the stack wraps as you'd expect.

I'd second the idea of using $02, or $42 - they're both unassigned, act as a jam (or KIL) instruction on 65c02, and on the 65816 they are escape instructions for extensions: COP and WDM.

Thanks for the answers!
I will use one of the "jam opcodes" then