65802/65816: Stack Overruns Program or Data

No longer limited to a single page, the native-mode stack will grow downward as far as your program pushes bytes onto it. Large programs should either retrieve every byte pushed on or reset the stack periodically (using TCS or TXS). The potential danger is when a stack grows uncontrollably until it overwrites variables, your program, or the operating system.

In this connection it is important to be aware that, although the high byte of the stack register is consistently forced to one, new 65816 opcodes executed in the emulation mode will not wrap the stack if the low byte over- or underflowed in the middle of an instruction. For example, if the stack pointer is equal to $101, and a JSL is executed, the final byte of the three bytes pushed on the stack will be at $FF, not $1FF; but the stack pointer at the end of the instruction will point to $1FE. However, if JSR (a 6502 instruction) is executed in the emulation mode with the stack pointer equal to $100, the second of the two bytes pushed will be stored at $1FF.