6502 general purpose calling convention.

The arguments or return values are first reordered as a list of bytes.

The first byte is in A, the second byte in Y, the third byte in X.
No function is ever expected to preserve A, X, nor Y, even if they are not used to pass arguments or return values.

If there are still more bytes, only THEN they are passed on the stack, and on the true hardware stack.
Arguments are pushed before the return address, by the caller. It is also the job of the called to pop them.
If Return values should be passed on the stack, then the caller should push dummy values if there is more bytes in the return than in the arguments.

In summary, a caller program should push
max (0, arguments_byte_count - 3, return_value_byte_count - 3)
bytes when calling a function.

Bytes 4 and up are accessed this way :

tsx
lda $103,X   ;Reads argument/return byte 4
sta $104,X   ;Write argument/return byte 5
etc....

For example :
void function1(u16 a, u16 b)
The low part of a is in A, the high part of a in Y, the low part of b in X and the high part of it on the stack

void fuction2(u16 a, u8 b)
The low part of a is in A, the high part of a in Y and b is in X

void function3(u8 b, u16 a)
b is in A, the low part of a in Y and the high part of a in X

The following optimisations are impossible for non static functions, as they could be written in a different language and compiled by a different compiler at a different time.

== optimisation 1 (static functions only) ==
of course, the code generated for function2 and function3 could be more efficient than the other while being strictly identical because of the call convention.

If the function is static, then the compiler could try to internally remap arguments of a function and see if the function is smaller or faster, and keep that version of the function if this is the case.


== optimisation 2 ==
If the compiler detects one of the arguments or return values is actually just a bool, then the argument uses one of the processor status flags instead to pass the arguments. Other (non bool) arguments are passed as if the bool argument was removed from the list.
Which flag exactly is used can depend. C and V are more "general purpose" (affected by few instructions), but Z or N could be good to pass conditions like (variable == 0) or (variable < 0) respectively.

== optimisation 3 ==
If a function can manage to spare completely X, Y or A registers, the caller function can be made "aware" of it and rely on this feature, so it avoids saving it's stuff in temporary storage

== optimisation 4 ==
(available only for argument, not for returns). If there is more than 3 bytes of arguments, but that *all* of them are "constant" i.e. in all calls the arguments evaluate into a simple numeric number, the arguments does not have to be passed to the stack.
Instead they can be included directly in the code like this :

jsr call_a_fun_with_lots_of_args
.db $8   ;Number of bytes of arguments
.dw funct
.dw $04, $05, $06, $ffff    ; A lot of arguments

This particular technique is slow but allows to save some precious bytes. It would only be used if optimisation for small code size is nessesarly.

== optimisation 4b (non 6502 specific) ==
If an argument not only reduces to a numeric expression, but also that the number is actually the same number every time, the argument is removed from the list, and hard-coded into the function.

== optimisation 5 (non 6502 specific) ==
If a function is only called once, then it has to be inlined.

    ; allocate block for potential results
    result .ds 16
    ; int myfunc(int a, int b) {
    ;     int c;
    ;     c = a + b;
    ;     return c;
    ; }

myfunc_a: .ds 2
myfunc_b: .ds 2
myfunc_c: .ds 2
myfunc:
    CLC
    LDA myfunc_a
    ADC myfunc_b
    STA myfunc_c
    LDA myfunc_a + 1
    ADC myfunc_b + 1
    STA myfunc_c + 1
    LDA myfunc_c
    STA result
    LDA myfunc_c + 1
    STA result + 1
    RTS
;
; void main() {
;     int a;
;     a = myfunc(1, 2);
; }
main_a: .ds 2
main:

    LDA #1
    STA myfunc_a
    LDA #0
    STA myfunc_a + 1
    LDA #2
    STA myfunc_b
    LDA #0
    STA myfunc_b + 1
    JSR myfunc
    LDA result
    STA main_a
    LDA result + 1
    STA main_a + 1
    RTS