Continued from this thread...


I can see a generic 6502 test program that is retargetable to different platforms/emulators as being very useful. It might be easier to write something new in assembler rather than try to get Py65's existing Python-based test suite to emit assembly.

There seems to be a fair amount of interest in the group here about developing a new processor that is a superset of the 6502. Having such a program would be useful for testing it, so perhaps a few people here would want to collaborate on it.

If that happened, I'd be happy to include that test program in the Py65 package and make running it part of the Py65 test suite.


That's odd. We probably shouldn't include his code in Py65 because of it. However, I don't see any reason we can't use it to find bugs in Py65 and then we can write our own test coverage for those bugs.


Great catch! I believe that I have fixed this now and I've added some new tests for it. Since Py65 is still very young, we'll probably find other issues. Please let me know if you find any others and I'll do my best to fix them.

Regards,
Mike

_________________
- Mike Naberezny (mike@naberezny.com) http://6502.org

Hi Mike
By way of experiment here's a test program which runs through all ADC operands, with carry set and clear, and compares the V flag with the expected result.

It passes on NMOS 6502, on 65816, on lib6502 and on my patched up py65. There's a problem with my copy of lib65816 which I haven't yet run to ground. (Edit: passes now on lib65816 too - BRK was mishandled.)

This is assembled with ORG=0:


Here's the source with ORG=4000:


Among the caveats: we use the machine to test itself, so there could be compensatory flaws; we haven't proved that we exercise all the interesting cases; we check the arithmetic but not say the addressing modes; not checking BCD mode.

I'm not sure how easily these ideas carry over to SBC. For the C flag, one might want also to test the various compare operations.

I'm thinking one could test the carry out by using ASR to right-shift the operands, accounting for the LSBs which land in C, and thus performing a 9-bit addition without relying on ADC's own C output. That is, bit 7 of that sum should match the C flag of the original sum.

But in any case, the V flag seems to cause the most trouble to get right, along with BCD mode. So it might not be worth checking C in this way: a few conventional tests would be much easier to write.

Comments welcome.

Cheers
Ed

Here's the test section for SBC - re-use the first part of the prgram, changing ADC to SBC of course.



This code runs OK on a real 6502, a 65816, and on lib65816. It fails on lib6502, py65 and one of the javascript emulators - evidently overlflow is even harder to get right for subtract than it is for addition.

I found a pointer (on the parallax forums) to an 1800-line assembly program to test a 6809 emulator - could be a useful template. It's by Wolfgang Schwotzer and is GPL.

This is the website, and here's the download - the program itself is src/tools/cputest.txt

An earlier thread points us to Wolfgang Lorenz' C64 testsuite

edit: Good news, it is in public domain. Bad news, it is in the form of p00 files (a pc64 format.) Good news, if a p00 file is binary and supposed to load at 0801 (see here) then we can load at 7e5 into py65 and jump to 0801 to ignore the header. Bad news, need to patch the i/o to suit py65. Good news, that shouldn't be too hard (see c64 rom)

I've written another self checking program based on dclxvi's observation that SBC is just an inverted ADC (when in binary mode) - and with it I've found some bugs in lib6502 and py65. lib65816 passes, as does an NMOS 6502 and a 65816.

Unfortunately we need a more complex test for decimal mode. As I understand it, I can expect the nmos 6502 and the 65816 to differ in decimal mode.



in srec format:


and in py65mon fill format:

There's a decimal mode checking program in Appendix B of my Decimal Mode write-up. As written, it tests documented and undocumented behavior on an NMOS 6502. There are instructions for modifying the program to test only documented behavior (in retrospect, I should have just supplied both routines, rather than a list of instructions). To test a 65C02 or 65816 (8-bit mode; decimal mode behaves the same in either 8-bit native or emulation mode) just substitute JSR A65C02 and JSR S65C02 (or A65816 and S65816) for JSR A6502 and JSR S6502. (I forgot to mention that in Appendix B.) I've tested it on actual hardware for all three flavors (NMOS 6502, 65C02, and 65816).

http://www.6502.org/tutorials/decimal_mode.html

Excellent, thanks! I hadn't stumbled across that code - it's just what we need!

I was about to embark on collecting 512k of results from each platform and comparing them, which felt like it was a sledgehammer approach, but at least not too much strain on the brain.

Ed

Hmm, the ADCA test fails for me on py65 and run6502, because it expects the B and the missing bit to be set in the status register as pushed by PHP. And yet the test runs OK on VICE, and apparently is OK on a real C64. So this would seem to be a visible difference of 6510 versus 6502, which I wasn't previously aware of. [Edit: or perhaps py65 and run6502 have bugs! See below.]

So, maybe these hundreds of tests are useful - they represent a lot of careful work, and seem to check each instruction exhaustively (80 minutes runtime!). But as each test chains in the following one, the suite benefits from being in a c64 emulation with some disk emulation capability. And it will need adjustment for the idiosyncrasies of the different 6502-like devices.

The sources are supplied, in a sense, but they are in a C64 turbo macro assembler format which I can't readily use. So tweaking the tests is a challenge: it took me an hour or two with a disassembly of ADCA to get to this point.

I have to disagree here. The 6510 was a 6502 core coupled with a 6-bit I/O port at $00/$01. There ought be no visible differences between a 6510 and 6502, except for the behavior of $00 and $01 during a memory test.

The VIC-20 was equipped with a genuine 6502 microprocessor, as were most of the PETs (at least one PET was equipped with a 6509 CPU, which used an ingenious technique to expand the user-visible address space to about 1MB). Do the tests behave differently under VICE in these environments?

I find it difficult to believe that the CPU emulation is faulty on x64 -- the KERNAL receives frequent interrupts and one of the first things it does is verify that the B bit is clear (a hardware interrupt). If B is set (indicating that BRK was the source of the interrupt), the end effect is the same as if you'd hit the RESTORE key.

I'm going to disagree too! I have no idea why I supposed that those two (relatively immature) emulators were correct. Although it might have been true that BRK and IRQ behaved differently from PHP, rather more likely that I hadn't got enough data.

Testing on an NMOS 6502, PHP + PLA got me $32. On a 65816, I got the same. That's consistent with VICE. lib65816 returned $22, which must therefore be a bug. Or have I missed something again?

My current view is that the 6510 test suite is an excellent resource, with the disadvantages (to me) of being a bit c64-specific and having inaccessible sources.

Assuming you're running in emulation-mode, I suspect it's a lib65816 bug. If you're running in native-mode, then the unused bits are now actually usable, as the m and x bits.

I wonder what the result is of this test on KEGS (which uses a different 65816 emulator than lib65816), and on running it on a SNES or live-hardware Apple IIgs. If I had a working Kestrel, I'd be able to confirm this myself.

Interesting question. It's 50 decimal, which is $32, as we'd hope:

The code was:


I don't have a 65C02 to hand: I suppose it's possible that it's different.

I'll have an answer for this later this evening, maybe tomorrow (at least, w.r.t. KEGS). I'm at work at the moment, and don't have X11 access to my box at home (and it likely wouldn't work anyway, because of frame rate issues).

I have not followed this thread too closely, but if you can provide the code, I can run it on my SBC-3 (65816).

I also have a 65C02 SBC-2 I can try.

Daryl