6502.org

Posted: **Thu Aug 06, 2009 11:46 pm**

Continued from this thread...

Quote:

I had an idea that one could write code - perhaps automatically, derived from the py65 testsuite - which was self-checking, hitting a BRK if it found a flag or a value in a different state from that expected.

I can see a generic 6502 test program that is retargetable to different platforms/emulators as being very useful. It might be easier to write something new in assembler rather than try to get Py65's existing Python-based test suite to emit assembly.

There seems to be a fair amount of interest in the group here about developing a new processor that is a superset of the 6502. Having such a program would be useful for testing it, so perhaps a few people here would want to collaborate on it.

If that happened, I'd be happy to include that test program in the Py65 package and make running it part of the Py65 test suite.

Quote:

I see now that Rob Finch has already written such a self-checking program! His bc6502 includes test6502.a65 which starts by checking branch and compare and then going on to more interesting instructions. He's got 40+ tests in 1000 lines of assembly.

(The license may be problematic: revocable, only for education and evaluation, only distributable unmodified and in full. So no snippets from me.)

That's odd. We probably shouldn't include his code in Py65 because of it. However, I don't see any reason we can't use it to find bugs in Py65 and then we can write our own test coverage for those bugs.

Quote:

(Oddly though, py65 fails several of the tests. At least one looks like a real bug - filed!)

Great catch! I believe that I have fixed this now and I've added some new tests for it. Since Py65 is still very young, we'll probably find other issues. Please let me know if you find any others and I'll do my best to fix them.

Regards,
Mike

Posted: **Sat Aug 08, 2009 4:06 pm**

Hi Mike
By way of experiment here's a test program which runs through all ADC operands, with carry set and clear, and compares the V flag with the expected result.

It passes on NMOS 6502, on 65816, on lib6502 and on my patched up py65. There's a problem with my copy of lib65816 which I haven't yet run to ground. (Edit: passes now on lib65816 too - BRK was mishandled.)

This is assembled with ORG=0:

Code: Select all

S1230000A90085A085A118201B4038201B40E6A0D0F4E6A1D0F000204F4B00A5A065A130DC
S122002014A5A01004A5A1300370036070FD00204641494C00A5A030F0A5A130EC10ED97

Here's the source with ORG=4000:

Code: Select all

        ;; test ADC's setting of V flag by running exhaustive test
        zpbase = $a0
        operanda = zpbase
        operandb = zpbase+1

        .org $4000

        lda #0
        sta operanda
        sta operandb
top
        clc
        jsr testadd
        sec
        jsr testadd
        inc operanda
        bne top
        inc operandb
        bne top
        brk     ;; or jsr putmsg
        .byte " OK"
        brk

testadd
        lda operanda
        adc operandb

testvflag
        ;; for add, overflow is when the inputs are same sign and output is different
        bmi resultisnegative
resultispositive
        lda operanda
        bpl assertvclear        ; no sign change so no overflow
        ;; a<0 and a+b>=0 so b ought to be positive
        lda operandb
        bmi assertvset
assertvclear
        bvs fail
ok
        rts

assertvset
        bvs ok
fail
        brk     ;; or jsr putmsg
        .byte " FAIL"
        brk

resultisnegative
        lda operanda
        bmi assertvclear        ; no sign change so no overflow
        ;; a>=0, a+b<0
        lda operandb
        bmi assertvclear
        bpl assertvset

Among the caveats: we use the machine to test itself, so there could be compensatory flaws; we haven't proved that we exercise all the interesting cases; we check the arithmetic but not say the addressing modes; not checking BCD mode.

I'm not sure how easily these ideas carry over to SBC. For the C flag, one might want also to test the various compare operations.

I'm thinking one could test the carry out by using ASR to right-shift the operands, accounting for the LSBs which land in C, and thus performing a 9-bit addition without relying on ADC's own C output. That is, bit 7 of that sum should match the C flag of the original sum.

But in any case, the V flag seems to cause the most trouble to get right, along with BCD mode. So it might not be worth checking C in this way: a few conventional tests would be much easier to write.

Comments welcome.

Cheers
Ed

Posted: **Mon Aug 10, 2009 4:07 pm**

Here's the test section for SBC - re-use the first part of the prgram, changing ADC to SBC of course.

Code: Select all

             testvflag
                ;; for subtract, overflow is when we
                ;; subtract positive from negative and get positive
                ;; or subtract negative from positive and get negative
                ;; that is, inputs differ in sign and sign of acc changes
30 14           bmi resultisnegative
             resultispositive
A5 A0           lda operanda
10 04           bpl assertvclear        ; no sign change so no overflow
A5 A1           lda operandb
10 03           bpl assertvset

             assertvclear
70 03           bvs fail
             ok
60              rts
             
             assertvset
70 FD           bvs ok
             fail

00              brk     ;; or jsr putmsg
20 46 41 49     .byte " FAIL"
4C           
00              brk
             
             resultisnegative
A5 A0           lda operanda
30 F0           bmi assertvclear        ; no sign change so no overflow
A5 A1           lda operandb
10 EC           bpl assertvclear
30 ED           bmi assertvset

This code runs OK on a real 6502, a 65816, and on lib65816. It fails on lib6502, py65 and one of the javascript emulators - evidently overlflow is even harder to get right for subtract than it is for addition.

Posted: **Sat Aug 15, 2009 8:18 pm**

I found a pointer (on the parallax forums) to an 1800-line assembly program to test a 6809 emulator - could be a useful template. It's by Wolfgang Schwotzer and is GPL.

This is the website, and here's the download - the program itself is src/tools/cputest.txt

Posted: **Sun Aug 16, 2009 9:17 pm**

An earlier thread points us to Wolfgang Lorenz' C64 testsuite

edit: Good news, it is in public domain. Bad news, it is in the form of p00 files (a pc64 format.) Good news, if a p00 file is binary and supposed to load at 0801 (see here) then we can load at 7e5 into py65 and jump to 0801 to ignore the header. Bad news, need to patch the i/o to suit py65. Good news, that shouldn't be too hard (see c64 rom)

Posted: **Sat Aug 29, 2009 2:35 pm**

I've written another self checking program based on dclxvi's observation that SBC is just an inverted ADC (when in binary mode) - and with it I've found some bugs in lib6502 and py65. lib65816 passes, as does an NMOS 6502 and a 65816.

Unfortunately we need a more complex test for decimal mode. As I understand it, I can expect the nmos 6502 and the 65816 to differ in decimal mode.

Code: Select all

	zpbase    = $a0
	operanda  = zpbase
	operandb  = zpbase+1
	result    = zpbase+2
	flags     = zpbase+3
	result2   = zpbase+4
	flags2    = zpbase+5
	temp      = zpbase+6

	.org $4000

	cld
	clc
	jsr testall
	
	sec
	jsr testall
	brk
	.byte "AOKbinary"
	brk
testall:
	php			; need a safe copy of the mode bits
	;; initialise the operands which we loop over
	lda #0
	sta operanda
	sta operandb

top:	
	plp			; we retrieve and re-save the mode bits
	php
	jsr testadd
	php			; we save the status byte
	sta result
	pla
	sta flags

	plp			; retrieve and re-save the mode bits
	php
	jsr testsub
	php
	sta result2
	cmp result
	bne failresult
	pla
	sta flags2
	cmp flags
	beq next
	brk
	.byte "Ffail flags"
	brk

failresult:
	brk
	.byte "Ffail result"
	brk

next:	
	inc operanda
	bne top
	inc operandb
	bne top
	plp
	rts
	
testsub:	
	lda operandb
	eor #$ff
	sta temp
	lda operanda
	sbc temp
	rts

testadd:	
	lda operanda
	adc operandb
	rts

in srec format:

Code: Select all

S1234000D8182022403820224000414F4B62696E61727900F8182022403820224000414F34
S12340204B0008A90085A085A128082077400885A26885A32808206C400885A4C5A2D014F7
S12340406885A5C5A3F01B00466661696C20666C6167730000466661696C20726573756C1B
S12340607400E6A0D0C3E6A1D0BF2860A5A149FF85A6A5A0E5A660A5A065A160A90085A3AC
S1114080A5A065A10885A29002E6A3286060B1

and in py65mon fill format:

Code: Select all

fill 4000  d8  18  20  22  40  38  20  22  40  00  41  4f  4b  62  69  6e  61  72
fill 4012  79  00  f8  18  20  22  40  38  20  22  40  00  41  4f  4b  00  08  a9
fill 4024  00  85  a0  85  a1  28  08  20  77  40  08  85  a2  68  85  a3  28  08
fill 4036  20  6c  40  08  85  a4  c5  a2  d0  14  68  85  a5  c5  a3  f0  1b  00
fill 4048  46  66  61  69  6c  20  66  6c  61  67  73  00  00  46  66  61  69  6c
fill 405a  20  72  65  73  75  6c  74  00  e6  a0  d0  c3  e6  a1  d0  bf  28  60
fill 406c  a5  a1  49  ff  85  a6  a5  a0  e5  a6  60  a5  a0  65  a1  60  a9  00
fill 407e  85  a3  a5  a0  65  a1  08  85  a2  90  02  e6  a3  28  60  60

Posted: **Tue Sep 01, 2009 12:56 am**

BigEd wrote:

Unfortunately we need a more complex test for decimal mode.

There's a decimal mode checking program in Appendix B of my Decimal Mode write-up. As written, it tests documented and undocumented behavior on an NMOS 6502. There are instructions for modifying the program to test only documented behavior (in retrospect, I should have just supplied both routines, rather than a list of instructions). To test a 65C02 or 65816 (8-bit mode; decimal mode behaves the same in either 8-bit native or emulation mode) just substitute JSR A65C02 and JSR S65C02 (or A65816 and S65816) for JSR A6502 and JSR S6502. (I forgot to mention that in Appendix B.) I've tested it on actual hardware for all three flavors (NMOS 6502, 65C02, and 65816).

http://www.6502.org/tutorials/decimal_mode.html

Posted: **Tue Sep 01, 2009 7:31 am**

Excellent, thanks! I hadn't stumbled across that code - it's just what we need!

I was about to embark on collecting 512k of results from each platform and comparing them, which felt like it was a sledgehammer approach, but at least not too much strain on the brain.

Ed

Posted: **Thu Sep 10, 2009 5:31 pm**

BigEd wrote:

An earlier thread points us to Wolfgang Lorenz' C64 testsuite

Hmm, the ADCA test fails for me on py65 and run6502, because it expects the B and the missing bit to be set in the status register as pushed by PHP. And yet the test runs OK on VICE, and apparently is OK on a real C64. So this would seem to be a visible difference of 6510 versus 6502, which I wasn't previously aware of. [Edit: or perhaps py65 and run6502 have bugs! See below.]

So, maybe these hundreds of tests are useful - they represent a lot of careful work, and seem to check each instruction exhaustively (80 minutes runtime!). But as each test chains in the following one, the suite benefits from being in a c64 emulation with some disk emulation capability. And it will need adjustment for the idiosyncrasies of the different 6502-like devices.

The sources are supplied, in a sense, but they are in a C64 turbo macro assembler format which I can't readily use. So tweaking the tests is a challenge: it took me an hour or two with a disassembly of ADCA to get to this point.

Posted: **Fri Sep 11, 2009 3:43 am**

BigEd wrote:

Hmm, the ADCA test fails for me on py65 and run6502, because it expects the B and the missing bit to be set in the status register as pushed by PHP. And yet the test runs OK on VICE, and apparently is OK on a real C64. So this would seem to be a visible difference of 6510 versus 6502, which I wasn't previously aware of.

I have to disagree here. The 6510 was a 6502 core coupled with a 6-bit I/O port at $00/$01. There ought be no visible differences between a 6510 and 6502, except for the behavior of $00 and $01 during a memory test.

The VIC-20 was equipped with a genuine 6502 microprocessor, as were most of the PETs (at least one PET was equipped with a 6509 CPU, which used an ingenious technique to expand the user-visible address space to about 1MB). Do the tests behave differently under VICE in these environments?

I find it difficult to believe that the CPU emulation is faulty on x64 -- the KERNAL receives frequent interrupts and one of the first things it does is verify that the B bit is clear (a hardware interrupt). If B is set (indicating that BRK was the source of the interrupt), the end effect is the same as if you'd hit the RESTORE key.

Posted: **Fri Sep 11, 2009 9:24 am**

kc5tja wrote:

BigEd wrote:

Hmm, the ADCA test fails for me on py65 and run6502, because it expects the B and the missing bit to be set in the status register as pushed by PHP. And yet the test runs OK on VICE, and apparently is OK on a real C64. So this would seem to be a visible difference of 6510 versus 6502, which I wasn't previously aware of.

I have to disagree here.

I'm going to disagree too! I have no idea why I supposed that those two (relatively immature) emulators were correct. Although it might have been true that BRK and IRQ behaved differently from PHP, rather more likely that I hadn't got enough data.

Testing on an NMOS 6502, PHP + PLA got me $32. On a 65816, I got the same. That's consistent with VICE. lib65816 returned $22, which must therefore be a bug. Or have I missed something again?

My current view is that the 6510 test suite is an excellent resource, with the disadvantages (to me) of being a bit c64-specific and having inaccessible sources.

Posted: **Fri Sep 11, 2009 2:48 pm**

BigEd wrote:

Testing on an NMOS 6502, PHP + PLA got me $32. On a 65816, I got the same. That's consistent with VICE. lib65816 returned $22, which must therefore be a bug. Or have I missed something again?

Assuming you're running in emulation-mode, I suspect it's a lib65816 bug. If you're running in native-mode, then the unused bits are now actually usable, as the m and x bits.

I wonder what the result is of this test on KEGS (which uses a different 65816 emulator than lib65816), and on running it on a SNES or live-hardware Apple IIgs. If I had a working Kestrel, I'd be able to confirm this myself.

Posted: **Fri Sep 11, 2009 4:11 pm**

kc5tja wrote:

I wonder what the result is of this test on KEGS

Interesting question. It's 50 decimal, which is $32, as we'd hope:

Code: Select all

POKE 5000,169
POKE 5001,0
POKE 5002,8
POKE 5003,104
POKE 5004,141
POKE 5005,135
POKE 5006,19
POKE 5007,96
CALL(5000)
?PEEK(4999)

The code was:

Code: Select all

1388 A9 00    LDA #0
138A 08       PHP
138B 68       PLA
138C 8D 87 13 STA 4999
138F 60       RTS

I don't have a 65C02 to hand: I suppose it's possible that it's different.

Posted: **Fri Sep 11, 2009 5:02 pm**

I'll have an answer for this later this evening, maybe tomorrow (at least, w.r.t. KEGS). I'm at work at the moment, and don't have X11 access to my box at home (and it likely wouldn't work anyway, because of frame rate issues).

Posted: **Fri Sep 11, 2009 11:42 pm**

I have not followed this thread too closely, but if you can provide the code, I can run it on my SBC-3 (65816).

I also have a 65C02 SBC-2 I can try.

Daryl

6502.org

Running test6502.a65 on Py65

Running test6502.a65 on Py65

self-testing code for emulators and homebrew cpus

SBC self-testing program

test program for 6809

A 6510 testsuite

comparing ADC with SBC

Re: comparing ADC with SBC

Re: A 6510 testsuite

Re: A 6510 testsuite

Re: A 6510 testsuite

Re: A 6510 testsuite

Re: A 6510 testsuite