6502.org Forum  Projects  Code  Documents  Tools  Forum
It is currently Fri Mar 29, 2024 1:16 pm

All times are UTC




Post new topic Reply to topic  [ 203 posts ]  Go to page Previous  1 ... 6, 7, 8, 9, 10, 11, 12 ... 14  Next
Author Message
PostPosted: Sat Aug 17, 2013 8:00 am 
Offline

Joined: Sat Jul 28, 2012 11:41 am
Posts: 442
Location: Wiesbaden, Germany
Michael,

also I don't know the timing for an individual test, the timings you have taken seem to be O.K. for the extent of each test. The ADC/SBC tests are so exhaustive because I didn't come up with a sensible set of test operands. Since I was lazy I decided to just test all combinations. Therefore your number of calls to decimal and binary ADC/SBC tests check routine is correct.

If you run into a timeout again, you could modify the initialization of the binary test. The test starts now at 0 for both operands and carry clear.
Code:
        cld
        ldx #ad2        ;for indexed test
        ldy #$ff        ;max range
        lda #0          ;start with adding zeroes & no carry
        sta adfc        ;carry in - for diag
        sta ad1         ;operand 1 - accumulator
        sta ad2         ;operand 2 - memory or immediate
        sta ada2        ;non zp
        sta adrl        ;expected result bits 0-7
        sta adrh        ;expected result bit 8 (carry out)
        lda #$ff        ;complemented operand 2 for subtract
        sta sb2
        sta sba2        ;non zp
        lda #2          ;expected Z-flag
        sta adrf
After you have checked ad2 on a timeout, you could preset ad2 to start with the ending ad2 value of the previous run. Since the first result expected is no longer 0 the resulting flags must be altered.
Code:
        cld
        ldx #ad2        ;for indexed test
        ldy #$ff        ;max range
        lda #0          ;start with adding zeroes & no carry
        sta adfc        ;carry in - for diag
        sta ad1         ;operand 1 - accumulator
        sta adrh        ;*** expected result bit 8 (carry out)
        lda #$f0        ;*** modify starting op2/result
        php             ;*** PS -> stack
        sta ad2         ;operand 2 - memory or immediate
        sta ada2        ;non zp
        sta adrl        ;expected result bits 0-7
;        sta adrh        ;*** moved ^^
        eor #$ff        ;*** now true complemented operand 2 for subtract
        sta sb2
        sta sba2        ;non zp
        pla             ;*** stack PS -> A
        and #$82        ;*** filter N,Z expected flags
        sta adrf
Changed or added lines are marked with ***.

You should always keep in mind, that the test was originally written for an emulator. For the non-exhaustive tests the predefined operands are choosen according to the need to test all possible combinations of resulting processor status bits.

Since logical operations in an emulator rely on the logical operations of the underlying processor, the test does not guarantee the logical functionality of each and every bit in the alu. It assumes the underlying logical operation to be correct if a limited number of patterns come out of the alu with a correct result.

In that sense it leaves a gap where a broken function in a single bit of an fpga core alu may go undetected. I don't know, how likely this is to happen but it is an exposure that you should be aware of.

_________________
6502 sources on GitHub: https://github.com/Klaus2m5


Top
 Profile  
Reply with quote  
PostPosted: Sat Aug 17, 2013 2:32 pm 
Offline
User avatar

Joined: Mon Apr 23, 2012 12:28 am
Posts: 760
Location: Huntsville, AL
Klaus:

Thanks for your pointer. Did not kick-off the binary mode test on the computer last night. But I will study your suggestions in the event I need to perform the tests in several passes.

I also like to perform exhaustive tests. The decimal mode arithmetic test for the BCD adder module of the core is exhaustive like your functional tests. I have opted to use your functional tests, although I recognize they were written for an emulator instead of a chip, because it provides a common basis for comparison among the various simulators, emulators, and cores. Ideally, the FPGA cores should have test benches that should determine if the various components meet their objectives.

In the case of the M65C02, I have such a test bench for every major module, and some not so major, including the BCD adder logic. The issue that your functional tests discovered was not associated with a fault in the logic of the M65C02_BCD.v module. Instead, when I recently changed the bus implementation in the ALU and core, it became possible that the results of one module, unless driven to logic 0, would corrupt the data from another component.

With multiplexer-based buses, the additional combinatorial delays were limiting the ability of the core to operate at a high rate. I failed to account for the extra cycle that the registered Decimal Unit (DU) enable FF (rEn) would remain set after its results were pulled off the ALU output bus and into A and P. Thus, it remained asserted beyond its allocated time on the ALU bus, and its LSB was set, i.e. A = $99 and P = $2C, so when P was pushed onto the stack, a value of $BD was written instead of $3C.

My regression test program is not sophisticated enough to identify this issue. It may be possible to expand it to address these types of issue, and I'll give that some thought. However, my inclination is to modify the HDL testbenches to perform this level of testing.

Your exhaustive testing approach may be time consuming and difficult to implement in an FPGA simulation environment because of its long runtimes, but it provides significant benefits that more limited targeted tests will not. To find timing faults like the one I found in my core with your functional tests simply requires test cycles. The fewer cycles of a targeted test is less likely to generate the conditions necessary to find bus overlaps.

The results reported by me, you, and others simply reinforce the need for exhaustive testing, and your functional tests provide a common set of tests that can be used in simulators, emulators, and FPGA cores. That versatility in invaluable and a testament to the quality of the tests.

Klaus2m5 wrote:
You should always keep in mind, that the test was originally written for an emulator. For the non-exhaustive tests the predefined operands are choosen according to the need to test all possible combinations of resulting processor status bits.

Since logical operations in an emulator rely on the logical operations of the underlying processor, the test does not guarantee the logical functionality of each and every bit in the alu. It assumes the underlying logical operation to be correct if a limited number of patterns come out of the alu with a correct result.

In that sense it leaves a gap where a broken function in a single bit of an fpga core alu may go undetected. I don't know, how likely this is to happen but it is an exposure that you should be aware of.

The nature of the HDL process fits in with the nature of the tests that you have built for the logic, shift, and rotate functions. Like the emulator, the logic used to implement the AND, EOR, ORA, ASL, LSR, ROL, and ROR instructions is a fundamental component of the HDL. There's no need to check that these functions have been correctly implemented for each bit. That being said, an implementation issue as I found with overlapping bus enables, could be more readily discovered with exhaustive tests of these functions.

However, to test for that type of timing issue requires intimate knowledge of the internal structure of the HDL design. Further, if the M65C02 was not pipelined and memory cycle optimized, then the overlap on the internal bus would not have been an issue at all. The overlap between the decimal adder unit and P on the ALU output bus occurred because there are no dead cycles in the M65C02's implementation. In a more standard implementation, the PHP instruction fetch cycle would have provided enough of a delay to allow the decimal adder unit registered output enable to be deasserted before P is required on the output bus to complete the PHP instruction.

Now that I've found this issue, I will try and determine a test sequence that can be used with the M65C02 core and soft-processor for regression testing of this fault. Another time where this overlap can be an issue is between the completion of an instruction and the initiation of the push of {PCH, PCL} or {PCH, PCL, P} onto the stack. If there is an overlap between the various data sources on the output bus, ALU output bus, dPCH, dPCL, and P, then there's a potential for the same kind of corruption of the value pushed onto the stack as occurred with your decimal mode ADC/SBC test. (Note: the binary mode adder is fully combinatorial, so there's no potential for the kind bus overlap that can occur with the decimal mode adder except as a design flaw in the FPGA mapper/router. The decimal mode adder has a fully combinatorial first stage and that is followed by decimal adjustment logic that is fed from the registered results of the combinatorial first stage.)

_________________
Michael A.


Top
 Profile  
Reply with quote  
PostPosted: Sat Aug 17, 2013 5:50 pm 
Offline
User avatar

Joined: Mon Apr 23, 2012 12:28 am
Posts: 760
Location: Huntsville, AL
Ed:

I have looked at Bruce's code several times. As a matter of fact, I looked at again last night and re-read the wiki he had added to the forum regarding its use.

However, I've not yet run it against the M65C02 soft-processor. Bruce states in the header of the file that it requires about 1 minute to execute on a 1 MHz 6502. Although the M65C02 core and soft-processor can be run (in simulation) at far higher rates, I have concluded that these routines need to be run on the target hardware itself. Since I've yet to attach the SPI Master and SSP UART from my other core to the M65C02 soft-processor, I have been focused on running a test suite like Klaus' that allows me to establish expected behavior of the core before moving to the M65C02/M16C5x HW development board for final verification.

With results from Klaus' functional tests in hand, I expect that moving to the HW will only result in the validation of the ISim results gathered and reported above. At that point, I expect to have the environment necessary to simply load and run Bruce's test code using the serial port and monitor. At that point, I'll report the results for both test suites.

_________________
Michael A.


Top
 Profile  
Reply with quote  
PostPosted: Sun Aug 18, 2013 9:52 am 
Offline

Joined: Sat Jul 28, 2012 11:41 am
Posts: 442
Location: Wiesbaden, Germany
BigEd wrote:
Hi Michael
Did you already run Bruce's ADC and SBC tests? It's possible that they take fewer cycles than Klaus' - it would be interesting indeed if they miss some errors which Klaus does not.
See http://www.6502.org/tutorials/decimal_mode.html#B
Cheers
Ed

Looking at Bruce Clark's code I can take an educated guess. He does
Code:
        ADC N2
        STA DA    ; actual accumulator result in decimal mode
        PHP
;and
        SBC N2
        STA DA    ; actual accumulator result in decimal mode
        PHP
So the result gets stored before anything else. Since it is the very same alu result jammed on the bus writing the accumulator, the problem wouldn't show. However, I have to admit that my test finds this bug is mere coincidence.

I remember from my 25 years of Mainframe experience that lots of things like this happen intentionally to take a shortcut for the pipelined next instruction, when it needs the result of the alu (called execution unit in the mf world) of the previous instruction before it is actually written to a register. That was called execute-execute bypass. Of course the bypass is supressed when the next instruction needs something else. 8)

_________________
6502 sources on GitHub: https://github.com/Klaus2m5


Top
 Profile  
Reply with quote  
PostPosted: Sat Aug 24, 2013 12:11 am 
Offline
User avatar

Joined: Mon Apr 23, 2012 12:28 am
Posts: 760
Location: Huntsville, AL
Klaus:

I have decided to move on to integrating the serial port and executing the binary mode ADC/SBC tests on a target board. I will probably work some on that effort this weekend.

The binary mode tests looked like they were going to require about 4+ hours to complete. They ran successfully for about the 1.5 hours, and then I pushed pause. That caused some kind of error in ISim, so I just decided to move ahead to a test on target.

This afternoon, I pulled some test logic from the core logic testbench, and re-ran the base tests: 1 - 40. On the M65C02, these tests require 73051 memory cycle to complete, and represent 49861 instructions. This is an average of 1.46 bytes/instruction. This is kind of interesting when compared to the weighted average instruction length of 2.15 bytes/instruction, if all instructions were equally likely.

In your program, the average instruction length is considerably below that value. I find it interesting that your test program is predominated by single byte instructions. This result may be skewed because your branching range tests require a significant percentage of the time required for execute these tests, and they consist mostly of single byte DEX instructions.

_________________
Michael A.


Top
 Profile  
Reply with quote  
PostPosted: Mon Aug 26, 2013 9:04 am 
Offline

Joined: Sat Jul 28, 2012 11:41 am
Posts: 442
Location: Wiesbaden, Germany
Hi Michael,

sorry for the late response. I spent the last 3 days on ferries with a bit of driving in between and arrived on Crete, Greece this morning.

MichaelM wrote:
This afternoon, I pulled some test logic from the core logic testbench, and re-ran the base tests: 1 - 40. On the M65C02, these tests require 73051 memory cycle to complete, and represent 49861 instructions. This is an average of 1.46 bytes/instruction. This is kind of interesting when compared to the weighted average instruction length of 2.15 bytes/instruction, if all instructions were equally likely.

In your program, the average instruction length is considerably below that value. I find it interesting that your test program is predominated by single byte instructions. This result may be skewed because your branching range tests require a significant percentage of the time required for execute these tests, and they consist mostly of single byte DEX instructions.
That is probably one reason. The other reason for the low bytes per instruction ratio has to do with how the test sets up and verifies the processor Status for each instruction it tests - PHA/PLP - PHP/PLA is used a lot in the test. The most used macros even do an extra PHP/PLP to preserve the initial processor status.

The ratio of test code to tested instruction is about 20 to 1 , maybe even more! Means you need about 20 instructions to verify 1 instruction under test.

cheers from Crete, Klaus

_________________
6502 sources on GitHub: https://github.com/Klaus2m5


Top
 Profile  
Reply with quote  
PostPosted: Mon Sep 02, 2013 5:56 pm 
Offline
User avatar

Joined: Mon Apr 23, 2012 12:28 am
Posts: 760
Location: Huntsville, AL
Sometimes it's difficult to know if a test suite is comprehensive or not. For those interested, Klaus' Functional Test for the NMOS 6502 is comprehensive, i.e. it tests all valid opcodes in the 6502. Anyone harboring doubts about the scope of Klaus' tests can rest assured that the test suite executes all 6502 instructions at least once.

In the past, I've generated a histogram of my test programs in the ISim simulation environment. Since I've not yet implemented a serial interface and Intel Hex file downloader for my M65C02 core's development board, I instrumented my simulation of Klaus' functional test. The result is that all tests that I allowed the suite to run were run to completion, and that all instructions except the ADC/SBC instructions were tested. In a previous run of the suite, I let it run the decimal mode ADC/SBC test to a successful conclusion. The run time of the decimal mode tests, which required 20000 iterations, was pretty long on my computer. The binary mode test, with an expected number of 131072 iterations, is apparently beyond the capability of ISim to successfully complete on my computer. I've run it successfully to about a quarter of the number of iterations.

Thus, by analysis and direct measurement, Klaus' functional test suite is comprehensive. I've yet to attempt to execute Klaus' 65C02 functional tests, but I expect that they too will be comprehensive and will fill in the remaining opcodes not covered by the 6502 tests, except the Rockwell instructions which his tests explicitly ignore. The Rockwell instructions are part of the latest WDC W65C02S microprocessor, but not part of the original 65C02. The following table gives the instruction counts for the 6502 Functional Test program up through test #40:
Attachment:
File comment: 6502_FT.a65 Histogram Results - Tests 1 to 40.
M65C02_Hist_File.txt [1.94 KiB]
Downloaded 175 times

_________________
Michael A.


Top
 Profile  
Reply with quote  
PostPosted: Tue Dec 17, 2013 10:49 pm 
Offline

Joined: Sun Oct 14, 2012 7:30 pm
Posts: 107
Has anyone made a binary file that can just be included into existing code so that we can run Klaus' test without having to setup a65 and such?


Top
 Profile  
Reply with quote  
PostPosted: Tue Dec 17, 2013 10:53 pm 
Offline

Joined: Thu Feb 28, 2013 6:34 am
Posts: 16
Location: San Francisco, California
https://github.com/zellyn/go6502/tree/master/tests - look for 6502_functional_test.bin, but it's a bit dated right now.


Top
 Profile  
Reply with quote  
PostPosted: Wed Dec 18, 2013 2:05 am 
Offline

Joined: Sun Oct 14, 2012 7:30 pm
Posts: 107
Thank you!


Top
 Profile  
Reply with quote  
PostPosted: Wed Dec 18, 2013 2:36 am 
Offline
User avatar

Joined: Mon Apr 23, 2012 12:28 am
Posts: 760
Location: Huntsville, AL
JimDrew:

The implementation of the Kingswood assembler, a65, puts out the binary image starting from the first declared address. It does not attempt to pad the binary image from 0x0000 to that address. The attached file is an edit of the file to which zellyn provide the link in a preceding post. I've added 10 leading nulls to pad out the file to the full 65536 address space. The page 2 data, the 0x1000 program start address, and the nmi, reset, and irq vectors are all at the correct locations if you load the binary file provided from 0. Alternatively, load the file zellyn provided from 0x000a.

Attachment:
File comment: Klaus2m5 6502 Functional Test Program - Binary, Load from 0x0000
6502_functional_test.zip [4.02 KiB]
Downloaded 167 times

_________________
Michael A.


Top
 Profile  
Reply with quote  
PostPosted: Wed Dec 18, 2013 10:25 am 
Offline

Joined: Sat Jul 28, 2012 11:41 am
Posts: 442
Location: Wiesbaden, Germany
A bin file doesn't do you any good, if you don't have the listing for it. Otherwise you would not be able to debug or even detect its successful completion, as every trap is detected only by looping on the same program count.

I have added a subdirectory with 2 pre-configured binaries and listings to GitHub, loadable at $0000 for full 64k: https://github.com/Klaus2m5/6502_65C02_ ... /bin_files

Since I can't load binaries in my project, the binaries have not been tested.

_________________
6502 sources on GitHub: https://github.com/Klaus2m5


Last edited by Klaus2m5 on Wed Dec 18, 2013 10:33 am, edited 1 time in total.

Top
 Profile  
Reply with quote  
PostPosted: Wed Dec 18, 2013 10:27 am 
Offline
User avatar

Joined: Thu Dec 11, 2008 1:28 pm
Posts: 10760
Location: England
Thanks Klaus!


Top
 Profile  
Reply with quote  
PostPosted: Wed Dec 18, 2013 9:03 pm 
Offline

Joined: Sun Oct 14, 2012 7:30 pm
Posts: 107
I just wanted to be able to incbin a binary file. The data is moved to whatever location in the virtual ROM space. I didn't want to have to setup yet another assembler on my machine, so thanks for the binary!


Top
 Profile  
Reply with quote  
PostPosted: Wed Oct 15, 2014 2:50 am 
Offline

Joined: Wed Oct 15, 2014 2:44 am
Posts: 4
Location: Chicago, IL
Big thanks to Klaus! I've been able to get my Javascript BBC Micro emulator passing the tests too - yay :)


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 203 posts ]  Go to page Previous  1 ... 6, 7, 8, 9, 10, 11, 12 ... 14  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 6 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: