How long should Klaus functional test take?

[Procrastin8]

Hello, world! I wrote an all Swift 65C02 simulator and am trying to run the Klaus functional tests. Most of it runs quite fast (I think?) but the comprehensive ADC/SBC section is taking an enormous amount of time. I tried to run the tests against lib6502 just as a baseline but was unsuccessful in getting it to run the tests at all. I have been running it for 9 min so far and it has only gone up to 178 on op2.

Some baselines just to make sure I'm doing this correctly:
- I execute 409 instructions on each innerloop from 0x3345 -> 0x3345 (this is incrementing op1)
- I execute ~103k instructions on each outerloop from 0x335F -> 0x335F (this is incrementing op2)

It takes ~6 seconds per outerloop iteration. So 256 * 6sec = ~1500sec (25 min!) to execute the test. I thought i read somewhere though their implementation ran in 52 seconds! That seems insane (ly good)!

Is my implementation just waaaay off?

[BigEd]

Welcome!

I just ran Klaus Dormann's testsuite on an approx 3MHz system and it took approx 24 seconds. So that's something under 100 million cycles.

It's just possible that a flaw in your emulation is causing it to run more instructions than it should - but I would hope such a flaw would already have shown up in failing the test. (The ADC/SBC tests are the last ones run.)

Here's a link to the claim of 52 seconds (on an approx 2MHz system)

[rwiker]

Procrastin8 wrote:

Hello, world! I wrote an all Swift 65C02 simulator and am trying to run the Klaus functional tests. Most of it runs quite fast (I think?) but the comprehensive ADC/SBC section is taking an enormous amount of time. I tried to run the tests against lib6502 just as a baseline but was unsuccessful in getting it to run the tests at all. I have been running it for 9 min so far and it has only gone up to 178 on op2.

Some baselines just to make sure I'm doing this correctly:
- I execute 409 instructions on each innerloop from 0x3345 -> 0x3345 (this is incrementing op1)
- I execute ~103k instructions on each outerloop from 0x335F -> 0x335F (this is incrementing op2)

It takes ~6 seconds per outerloop iteration. So 256 * 6sec = ~1500sec (25 min!) to execute the test. I thought i read somewhere though their implementation ran in 52 seconds! That seems insane (ly good)!

Is my implementation just waaaay off?

Assuming an average of 2.3 clock cycles per instruction, it looks like your code is executing at about 40kHz. If the implementation that ran this at 52 seconds was a 1MHz processor (or virtual equivalent), then that has a 25x clock rate, and approximately 29 times the performance. Does this sound reasonable?

[dmsc]

Hi!

Procrastin8 wrote:

Hello, world! I wrote an all Swift 65C02 simulator and am trying to run the Klaus functional tests. Most of it runs quite fast (I think?) but the comprehensive ADC/SBC section is taking an enormous amount of time. I tried to run the tests against lib6502 just as a baseline but was unsuccessful in getting it to run the tests at all. I have been running it for 9 min so far and it has only gone up to 178 on op2.

Some baselines just to make sure I'm doing this correctly:
- I execute 409 instructions on each innerloop from 0x3345 -> 0x3345 (this is incrementing op1)
- I execute ~103k instructions on each outerloop from 0x335F -> 0x335F (this is incrementing op2)

It takes ~6 seconds per outerloop iteration. So 256 * 6sec = ~1500sec (25 min!) to execute the test. I thought i read somewhere though their implementation ran in 52 seconds! That seems insane (ly good)!

Is my implementation just waaaay off?

IMHO, your emulator is really slow.

My stats for the Klaus 6502 test:

Code: Select all

Total Cycles:            96243729
Total Instructions:      30646866
Total Branches:           5206450 (17.0% of instructions)
Total Branches Taken:      258795 (5.0% of branches)
Branches cross-page:          285 (0.1% of taken branches)
Absolute X cross-page:     302152
Absolute Y cross-page:     302152
Indirect Y cross-page:     302148

That includes 5 extra cycles for my the emulator exit - a "JMP (DOS)".

My emulator executes all 30M instructions in 0.25 seconds here, or if I enable tracing (that writes each disassembled executed instruction to a log file) it takes 1.8 seconds.

A 2MHz 6502, without any wait states, should complete the test in 96243729 / 2000000 = 48.12 seconds.

Have Fun!

[BigEd]

Some useful timings from this earlier post:
viewtopic.php?p=26265#p26265

> Is my implementation just waaaay off?

I see py65 could run the tests in a couple of minutes. It's possible that you'll have to construct your Swift code very carefully if you want such performance.

[Procrastin8]

Yeah I thought mine was pretty far off. Thanks for confirming! I did some profiling and I was doing some really slow pattern matching to decode the mnemonic and addressing modes. After changing the data structures I was using I get up to the decimal tests (where I currently fail) in ~100 seconds. So improvement, but still a little ways to go, though for my project fast execution is not strictly critical. I do want this suite to complete in my lifetime, however.

[BigEd]

That's good! You might want to run and pass Bruce's decimal tests - it's faster and clearer, I think, to do that, and then pass Klaus' suite after.
http://www.6502.org/tutorials/decimal_mode.html

[org]

If we're talking about slow... Our 6502 model in Logisim will run the Klaus tests for about 44 days. (Measurements have shown that the speed in Logisim is about 1500 CLK per minute).

https://github.com/emu-russia/breaks/tr ... /6502.circ

[tafens]

dmsc wrote:

Hi!

My emulator executes all 30M instructions in 0.25 seconds here, or if I enable tracing (that writes each disassembled executed instruction to a log file) it takes 1.8 seconds.

A 2MHz 6502, without any wait states, should complete the test in 96243729 / 2000000 = 48.12 seconds.

Have Fun!

That was really quick! My emulator written in C# runs the test in 22 seconds in debug mode and in 3.5 seconds in release-mode (on a 3GHz i5-8500).
It's not optimised for speed though, as it tries to be cycle-exact to a real 6502.

I get a cycle count of 96241363 including the initial reset sequence with the reset vector pointing at the start of the test at 0x400 and runs until the PC hits 0x3469 which is where it goes into an infinite JMP-loop after a successful test.

I notice our total cycles differs with about 2500 cycles.
Does anyone know the exact cycle count from start to successful finish of the functional test? It would be interesting to know as that would be a good indicator of emulator cycle accuracy (if one is so inclined ).

[org]

I got a value of 96241361 cycles, not including the initial BRK reset sequence. The test stopped as soon as the PC was 0x3469 (`success`).

[tafens]

Looking closer at my code that sets up the functional test (it was a while ago), I see that I indeed zero the cycle count after the reset-sequence (so cycle=0 when PC it at 0x400 and the SYNC-pin is active). Then the emulation is run one cycle at a time until PC hits 0x3469. So my cycle count did not include the reset sequence after all. Also, on closer examination, the cycle count actually gets to 96241364 when the SYNC pin also goes active at PC 0x3469 (which should be when the final STA is fully completed and the JMP instruction is only just reached).

[nelbr]

I run the functional test on 0.3s on my i3-8100 (3.5Ghz) computer and I counted 96,241,364 cycles (on my code).

My initial runs were taking around 52s because I was printing the intermediate results to stdout. Once I got the test working, removing the printf improved performance to under 1s, so huge impact. Just in case you were doing something similar.

I have recently found out software on the Apple II that uses undocummented code. Ms. Pacman uses undocummented NOPs, and Ultima 1 uses an undocummented opcode in the space battle routines that makes it impossible to score a hit.

So, I started to add undocummented codes to my emulator. Is there a functional test for those that anyone is aware and can recommend ?

Thx,
Nelson

[John West]

nelbr wrote:

So, I started to add undocummented codes to my emulator. Is there a functional test for those that anyone is aware and can recommend ?

VICE has a collection of tests at https://sourceforge.net/p/vice-emu/code ... progs/CPU/

The best description of these instructions that I know of is "No More Secrets" https://csdb.dk/release/download.php?id=261988. It has regular 6502 equivalents for them, and the accompanying test code appears to be in the Vice collection under https://sourceforge.net/p/vice-emu/code ... -2.15/src/. It seems to be C64-specific (I see "jsr $ffd2" for printing characters).

[tafens]

nelbr wrote:

I run the functional test on 0.3s on my i3-8100 (3.5Ghz) computer and I counted 96,241,364 cycles (on my code).

My initial runs were taking around 52s because I was printing the intermediate results to stdout. Once I got the test working, removing the printf improved performance to under 1s, so huge impact. Just in case you were doing something similar.

No printf()s here, sad to say, but I'm guessing you use C/C++ and I'm in C# which is inherently slower due to it being run in a CLR much like Java.
But mostly, I guess your code is just more clever and optimised than mine

I'm happy we get the same cycle count though, that means I'm probably not totally off with the emulation

[sclucas]

Hi, I created a python based 6502 emulator. It runs slowly but I made it to learn the processor. It successfully runs the Klaus functional test (well I assume successful as it exits nicely at address 0x3469). However, it runs thought with around1M cycles less that stated above. This is fine as I have not pinned all of the instructions to be correct. Does anyone have an idea of how many instructions should be executed?