Mystery with RTS off by one page?!

[pdragon]

Rebuilt a simple breadboard setup with address decoding based on Michael's diagram here viewtopic.php?f=12&t=6844&p=108973#p88176 and having a mysterious problem that I can't figure out. Writing this in the hope it might trigger some suggestions for how to debug.

The setup decodes RAM from $0000-bfff, an IO page at $c000-c0ff and ROM from $c100-ffff. I use Michael's clock-qualified read and write along with the /io flag to access ROM and RAM as shown in the diagram. The VIA is selected with the same /io flag and has registers mapped to A0-3 (so appear 16x in the $c1xx page). I have the 65c02 r/w connected directly to VIA's corresponding pin - afaik it handles the clock-qualified access?

I use the second half of the '139 like Michael shows so that A4-5 (not 6-7) generate four /iox signals. I use /io1 for lcd, /io2 for kbd shift register and /io3 for sd shift register (not wired yet). I also have a speaker set up on a portb pin which I can drive with a timer. /io0 is not connected so when I r/w to $c000-c00f I can manipulate VIA registers with no device active. For example with the LCD I set up portb control pins and porta data pins without enabling the device, and then BIT $c01x to enable the LCD while presenting the correct data and control values.

This all seems to work fine for several thousand instructions including initializing the LCD, clearing the screen and writing some text, but then at one point it returns from a JSR (which is called multiple times earlier) and unstacks $f11d after originally storing $f01d, so returns exactly a page beyond where intended. I have a simple monitor trace (below) where I can't see any other access to those stack locations. I've inspected memory at several intervening points and it seems like the stored value $f0 is correct until some time just before it reads the wrong value. My best guess is some kind of timing issue but can't understand why it happens in this repeatable way after so much other code runs correctly, nor any theory for how it gets that particular value wrong.

I've run the same binary image in a simulator without seeing the problem, so it doesn't appear to be a software issue. I've run various memory tests for data bus, addr bus including extensive use of the stack and recursive JSR and that all seems to be fine as well (code here viewtopic.php?f=4&t=144&p=109620#p109620 ).

Please lmk if you have any ideas on what else I could try.

Code: Select all

=== trace begin ===

* f207 r 60        RTS 
  f208 r a5
  01fd r f1 17 f0
  f017 r f1
* f018 r 20 08
  01ff r f0        *******************   
  01ff W f0        *******************   
  01fe W 1a        *******************  
  f01a r f2        JSR  $f208
* f208 r a5 03     LDA  $0003
  0003 r 00
  ...
* f20a r 10 fc     BPL  $f208
  f20c r 29
* f208 r a5 03     LDA  $0003
  0003 r 00
* f20a r 10
  f20a r 10
  01fd W f2        <========== KB IRQ
  01fc W 0a
  01fb W 23
  fffe r 11 f2
* f211 r 48        PHA 
  f212 r ad
  01fa W 00
* f212 r ad 03 c0  LDA  $c003
  c003 r ff
* f215 r 48        PHA 
  f216 r 9c
  01f9 W ff
* f216 r 9c 03 c0  STZ  $c003
  c003 W 00
* f219 r ad 21 c0  LDA  $c021
  c021 r 35
* f21c r 85 04     STA  $0004
  0004 W 35
* f21e r 09 80     ORA  #$80
* f220 r 85 03     STA  $0003
  0003 W b5
* f222 r 68        PLA 
  f223 r 8d
  01f8 r fc ff
* f223 r 8d 03 c0  STA  $c003
  c003 W ff
* f226 r 68        PLA 
  f227 r 40
  01f9 r ff 00
* f227 r 40        RTI 
  f228 r a9
  01fa r 00 23 0a f2
* f20a r 10 fc     BPL  $f208
  f20c r 29
* f208 r a5 03     LDA  $0003
  0003 r b5
* f20a r 10 fc     BPL  $f208
* f20c r 29 7f     AND  #$7f
* f20e r 64 03     STZ  $0003
  0003 W 00
* f210 r 60        RTS 
  f211 r 48
  01fd r f2 1a f0
  f01a r f2
* f01b r 20 70      <= start of JSR, should return to $f01e
  01ff r f0        *******************   
  01ff W f0        *******************   <= write f0 TOS
  01fe W 1d        *******************   <= write 1d = %0001_1001 NOS
  f01d r f1        JSR  $f170             
* f170 r 38        SEC 
  f171 r e9
* f171 r e9 20
* f173 r 85 00     STA  $0000
  0000 W 15
* f175 r a0 c0     LDY  #$c0
* f177 r 18        CLC 
  f178 r da
* f178 r da        PHX 
  f179 r a2
  01fd W ff
* f179 r a2 00     LDX  #$00
* f17b r 20 c7
  01fc r 0a
  01fc W f1
  01fb W 7d
  f17d r f1        JSR  $f1c7
* f1c7 r 9c 03 c0  STZ  $c003
  c003 W 00
* f1ca r a9 05     LDA  #$05
* f1cc r 8d 00 c0  STA  $c000
  c000 W 05
* f1cf r ad 11 c0  LDA  $c011
  c011 r 2b
* f1d2 r 29 03     AND  #$03
* f1d4 r 49 03     EOR  #$03
* f1d6 r d0 f7     BNE  $f1cf
* f1d8 r a9 ff     LDA  #$ff
* f1da r 8d 03 c0  STA  $c003
  c003 W ff
* f1dd r 60        RTS 
  f1de r 9c
  01fa r 00 7d f1
  f17d r f1
* f17e r a9 02     LDA  #$02
* f180 r 8d 00 c0  STA  $c000
  c000 W 02
* f183 r b5 00     LDA  $0000,X
  f184 r 00
  0000 r 15
* f185 r 8d 01 c0  STA  $c001
  c001 W 15
* f188 r 2c 11 c0  BIT  $c011
  c011 r 15
* f18b r 90 04     BCC  $f191
  f18d r 18
* f191 r fa        PLX 
  f192 r 20
  01fc r f1 ff
* f192 r 20 c7
  01fd r ff
  01fd W f1
  01fc W 94
  f194 r f1        JSR  $f1c7
* f1c7 r 9c 03 c0  STZ  $c003
  c003 W 00
* f1ca r a9 05     LDA  #$05
* f1cc r 8d 00 c0  STA  $c000
  c000 W 05
* f1cf r ad 11 c0  LDA  $c011
  c011 r a3
* f1d2 r 29 03     AND  #$03
* f1d4 r 49 03     EOR  #$03
* f1d6 r d0 f7     BNE  $f1cf
* f1d8 r a9 ff     LDA  #$ff
* f1da r 8d 03 c0  STA  $c003
  c003 W ff
* f1dd r 60        RTS 
  f1de r 9c
  01fb r 7d 94 f1
  f194 r f1
* f195 r a9 03     LDA  #$03
* f197 r 8d 00 c0  STA  $c000
  c000 W 03
* f19a r 8c 01 c0  STY  $c001
  c001 W c0
* f19d r 2c 11 c0  BIT  $c011
  c011 r c0
* f1a0 r 60        RTS 
  f1a1 r a0
  01fd r f1 1d f1  *******************  TOS is now 1d f1 instead of 1d f0
  f11d r f1
* f11e r a0 84     LDY  #$84            <= returns to $f11e instead of $f01e
* f120 r 20 92
  01ff r f1        *******************
  01ff W f1        *******************
  01fe W 22
  f122 r f1        JSR  $f192
* f192 r 20 c7
  01fd r f1
  01fd W f1
  01fc W 94
  f194 r f1        JSR  $f1c7
* f1c7 r 9c 03 c0  STZ  $c003
  c003 W 00
* f1ca r a9 05     LDA  #$05
* f1cc r 8d 00 c0  STA  $c000
  c000 W 05
* f1cf r ad 11

=== trace end ===

[BigDumbDinosaur]

A schematic (not in color, please) would be better than a description of who is connected to what.

[BigEd]

I'd agree that it's probably a timing issue. Broadly, there are perhaps three flavours of data error, as opposed to address error
- a read error - the RAM contents is correct but the CPU got something different
- a write error - when the RAM was written the wrong value got written
- a misdirected write - some other write to RAM affected this location (possibly in addition to what it should have done

If you read up on decoding logic, you'll see the need to gate the RAM's write enable (or chip enable) until after the address, and preferably also the data, are stable. Conventionally, the system clock is used for that.

Because the address will change on the cycle after a write, it's important that the RAM is no longer trying to write at that point. In other words, the write enable (or chip enable) must be deasserted before the CPU (or any other logic) can affect the address bus.

[Paganini]

What bus monitor are you using? It might be worth spending $12 or so to pick up an FX2 board and learning to use Hoglet's decoder. Among many nice things, one thing it does is number the bus cycles in a capture, which makes it a lot easier to talk about what's going on!

I've had similar problems with some of my builds, where it seems like things are generally working but there are intermittent glitches. They're frustrating to debug because they're repeatable (in the sense that the glitch happens the same way every time), but not reproducible (in the sense that you can't make it glitch on command by calling a specific subroutine or using a particular instruction). This usually seems to be noise related. How fast are you running this? Noise problems usually get worse if you go faster. Also what specific parts are you using? Some forum members have had similar head-bangingly frustrating problems with some specific RAM ICs. The conclusion was, I believe, that the RAM is internally a 3.3v part and the external interface wasn't quite adequate for a 5v system. I believe the thread was called "Strange Memory Write Problems." Meanwhile, I'd like to see a photo of your breadboards.

[BigEd]

(Good point, a problem like this could be a noise problem: supply noise, ground bounce, crosstalk. A solid power supply, adequate decoupling, a good ground plane or grid, might be helpful if not already present. Overfast edges can be a problem too, so watch out for various logic families.)

[pdragon]

Here's a current picture. I pulled out the arduino address and databus monitor cables for clarity, but left some of the individual pin jumpers.

I'm currently using 65c02-Tool for monitoring.

iirc I did buy an fx2 board for hoglet but haven't tried to use it yet - good excuse for the weekend!

I'm working on a simple schematic to add here later. One thing I noticed while sketching was (a) Michael's drawing says "untested" and (b) it's using the phi1 output for qualified read/write. I know the W65c02 datasheet says this is no longer spec'd/tested, so maybe it is lagging phi2 enough that it causes some problem when i'm accessing one of the IO chips immediately before/after RAM? I can try looking with the scope later on, and i guess an easy test would be to run phi2 thru a nand gate instead (isn't that what they'd do inside the chip h/w anyway tho??).

I've been running super slow for monitoring - you can see my beneater clock hooked up in the picture. So only a few Hz or even single step, but seems to behave similarly if I drive with a 1Mhz clock, so more likely edge-related thing I guess?

Specific parts:

WDC 65c02 and 6522
RAM: AS6C1008-55PCN - i have a16 wired high, i'll try pulling low instead to use the other half but I don't think it's a memory problem.
eeprom: AT28C256-15PU

demux - SN74LS139AN <= need to double-check but I might be mixing LS and HC here which is probably a bad idea
shift reg x 2 - SN74HC595N
comparator - SN74HC682N

also econoreset which is nice for clean startup once I had adequate over-current threshold on my power supply.

I think that's all the active parts currently.

[BigDumbDinosaur]

pdragon wrote:

Here's a current picture...

How about a schematic?

Quote:

demux - SN74LS139AN <= need to double-check but I might be mixing LS and HC here which is probably a bad idea

Not probably...if you are going to use a 74LS part in a CMOS system, you need to use 74HCT or similar if any 74LS part is driving a CMOS part.  Otherwise, your system will be noise-sensitive, if not actually DOA.

That said, use of 74LS logic in a new design is not recommended.  WDC’s MPUs are not TTL-compatible.

[BigEd]

Ah, if you qualify writes by a signal that's delayed relative to the CPU's activity, you may find you are still writing when the address lines change at the beginning of the next cycle. This could cause a stray write, or a RAM malfunction.

[pdragon]

Schematic coming once I finish drawing it…

[pdragon]

Here's a rough sketch of the schematic (gdocs source).

I did confirm the '139 is LS so will find an HC version to replace it on the weekend.

[Paganini]

pdragon wrote:

One thing I noticed while sketching was (a) Michael's drawing says "untested" and (b) it's using the phi1 output for qualified read/write. I know the W65c02 datasheet says this is no longer spec'd/tested, so maybe it is lagging phi2 enough that it causes some problem when i'm accessing one of the IO chips immediately before/after RAM?

I doubt it. Michael's drawing says "untested," but many of them have been tested by me ( ) and they work great! I haven't built this particular one, because I don't have any `682s, but I have built several very similar ones using `688s.

Although the WDC docs do discourage folks from using the Ø1 output in new designs, one of the stated purposes for these chips is to keep vintage machines running by replacing their old dead CPUs. Those old designs often rely on the Ø1 output. As Garth says in the Primer, it's silly (especially for retro hobbyists!) to abandon a 50 year old time tested design just because WDC has stopped a production test. The transistors are still there.

That being said, there's a better way to get a symmetrical two-phase clock, which is use a double-speed oscillator and run it into a flip flop (such as half of a 74AHC 74). Use the Q output as Ø2 and the ~Q output as Ø1. Like this picture from the Primer:

Quote:

RAM: AS6C1008-55PCN - i have a16 wired high, i'll try pulling low instead to use the other half but I don't think it's a memory problem.

Aha! That is the same series that other members have had trouble with. In the thread I mentioned it was a smaller capacity chip (the Alliance AS6C62256) but I wonders, I does. Do you have a RAM chip from a different manufacturer you could try switching out?

Quote:

demux - SN74LS139AN <= need to double-check but I might be mixing LS and HC here which is probably a bad idea

Yeah, unless you want to do a lot of thinking about what's driving what and adding strategic pullup resistors. Also, that `139 is doing a pretty important job. Ideally it would be a 74AHC (although HC will probably work fine at your 1MHz and slower speed).

Your wiring is beautifully tidy! I'm going to give you some suggestions. Unfortunately my "shop" is a wreck (thanks to being in the process of building an actual shop) so my electronics stuff isn't really available for photo demonstrations. I'll just have to try and describe verbally.

In the top, main part, of your system, you have power and ground rails running vertically. All the rails are connected together by horizontal rails at the top and the bottom, so your layout looks like a sideways ladder. That's good! However, there aren't enough horizontal connections between those vertical "rungs." Look at, for example, your `139. There's a short black jumper that connects its GND pin to the GND rail on its left side. You should add a longer black jumper that connects that same pin to the GND rail on its right side (where the two yellow wires are running). You should do that for every IC so that instead of it being like your chips are hanging off of a ladder, it's like they're living in a grid with a resolution of one "GND square" per IC.

The other thing is that your bus wiring generally runs horizontally, while your power wiring generally runs vertically. Ideally, wherever you have those horizontal arches of signal wires, there would also be arches of GND wires - one every other signal wire. That might not be practical, but you could try adding one or two GND arches per bus "bundle," especially to the ones that sort of "submerge" in the breadboard to go under some other signals and come out the other side. When I'm doing breadboard wiring I try as much as possible to have the signal wires running directly above the power rails. If they have to cross a breadboard I put a power rail there using jumpers.

Here's a picture of one of my projects: download/file.php?id=20708&mode=view If you look at all the little green jumper wires you can see how every IC has a row of them right next to its GND pin connecting the adjacent GND rails. If you look at the white signal bus wires right in the middle of the board you can see that right under them is a vertical row of green staples so that there is a GND path that directly tracks them.

Edit: One really important thing I meant to mention and forgot to include: your clock signal is REALLY long. Give it its own dedicated ground wire that runs right next to it over the breadboard surface, and where it's airborn make it into a twisted pair of CLK/GND. Your clock module only has ONE connection each of GND and VCC. Do the same thing to it as to the main part of your system (give every IC a connection to both GND rails) and then connect its top rail to the main system's bottom rail 4 or 5 times along its length (basically, wherever one of the ladder "rungs" connects to the bottom rail of the main system).

[BigDumbDinosaur]

Some comments about clocking:

2. WDC specifies a maximum rise/fall time for the Ø0 input of 5 nanoseconds.  That is not achievable with 74HC logic and is marginally achievable with 74AHC.  This is an area in which I recommend the use of 74AC logic.  Only thing is to watch out for ringing, which if sufficiently severe, may trigger double-clocking.
   
   A small amount of series resistance, with the resistor placed as physically close as possible to the flip-flop’s Q output, can help to suppress excessive ringing.  Additional suppression may be gotten with a pair of fast Schottky diodes attached to the output side of the series resistance, with the junction of the diodes as physically close as possible to the resistor.  A suitable discrete diode is a BAT85, which has a tRR of 4ns.
    
   
3. The Ø1O and Ø2O outputs should not be used at elevated Ø2 rates.  Yes, the circuitry is still there and still functions.  The problem is that if your system timing is slaved to Ø1O or Ø2O, everything will lag what the 65C02 is doing by an unknown amount—the MPU’s activity is slaved to Ø0.  While this lag is not going to cause any grief at 1 MHz, it definitely may once the clock rate is elevated.  In particular, 65xx peripherals that are slaved to the clock will be out of sync with the MPU and may fail to respond due to timing skew.
    
4. As Ed noted, a delayed rise of the /WD signal at the end of write cycle can result in data corruption.  Such a malady can be very difficult to troubleshoot.  /WD must rise as soon as possible after the fall of the clock.  Otherwise, the addressed device may see a sudden change in the state of A0-A15 while /WD is still low, which will likely cause a wild write.  The only way to assure that /WD closely tracks the clock is to exclusively use the clock generator’s output to drive all devices that are clock-dependent.

Succinctly, Ø1O and Ø2O should not be used in any new design.  As WDC says in the C02’s data sheet, those outputs are only intended for applications in which the 65C02 is used in an application that originally had an NMOS 6502.

bat85_npx.pdf

BAT85 Schottky Diode

(126.05 KiB) Downloaded 38 times

[pdragon]

Here are a few more data points.

- Replacing the LS139 with an HC version made no difference. Fails the same way.

- I changed the .org of my ROM code from $f000 to $e111 to flip the parity of all the jsr addresses, with no other changes. The IO addresses remained in the same place in the $c0xx page. The failure mode is identical - it returns $100 beyond where it should, at the same call location. In the original case it read $f113 instead of $f013. With the modified org it read $e224 instead of $e124. See side-by-side traces attached

- trying to understand how it could be a software issue I sprinkled in some `lda $1ff` instructions so I'd see where the value changed. Here's a fragment of a trace where you can see the value morph. This is after successfully doing several jsr/rts involving the top-of-stack at $1ff, and passing my memory tests which heavily exercise the stack in a number ways.

It's as if somehow interactions with the VIA are affecting that memory location, but I'm having trouble imagining a scenario where the byte at $1ff magically increments when $ff doesn't appear on the LSB of the address bus at any intervening point.

Any other debugging ideas? I'll try to organize a PC with hoglet on the weekend. I'll probably also try some clock alternatives - all these tests are still clocked super slow, <1KHz. I'll also try to dig out a different memory chip - I think it's only 32Kb so will leave a hole from $8000-$bfff but I guess that doesn't matter for this purpose.

I've also attached the org .f000 listing if anyone wants to stare at it - it's a bit of mess as chop and change...

Code: Select all

* f108 r 85 00     STA  $0000
  0000 W 41
* f10a r a0 c0     LDY  #$c0
* f10c r ad ff 01  LDA  $01ff
  01ff r f0                     <= TOS OK with value $f0
* f10f r 18        CLC
  f110 r da
* f110 r da        PHX
  f111 r a2
  01fd W 00
* f111 r a2 00     LDX  #$00
* f113 r 20 68
  01fc r c0
  01fc W f1
  01fb W 15
  f115 r f1        JSR  $f168
* f168 r 9c 03 c0  STZ  $c003
  c003 W 00
* f16b r a9 05     LDA  #$05
* f16d r 8d 00 c0  STA  $c000
  c000 W 05
* f170 r ad 11 c0  LDA  $c011
  c011 r 2b
* f173 r 29 03     AND  #$03
* f175 r 49 03     EOR  #$03
* f177 r d0 f7     BNE  $f170
* f179 r a9 ff     LDA  #$ff
* f17b r 8d 03 c0  STA  $c003
  c003 W ff
* f17e r 60        RTS
  f17f r 9c
  01fa r 32 15 f1
  f115 r f1
* f116 r ad ff 01  LDA  $01ff
  01ff r f1                       <= TOS is now wrong, $f1 instead of $f0
* f119 r a9 02     LDA  #$02
* f11b r 8d 00 c0  STA  $c000
  c000 W 02

[pdragon]

woohoo!

Dinosaur's suggestion of using symmetric Q, /Q outputs from a '74 flipflop halving the input clock (replacing the w65c02's phi1 output) seems to fix the problem, at least at for the sub-1khz speeds i'm experimenting with.

I also narrowed down where the phantom write happens (below). Surprisingly to me, read/write to IO page ($c0xx) doesn't seem to be implicated, but I guess phi1-qualified /mr and /mw signals are enough out of phase with the /phi2, /hi and /io signals that we still have the write enabled as the address bits change?

I find it odd that so much seems to work correctly before we get to this repeatable failure point but the hardware details are still very mysterious to me.

Is there a good tutorial that would help me understand the two write-cycle timing diagrams in the RAM datasheet https://www.alliancememory.com/wp-conte ... eb2007.pdf ? I haven't had enough practice to grok them on their own.

Code: Select all

* f114 r ad ff 01  LDA  $01ff
  01ff r f0        <= TOS is still $f0 here
* f117 r a2 00     LDX  #$00
* f119 r 20 6e
  01fc r c0
  01fc W f1       <= ?? perhaps this $f1 also gets written to 01ff ??
  01fb W 1b
  f11b r f1        JSR  $f16e
* f16e r ad ff 01  LDA  $01ff
  01ff r f1         <= now TOS is $f1

[Paganini]

pdragon wrote:

woohoo!

Dinosaur's suggestion of using symmetric Q, /Q outputs from a '74 flipflop halving the input clock (replacing the w65c02's phi1 output) seems to fix the problem, at least at for the sub-1khz speeds i'm experimenting with.

Progress!

Quote:

I also narrowed down where the phantom write happens (below). Surprisingly to me, read/write to IO page ($c0xx) doesn't seem to be implicated, but I guess phi1-qualified /mr and /mw signals are enough out of phase with the /phi2, /hi and /io signals that we still have the write enabled as the address bits change? I find it odd that so much seems to work correctly before we get to this repeatable failure point but the hardware details are still very mysterious to me.

I still suspect a noise issue of some kind. The W65C02's clock outputs don't lag the clock generator by enough to cause logic problems at your very slow speeds. But the W65C02 is fussy about edges (especially clock edges) at all speeds! Sometimes noise glitches like crosstalk happen at only very specific emergent combinations of conditions and can be really tough to pin down. Can you look at your clock and address lines on a scope? Now that you have a potentially working configuration you could do a before/after comparison.

Quote:

Is there a good tutorial that would help me understand the two write-cycle timing diagrams in the RAM datasheet https://www.alliancememory.com/wp-conte ... eb2007.pdf ? I haven't had enough practice to grok them on their own.

Jeff's excellent timing diagrams here might help: viewtopic.php?f=4&t=2909
One of Ben Eater's "Build a 6502 Computer" videos also has a discussion of RAM timings: https://www.youtube.com/watch?v=i_wrxBdXTgM
I've learned a lot from threads like this one: viewtopic.php?f=4&t=5180
I also *really* like this article from an old magazine: https://www.atarimagazines.com/computei ... /page9.php Of course, it's talking about the old NMOS6502, but many of the principles are unchanged.

None of these will precisely explain the memory timing diagrams you're talking about, but once you get comfortable with the 6502 timing diagrams, the memory ones aren't that much of a challenge.