6502.org Forum  Projects  Code  Documents  Tools  Forum
It is currently Thu May 23, 2024 8:08 am

All times are UTC




Post new topic Reply to topic  [ 38 posts ]  Go to page Previous  1, 2, 3  Next
Author Message
 Post subject: Re: SRAM mystery
PostPosted: Thu Oct 07, 2021 8:24 pm 
Offline

Joined: Tue Jul 05, 2005 7:08 pm
Posts: 995
Location: near Heidelberg, Germany
At one point I sprayed the chip with a cold spray - but it did not trigger any immediate error. I can not test if it prevents errors, as these may appear only after a few minutes...

In the picture you can see two errors, one appeared around the 60th cycle, the next one at around the 110th, with each cycle running about 5 sec. (Note that these appear in the dRAM test, but that comes from the fact that the actual test program runs in the problematic SRAM)
Attachment:
File comment: Two errors shown, one from the 60th cycle (~5min in), the other from the 110th (~9:30 min).
20211007_215426_1920x1080.jpg
20211007_215426_1920x1080.jpg [ 1.74 MiB | Viewed 513 times ]


I was actually more inclined to look at the address line bounce that appears shortly after Phi2 goes high in the above scope shot. I was wondering if I have a problem with ground bounce. However, measuring this is difficult... See the next picture for my setup
Attachment:
20211007_215600_1920x1080.jpg
20211007_215600_1920x1080.jpg [ 1.77 MiB | Viewed 513 times ]

The colored flat ribbon cable on the lower right is attached to the bus terminator board, and has three scope probes attached. So measuring ground bounce based on that ground is ... problematic I'd guess.

Nevertheless it seems there is an issue
Attachment:
File comment: Ground bounce
20211007_215411_1920x1080.jpg
20211007_215411_1920x1080.jpg [ 1.79 MiB | Viewed 513 times ]

Note the dark blue line which is GND of the SRAM chip, based on the ground from above setup (from the terminator board).

What I tried to do then, as a first test, was to remove all the terminators to VCC, and keep the 3k3 terminators to GND on the address lines. This let the address line on the scope look a bit better, but did not prevent the errors from happening.

_________________
Author of the GeckOS multitasking operating system, the usb65 stack, designer of the Micro-PET and many more 6502 content: http://6502.org/users/andre/


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Thu Oct 07, 2021 8:45 pm 
Offline

Joined: Tue Jul 05, 2005 7:08 pm
Posts: 995
Location: near Heidelberg, Germany
Interestingly, NOT qualifying /CS with Phi2 seems to remove the DOUT flips completely!

So, while running this "JMP *" loop, /CS is constant low (it used to be qualified by Phi2), /WE is constant high (as it was before) and /OE is still qualified with Phi2.
Now the data lines do not flip like above!
Attachment:
20211007_223611_1920x1080.jpg
20211007_223611_1920x1080.jpg [ 1.65 MiB | Viewed 511 times ]

This has the three lines Phi2, A0 and D0 as usual, and the dark blue one is SRAM /CS.

The RAM test is running for about 5 mins now without error, but given the symptoms above that does not say anything
I'll have to check if there is a setup time we have overseen tomorrow, as I have to break for the night.

(note to self: still running with just 3k3 pulldown terminators only)

_________________
Author of the GeckOS multitasking operating system, the usb65 stack, designer of the Micro-PET and many more 6502 content: http://6502.org/users/andre/


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Thu Oct 07, 2021 11:34 pm 
Offline
User avatar

Joined: Sun Jun 30, 2013 10:26 pm
Posts: 1930
Location: Sacramento, CA, USA
fachat wrote:
Interestingly, NOT qualifying /CS with Phi2 seems to remove the DOUT flips completely!

So, while running this "JMP *" loop, /CS is constant low (it used to be qualified by Phi2) ...

I seem to recall some members here with far better hardware chops than mine claim that qualifying /CS with phi2 was an invitation for trouble.

[Edit: it's quite possible that I am mistaken ... my biological memory has a tendency to become less reliable with age ...]

_________________
Got a kilobyte lying fallow in your 65xx's memory map? Sprinkle some VTL02C on it and see how it grows on you!

Mike B. (about me) (learning how to github)


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Fri Oct 08, 2021 1:08 am 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8191
Location: Midwestern USA
barrym95838 wrote:
fachat wrote:
Interestingly, NOT qualifying /CS with Phi2 seems to remove the DOUT flips completely!

So, while running this "JMP *" loop, /CS is constant low (it used to be qualified by Phi2) ...

I seem to recall some members here with far better hardware chops than mine claim that qualifying /CS with phi2 was an invitation for trouble.

[Edit: it's quite possible that I am mistaken ... my biological memory has a tendency to become less reliable with age ...]

Your memory is not failing you. Qualifying chip selects with Ø2 needlessly reduces the available setup time and does have the potential to inadvertently create timing violations. In the case of 65xx peripherals, qualifying anything with Ø2 will likely result in non-operation. 65xx peripherals "understand" the 6502 bus cycle and must have all control inputs set up before the rise of Ø2.

When using peripherals that don't "understand" the 6502 bus cycle, it is necessary as a minimum, to qualify writes. Qualifying reads with Ø2 is a necessity in a 65C816 that lacks a data bus transceiver. Doing so is optional with the 65C02.

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Fri Oct 08, 2021 5:58 am 
Offline

Joined: Tue Jul 05, 2005 7:08 pm
Posts: 995
Location: near Heidelberg, Germany
It seems indeed that both datasheets, Alliance and BSI allow some garbage to appear on the data bus after /CE going low (tCLZ << tAA).
Those are both measured from /CE going low.

Just it seems
a) the Alliance chip does not produce such garbage and
b) I wasn't aware that this would create such problems (at least I hope that's the culprit)

I'm trying to run some burnin tests over the day today.

_________________
Author of the GeckOS multitasking operating system, the usb65 stack, designer of the Micro-PET and many more 6502 content: http://6502.org/users/andre/


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Fri Oct 08, 2021 7:12 pm 
Offline

Joined: Tue Jul 05, 2005 7:08 pm
Posts: 995
Location: near Heidelberg, Germany
So, I did some burn-in tests today as promised. With the BSI SRAM, with full terminators (3k3 to GND, 5k6 to VCC), without terminators, and even with ALS bus drivers instead of HCT - no problem whatsoever!
(I am repeating with the Alliance chip as well, just to be sure)

It's amazing. This circuit has been in the system for 32+ years ... (see this schematic from 1989: http://www.6502.org/users/andre/csa/bio ... -schem.png ). In the high level diagrams of the 32k SRAMs I originally used the /CS and /OE or /WE were simply "ORd" to give the right signal, but this BSI chip is obviously different.

And that probably was big part of the, maybe I should call it "instability crisis", I had with this machine. When I started working on "newer" tech (like ALS chips, larger RAMs, etc) I experienced some instability that I largely assumed to be on the dRAM part - who would have thought that the SRAM has these problems too...

Over the weekend I will update the schematics on the web page....

Many thanks for your help & feedback!

_________________
Author of the GeckOS multitasking operating system, the usb65 stack, designer of the Micro-PET and many more 6502 content: http://6502.org/users/andre/


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Fri Oct 08, 2021 8:29 pm 
Offline
User avatar

Joined: Fri Dec 11, 2009 3:50 pm
Posts: 3354
Location: Ontario, Canada
Good to hear about your success! And it was Hoglet's suggestion (to remove the Φ2 term from the RAM nCS signal) that made the difference, is that right?

It's true, as BDD said, that qualifying chip selects with Ø2 may reduce the available setup time. But in the lead post you said switching between 1MHz and 2MHz did not change anything, which to me suggests a lot of leeway for the setup time. Odd.

-- Jeff

_________________
In 1988 my 65C02 got six new registers and 44 new full-speed instructions!
https://laughtonelectronics.com/Arcana/ ... mmary.html


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Fri Oct 08, 2021 9:08 pm 
Offline
User avatar

Joined: Fri Dec 11, 2009 3:50 pm
Posts: 3354
Location: Ontario, Canada
hoglet wrote:
This will allow a bit more time for the address to settle internally in the SRAM before the output is enabled, which might avoid the additional transitions on the data bus.

If that helps, we can start to theorise why....
Yes, why would avoiding the additional transitions make a difference? Maybe I missed something, but I thought that, even with the extra transitions, the data still settled soon enough.

_________________
In 1988 my 65C02 got six new registers and 44 new full-speed instructions!
https://laughtonelectronics.com/Arcana/ ... mmary.html


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Fri Oct 08, 2021 9:51 pm 
Offline

Joined: Sun Jun 29, 2014 5:42 am
Posts: 337
Dr Jefyll wrote:
Yes, why would avoiding the additional transitions make a difference? Maybe I missed something, but I thought that, even with the extra transitions, the data still settled soon enough.

I agree, there should be plenty of time for the signals to settle. So something else has to be going on here.

Those extra transitions are probably happening on all the RAM outputs simultaneously. So I was wondering if the actual issue is ground bounce bad enough to cause corruption of the RAM contents.

I had one question for fachat: does your RAM test indicate whether it was a read or a write that failed? It would be very useful to know whether re-reading the failed address produces consistently wrong data.

Dave


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Fri Oct 08, 2021 10:07 pm 
Offline

Joined: Tue Jul 05, 2005 7:08 pm
Posts: 995
Location: near Heidelberg, Germany
hoglet wrote:
Dr Jefyll wrote:
Yes, why would avoiding the additional transitions make a difference? Maybe I missed something, but I thought that, even with the extra transitions, the data still settled soon enough.

I agree, there should be plenty of time for the signals to settle. So something else has to be going on here.


Indeed, that was my assumption as well, there should have been enough time to settle

Quote:

Those extra transitions are probably happening on all the RAM outputs simultaneously. So I was wondering if the actual issue is ground bounce bad enough to cause corruption of the RAM contents.

I had one question for fachat: does your RAM test indicate whether it was a read or a write that failed? It would be very useful to know whether re-reading the failed address produces consistently wrong data.

Dave


The RAM test actually indicates that. However, I have just looked through my pictures I took from the test, and found I only had what I call "secondary" errors - problems that appear in the dRAM test. Those were definitely of read and of write type (where the test just re-checks the memory content if it did not match. If the second read matches the should-be value, I classify as read error, if not, independent of value, I classify as write). These completely go away as well now.

My assumption is that they were caused by wrong memory accesses during the test, that actually runs in the SRAM that has the problem. It might be something else though, as I am actually wondering why actual crashes appear so seldomly (if at all)

One picture above actually shows quite some signal bouncing on the /CS line shortly after Phi2 goes high - maybe that is caused by ground bounce, that may cause other havoc.

Edit: it's actually the picture in the post on the top of this page, and it actually does show SRAM ground bouncing, but my problem is what is the reference?

I don't really have experience measuring the ground bounce though - what is the best way to measure that? Use two probes between the different (which) ground points, or between ground and VCC, and let the scope do a diff?

André

_________________
Author of the GeckOS multitasking operating system, the usb65 stack, designer of the Micro-PET and many more 6502 content: http://6502.org/users/andre/


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Fri Oct 08, 2021 10:45 pm 
Offline

Joined: Fri Dec 21, 2018 1:05 am
Posts: 1076
Location: Albuquerque NM USA
I also agree that the "fix" does not explain the problem; there is something else going on. A 55nS RAM operating in 1MHz system where clock high phase is 500nS doesn't care whether CS is qualified by clock or not; there are plenty of time for signals to settle out by the falling edge of the clock. I am wondering whether there are other activities during the clock high period that can upset the signals near the falling edge of the clock. One thing comes to mind is DRAM access and refresh. Active edge of /RAS is a particularly noisy event, CAS-before-RAS refresh can also generate lots of noise. That may be an area of investigation.

When troubleshooting intermittent problems, do not do something to make the problem go away, but do something to make the problem show up more frequently. Once the problem becomes deterministic, troubleshooting become much easier.
Bill


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Sat Oct 09, 2021 12:46 am 
Offline
User avatar

Joined: Thu May 28, 2009 9:46 pm
Posts: 8191
Location: Midwestern USA
fachat wrote:
I don't really have experience measuring the ground bounce though - what is the best way to measure that? Use two probes between the different (which) ground points, or between ground and VCC, and let the scope do a diff?

That's basically it. The critical aspect of the test is to be sure the reference ground point is very solidly tied to power supply ground. The rest is a matter of observation.

SRAMs can significantly spike the power supply as they come out of the high-Z state, which would be when they are selected and then read- or write-enabled. At that point, current consumption increases from microamps to multiple tens of milliamps. If the SRAM's ground is marginal the sudden increase will trigger bounce, more so during a read cycle which is when the outputs are all enabled at once.

See attached for some info from Cypress. This stuff would also apply to E(E)PROMs.

Attachment:
SRAM_board_design_guidelines.pdf [822.24 KiB]
Downloaded 31 times
Attachment:
SRAM_system_design_guidelines.pdf [473.64 KiB]
Downloaded 26 times

_________________
x86?  We ain't got no x86.  We don't NEED no stinking x86!


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Sat Oct 09, 2021 8:21 am 
Offline

Joined: Sun Jun 29, 2014 5:42 am
Posts: 337
Personally I have always found ground-bounce very difficult to measure directly.

The problem is the scope probe together with its ground lead foms a loop antenna. In a system using modern devices with fast (<5ns) edges, much of what you see on the scope is switching noise being picked up by the loop antenna. i.e. it makes the system look far noisier than it actually is.

The other problem is that the older 24/28/32-pin DIP packages have a single ground pin at the corner of the package. There is considerable lead inductance due to this, which makes the internal ground bounce on the die worse than what you would see on the PCB.

The worst case is during a RAM read, where the RAM is directly driving a highly capacitive load, for example a backplane, and when all 8 outputs switch simultaneously, changing the value on the data bus from 0xFF to 0x00. In your system, I'm wondering if at this point the voltage on the nWE pin (referenced to the internal ground of the SRAM die) falls low enough to trigger a write cycle.

One way to indirectly measure the ground bounce on the die is to setup an experiment where one output remains low and the other seven outputs switch from high to low. By placing the scope probe between the RAM ground and the static output, you get a view of then amount of internal ground bounce.

A carefully crafted JMP loop can be used for this test:
Code:
00FD : 4C FD 00 : JMP $00FD


This causes the RAM to cycle through the following values:
Code:
01001100
11111101
00000000

By looking at bit D1 you can observe the effect of the other seven bits switching simultaneously from 1 to 0.

It's important to do this test with as short a scope ground connection as possible. I usually resort to soldering a ground wire directly to the IC itself and wrapping this around the scope probe (without the clip). You also need a reasoably fast scope. I would say a minumum of 100MHz.

Good luck tracking this one down!

Dave


Last edited by hoglet on Sat Oct 09, 2021 2:21 pm, edited 1 time in total.

Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Sat Oct 09, 2021 10:54 am 
Offline

Joined: Tue Jul 05, 2005 7:08 pm
Posts: 995
Location: near Heidelberg, Germany
BigDumbDinosaur wrote:
fachat wrote:
I don't really have experience measuring the ground bounce though - what is the best way to measure that? Use two probes between the different (which) ground points, or between ground and VCC, and let the scope do a diff?

That's basically it. The critical aspect of the test is to be sure the reference ground point is very solidly tied to power supply ground. The rest is a matter of observation.

SRAMs can significantly spike the power supply as they come out of the high-Z state, which would be when they are selected and then read- or write-enabled. At that point, current consumption increases from microamps to multiple tens of milliamps. If the SRAM's ground is marginal the sudden increase will trigger bounce, more so during a read cycle which is when the outputs are all enabled at once.

See attached for some info from Cypress. This stuff would also apply to E(E)PROMs.

Attachment:
SRAM_board_design_guidelines.pdf
Attachment:
SRAM_system_design_guidelines.pdf


Really interesting reads! Many thanks for sharing!

So, instead of adding up more capacitance, the main point is to reduce inductance. In the given scenario (switching 18 outputs to a 50Ohm line to limit power droop to 300mV) a 0.1uF seems already oversized by capacitance!
I have actually planned - in newer designs - to use 2 bypass caps per power supply pin, one with 1uF and one with 0.1uF - but that would most likely not help. So, instead two 0.1uF would be better (given the scenario in the paper). And I should probably do more maths before just slapping bypass caps there...

And while older chips had rise/fall times of 5ns or more, the bypass frequencies were lower. On the other hand through-hole caps sure have a higher inductance as well.

I'll have to think deeper about this...

_________________
Author of the GeckOS multitasking operating system, the usb65 stack, designer of the Micro-PET and many more 6502 content: http://6502.org/users/andre/


Top
 Profile  
Reply with quote  
 Post subject: Re: SRAM mystery
PostPosted: Sat Oct 09, 2021 1:46 pm 
Offline
User avatar

Joined: Fri Dec 11, 2009 3:50 pm
Posts: 3354
Location: Ontario, Canada
Andre, I think you'll do well to give hoglet's post a good read, beginning with his observation that ground bounce is very difficult to measure. I suspect he'd agree that even the very clever, carefully crafted JMP loop poses some challenges.

So, rather than trying to measure the ground bounce, maybe we can contrive to vary the conditions in a way that's calculated to decidedly improve or worsen the symptoms if indeed it's ground bounce that's responsible.

I can think of several ways to improve or worsen ground bounce, but, before I get to that, here's an important question.

Referring to the image below, is the NAND gate that drives the RAM's nWE pin a CMOS type, or is it a TTL type such as LS or ALS? This is highly pertinent, as a CMOS gate will have a higher Voh, and our concern relates to the voltage difference between nWE and the RAM die's internal ground.

If the gate is TTL then Dave's theory seems plausibe -- ie, that the voltage on the nWE pin (referenced to the internal ground of the SRAM die) is low enough that a write cycle results when the internal ground momentarily bounces higher in voltage. But if the gate is CMOS then I think we need a different theory (because it's unlikely that the internal ground would jump sufficiently high to make nWE seem like a logic low).

As for other theories, maybe internal Vcc bounces (downward) to such an extent that, combined with ground bounce (upward), the difference between the internal supply rails falls below the level needed to retain data. So, more of a power failure, as opposed to an inadvertent write cycle. (But I've never heard of such a thing happening, so as a theory it may be far fetched.)

-- Jeff


Attachments:
20211006_180828_1920x1080 jwl.png
20211006_180828_1920x1080 jwl.png [ 661.03 KiB | Viewed 370 times ]

_________________
In 1988 my 65C02 got six new registers and 44 new full-speed instructions!
https://laughtonelectronics.com/Arcana/ ... mmary.html
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 38 posts ]  Go to page Previous  1, 2, 3  Next

All times are UTC


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to: