Your above description smacks of a flow-control error on the serial link. However, I suppose an obscure hardware problem could be present in the computer itself. Here are some things I would consider in the latter case:
- 74HCT logic is relatively slow, usually no faster than 74LS, with only slightly better fanout. The 74HCT245 you are using as a data bus transceiver may be taking too long to become active after Ø1 goes low and thus may be inadvertently violating the UART's timing requirements, especially during writes. Try replacing your 74HCT245 with a 74ACT245, which is about three times faster than the HCT device.
Hmm that's the best idea I've heard so far. I'm actually going to be ordering some 74ACT245's this week since I don't have any in my collection and I've just been working with what I had. In fact one reason the prototype wiring is messy is because I originally tried building it without the buffer, figuring it wasn't needed since nothing else would be on the bus during PHI2 low anyway...and then I discovered my SRAM only outputs TTL compatible logic levels and the CPU wasn't able to read RAM at all...
[*]Most GALs generate TTL-compatible output levels. A check of one of your quiescent chip selects with a DVM will tell you this. The 28L92 requires 2.4 volts minimum on control inputs (excepting the X1 clock) to recognize a valid logic 1. There's a slight chance that the GAL output driving the UART's control inputs is right on the ragged edge.
I did notice that; the GAL's high values are more like 4.2ish volts than 5V; a bit low but definitely strong enough for the UART.
[*]While on the subject of GALs, they can be power hogs when the outputs are switching states. Be sure your Vcc and ground connections to the GALs are robust and properly bypassed.
Well it's a breadboard so I dunno how robust they can be.
What I can do is try running the 5V lines directly from the power bus strips at the top instead of the vertical ones, just to eliminate some power loss.
[*]Verify that when the X1 (3.6864 MHz) clock signal is high it is greater than 4 volts. Aside from providing the time base for setting data reception/transmission bit rates and driving the counter/timer, the X1 clock is what sequences the UART's internal state machine logic. If your oscillator has a TTL output you are not driving X1 hard enough, which will likely cause the UART to misbehave.
Funny you mention that because I was checking that last night; it turns out the batch of oscillators I bought isn't guaranteed to put out quite enough voltage for the UART, so I rewired the oscillator output through a 74hct125 buffer and the clock is now definitely a healthy 5V. Unfortunately that didn't seem to make a difference.
[*]The 28L92 is fussy about its power source. As with the GAL, be sure your Vcc and ground connections are robust and properly bypassed.
This one I'm pretty confident about; since the UART is on its own little breaboard it is powered directly from the top strip of the main board. Everything has a bypass cap but it may be worth trying to toss a larger electrolytic up near the power input just to make sure everything is smooth. Thing thing is already drawing about 0.275 amps according to my bench supply, largely due to the two GALs...and the final version will draw even more, because the prototype only has half the RAM, and I'm also toying with adding a second VIA.
[*]Although the NXP data sheet says the 28L92's I/M mode input can be floated when operating in x86 mode (which I know will work from testing in POC V1.1), I'm always leary of leaving an input on a CMOS device unconnected. Try tying I/M to Vcc and see if that has any effect.
No difference, unfortunately.
Not knowing anything about how you have your unit interfaced to your console device, I can't offer any advice except to see if you can implement flow control. Any speed above 9600 bps will most likely require flow control to avoid mangled data. Typically, flow control is implemented in hardware (CTS/RTS handshaking), as software flow control (XOFF/XON handshaking) at higher speeds is generally unreliable.
I'm using some generic USB->TTL serial cables, not sure where I picked them up (Amazon, I think). The far end is my Linux workstations running Minicom, set to 38.4k bps, 8N1, and hardware handshaking. This exact same cable works flawlessly when connected to my COLE-1 SBC, which runs 115.2k with no flow control through a 6850.
On the software side, carefully review your setup data table to verify that you are writing the correct parameters into the UART registers in the correct order. Try working with the unedited setup data that I published, which has been proven to work at 115.2 Kbps on POC versions 1.1 and 2.1 (both equipped with the 28L92). It could be you accidentally changed something that is critical to achieving proper operation.
The 28L92 will reliably perform at high speeds with a good circuit design and correct setup. My POC units drive the console at 115.2Kbps and also the data link to my Linux development box at that speed—without error. Knowing that should reassure you that a solution can be found.
I'll try copying back in your original file and table just to be safe; they should only differ in baud rate and the removal of the disable powerdown mode command, but it's also possible the magic I did in vim to convert the file over to ca65 format may have inadvertently removed something.
I too am confident it will eventually work; if it works on my original SBC with a 6850 I'm sure it will work with the 28L92.
AC / ACT series does have its place (such as when driving low-impedance transmission lines) but that implies a special need and a well-controlled environment. For our purposes, AHC / AHCT is usually a much better choice. 
... This is a good chance to remind everyone that it's impossible for signal wiring to carry current without the ground and VCC lines also carrying current (in order to establish a complete circuit). In other words, for every signal there will inevitably be a return path. Most of us know enough to avoid a signal path that's longer than necessary, but what's sometimes forgotten is that the ground / VCC wiring, in addition to supplying DC power to IC's, also participates in carrying the AC signal -- ie, it provides the AC return path. And it's best to ensure that the return path is close to the signal path. ("Close to the signal path" generally means "short," because hopefully the signal path is short.)
