Yuri, thanks for sharing your code!
OK, let's regroup.
I have a 4MHz breadboard computer. Using a Rockwell 65C51 I built a 3-wire serial connection. That one works fine. I duplicated that on an I/O board for Blue August, my faster wire-wrap-on-protoboard computer. Using the same Rockwell 65C51 and running Blue August at 4MHz, the duplicate 3-wire serial connection works fine. I replaced the Rockwell part with a WDC65C51 and clocked Blue August back up to 16MHz. Now the 3-wire serial connection mostly works, but with intermittent glitches when receiving. In all of these cases, the baud rate is set to 19,200. Yuri suggests that the bit rate on the laptop might be too fast and suggests I add flow control. That seems like a good idea to me! There does not seem to be a lot of info about how to do this floating around the web, though. Here's how I think it works, but I must be making a mistake somewhere, because when I do this the serial connection doesn't work at all.
CTS\: This is an
input on the ACIA - pin 9. It needs to connect to an output pin on the logic side of the MAX232. R1OUT is connected to RxD (pin 12 -> 12!), but R2OUT is free. That is pin 9 on the MAX232, so pin 9 goes to pin 9. This signal comes into the MAX232 on the RS232 side on pin 8. That pin connects to pin 7 on my serial header, but might need to connect to pin 8 there depending on whether or not I'm using a crossover cable. I connect it with a Dupont wire so I can try both ways. The cable comes from the late 90s when I used it to play Quake with a friend.
There is no way to interact with CTS\ in software; it's not reported in any register. It seems to work like a G\ or E\ input - when it is asserted low, the ACIA simply stops.
RTS\: This is an
output on the ACIA - pin 8. It needs to connect to an input pin on the logic side of the MAX232. T2IN is free. That is pin 10. This signal emerges from the MAX232 on the RS232 side on pin 7. This should go to pin 8 on my serial header, but might need to go to pin 7, depending on my cable, again. RTS\ is software controlled. Asserting it low by writing the appropriate value to bits 2-3 of the ACIA command register tells the other end of the connection that the ACIA is ready to receive. Right now my firmware sets RTS\ low when the ACIA is initialized, and then leaves it alone.
I would think that, after changing my terminal emulator settings to use RTS/CTS flow control, the above would mimic the situation where there is no flow control at all. Since RTS\ is always low, the laptop will send to the ACIA as fast as it can, and I should see it mostly working with the occasional timing glitch. What do I see is nothing at all.
If I manually ground CTS\ (pin 9) on the ACIA, then I can send and receive, even with hardware flow control set in my terminal emulator. But there doesn't seem to be any pin on my DB9 header that is ever driven low by the host laptop.
Also, I can't get the board to glitch today, which makes me think the problem might be temperature-related. My "workshop" is in the garage now, which is not climate controlled. Still, it was only in the 90˚F range yesterday. Not close to 70˚C!