Here's a more detailed explanation of what I'm thinking. AIUI, the computer is always OK except when the CF card is plugged in, and then it's never OK. The card's /Chip-Select has been measured, and seems to be high. (I'm not sure what instrument you used for the measurement, but some instruments are inappropriate, and all can suffer from human factors and even outright flukes. Myself, I would simply tie /CS high and thus remove any doubt.)
Alright, so let's say the /CS pin on the socket is tied high and the computer still fails; it will only work if the card is physically removed from the socket. Here's one theory to explain that.
Trouble in a new project often is the result of a wiring error. The CF card itself probably has several pins which serve as Ground, and all these pins are connected together... inside the CF card itself. So, suppose on the CPU side there's a data line (or other) which, due to error, connects to a socket pin which in turn is bused together with the other Grounds inside the CF. That means the data line (or whatever) gets shorted to Gnd whenever the card is plugged in. But the short disappears if the card is physically removed from the socket.
Other example are possible. If I were in your place I'd check the wiring. And/or do some single-stepping right from reset, or get a LA trace from reset as Ed suggested.
Have fun and keep us posted!
-- Jeff