Classic goof-up, simple to do, should have caught it earlier, didn't, 
but it's the strange symptoms that I thought were worth sharing.

Breadboard prototype worked great.  Contained two PICs talking over a 
pair of serial lines, using a simple asynchronous protocol, one line for 
transmit and one for receive.  One 16F688 for main processing, and a 
16F916 to drive a six-digit LED display in the usual multiplexed manner.

Got the boards.  Populated (or shall I say, 'stuffed') one.  Powered up. 
  The display is all 8's for a few seconds (its normal startup mode), 
then changes to one much brighter 8.  The specific digit varies among 
trials.

My initial guess was that the LED driver code was hanging, and this 
appeared to be borne out by further experiments.  Pared the code down to 
an absolute minimum; still hung.

So I'm thinking bypass capacitors.  Added more, and larger.  They change 
the behavior so I think I'm on the right track, but eventually I'm up to 
one 10 uF tantalum each and it still hangs regularly.

Then I'm questioning the power supply.  Too much ripple?  Any drains 
that would be bigger on the PCB than the breadboard?  Less stray 
capacitance in general on the PCB...  I had chosen not to do a ground 
plane because I thought I wouldn't need it - mostly low-frequency stuff 
- but maybe the LED multiplexing was switching too many mA too fast? 
Seemed unlikely.  But maybe...

Pins left floating?  No, all are either outputs or pulled up internally.

Watchdog reset?  Doesn't really fit.  I had watchdogs turned off on both 
chips at this point.  Turned them on; the watchdog barked every time, 
after the hang, and the chip happily restarted.  So it wasn't the problem.

And why wasn't the main controller sending anything to the display, 
anyway?  What was it doing while the LED driver was locked up?

After nearly eight hours of this...

TX on PIC #1 was going to TX on PIC #2, and RX to RX.  A brain fart in 
the schematic, and I never caught it before the board went out.

So basically, one PIC was lowering the TX line to start transmitting a 
byte, and there's no resistor between the two, and the other was still 
holding it high.  So current was unlimited, and one or both chips 
latched up?  Make sense?

Luckily, there were a couple of vias I could use to insert some jumpers 
without too much pain.  And it's only a run of 10, so fixing all the 
boards will be annoying but not too slow.

So basically, the weird lockup symptom got me sidetracked thinking about 
bypass capacitors and fiddly voodoo analog stuff, instead of going back 
to the basics and checking continuity.  Checking continuity while 
thinking carefully, I should say.

This was yesterday, so it's still a bit painful, but today the whole 
product is working, so I'm cheered up and able to get it off my chest.
-- 
Timothy J. Weber
http://timothyweber.org
-- 
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist