Gerhard Fiedler wrote:
> In some ways, the test program for the test program is the target
> program. But this means that you need to debug both, observe both,
> have a means to verify results that is outside of both. (This may be
> data files produced that you manually verify occasionally, this may
> be a scope hooked up to the inputs and outputs where you manually
> verify the timing of certain events, etc.)

That's what I see most of the time.  The test and target are developed
independently by different people from the same spec.  Each will of course
do some basic testing using stubs or simple data or whatever, but the real
testing starts when they get together and try to make the combination work.
That's when scopes and protocol analysers and the like get used.  Of course
the data gets looked at too, to the extent a human can look at it and tell
bad from good.

On one project I worked on many years ago, we had a team of people designing
the real hardware, one person creating a automated test framework for it,
several others designing test suites for the automated framework to run, and
someone else creating a simulator of the hardware to independently verify
the tests and provide a reference for the hardware.  The system was too
complex to easily specify the desired result for each test.  Instead the
test framework would run a test on the functional simulator, then on the
gate-level simulated hardware and compare the two.  Discrepancies were
analyzed by humans.  Sometimes the functional simulator got it wrong,
sometimes the gate level design was wrong.  That's all part of testing.  In
the end the simulator turned out to be useful for the software people to
test their code on before real hardware was availble and to get more
diagnostic information than the real hardware supplied.

The important part binding all the efforts together was the functional spec.
Everybody referred to it as the reference on how the system was supposed to
function.  It was a critical document that initially took a full time person
to maintain.  There was no way you could have jumped into the middle, hacked
up something that partially worked, then refined it from there without
having a clear idea of how you were going to get to the full picture.  We
had three people initially working on the architecture before anyone started
laying gates.  The early versions of the functional simulator were used a
test bed for different architectures.  Most of the engineers that were to
design the final hardware weren't even hired until we had a decent idea of
the overall architecture.


********************************************************************
Embed Inc, Littleton Massachusetts, http://www.embedinc.com/products
(978) 742-9014.  Gold level PIC consultants since 2000.
-- 
http://www.piclist.com PIC/SX FAQ & list archive
View/change your membership options at
http://mailman.mit.edu/mailman/listinfo/piclist