On Thu, 22 Jul 1999, Russell McMahon wrote:

> I've just looked at your results and yes, they are similar in error
> to mine.
> It occurs to me that I've not been comparing apples with apples here.
> It is important to me what the absolute value of the error is at any
> given reading
> eg (Reading-Actiual)/Actual * 100 %
> So, my error percentages are per reading. without reference to other
> readings.
> But, A2Ds are normally specd as accuracy in bits with respect to full
> scale. ie 1 5 volt convetre with an error of 50 millivolts would be
> classed as an error of 1% whether the true value was 5 volts or 0.5
> volts. However, at 0.5 volts thids apparent error of 1% would be an
> actual error of 10%. Nothing startling here but I had got so tied up
> in what I was trying to achieve for my purposes that I was reporting
> in absolute terms without reflecting that others may be confused by
> it.
>

I know what you mean. In case anyone's interested, this is how you measure
linearity error.

1) Collect a whole bunch of data and record a) the voltage applied and b)
and the converted digital value. You also should collect several samples
at each voltage value and use the mean of the digital values to represent
converted digital value. There's another test for measuring the spread.

2) Find the least square's curve fit of the data. In other words, find the
line that best fits the data.

3) Find the sample that most deviates from this line.

The line found by the least square's curve fit is of the form:

   y = m*x + b

where
 x = true value of the input signal (the applied voltage)

 y = expected value when the true value is applied (the actual value is
     what you measure)

 m & b are the slope and intercept of the line and are calculated:


m = [s(x) * s(x * d) - s(x*x) s(d)] / den

b = [s(x) * s(d) - n * s(x * d) ] / den

den = s(x)*s(x) - n * s(x*x)

where,

d = digitized value
n = total number of samples

and
          n
        -----
        \
s( ) =  /     (   )
        -----
         i=1


Once you have m and b, you can perform step 3 by running the applied
voltage through the line representing the data and comparing the expected
value output with the value you actually measured. The Linearity Error is
then:

L = max( ( d - y ) / x_fr ) * 100%

where x_fr is the full range of the data, or 4096 for a 12-bit A/D.

Like I said earlier, I've got an octave program to do this. If anyone
wants it, drop me a line. You probably can find a lsf function in excel
too.

> Here are the full scale errors % that I get from your data - very
> nice.
> Two key questions are -
>
> 1.    How many counts in your loop? - whoops - you say that its 4096.
> Mine are 2048 counts.

4096

>
> 2.    What are you using as your "gate" input - standard PIC pin /
> PIC Schmitt pin / external gate  or an internal/external comparator
> ?
>     What PIC?

I'm using RA0 for the input gate and RA1 for the output and the pic is a
16F84 running at 4MHz.

Scott