On Fri, 9 May 2008 21:05:46 -0700 "William \"Chops\" Westfield" wrote: > Sounds good, doesn't it? But 14/14 = 0 is 100% error ! Ok... But the _range_ of the output is 0-4000 (approx), so 14/14 = 0 is only an error of 1 in 4000 (if you implement the 16-bit algorithm). That's like blaming a DVM when you're measuring on the 4.000 volt range for changing the last digit (1mV). It's probable that if you divide 14.01 by 14, you'd get 1. In fact, I'm sure, as the approximation the algorithm does is really dividing by 14.00042726, not 14. > over the input range (1..65535) gives only 1/3 the number of errors > (about 7000) as adding the set of integer divisions. Note that you may be seeing approximation errors in the floating point values wherever you're simulating. John -- http://www.piclist.com PIC/SX FAQ & list archive View/change your membership options at http://mailman.mit.edu/mailman/listinfo/piclist