Sunday, June 26, 2011

About accuracy

One thing that gets beat into engineers is that you are only as accurate as your least accurate number.

If you see a calculation:

2.2 * 2349014 * 2033 = ?

The incorrect answer is 10506200016.

The correct answer is 11x109.

The answer is only good to 2 significant figures.

(an argument could be made that 10.5 x109 is acceptable).

For more on significant numbers:
Significant Figures
Significant Figures in Math Operations

No comments:

Post a Comment