Electronics World articles Popular Electronics articles QST articles Radio & TV News articles Radio-Craft articles Radio-Electronics articles Short Wave Craft articles Wireless World articles Google Search of RF Cafe website Sitemap Electronics Equations Mathematics Equations Equations physics Manufacturers & distributors Engineer Jobs LinkedIn Crosswords Engineering Humor Kirt's Cogitations RF Engineering Quizzes Notable Quotes Calculators Education Engineering magazine articles Engineering software Engineering smorgasbord RF Cafe Archives RF Cascade Workbook 2018 RF Symbols for Visio - Word Advertising RF Cafe Forums Magazine USAF Radr Shop Sponsor RF Cafe RF Electronics Symbols for Visio RF Electronics Symbols for Office Word RF Electronics Stencils for Visio Thank you for visiting RF Cafe!
everythingRF RF & Microwave Parts Database - RF Cafe
RF Cascade Workbook 2018 by RF Cafe

About RF Cafe

Kirt Blattenberger - RF Cafe Webmaster

Copyright: 1996 - 2024


    Kirt Blattenberger,


RF Cafe began life in 1996 as "RF Tools" in an AOL screen name web space totaling 2 MB. Its primary purpose was to provide me with ready access to commonly needed formulas and reference material while performing my work as an RF system and circuit design engineer. The World Wide Web (Internet) was largely an unknown entity at the time and bandwidth was a scarce commodity. Dial-up modems blazed along at 14.4 kbps while tying up your telephone line, and a nice lady's voice announced "You've Got Mail" when a new message arrived...

All trademarks, copyrights, patents, and other rights of ownership to images and text used on the RF Cafe website are hereby acknowledged.

My Hobby Website:


Copper Mountain Technologies (VNA) - RF Cafe

Splitting Hairs over Precision
Kirt's Cogitations™ #207

RF Cafe University"Factoids," "Kirt's Cogitations," and "Tech Topics Smorgasbord" are all manifestations of my rantings on various subjects relevant (usually) to the overall RF Cafe theme. All may be accessed on these pages:

 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36

< Previous                      Next >


Splitting Hairs over Precision

There is a really interesting short article in the February 13, 2006, edition of EETimes titled, "Humans Still Thrive on Decimals," by Clive (Max) Maxfield, that illustrates by example how binary mathematics can make simple numerical operations troublesome. Drawing on the extensive collection of works by IBM Fellow Mike Cowlishaw's "General Decimal Arithmetic" website, Max shows how, for instance, the "double" floating point real number type cannot precisely represent 0.1. Multiplying 0.1 x 8 returns, to full precision, 0.8000000000000000444089209850062616169452667236328125. However, adding 0.1 to itself 8 times yields an entirely different result, 0.7999999999999999333866185224906075 458209991455078125.

Theoretically, an infinite number of binary bits is required to precisely represent many seemingly simple decimal values. Having written a fair amount of engineering software that requires carrying calculations to many decimal places, I have run into this problem often. It can be a real headache to resolve in a manner that does not cause users to question the validity of results. For instance, consider the case of calculating frequency conversions where both very high and very low frequencies are employed in the same calculation. The two sets of numbers (maybe three sets, depending on the relative frequency of the local oscillator) may be separated by six or even nine orders of magnitude. Many decimal places are required to maintain precision at both extremities. Nobody I work with will accept an answer that turns what should be, say, 5.000000 kHz into 4.999999 kHz. It just doesn't feel right even though we know it is "close enough."

Because of the limitations imposed by our finite systems, financial calculations are by law required to carry out using binary coded decimal (BCD) operations. Doing so adding more bits to represent each number, but the phenomenon of binary approximation is eliminated. Financials can face the same number extremities as engineering calculations when you consider the case of calculating the trillion dollar gross domestic product of a country like the U.S. down to the penny ($1,000,000,000,000.01). Calculation times increases significantly when using BCD arithmetic, and can take 1000 times or more longer than floating point operations. Of course, special circuitry could be used (FPGAs) to implement calculators whose only function is to do binary math and not floating point.

The story reminds me of a simple test that was popular in the 1980s as a cursory means of determining whether a particular handheld calculator was worthy of engineering pursuits. Perform the operation 1 / 3 * 3 =. If the answer was 1.00000000, then you could buy the calculator. If it was 0.9999999, then it was time to scoff at it in an elitist way and move on – let the knaves use those.

Innovative Power Products Couplers
RF Cafe Sponsors

Innovative Power Products Passive RF Products - RF Cafe

Please Support RF Cafe by purchasing my  ridiculously low−priced products, all of which I created.

These Are Available for Free