string_helpers: fix precision loss for some inputs

Browse source

commit 564b026fbd upstream.

It was noticed that we lose precision in the final calculation for some
inputs.  The most egregious example is size=3000 blk_size=1900 in units
of 10 should yield 5.70 MB but in fact yields 3.00 MB (oops).

This is because the current algorithm doesn't correctly account for
all the remainders in the logarithms.  Fix this by doing a correct
calculation in the remainders based on napier's algorithm.

Additionally, now we have the correct result, we have to account for
arithmetic rounding because we're printing 3 digits of precision.  This
means that if the fourth digit is five or greater, we have to round up,
so add a section to ensure correct rounding.  Finally account for all
possible inputs correctly, including zero for block size.

Fixes: b9f28d8635
Signed-off-by: James Bottomley <JBottomley@Odin.com>
Reported-by: Vitaly Kuznetsov <vkuznets@redhat.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

...

This commit is contained in:

James Bottomley 2016-01-20 14:58:29 -08:00 • committed by Greg Kroah-Hartman

parent 5c73252f74

commit 726ecfc321

1 changed files with 43 additions and 20 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download patch file Download diff file Expand all files Collapse all files

69

lib/string_helpers.c

View file