Skip to content

Commit

Permalink
Fix precision
Browse files Browse the repository at this point in the history
  • Loading branch information
kg583 committed Nov 15, 2024
1 parent 761e3c4 commit bf594bf
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 6 deletions.
2 changes: 1 addition & 1 deletion tivars/numeric.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
from .data import *


pi = dec.Decimal("3.141592653589792")
pi = dec.Decimal("3.1415926535898")


def replacer(string: str, replacements: dict[str, str]) -> str:
Expand Down
8 changes: 3 additions & 5 deletions tivars/types/real.py
Original file line number Diff line number Diff line change
Expand Up @@ -614,12 +614,10 @@ def load_string(self, string: str):
super().load_string(string.strip("π"))

def string(self) -> str:
string = f"{self.decimal() / pi:.14g}".rstrip("0").rstrip(".")

if string.startswith("0e"):
if self.decimal() == 0:
return "0"
else:
return string + "π"

return f"{self.decimal() / pi:.14f}".rstrip("0").rstrip(".") + "π"

This comment has been minimized.

Copy link
@rpitasky

rpitasky Nov 15, 2024

Member

This math has me worried- wouldn't it be better to store the part being multiplied by pi to avoid precision issues?

This comment has been minimized.

Copy link
@kg583

kg583 Nov 15, 2024

Author Member

Well, the part being multiplied by pi is what's stored... and I just realized the call chain here does a multiply followed by a divide.



class TIRealPiFraction(TIRealPi, TIRealFraction, register=True):
Expand Down

0 comments on commit bf594bf

Please sign in to comment.