| View previous topic :: View next topic |
| Author |
Message |
Matthias
Joined: 18 Sep 2007
Posts: 6
|
 Posted: Sat Apr 04, 2009 10:36 pm Post subject: How to: format/float precision? |
 |
|
Beginner's question here, as I usually just make small applications to help me do something. I have a program that takes a number from a text box and adds it to another number, then spits it out as text. I noticed that sometimes they were getting cut off, and after poking around I traced it to doFormat's default of six significant digits. This is the relevant line, how would I go about raising the precision?
| Code: |
newSeconds = std.string.toString(atof(seconds) + displacement);
|
|
|
| Back to top |
|
 |
csauls
Joined: 27 Mar 2004
Posts: 278
|
| Posted: Sun Apr 05, 2009 6:27 pm Post subject: |
|
|
| Code: | | newSeconds = std.string.format("%13.9f", std.conv.toReal(seconds) + displacement); |
The magic is in "%13.9f". The first number is the number of characters to emit, the second is how many should go after the decimal point. So, in this case, it emits three for the whole part, then the dot, then nine for the fraction (3 + 1 + 9 = 13).
_________________
Chris Nicholson-Sauls |
|
| Back to top |
|
|
Matthias
Joined: 18 Sep 2007
Posts: 6
|
| Posted: Mon Apr 06, 2009 9:36 pm Post subject: |
|
|
| Awesome, thanks a lot. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
