 Notice. New forum software under development. It's going to miss a few functions and look a bit ugly for a while, but I'm working on it full time now as the old forum was too unstable. Couple days, all good. If you notice any issues, please contact me. |
Forum Index : Microcontroller and PC projects : limiting decimal places to two when reding AIN port
| Author | Message | ||||
| desitech Newbie  Joined: 13/12/2023 Location: AustraliaPosts: 2 |
I am reading the input from a Analogue port and want to restrict the reading to two decimal places. I cant find a math function to do this. Can anyone assist? Regards - Graham Hobbs |
||||
| Volhout Guru  Joined: 05/03/2018 Location: NetherlandsPosts: 4237 |
In displaying the value: print str$(pin(gpx),2,2) In limitting the value's resolution value=int(100*pin(gpx))/100 Volhout Edited 2023-12-21 18:27 by Volhout PicomiteVGA PETSCII ROBOTS |
||||
| matherp Guru Joined: 11/12/2012 Location: United KingdomPosts: 9118 |
Assuming this is for output then use STR$(). Otherwise a2=cint(a*100)/100 |
||||
| JohnS Guru Joined: 18/11/2011 Location: United KingdomPosts: 3802 |
If you store the value in floating point, you can't. There will always (*) be a slight deviation from the exact number you'd like to store. Typically what is stored is plenty good enough, especially as in your case where the thing you're reading is itself not entirely accurate. If you really wanted to store it exactly you'd have to jump through some hoops e.g. store 100 times the value as an integer. (*) nearly always, i.e. except in a few cases where what can be stored is the same as what you want, but these are few Note: the above is about an inherent issue with floating point on all computers using it. You may have seen the same issue using a calculator and doing such as division by 3 then multiplying by 3 and not getting back the original number. John Edited 2023-12-21 20:06 by JohnS |
||||
| desitech Newbie Joined: 13/12/2023 Location: AustraliaPosts: 2 |
Thanks to everyone for your prompt and great assistance Regards Graham Hobbs |
||||
 Print this page |
