PRECISION

PRECISION

Top  Previous  Next

 

The PRECISION statement sets the maximum number of decimal places to appear when converting numeric values to strings.

 

 

Format

 

PRECISION expr

PRECISION INHERIT

 

where

 

expris an expression specifying the number of decimal places. This value must be between zero and fourteen. Negative values are treated as zero; values greater than fourteen are treated as fourteen.

 

 

Arithmetic operations performed by QM always work to the maximum precision of the computer system. The precision value determines the number of decimal places when numeric values are converted to strings, for example, when printing.

 

Values are converted with rounding on the last digit. Trailing zero digits are removed from the decimal places and, if the resultant value is an integer, the decimal point is also removed.

 

The precision value is associated with each program and subroutine and is initially set to 4. A program which sets a precision of 6 and calls a subroutine will use precision 6 up to the call, the subroutine will use precision 4 and, on return to the calling program, the precision reverts to 6.

 

Use of the INHERIT option to the PRECISION statement causes the program to inherit the precision value of the program from which it was called.

 

 

Example

 

X = 333.33333

Y = 666.66666

PRINT X, Y

PRECISION 4

PRINT X, Y

PRECISION 1

PRINT X, Y

PRECISION 0

PRINT X, Y

 

This program fragment would print

333.3333  666.6667

333.3     666.7

333       667