blob: 4fa675cc1b435e9f486526581ae5273681a9578d [file] [log] [blame]
lh9ed821d2023-04-07 01:36:19 -07001README for libm-test math test suite
2====================================
3
4The libm-test math test suite tests a number of function points of
5math functions in the GNU C library. The following sections contain a
6brief overview. Please note that the test drivers and the Perl script
7"gen-libm-test.pl" have some options. A full list of options is
8available with --help (for the test drivers) and -h for
9"gen-libm-test.pl".
10
11
12What is tested?
13===============
14The tests just evaluate the functions at specified points and compare
15the results with precomputed values and the requirements of the ISO
16C99 standard.
17
18Besides testing the special values mandated by IEEE 754 (infinities,
19NaNs and minus zero), some more or less random values are tested.
20
21Files that are part of libm-test
22================================
23
24The main file is "libm-test.inc". It is independent of the target
25platform and the specific real floating type and format and contains
26placeholder test "templates" for math functions defined in libm.
27The file, along with a generated file named "auto-libm-test-out",
28is preprocessed by the Perl script "gen-libm-test.pl" to expand
29the templates and produce a set of test cases for each math function
30that are specific to the target platform but still independent of
31the real floating type. The results of the processing are
32"libm-test.c" and a file "libm-test-ulps.h" with platform specific
33deltas by which the actual math function results may deviate from
34the expected results and still be considered correct.
35
36The test drivers "test-double.c", "test-float.c", and "test-ldouble.c"
37test the normal double, float and long double implementation of libm.
38The test drivers with an 'i' in their name ("test-idouble.c",
39"test-ifloat.c", and "test-ildoubl.c") test the corresponding inline
40functions (where available - otherwise they also test the real
41functions in libm). Each driver selects the desired real floating
42type to exercise the math functions to test with (float, double, or
43long double) by defining a small set of macros just before including
44the generic "libm-test.c" file. Each driver also either defines or
45undefines the __NO_MATH_INLINES macro just before including
46"libm-test.c" to select either the real or inline functions,
47respectively. Each driver is compiled into a single executable test
48program with the corresponding name.
49
50As mentioned above, the "gen-libm-test.pl" script looks for a file
51named "libm-test-ulps" in the platform specific sysdep directory (or
52its fpu or nofpu subdirectory) and for each variant (real floating
53type and rounding mode) of every tested function reads from it the
54maximum difference expressed as Units of Least Precision (ULP) the
55actual result of the function may deviate from the expected result
56before it's considered incorrect.
57
58The "auto-libm-test-out" file contains sets of test cases to exercise,
59the conditions under which to exercise each, and the expected results.
60The file is generated by the "gen-auto-libm-tests" program from the
61"auto-libm-test-in" file. See the comments in gen-auto-libm-tests.c
62for details about the content and format of the -in and -out files.
63
64How can I generate "libm-test-ulps"?
65====================================
66
67To automatically generate a new "libm-test-ulps" run "make regen-ulps".
68This generates the file "math/NewUlps" in the build directory. The file
69contains the sorted results of all the tests. You can use the "NewUlps"
70file as the machine's updated "libm-test-ulps" file. Copy "NewUlps" to
71"libm-test-ulps" in the appropriate machine sysdep directory. Verify
72the changes, post your patch, and check it in after review.
73
74To manually generate a new "libm-test-ulps" file, first remove "ULPs"
75file in the current directory, then you can execute for example:
76 ./testrun.sh math/test-double -u --ignore-max-ulp=yes
77This generates a file "ULPs" with all double ULPs in it, ignoring any
78previously calculated ULPs, and running with the newly built dynamic
79loader and math library (assumes you didn't install your build). Now
80generate the ULPs for all other formats, the tests will be appending the
81data to the "ULPs" file. As final step run "gen-libm-test.pl" with the
82file as input and ask to generate a pretty printed output in the file
83"NewUlps":
84 gen-libm-test.pl -u ULPs -n
85Copy "NewUlps" to "libm-test-ulps" in the appropriate machine sysdep
86directory.
87
88Note that the test drivers have an option "-u" to output an unsorted
89list of all epsilons that the functions have. The output can be read
90in directly but it's better to pretty print it first.
91"gen-libm-test.pl" has an option to generate a pretty-printed and
92sorted new ULPs file from the output of the test drivers.
93
94Contents of libm-test-ulps
95==========================
96
97Since libm-test-ulps can be generated automatically, just a few notes.
98The file contains lines for maximal errors of single functions, like:
99
100Function "yn":
101idouble: 6
102
103The keywords are float, ifloat, double, idouble, ldouble and ildouble
104(the prefix i stands for inline).
105
106Adding tests to libm-test.inc
107=============================
108
109The tests are evaluated by a set of special test macros. The macros
110start with "TEST_" followed by a specification the input values, an
111underscore and a specification of the output values. As an example,
112the test macro for a function with input of type FLOAT (FLOAT is
113either float, double, long double) and output of type FLOAT is
114"TEST_f_f". The macro's parameter are the name of the function, the
115input parameter, output parameter and optionally one exception
116parameter.
117
118The accepted parameter types are:
119- "f" for FLOAT
120- "b" for boolean - just tests if the output parameter evaluates to 0
121 or 1 (only for output).
122- "c" for complex. This parameter needs two values, first the real,
123 then the imaginary part.
124- "i" for int.
125- "l" for long int.
126- "L" for long long int.
127- "F" for the address of a FLOAT (only as input parameter)
128- "I" for the address of an int (only as input parameter)
129
130How to read the test output
131===========================
132
133Running each test on its own at the default level of verbosity will
134print on stdout a line describing the implementation of math functions
135exercised by the test (float, double, or long double), along with
136whether the inline set has been selected, regardless of whether or
137not any inline functions actually exist. This is then followed by
138the details of test failures (if any). The output concludes by
139a summary listing the number of test cases exercised and the number
140of test failures uncovered.
141
142For each test failure (and for each test case at higher levels of
143verbosity), the output contains the name of the function under test
144and its arguments or conditions that triggered the failure. Note
145that the name of the function in the output need not correspond
146exactly to the name of the math function actually invoked. For example,
147the output will refer to the "acos" function even if the actual function
148under test is acosf (for the float version) or acosl (for the long
149double version). Also note that the function arguments may be shown
150in either the decimal or the hexadecimal floating point format which
151may or may not correspond to the format used in the auto-libm-test-in
152file. Besides the name of the function, for each test failure the
153output contains the actual and expected results and the difference
154between the two, printed in both the decimal and hexadecimal
155floating point format, and the ULP and maximum ULP for the test
156case.