It has been noted that test-tgmath3 is slow to compile, and to link on
some systems
<https://sourceware.org/ml/libc-alpha/2018-02/msg00477.html>, because
of the size of the test.
I'm working on tgmath.h support for the TS 18661-1 / 18661-3 functions
that round their results to a narrower type. For the functions
already present in glibc, this wouldn't make test-tgmath3 much bigger,
because those functions only have two arguments. For the narrowing
versions of fma (for which I've not yet added the functions to glibc),
however, it would result in many configurations building tests of the
type-generic macros f32fma, f64fma, f32xfma, f64xfma, each with 21
possible types for each of three arguments (float, double, long double
aren't valid argument types for these macros when they return a
_FloatN / _FloatNx type), so substantially increasing the size of the
testcase.
To avoid further increasing the size of a single test when adding the
type-generic narrowing fma macros, this patch arranges for the
test-tgmath3 tests to be run separately for each function tested. The
fma tests are still by far the largest (next is pow, as that has two
arguments that can be real or complex; after that, the two-argument
real-only functions), but each type-generic fma macro for a different
return type would end up with its tests being run separately, rather
than increasing the size of a single test.
To avoid accidentally missing testing a macro because
gen-tgmath-tests.py supports testing it but the makefile fails to call
it for that function, a test is also added that verifies that the
lists of macros in the makefile and gen-tgmath-tests.py agree.
Tested for x86_64.
* math/gen-tgmath-tests.py: Import sys.
(Tests.__init__): Initialize macros_seen.
(Tests.add_tests): Add macro to macros_seen. Only generate tests
if requested to do so for this macro.
(Tests.add_all_tests): Take argument for macro for which to
generate tests.
(Tests.check_macro_list): New function.
(main): Handle check-list argument and argument specifying macro
for which to generate tests.
* math/Makefile [PYTHON] (tgmath3-macros): New variable.
[PYTHON] (tgmath3-macro-tests): Likewise.
[PYTHON] (tests): Add $(tgmath3-macro-tests) not test-tgmath3.
[PYTHON] (generated): Add $(addsuffix .c,$(tgmath3-macro-tests))
not test-tgmath3.c.
[PYTHON] (CFLAGS-test-tgmath3.c): Remove.
[PYTHON] ($(tgmath3-macro-tests:%=$(objpfx)%.o): Add -fno-builtin
to CFLAGS.
[PYTHON] ($(objpfx)test-tgmath3.c): Replace rule by....
[PYTHON] ($(foreach
m,$(tgmath3-macros),$(objpfx)test-tgmath3-$(m).c): ... this. New
rule.
[PYTHON] (tests-special): Add
$(objpfx)test-tgmath3-macro-list.out.
[PYTHON] ($(objpfx)test-tgmath3-macro-list.out): New rule.
README for libm-test math test suite
====================================
The libm-test math test suite tests a number of function points of
math functions in the GNU C library. The following sections contain a
brief overview. Please note that the test drivers and the Perl script
"gen-libm-test.pl" have some options. A full list of options is
available with --help (for the test drivers) and -h for
"gen-libm-test.pl".
What is tested?
===============
The tests just evaluate the functions at specified points and compare
the results with precomputed values and the requirements of the ISO
C99 standard.
Besides testing the special values mandated by IEEE 754 (infinities,
NaNs and minus zero), some more or less random values are tested.
Files that are part of libm-test
================================
The main files are "libm-test-<func>.inc". They are independent of
the target platform and the specific real floating type and format and
contain placeholder test "templates" for math functions defined in
libm. These files, along with generated files named
"auto-libm-test-out-<func>", are preprocessed by the Perl script
"gen-libm-test.pl" to expand the templates and produce a set of test
cases for each math function that are specific to the target platform
but still independent of the real floating type. The results of the
processing are "libm-test-<func>.c" and a file "libm-test-ulps.h" with
platform specific deltas by which the actual math function results may
deviate from the expected results and still be considered correct.
The test drivers "test-double-<func>.c", "test-float-<func>.c", and
"test-ldouble-<func>.c", generated by the Makefile, test the normal
double, float and long double implementation of libm. The test
drivers with an 'i' in their name ("test-idouble-<func>.c",
"test-ifloat-<func>.c", and "test-ildoubl-<func>.c") test the
corresponding inline functions (where available - otherwise they also
test the real functions in libm). Each driver selects the desired
real floating type to exercise the math functions to test with (float,
double, or long double) by defining a small set of macros just before
including the generic "libm-test.c" file. Each driver also either
defines or undefines the __NO_MATH_INLINES macro just before including
"libm-test-<func>.c" to select either the real or inline functions,
respectively. Each driver is compiled into a single executable test
program with the corresponding name.
As mentioned above, the "gen-libm-test.pl" script looks for a file
named "libm-test-ulps" in the platform specific sysdep directory (or
its fpu or nofpu subdirectory) and for each variant (real floating
type and rounding mode) of every tested function reads from it the
maximum difference expressed as Units of Least Precision (ULP) the
actual result of the function may deviate from the expected result
before it's considered incorrect.
The "auto-libm-test-out-<func>" files contain sets of test cases to
exercise, the conditions under which to exercise each, and the
expected results. The files are generated by the
"gen-auto-libm-tests" program from the "auto-libm-test-in" file. See
the comments in gen-auto-libm-tests.c for details about the content
and format of the -in and -out files.
How can I generate "libm-test-ulps"?
====================================
To automatically generate a new "libm-test-ulps" run "make regen-ulps".
This generates the file "math/NewUlps" in the build directory. The file
contains the sorted results of all the tests. You can use the "NewUlps"
file as the machine's updated "libm-test-ulps" file. Copy "NewUlps" to
"libm-test-ulps" in the appropriate machine sysdep directory. Verify
the changes, post your patch, and check it in after review.
To manually generate a new "libm-test-ulps" file, first remove "ULPs"
file in the current directory, then you can execute for example:
./testrun.sh math/test-double -u --ignore-max-ulp=yes
This generates a file "ULPs" with all double ULPs in it, ignoring any
previously calculated ULPs, and running with the newly built dynamic
loader and math library (assumes you didn't install your build). Now
generate the ULPs for all other formats, the tests will be appending the
data to the "ULPs" file. As final step run "gen-libm-test.pl" with the
file as input and ask to generate a pretty printed output in the file
"NewUlps":
gen-libm-test.pl -u ULPs -n NewUlps
Copy "NewUlps" to "libm-test-ulps" in the appropriate machine sysdep
directory.
Note that the test drivers have an option "-u" to output an unsorted
list of all epsilons that the functions have. The output can be read
in directly but it's better to pretty print it first.
"gen-libm-test.pl" has an option to generate a pretty-printed and
sorted new ULPs file from the output of the test drivers.
Contents of libm-test-ulps
==========================
Since libm-test-ulps can be generated automatically, just a few notes.
The file contains lines for maximal errors of single functions, like:
Function "yn":
idouble: 6
The keywords are float, ifloat, double, idouble, ldouble and ildouble
(the prefix i stands for inline).
Adding tests to libm-test-<func>.inc
====================================
The tests are evaluated by a set of special test macros. The macros
start with "TEST_" followed by a specification the input values, an
underscore and a specification of the output values. As an example,
the test macro for a function with input of type FLOAT (FLOAT is
either float, double, long double) and output of type FLOAT is
"TEST_f_f". The macro's parameter are the name of the function, the
input parameter, output parameter and optionally one exception
parameter.
The accepted parameter types are:
- "f" for FLOAT
- "j" for long double.
- "a" for ARG_FLOAT, the argument type for narrowing functions.
- "b" for boolean - just tests if the output parameter evaluates to 0
or 1 (only for output).
- "c" for complex. This parameter needs two values, first the real,
then the imaginary part.
- "i" for int.
- "l" for long int.
- "L" for long long int.
- "u" for unsigned int.
- "M" for intmax_t.
- "U" for uintmax_t.
- "p" for an argument (described in the previous character) passed
through a pointer rather than directly.
- "F" for the address of a FLOAT (only as input parameter)
- "I" for the address of an int (only as input parameter)
- "1" for an additional output (either output through a pointer passed
as an argument, or to a global variable such as signgam).
How to read the test output
===========================
Running each test on its own at the default level of verbosity will
print on stdout a line describing the implementation of math functions
exercised by the test (float, double, or long double), along with
whether the inline set has been selected, regardless of whether or
not any inline functions actually exist. This is then followed by
the details of test failures (if any). The output concludes by
a summary listing the number of test cases exercised and the number
of test failures uncovered.
For each test failure (and for each test case at higher levels of
verbosity), the output contains the name of the function under test
and its arguments or conditions that triggered the failure. Note
that the name of the function in the output need not correspond
exactly to the name of the math function actually invoked. For example,
the output will refer to the "acos" function even if the actual function
under test is acosf (for the float version) or acosl (for the long
double version). Also note that the function arguments may be shown
in either the decimal or the hexadecimal floating point format which
may or may not correspond to the format used in the auto-libm-test-in
file. Besides the name of the function, for each test failure the
output contains the actual and expected results and the difference
between the two, printed in both the decimal and hexadecimal
floating point format, and the ULP and maximum ULP for the test
case.
7c67e6e8b9