From a8107a983fb78418329a23574f8b915b7d909f04 Mon Sep 17 00:00:00 2001 From: Jonathan Wakely Date: Thu, 18 Aug 2016 14:47:28 +0100 Subject: [PATCH] Improve markup in libstdc++ manual * doc/xml/manual/build_hacking.xml: Improve markup. * doc/xml/manual/test.xml: Likewise. Change section title from "Test" to "Testing". * doc/xml/faq.xml: Change link text to "Testing". From-SVN: r239571 --- libstdc++-v3/ChangeLog | 5 + libstdc++-v3/doc/xml/faq.xml | 2 +- libstdc++-v3/doc/xml/manual/build_hacking.xml | 118 +++--- libstdc++-v3/doc/xml/manual/test.xml | 363 ++++++++++-------- 4 files changed, 266 insertions(+), 222 deletions(-) diff --git a/libstdc++-v3/ChangeLog b/libstdc++-v3/ChangeLog index 281ee981c9f..a65d040e8c2 100644 --- a/libstdc++-v3/ChangeLog +++ b/libstdc++-v3/ChangeLog @@ -1,5 +1,10 @@ 2016-08-18 Jonathan Wakely + * doc/xml/manual/build_hacking.xml: Improve markup. + * doc/xml/manual/test.xml: Likewise. Change section title from "Test" + to "Testing". + * doc/xml/faq.xml: Change link text to "Testing". + * testsuite/tr1/3_function_objects/function/10.cc: Remove unintended dg-options directive. diff --git a/libstdc++-v3/doc/xml/faq.xml b/libstdc++-v3/doc/xml/faq.xml index b24ee228894..94662314928 100644 --- a/libstdc++-v3/doc/xml/faq.xml +++ b/libstdc++-v3/doc/xml/faq.xml @@ -328,7 +328,7 @@ performance testing. Please consult the testing documentation for GCC and - Test in the libstdc++ + Testing in the libstdc++ manual for more details. diff --git a/libstdc++-v3/doc/xml/manual/build_hacking.xml b/libstdc++-v3/doc/xml/manual/build_hacking.xml index 1b789d32028..0bcd8791c6f 100644 --- a/libstdc++-v3/doc/xml/manual/build_hacking.xml +++ b/libstdc++-v3/doc/xml/manual/build_hacking.xml @@ -93,7 +93,7 @@ in the build directory starts the build process. The all targ Regenerate all generated files by using the command - autoreconf at the top level of the libstdc++ source + autoreconf at the top level of the libstdc++ source directory. @@ -108,10 +108,10 @@ in the build directory starts the build process. The all targ - Until that glorious day when we can use AC_TRY_LINK with a - cross-compiler, we have to hardcode the results of what the tests + Until that glorious day when we can use AC_TRY_LINK + with a cross-compiler, we have to hardcode the results of what the tests would have shown if they could be run. So we have an inflexible - mess like crossconfig.m4. + mess like crossconfig.m4. @@ -119,8 +119,8 @@ in the build directory starts the build process. The all targ like configure.host, which can be modified without needing to regenerate anything, and can even be tweaked without really knowing how the configury all works? Perhaps break the pieces of - crossconfig.m4 out and place them in their appropriate - config/{cpu,os} directory. + crossconfig.m4 out and place them in their appropriate + config/{cpu,os} directory. @@ -141,33 +141,37 @@ in the build directory starts the build process. The all targ Most comments should use {octothorpes, shibboleths, hash marks, - pound signs, whatever} rather than "dnl". Nearly all comments in - configure.ac should. Comments inside macros written in ancillary - .m4 files should. About the only comments which should - not use #, but use dnl instead, are comments - outside our own macros in the ancillary - files. The difference is that # comments show up in - configure (which is most helpful for debugging), - while dnl'd lines just vanish. Since the macros in ancillary - files generate code which appears in odd places, their "outside" - comments tend to not be useful while reading - configure. + pound signs, whatever} rather than "dnl". + Nearly all comments in configure.ac should. + Comments inside macros written in ancillary + .m4 files should. + About the only comments which should not + use #, but use dnl instead, + are comments outside our own macros in the ancillary + files. The difference is that # comments show up in + configure (which is most helpful for debugging), + while dnl'd lines just vanish. Since the macros + in ancillary files generate code which appears in odd places, + their "outside" comments tend to not be useful while reading + configure. Do not use any $target* variables, such as - $target_alias. The single exception is in - configure.ac, for automake+dejagnu's sake. + $target_alias. The single exception is in + configure.ac, for automake+dejagnu's sake.
The acinclude.m4 layout - The nice thing about acinclude.m4/aclocal.m4 is that macros aren't + The nice thing about + acinclude.m4/aclocal.m4 + is that macros aren't actually performed/called/expanded/whatever here, just loaded. So we can arrange the contents however we like. As of this writing, - acinclude.m4 is arranged as follows: + acinclude.m4 is arranged as follows: GLIBCXX_CHECK_HOST @@ -175,7 +179,8 @@ in the build directory starts the build process. The all targ GLIBCXX_CONFIGURE - All the major variable "discovery" is done here. CXX, multilibs, + All the major variable "discovery" is done here. + CXX, multilibs, etc. @@ -263,7 +268,8 @@ in the build directory starts the build process. The all targ - Builds the call to the AC_ARG_ENABLE macro, with --help text + Builds the call to the AC_ARG_ENABLE macro, with + text properly quoted and aligned. (Death to changequote!) @@ -305,21 +311,25 @@ in the build directory starts the build process. The all targ - FEATURE is the string that follows --enable. The results of the - test (such as it is) will be in the variable $enable_FEATURE, - where FEATURE has been squashed. Example: - [extra-foo], controlled by the --enable-extra-foo - option and stored in $enable_extra_foo. + FEATURE is the string that follows + . The results of the + test (such as it is) will be in the variable + $enable_FEATURE, + where FEATURE has been squashed. Example: + [extra-foo], controlled by the + + option and stored in $enable_extra_foo. - DEFAULT is the value to store in $enable_FEATURE if the user does - not pass --enable/--disable. It should be one of the permitted - values passed later. Examples: [yes], or - [bar], or [$1] (which passes the - argument given to the GLIBCXX_ENABLE_FOO macro - as the default). + DEFAULT is the value to store in + $enable_FEATURE if the user does + not pass /. + It should be one of the permitted values passed later. + Examples: [yes], or [bar], or + [$1] (which passes the argument given to the + GLIBCXX_ENABLE_FOO macro as the default). For cases where we need to probe for particular models of things, @@ -329,11 +339,11 @@ in the build directory starts the build process. The all targ - HELP-ARG is any text to append to the option string itself in the - --help output. Examples: [] (i.e., an empty string, - which appends nothing), [=BAR], which produces - --enable-extra-foo=BAR, and - [@<:@=BAR@:>@], which produces + HELP-ARG is any text to append to the option string + itself in the output. Examples: + [] (i.e., an empty string, which appends nothing), + [=BAR], which produces --enable-extra-foo=BAR, + and [@<:@=BAR@:>@], which produces --enable-extra-foo[=BAR]. See the difference? See what it implies to the user? @@ -345,18 +355,19 @@ in the build directory starts the build process. The all targ - HELP-STRING is what you think it is. Do not include the + HELP-STRING is what you think it is. Do not include the "default" text like we used to do; it will be done for you by - GLIBCXX_ENABLE. By convention, these are not full English - sentences. Example: [turn on extra foo] + GLIBCXX_ENABLE. By convention, these are not full English + sentences. Example: [turn on extra foo] With no other arguments, only the standard autoconf patterns are - allowed: "--{enable,disable}-foo[={yes,no}]" The - $enable_FEATURE variable is guaranteed to equal either "yes" or "no" + allowed: "" The + $enable_FEATURE variable is guaranteed to equal + either "yes" or "no" after the macro. If the user tries to pass something else, an explanatory error message will be given, and configure will halt. @@ -365,11 +376,14 @@ in the build directory starts the build process. The all targ The second signature takes a fifth argument, "[permit a | b | c | ...]" This allows a or b or - ... after the equals sign in the option, and $enable_FEATURE is + ... after the equals sign in the option, and + $enable_FEATURE is guaranteed to equal one of them after the macro. Note that if you - want to allow plain --enable/--disable with no "=whatever", you must - include "yes" and "no" in the list of permitted values. Also note - that whatever you passed as DEFAULT must be in the list. If the + want to allow plain / + with no "=whatever", you must + include "yes" and "no" in the + list of permitted values. Also note that whatever you passed as + DEFAULT must be in the list. If the user tries to pass something not on the list, a semi-explanatory error message will be given, and configure will halt. Example: [permit generic|gnu|ieee_1003.1-2001|yes|no|auto] @@ -380,8 +394,8 @@ in the build directory starts the build process. The all targ code to execute if the user actually passes the enable/disable option. (If the user does not, the default is used. Duh.) No argument checking at all is done in this signature. See - GLIBCXX_ENABLE_CXX_FLAGS for an example of handling, and an error - message. + GLIBCXX_ENABLE_CXX_FLAGS for an example of handling, + and an error message.
@@ -430,7 +444,7 @@ in the build directory starts the build process. The all targ Generates a libtool convenience library, libc++98convenience with language-support - routines. Uses the -std=gnu++98 dialect. + routines. Uses the dialect. @@ -440,7 +454,7 @@ in the build directory starts the build process. The all targ Generates a libtool convenience library, libc++11convenience with language-support - routines. Uses the -std=gnu++11 dialect. + routines. Uses the dialect. diff --git a/libstdc++-v3/doc/xml/manual/test.xml b/libstdc++-v3/doc/xml/manual/test.xml index 456c48b2639..740323ab57c 100644 --- a/libstdc++-v3/doc/xml/manual/test.xml +++ b/libstdc++-v3/doc/xml/manual/test.xml @@ -2,7 +2,7 @@ xml:id="manual.intro.setup.test" xreflabel="Testing"> -Test +Testing ISO C++ test @@ -19,66 +19,104 @@ The libstdc++ testsuite includes testing for standard conformance, regressions, ABI, and performance. -
Organization +
Test Organization
Directory Layout - The directory libsrcdir/testsuite contains the - individual test cases organized in sub-directories corresponding to - clauses of the C++ standard (detailed below), the dejagnu test - harness support files, and sources to various testsuite utilities - that are packaged in a separate testing library. + The directory + gccsrcdir/libstdc++-v3/testsuite + contains the individual test cases organized in sub-directories + corresponding to clauses of the C++ standard (detailed below), + the DejaGnu test harness support files, and sources to various + testsuite utilities that are packaged in a separate testing library. All test cases for functionality required by the runtime components of the C++ standard (ISO 14882) are files within the following - directories. - + directories: -17_intro -18_support -19_diagnostics -20_util -21_strings -22_locale -23_containers -25_algorithms -26_numerics -27_io -28_regex -29_atomics -30_threads + 17_intro + 18_support + 19_diagnostics + 20_util + 21_strings + 22_locale + 23_containers + 24_iterators + 25_algorithms + 26_numerics + 27_io + 28_regex + 29_atomics + 30_threads + In addition, the following directories include test files: - - -tr1 Tests for components as described by the Technical Report on Standard Library Extensions (TR1). -backward Tests for backwards compatibility and deprecated features. -demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler -ext Tests for extensions. -performance Tests for performance analysis, and performance regressions. - + + + tr1 + Tests for components as described by the Technical Report + on Standard Library Extensions (TR1). + + + + backward + Tests for backwards compatibility and deprecated features. + + + + demangle + Tests for __cxa_demangle, the IA-64 C++ ABI + demangler. + + + + ext + Tests for extensions. + + + performance + Tests for performance analysis, and performance regressions. + + + + Some directories don't have test files, but instead contain auxiliary information: - - -config Files for the dejagnu test harness. -lib Files for the dejagnu test harness. -libstdc++* Files for the dejagnu test harness. -data Sample text files for testing input and output. -util Files for libtestc++, utilities and testing routines. - + + + config + Files for the DejaGnu test harness. + + + lib + Files for the DejaGnu test harness. + + + libstdc++* + Files for the DejaGnu test harness. + + + data + Sample text files for testing input and output. + + + util + Files for libtestc++, utilities and testing routines. + + + Within a directory that includes test files, there may be @@ -86,13 +124,9 @@ util Files for libtestc++, utilities and testing routines. were appended to one file that represented a particular section of the chapter under test, and was named accordingly. For instance, to test items related to 21.3.6.1 - - basic_string::find [lib.string::find] in the standard, - the following was used: - - -21_strings/find.cc - - + basic_string::find [lib.string::find] + in the standard, the following was used: + 21_strings/find.cc However, that practice soon became a liability as the test cases became huge and unwieldy, and testing new or extended functionality (like wide characters or named locales) became @@ -102,19 +136,17 @@ util Files for libtestc++, utilities and testing routines. above issues and gives finer grained results and more manageable error debugging. As an example, the test case quoted above becomes: + 21_strings/basic_string/find/char/1.cc + 21_strings/basic_string/find/char/2.cc + 21_strings/basic_string/find/char/3.cc + 21_strings/basic_string/find/wchar_t/1.cc + 21_strings/basic_string/find/wchar_t/2.cc + 21_strings/basic_string/find/wchar_t/3.cc - -21_strings/basic_string/find/char/1.cc -21_strings/basic_string/find/char/2.cc -21_strings/basic_string/find/char/3.cc -21_strings/basic_string/find/wchar_t/1.cc -21_strings/basic_string/find/wchar_t/2.cc -21_strings/basic_string/find/wchar_t/3.cc - - All new tests should be written with the policy of one test - case, one file in mind. + All new tests should be written with the policy of "one test + case, one file" in mind.
@@ -128,87 +160,72 @@ util Files for libtestc++, utilities and testing routines. tests. - - - - _xin.cc - - + + + _xin.cc + This test case expects some kind of interactive input in order to finish or pass. At the moment, the interactive tests are not run by default. Instead, they are run by hand, like: - g++ 27_io/objects/char/3_xin.cc -cat 27_io/objects/char/3_xin.in | a.out - - - - - .in - - +cat 27_io/objects/char/3_xin.in | a.out + + + + .in + This file contains the expected input for the corresponding _xin.cc test case. - - - - - _neg.cc - - + + + + _neg.cc + This test case is expected to fail: it's a negative test. At the moment, these are almost always compile time errors. - - - - - char - - + + + + char + This can either be a directory name or part of a longer file name, and indicates that this file, or the files within this directory are testing the char instantiation of a template. - - - - - wchar_t - - + + + + wchar_t + This can either be a directory name or part of a longer file name, and indicates that this file, or the files within this directory are testing the wchar_t instantiation of a template. Some hosts do not support wchar_t functionality, so for these targets, all of these tests will not be run. - - - - - thread - - + + + + thread + This can either be a directory name or part of a longer file name, and indicates that this file, or the files within this directory are testing situations where multiple threads are being used. - - - - - performance - - + + + + performance + This can either be an enclosing directory name or part of a specific file name. This indicates a test that is used to analyze runtime performance, for performance regression testing, or for other optimization related analysis. At the moment, these test cases are not run by default. - - - + + +
@@ -222,24 +239,29 @@ cat 27_io/objects/char/3_xin.in | a.out You can check the status of the build without installing it - using the dejagnu harness, much like the rest of the gcc - tools. - make check - in the libbuilddir directory. - or - make check-target-libstdc++-v3 - in the gccbuilddir directory. - + using the DejaGnu harness, much like the rest of the gcc + tools, i.e. + make check + in the + libbuilddir + directory, or + make check-target-libstdc++-v3 + in the + gccbuilddir + directory. + These commands are functionally equivalent and will create a - 'testsuite' directory underneath - libbuilddir containing the results of the - tests. Two results files will be generated: - libstdc++.sum, which is a PASS/FAIL summary for each - test, and libstdc++.log which is a log of - the exact command line passed to the compiler, the compiler - output, and the executable output (if any). + 'testsuite' directory underneath + libbuilddir + containing the results of the + tests. Two results files will be generated: + libstdc++.sum, which is a PASS/FAIL summary + for each test, and + libstdc++.log which is a log of + the exact command-line passed to the compiler, the compiler + output, and the executable output (if any) for each test. @@ -262,88 +284,88 @@ cat 27_io/objects/char/3_xin.in | a.out checking the exported symbols of the shared library. - To debug the dejagnu test harness during runs, try invoking with a - specific argument to the variable RUNTESTFLAGS, as below. - - + To debug the DejaGnu test harness during runs, try invoking with a + specific argument to the variable RUNTESTFLAGS, + like so: -make check-target-libstdc++-v3 RUNTESTFLAGS="-v" + make check-target-libstdc++-v3 RUNTESTFLAGS="-v" - - or - - -make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v" + make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v" + To run a subset of the library tests, you can either generate the - testsuite_files file (described below) by running - make testsuite_files in the - libbuilddir/testsuite directory, then edit the + testsuite_files file (described below) by running + make testsuite_files in the + libbuilddir/testsuite + directory, then edit the file to remove the tests you don't want and then run the testsuite as normal, or you can specify a testsuite and a subset of tests in the - RUNTESTFLAGS variable. + RUNTESTFLAGS variable. For example, to run only the tests for containers you could use: - -make check-target-libstdc++-v3 RUNTESTFLAGS="conformance.exp=23_containers/*" + make check-target-libstdc++-v3 RUNTESTFLAGS="conformance.exp=23_containers/*" + - When combining this with other options in RUNTESTFLAGS the - testsuite.exp=testfiles options must come first. + When combining this with other options in RUNTESTFLAGS + the options must come first. - There are two ways to run on a simulator: set up DEJAGNU to point to a - specially crafted site.exp, or pass down --target_board flags. + There are two ways to run on a simulator: set up DEJAGNU + to point to a specially crafted site.exp, + or pass down flags. Example flags to pass down for various embedded builds are as follows: - - --target=powerpc-eabism (libgloss/sim) -make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim" + --target=powerpc-eabisim (libgloss/sim) + make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim" ---target=calmrisc32 (libgloss/sid) -make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid" + --target=calmrisc32 (libgloss/sid) + make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid" ---target=xscale-elf (newlib/sim) -make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim" + --target=xscale-elf (newlib/sim) + make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim" + Also, here is an example of how to run the libstdc++ testsuite for a multilibed build directory with different ABI settings: - -make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"' + make check-target-libstdc++-v3 RUNTESTFLAGS='--target_board \"unix{-mabi=32,,-mabi=64}\"' + You can run the tests with a compiler and library that have already been installed. Make sure that the compiler (e.g., - g++) is in your PATH. If you are + g++) is in your PATH. If you are using shared libraries, then you must also ensure that the directory containing the shared version of libstdc++ is in your - LD_LIBRARY_PATH, or equivalent. If your GCC source - tree is at /path/to/gcc, then you can run the tests - as follows: - + LD_LIBRARY_PATH, or + equivalent. + If your GCC source tree is at + /path/to/gcc, + then you can run the tests as follows: -runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite + runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite + The testsuite will create a number of files in the directory in @@ -356,12 +378,13 @@ runtest --tool libstdc++ --srcdir=/path/to/gcc/libstdc++-v3/testsuite In addition, there are some testing options that are mostly of interest to library maintainers and system integrators. As such, - these tests may not work on all cpu and host combinations, and + these tests may not work on all CPU and host combinations, and may need to be executed in the - libbuilddir/testsuite directory. These + libbuilddir/testsuite + directory. These options include, but are not necessarily limited to, the following: - + make testsuite_files @@ -626,7 +649,8 @@ up in the normal.exp file. - More examples can be found in the libstdc++-v3/testsuite/*/*.cc files. + More examples can be found in the + libstdc++-v3/testsuite/*/*.cc files. @@ -634,22 +658,23 @@ up in the normal.exp file.
Test Harness and Utilities -
Dejagnu Harness Details +
DejaGnu Harness Details Underlying details of testing for conformance and regressions are - abstracted via the GNU Dejagnu package. This is similar to the + abstracted via the GNU DejaGnu package. This is similar to the rest of GCC. This is information for those looking at making changes to the testsuite -structure, and/or needing to trace dejagnu's actions with --verbose. This -will not be useful to people who are "merely" adding new tests to the existing -structure. +structure, and/or needing to trace DejaGnu's actions with +. +This will not be useful to people who are "merely" adding new tests +to the existing structure. -The first key point when working with dejagnu is the idea of a "tool". +The first key point when working with DejaGnu is the idea of a "tool". Files, directories, and functions are all implicitly used when they are named after the tool in use. Here, the tool will always be "libstdc++". @@ -665,7 +690,7 @@ our own. Callbacks must be prefixed with the name of the tool. To easily distinguish the others, by convention our own routines are named "v3-*". -The next key point when working with dejagnu is "test files". Any +The next key point when working with DejaGnu is "test files". Any directory whose name starts with the tool name will be searched for test files. (We have only one.) In those directories, any .exp file is considered a test file, and will be run in turn. Our main test file is called -- 2.30.2