+2003-05-20 Benjamin Kosnik <bkoz@redhat.com>
+
+ * testsuite/README: Move to...
+ * docs/html/test.html: ...here. Add documentation.
+ * docs/html/install.html: Move testing bits out..
+ * docs/html/documentation.html: Add separate testing link.
+ * testsuite/performance: Add.
+ * testsuite/performance/allocator.cc: New.
+ * testsuite/performance/complex_norm.cc: New.
+ * testsuite/performance/cout_insert_int.cc: New.
+ * testsuite/performance/fstream_seek_write.cc: New.
+ * testsuite/performance/ifstream_getline.cc: New.
+ * testsuite/performance/map_create_fill.cc: New.
+ * testsuite/performance/ofstream_insert_float.cc: New.
+ * testsuite/performance/ofstream_insert_int.cc: New.
+ * testsuite/performance/string_append.cc: New.
+ * testsuite/lib/libstdc++-v3-dg.exp (v3-compute-tests): Filter
+ performance tests.
+
2003-05-20 Gabriel Dos Reis <gdr@integrable-solutions.net>
PR libstdc++/10689
<hr />
<br />
-<h2><a name="2">Configuring, Building, Installing</a></h2>
+<h2><a name="2">Configuring, Building, Testing, Installing</a></h2>
<ul>
<li><a href="configopts.html">Configure options</a></li>
<li><a href="install.html">Getting started: configure, build, install</a>
</li>
+ <li><a href="test.html">Testing details</a>
<li><a href="debug.html">Debugging schemes and strategies</a>
</li>
</ul>
</head>
<body>
-<h1 class="centered"><a name="top">libstdc++-v3 INSTALL</a></h1>
+<h1 class="centered"><a name="top">Getting started: configure, build, install</a></h1>
<p class="fineprint"><em>
The latest version of this document is always available at
the headers and library files will be moved under
<code>lib/gcc-lib/</code> instead.
</p>
- <p>You can check the status of the build without installing it using</p>
- <pre>
- make check</pre>
- <p>or you can check the status of the installed library using</p>
- <pre>
- make check-install</pre>
- <p>in the <em>libbuilddir</em> directory.
- These commands will create a 'testsuite' directory underneath
- <em>libbuilddir</em> containing the results of the tests. We are
- interested in any strange failures of the testsuite; please see
- <a href="faq/index.html#2_4">FAQ 2.4</a> for which files to examine.
- </p>
-
- <p> In addition, there are some testing options that are mostly of
- interest to library maintainers and system integrators. As such,
- these tests may not work on all cpu and host combinations. These
- options include, but are not necessarily limited to, the following:
- </p>
-
- <p>The library ABI can be tested using</p>
- <pre>
- make check-abi</pre>
-
- <p>The library can also be tested using a bash script, instead of
- the default dejagnu test harness</p>
- <pre>
- make check-script</pre>
- <p>or</p>
- <pre>
- make check-script-install</pre>
<hr />
<h2><a name="usage">Using the library</a></h2>
--- /dev/null
+<?xml version="1.0" encoding="ISO-8859-1"?>
+<!DOCTYPE html
+ PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
+
+<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
+<head>
+ <meta name="AUTHOR" content="bkoz@gcc.gnu.org (Benjamin Kosnik)" />
+ <meta name="KEYWORDS" content="c++, libstdc++, test, regression, g++" />
+ <meta name="DESCRIPTION" content="README for the GNU libstdc++ effort." />
+ <meta name="GENERATOR" content="vi and eight fingers" />
+ <title>libstdc++-v3 Testing Instructions</title>
+<link rel="StyleSheet" href="lib3styles.css" />
+</head>
+<body>
+
+<h1 class="centered"><a name="top">Testing Details</a></h1>
+
+<p class="fineprint"><em>
+ The latest version of this document is always available at
+ <a href="http://gcc.gnu.org/onlinedocs/libstdc++/test.html">
+ http://gcc.gnu.org/onlinedocs/libstdc++/test.html</a>.
+</em></p>
+
+<p><em>
+ To the <a href="http://gcc.gnu.org/libstdc++/">libstdc++-v3 homepage</a>.
+</em></p>
+
+<!-- ####################################################### -->
+<hr />
+<h2>Contents</h2>
+<ul>
+ <li><a href="#org">Testsuite organization and naming conventions</a></li>
+ <li><a href="#util">Utilities: abicheck and libv3test</a></li>
+ <li><a href="#new">How to write a new test case</a></li>
+ <li><a href="#check">Options for running the tests</a></li>
+ <li><a href="#future">Future</a></li>
+</ul>
+
+<hr />
+
+<!-- ####################################################### -->
+
+<h2><a name="org">Testsuite organization and naming conventions</a></h2>
+ <p>
+ The directory <em>libsrcdir/testsuite</em> contains the test
+ files, test harness, and utility information for verifying the
+ correctness of C++ library on a given host. It includes the
+ following directories, each named after a specific chapter of
+ the C++ standard, and each containing test files or
+ subdirectories of test files that test for that particular part
+ of the standard.
+ <p>
+
+ <pre>
+17_intro
+18_support
+19_diagnostics
+20_util
+21_strings
+22_locale
+23_containers
+25_algorithms
+26_numerics
+27_io
+ </pre>
+
+ <p>
+ In addition, the following directories include test files:
+ </p>
+
+ <pre>
+backward Tests for backwards compatibility and deprecated features.
+demangle Tests for __cxa_demangle, the IA 64 C++ ABI demangler
+ext Tests for extensions.
+performance Tests for performance analysis, and performance regressions.
+thread Tests for threads.
+ </pre>
+
+ <p>
+ Some directories don't have test files, but instead contain
+ auxiliary information:
+ </p>
+
+ <pre>
+config Files for the dejagnu test harness.
+lib Files for the dejagnu test harness.
+libstdc++-v3.dg Files for the dejagnu test harness.
+data Sample text files for testing input and output.
+ </pre>
+
+ <p>
+ Within a directory that includes test files, there may be
+ additional subdirectories, or files: this particular point is in
+ flux. Originally, test cases were appended to one file that
+ represented a particular section of the chapter under test, and
+ was named accordingly. For instance, to test items related to
+ <code> 21.3.6.1 - basic_string::find [lib.string::find]</code>
+ in the standard, the following was used:
+ <p>
+ <pre>
+21_strings/find.cc
+ </pre>
+ <p>
+ However, that practice soon became a liability as the test cases
+ became huge and unwieldy, and testing new or extended
+ functionality (like wide characters or named locales) became
+ frustrating, leading to aggressive pruning of test cases on some
+ platforms that covered up implementation errors. Now, the test
+ suite is converging on a policy of one file, one test case,
+ which solves the above issues and gives finer grained results
+ and more manageable error debugging. As an example, the test case
+ quoted above becomes:
+ </p>
+ <pre>
+21_strings/basic_string/find/char/1.cc
+21_strings/basic_string/find/char/2.cc
+21_strings/basic_string/find/char/3.cc
+21_strings/basic_string/find/wchar_t/1.cc
+21_strings/basic_string/find/wchar_t/2.cc
+21_strings/basic_string/find/wchar_t/3.cc
+ </pre>
+
+ <p>
+ All new tests should be written with the policy of one test
+ case, one file in mind. At some point the entire testsuite will
+ be converted: the current status is that the 21_string,
+ 22_locale, 27_io, and demangle directories have all been
+ transitioned.
+ </p>
+
+ <p>
+ In addition, there are some special names and suffixes that are
+ used within the testsuite to designate particular kinds of
+ tests.
+ </p>
+
+<ul>
+<li>
+ <em>_xin.cc</em>
+ <p>
+ This test case expects some kind of interactive input in order
+ to finish or pass. At the moment, the interactive tests are not
+ run by default. Instead, they are run by hand, like:
+ <pre>
+g++ 27_io/objects/char/3_xin.cc
+cat 27_io/objects/char/3_xin.in | a.out
+ </pre>
+ </p>
+</li>
+<li>
+ <em>.in</em>
+ <p>
+ This file contains the expected input for the corresponding <em>
+ _xin.cc</em> test case.
+ </p>
+</li>
+<li>
+ <em>_neg.cc</em>
+ <p>
+ This test case is expected to fail: it's a negative test. At the
+ moment, these are almost always compile time errors.
+ </p>
+</li>
+<li>
+ <em>char</em>
+ <p>
+ This can either be a directory name or part of a longer file
+ name, and indicates that this file, or the files within this
+ directory are testing the <code>char</code> instantiation of a
+ template.
+ </p>
+</li>
+<li>
+ <em>wchar_t</em>
+ <p>
+ This can either be a directory name or part of a longer file
+ name, and indicates that this file, or the files within this
+ directory are testing the <code>wchar_t</code> instantiation of
+ a template. Some hosts do not support <code>wchar_t</code>
+ functionality, so for these targets, all of these tests will not
+ be run.
+ </p>
+</li>
+<li>
+ <em>performance</em>
+ <p>
+ This can either be an enclosing directory name or part of a
+ specific file name. This indicates a test that is used to
+ analyze runtime performance, for performance regression testing,
+ or for other optimization related analysis. At the moment, these
+ test cases are not run by default, and instead assumed to be run
+ manually.
+ </p>
+</li>
+</ul>
+
+<hr />
+<h2><a name="util">Utilities: abicheck and libv3test</a></h2>
+ <p>
+ The testsuite directory also contains some files that implement
+ functionality that is intended to make writing test cases easier,
+ or to avoid duplication, or to provide error checking in a way that
+ is consistent across platforms and test harnesses. A stand-alone
+ executable, called <em>abi_check</em>, and a static library called
+ <em>libv3test</em> are constructed during the build. Both of these
+ items are not installed, and only used during testing.
+ </p>
+
+ <p>
+ These files include the following functionality:
+ </p>
+
+ <ul>
+ <li>
+ <em>abi_check.cc</em>
+ <p>
+ Creates the executable <em>abi_check</em>.
+ Used to check correctness of symbol versioning, visibility of
+ exported symbols, and compatibility on symbols in the shared
+ library, for hosts that support this feature. More information
+ can be found in the ABI documentation <a href="abi.txt"> here</a>
+ </p>
+ </li>
+ <li>
+ <em>testsuite_allocator.h and </em>
+ <em>testsuite_allocator.cc</em>
+ <p>
+ Specialized allocators that keep track of construction and destruction
+ </p>
+ </li>
+ <li>
+ <em>testsuite_hooks.h and </em>
+ <em>testsuite_hooks.cc</em>
+ <p>
+ A large number of utilities, including:
+ </p>
+ <ul>
+ <li>VERIFY</li>
+ <li>set_memory_limits</li>
+ <li>verify_demangle</li>
+ <li>run_tests_wrapped_locale</li>
+ <li>run_tests_wrapped_env</li>
+ <li>try_named_locale</li>
+ <li>counter</li>
+ <li>copy_constructor</li>
+ <li>assignment_operator</li>
+ <li>destructor</li>
+ <li>copy_tracker</li>
+ <li>pod_char, pod_int and associated char_traits specializations</li>
+ </ul>
+ <p></p>
+ </li>
+ <li>
+ <em>printnow.c</em>
+ <p>
+ A cross-platform timer for use in one of the older harnesses
+ to determine compilation and link time.
+ </p>
+ </li>
+ </ul>
+
+<hr />
+<h2><a name="new">How to write a new test case</a></h2>
+
+ <p>
+ The first step in making a new test case is to choose the correct
+ directory and file name, given the organization as previously
+ described.
+ </p>
+
+ <p>
+ All files are copyright the FSF, and GPL'd: this is very
+ important. The first copyright year should correspond to the date
+ the file was checked in to CVS.
+ </p>
+
+ <p>
+ As per the dejagnu instructions, always return 0 from main to
+ indicate success.
+ </p>
+
+ <p>
+ A bunch of utility functions and classes have already been
+ abstracted out into the testsuite utility library, <code>
+ libv3test</code>. To use this functionality, just include the
+ appropriate header file: the library will automatically be linked
+ in as part of the testsuite run.
+ </p>
+
+ <p>
+ For a test that needs to take advantage of the dejagnu test
+ harness, what follows below is a list of special keyword that
+ harness uses. Basically, a test case contains dg-keywords (see
+ dg.exp) indicating what to do and what kinds of behavior are to be
+ expected. New test cases should be written with the new style
+ DejaGnu framework in mind.
+ </p>
+
+ <p>
+ To ease transition, here is the list of dg-keyword documentation
+ lifted from dg.exp.
+ </p>
+
+<pre>
+# The currently supported options are:
+#
+# dg-prms-id N
+# set prms_id to N
+#
+# dg-options "options ..." [{ target selector }]
+# specify special options to pass to the tool (eg: compiler)
+#
+# dg-do do-what-keyword [{ target/xfail selector }]
+# `do-what-keyword' is tool specific and is passed unchanged to
+# ${tool}-dg-test. An example is gcc where `keyword' can be any of:
+# preprocess|compile|assemble|link|run
+# and will do one of: produce a .i, produce a .s, produce a .o,
+# produce an a.out, or produce an a.out and run it (the default is
+# compile).
+#
+# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]]
+# indicate an error message <regexp> is expected on this line
+# (the test fails if it doesn't occur)
+# Linenum=0 for general tool messages (eg: -V arg missing).
+# "." means the current line.
+#
+# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]]
+# indicate a warning message <regexp> is expected on this line
+# (the test fails if it doesn't occur)
+#
+# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]]
+# indicate a bogus error message <regexp> use to occur here
+# (the test fails if it does occur)
+#
+# dg-build regexp comment [{ target/xfail selector }]
+# indicate the build use to fail for some reason
+# (errors covered here include bad assembler generated, tool crashes,
+# and link failures)
+# (the test fails if it does occur)
+#
+# dg-excess-errors comment [{ target/xfail selector }]
+# indicate excess errors are expected (any line)
+# (this should only be used sparingly and temporarily)
+#
+# dg-output regexp [{ target selector }]
+# indicate the expected output of the program is <regexp>
+# (there may be multiple occurrences of this, they are concatenated)
+#
+# dg-final { tcl code }
+# add some tcl code to be run at the end
+# (there may be multiple occurrences of this, they are concatenated)
+# (unbalanced braces must be \-escaped)
+#
+# "{ target selector }" is a list of expressions that determine whether the
+# test succeeds or fails for a particular target, or in some cases whether the
+# option applies for a particular target. If the case of `dg-do' it specifies
+# whether the test case is even attempted on the specified target.
+#
+# The target selector is always optional. The format is one of:
+#
+# { xfail *-*-* ... } - the test is expected to fail for the given targets
+# { target *-*-* ... } - the option only applies to the given targets
+#
+# At least one target must be specified, use *-*-* for "all targets".
+# At present it is not possible to specify both `xfail' and `target'.
+# "native" may be used in place of "*-*-*".
+
+Example 1: Testing compilation only
+// { dg-do compile }
+
+Example 2: Testing for expected warnings on line 36, which all targets fail
+// { dg-warning "string literals" "" { xfail *-*-* } 36
+
+Example 3: Testing for expected warnings on line 36
+// { dg-warning "string literals" "" { target *-*-* } 36
+
+Example 4: Testing for compilation errors on line 41
+// { dg-do compile }
+// { dg-error "no match for" "" { target *-*-* } 41 }
+</pre>
+
+ <p>
+ More examples can be found in the libstdc++-v3/testsuite/*/*.cc files.
+ </p>
+
+<hr />
+<h2><a name="check">Options for running the tests</a></h2>
+
+ <p> There are several ways to run the testsuite. There are two
+ harnesses, one using dejagnu and one using bash. In addition, there
+ is a special rule for checking the ABI of the shared library.
+ </p>
+
+ <p>You can check the status of the build without installing it
+ using the dejagnu harness, much like the rest of the gcc tools.</p>
+ <p>
+ <pre> make check</pre> in the <em>libbuilddir</em> directory.</p>
+ <p>or</p>
+ <p><pre> make check-target-libstdc++-v3</pre> in the
+ <em>gccbuilddir</em> directory.</p>
+
+ <p>
+ These commands are equivalent and will create a 'testsuite'
+ directory underneath <em>libbuilddir</em> containing the results
+ of the tests. Two results files will be generated: <em>
+ libstdc++-v3.sum</em>, which is a PASS/FAIL summary for each
+ test, and <em>libstdc++.log</em> which is a log of the exact
+ command line passed to the compiler, the compiler output, and
+ the executable output (if any). In addition, four files are
+ generated that determine what test files are run. These files
+ are:
+ </p>
+
+ <ul>
+ <li>
+ <em>testsuite_files </em>
+ <p> This is a list of all the test cases that will be run. Each
+ test case is on a separate line, given with an absolute path
+ from the <em>libsrcdir/testsuite</em> directory.
+ </p>
+ </li>
+
+ <li>
+ <em>testsuite_files_interactive </em>
+ <p> This is a list of all the interactive test cases, using the
+ same format as the file list above. These tests are not run by default.
+ </p>
+ </li>
+
+ <li>
+ <em>testsuite_files_performance</em>
+ <p> This is a list of all the performance test cases, using the
+ same format as the file list above. These tests are not run by default.
+ </p>
+ </li>
+
+ <li>
+ <em>testsuite_wchar_t </em>
+ <p> This file indicates that the host system can run the wchar_t
+ tests, and corresponds to the macro definition <code>
+ _GLIBCPP_USE_WCHAR_T</code> in the file c++config.h.
+ </p>
+ </li>
+ </ul>
+
+<p>
+To debug the dejagnu test harness during runs, try invoking with a
+specific argument to the variable RUNTESTFLAGS, as below.
+</p>
+
+<pre>
+make check-target-libstdc++-v3 RUNTESTFLAGS="-v"
+</pre>
+or
+<pre>
+make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v"
+</pre>
+
+There are two ways to run on a simulator: set up DEJAGNU to point to a
+specially crafted site.exp, or pass down --target_board flags.
+
+Example flags to pass down for various embedded builds are as follows:
+
+<pre>
+--target=powerpc-eabism (libgloss/sim)
+make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim"
+
+--target=calmrisc32 (libgloss/sid)
+make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid"
+
+--target=xscale-elf (newlib/sim)
+make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim"
+</pre>
+
+ <p> In addition, there are some testing options that are mostly of
+ interest to library maintainers and system integrators. As such,
+ these tests may not work on all cpu and host combinations, and must
+ be executed in the <em>libbuilddir</em> directory.</p> These options
+ include, but are not necessarily limited to, the following:
+ </p>
+
+ <p>
+ The library can also be tested using a bash script, instead of
+ the default dejagnu test harness.
+ </p>
+ <pre>
+ make check-script</pre>
+ <p>
+ These commands use the generated test_file lists as above, but
+ run all the tests using both shared and static linking, and in
+ addition provide some additional diffing of expected output
+ files for the input/output tests. (This added diff may or may
+ not be useful or necessary at the moment.) In addition, these
+ tests provide size information for all the generated test cases,
+ so that size data for new compiler or linker features can be
+ collected. At one time timing information was attempted, so that
+ compile speeds, link speeds, etc. could be measured, however at
+ the moment all timing information is currently disabled.
+ </p>
+
+ <pre>
+ make check-script-install</pre>
+ <p> As directly above, but tests an installed library, not the
+ library and compiler in the build tree.
+ </p>
+
+ <pre>
+ make check-abi</pre>
+ <p>The library ABI can be tested. This involves testing the shared
+ library against an ABI-defining previous version.</p>
+
+ <p>
+ We are interested in any strange failures of the
+ testsuite; please see <a href="faq/index.html#2_4">FAQ 2.4</a>
+ for which files to examine.
+ </p>
+
+<hr />
+<h2><a name="future">Future</a></h2>
+
+<p>
+Shared runs need to be implemented, for targets that support shared libraries.
+</p>
+
+<p>
+Diffing of expected output to standard streams needs to be finished off.
+</p>
+
+<p>
+The V3 testing framework supports, or will eventually support,
+additional keywords for the purpose of easing the job of writing
+test cases. All V3-keywords are of the form @xxx@. Currently plans
+for supported keywords include:
+</p>
+
+ @require@ <files>
+ The existence of <files> is essential for the test to complete
+ successfully. For example, a test case foo.C using bar.baz as
+ input file could say
+ // @require@ bar.baz
+ The special variable % stands for the rootname, e.g. the
+ file-name without its `.C' extension. Example of use (taken
+ verbatim from 27_io/filebuf.cc)
+ // @require@ %-*.tst %-*.txt
+
+ @diff@ <first-list> <second-list>
+ After the test case compiles and ran successfully, diff
+ <first-list> against <second-list>, these lists should have the
+ same length. The test fails if diff returns non-zero a pair of
+ files.
+
+<!-- ####################################################### -->
+
+<hr />
+<p class="fineprint"><em>
+See <a href="17_intro/license.html">license.html</a> for copying conditions.
+Comments and suggestions are welcome, and may be sent to
+<a href="mailto:libstdc++@gcc.gnu.org">the libstdc++ mailing list</a>.
+</em></p>
+
+
+</body>
+</html>
+++ /dev/null
-We're in the process of converting the existing testsuite machinery to
-use the new style DejaGnu framework. Eventually, we'll abandon
-../mkcheck.in in favor of this new testsuite framework.
-
-// 1: Thoughts on naming test cases, and structuring them.
-The testsuite directory has been divided into 11 directories, directly
-correlated to the relevant chapters in the standard. For example, the
-directory testsuite/21_strings contains tests related to "Chapter 21,
-Strings library" in the C++ standard.
-
-So, the first step in making a new test case is to choose the correct
-directory. The second item is seeing if a test file exists that tests
-the item in question. Generally, within chapters test files are named
-after the section headings in ISO 14882, the C++ standard. For instance,
-
-21.3.7.9 Inserters and Extractors
-
-Has a related test case:
-21_strings/inserters_extractors.cc
-
-Not so hard. Some time, the words "ctor" and "dtor" are used instead
-of "construct", "constructor", "cons", "destructor", etc. Other than
-that, the naming seems mostly consistent. If the file exists, add a
-test to it. If it does not, then create a new file. All files are
-copyright the FSF, and GPL'd: this is very important.
-
-In addition, some of the locale and io code tests different
-instantiating types: thus, 'char' or 'wchar_t' is appended to the name
-as constructed above.
-
-Also, some test files are negative tests. That is, they are supposed
-to fail (usually this involves making sure some kind of construct gets
-an error when it's compiled.) These test files have 'neg' appended to
-the name as constructed above.
-
-Inside a test file, the plan is to test the relevant parts of the
-standard, and then add specific regressions as additional test
-functions, ie test04() can represent a specific regression noted in
-GNATS. Once test files get unwieldy or too big, then they should be
-broken up into multiple sub-categories, hopefully intelligently named
-after the relevant (and more specific) part of the standard.
-
-
-// 2: How to write a test case, from a dejagnu perspective
-As per the dejagnu instructions, always return 0 from main to indicate
-success.
-
-Basically, a test case contains dg-keywords (see dg.exp) indicating
-what to do and what kinds of behaviour are to be expected. New
-testcases should be written with the new style DejaGnu framework in
-mind.
-
-To ease transition, here is the list of dg-keyword documentation
-lifted from dg.exp -- eventually we should improve DejaGnu
-documentation, but getting checkin account currently demands Pyrrhic
-effort.
-
-# The currently supported options are:
-#
-# dg-prms-id N
-# set prms_id to N
-#
-# dg-options "options ..." [{ target selector }]
-# specify special options to pass to the tool (eg: compiler)
-#
-# dg-do do-what-keyword [{ target/xfail selector }]
-# `do-what-keyword' is tool specific and is passed unchanged to
-# ${tool}-dg-test. An example is gcc where `keyword' can be any of:
-# preprocess|compile|assemble|link|run
-# and will do one of: produce a .i, produce a .s, produce a .o,
-# produce an a.out, or produce an a.out and run it (the default is
-# compile).
-#
-# dg-error regexp comment [{ target/xfail selector } [{.|0|linenum}]]
-# indicate an error message <regexp> is expected on this line
-# (the test fails if it doesn't occur)
-# Linenum=0 for general tool messages (eg: -V arg missing).
-# "." means the current line.
-#
-# dg-warning regexp comment [{ target/xfail selector } [{.|0|linenum}]]
-# indicate a warning message <regexp> is expected on this line
-# (the test fails if it doesn't occur)
-#
-# dg-bogus regexp comment [{ target/xfail selector } [{.|0|linenum}]]
-# indicate a bogus error message <regexp> use to occur here
-# (the test fails if it does occur)
-#
-# dg-build regexp comment [{ target/xfail selector }]
-# indicate the build use to fail for some reason
-# (errors covered here include bad assembler generated, tool crashes,
-# and link failures)
-# (the test fails if it does occur)
-#
-# dg-excess-errors comment [{ target/xfail selector }]
-# indicate excess errors are expected (any line)
-# (this should only be used sparingly and temporarily)
-#
-# dg-output regexp [{ target selector }]
-# indicate the expected output of the program is <regexp>
-# (there may be multiple occurrences of this, they are concatenated)
-#
-# dg-final { tcl code }
-# add some tcl code to be run at the end
-# (there may be multiple occurrences of this, they are concatenated)
-# (unbalanced braces must be \-escaped)
-#
-# "{ target selector }" is a list of expressions that determine whether the
-# test succeeds or fails for a particular target, or in some cases whether the
-# option applies for a particular target. If the case of `dg-do' it specifies
-# whether the testcase is even attempted on the specified target.
-#
-# The target selector is always optional. The format is one of:
-#
-# { xfail *-*-* ... } - the test is expected to fail for the given targets
-# { target *-*-* ... } - the option only applies to the given targets
-#
-# At least one target must be specified, use *-*-* for "all targets".
-# At present it is not possible to specify both `xfail' and `target'.
-# "native" may be used in place of "*-*-*".
-
-Example 1: Testing compilation only
-(to just have a testcase do compile testing, without linking and executing)
-// { dg-do compile }
-
-Example 2: Testing for expected warings on line 36
-// { dg-warning "string literals" "" { xfail *-*-* } 36
-
-Example 3: Testing for compilation errors on line 41
-// { dg-do compile }
-// { dg-error "no match for" "" { xfail *-*-* } 41 }
-
-More examples can be found in the libstdc++-v3/testsuite/*/*.cc files.
-
-
-// 3: Test harness notes, invocation, and debugging.
-Configuring the dejagnu harness to work with libstdc++-v3 in a cross
-compilation environment has been maddening. However, it does work now,
-and on a variety of platforms. Including solaris, linux, and cygwin.
-
-To debug the test harness during runs, try invoking with
-
-make check-target-libstdc++-v3 RUNTESTFLAGS="-v"
-or
-make check-target-libstdc++-v3 RUNTESTFLAGS="-v -v"
-
-There are two ways to run on a simulator: set up DEJAGNU to point to a
-specially crafted site.exp, or pass down --target_board flags.
-
-Example flags to pass down for various embedded builds are as follows:
-
---target=powerpc-eabism (libgloss/sim)
-make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=powerpc-sim"
-
---target=calmrisc32 (libgloss/sid)
-make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=calmrisc32-sid"
-
---target=xscale-elf (newlib/sim)
-make check-target-libstdc++-v3 RUNTESTFLAGS="--target_board=arm-sim"
-
-
-// 4: Future plans, to be done
-Shared runs need to be implemented, for targets that support shared libraries.
-
-Diffing of expected output to standard streams needs to be finished off.
-
-The V3 testing framework supports, or will eventually support,
-additional keywords for the purpose of easing the job of writing
-testcases. All V3-keywords are of the form @xxx@. Currently plans
-for supported keywords include:
-
- @require@ <files>
- The existence of <files> is essential for the test to complete
- successfully. For example, a testcase foo.C using bar.baz as
- input file could say
- // @require@ bar.baz
- The special variable % stands for the rootname, e.g. the
- file-name without its `.C' extension. Example of use (taken
- verbatim from 27_io/filebuf.cc)
- // @require@ %-*.tst %-*.txt
-
- @diff@ <first-list> <second-list>
- After the testcase compiles and ran successfully, diff
- <first-list> against <second-list>, these lists should have the
- same length. The test fails if diff returns non-zero a pair of
- files.
-
-Current testing problems with cygwin-hosted tools:
-
-There are two known problems which I have not addressed. The first is
-that when testing cygwin hosted tools from the unix build dir, it does
-the wrong thing building the wrapper program (testglue.c) because host
-and target are the same in site.exp (host and target are the same from
-the perspective of the target libraries)
-
-Problem number two is a little more annoying. In order for me to make
-v3 testing work on Windows, I had to tell dejagnu to copy over the
-debug_assert.h file to the remote host and then set the includes to
--I./. This is how all the other tests like this are done so I didn't
-think much of it. However, this had some unfortunate results due to
-gcc having a testcase called "limits" and C++ having an include file
-called "limits". The gcc "limits" binary was in the temporary dir
-when the v3 tests were being built. As a result, the gcc "limits"
-binary was being #included rather than the intended one. The only way
-to fix this is to go through the testsuites and make sure binaries are
-deleted on the remote host when testing is done with them. That is a
-lot more work than I want to do so I worked around it by cleaning out
-D:\kermit on compsognathus and rerunning tests.
set tests_file "${outdir}/testsuite_files"
set tests_file_inter "${outdir}/testsuite_files_interactive"
+ set tests_file_perf "${outdir}/testsuite_files_performance"
set sfiles ""
# If there is a testsuite_file, use it.
}
set sfiles ${restwo}
+ # Filter performance analysis tests.
+ # Use special rules to run these tests.
+ set resthree {}
+ set sfiles_perf {}
+ foreach i $sfiles {
+ if [regexp "performance" $i] {
+ verbose "element performance list is $i"
+ lappend sfiles_perf $i
+ } else {
+ verbose "element non-performancet list is $i"
+ lappend resthree $i
+ }
+ }
+ set sfiles ${resthree}
+
# Write out testsuite_files.
set f [open $tests_file "w"]
foreach t $sfiles {
puts $f $t
}
close $f
+
+ # Write out testsuite_files_performance.
+ set f [open $tests_file_perf "w"]
+ foreach t $sfiles_perf {
+ puts $f $t
+ }
+ close $f
}
proc v3-list-tests { filename } {
--- /dev/null
+// Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+// As a special exception, you may use this file as part of a free software
+// library without restriction. Specifically, if other files instantiate
+// templates or use macros or inline functions from this file, or you compile
+// this file and link it with other files to produce an executable, this
+// file does not by itself cause the resulting executable to be covered by
+// the GNU General Public License. This exception does not however
+// invalidate any other reasons why the executable file might be covered by
+// the GNU General Public License.
+
+#include <list>
+
+// Primarily a test of the default allocator.
+// http://gcc.gnu.org/ml/libstdc++/2001-05/msg00105.html
+int main ()
+{
+ std::list<int> List;
+ for (int i = 0; i < 10000000; ++i )
+ List.push_back( int() );
+ return 0;
+}
+
--- /dev/null
+// Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+// As a special exception, you may use this file as part of a free software
+// library without restriction. Specifically, if other files instantiate
+// templates or use macros or inline functions from this file, or you compile
+// this file and link it with other files to produce an executable, this
+// file does not by itself cause the resulting executable to be covered by
+// the GNU General Public License. This exception does not however
+// invalidate any other reasons why the executable file might be covered by
+// the GNU General Public License.
+
+#include <complex>
+
+// based on libstdc++/5730, use --fast-math
+int main()
+{
+ typedef std::complex<double> complex_type;
+ complex_type u[2048];
+
+ for (int i = 0; i < 2048; ++i)
+ u[i] = 1.0;
+
+ for (int i = 0; i < 2000; ++i)
+ {
+ complex_type * p = u;
+ for (int j = 0; j < 2048; ++j)
+ {
+#if 1
+ double u2 = norm(*p);
+#else
+ // Shouldn't be slower than the above.
+ double ur = real(*p);
+ double ui = imag(*p);
+ double u2 = ur * ur + ui * ui;
+#endif
+ double t = u2 * 0.1;
+ *p *= complex_type(cos(t), sin(t));
+ ++p;
+ }
+ }
+ return 0;
+}
+
--- /dev/null
+// Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+// As a special exception, you may use this file as part of a free software
+// library without restriction. Specifically, if other files instantiate
+// templates or use macros or inline functions from this file, or you compile
+// this file and link it with other files to produce an executable, this
+// file does not by itself cause the resulting executable to be covered by
+// the GNU General Public License. This exception does not however
+// invalidate any other reasons why the executable file might be covered by
+// the GNU General Public License.
+
+#include <iostream>
+
+// libstdc++/7076
+int main()
+{
+ for (int i = 0; i < 150000; i++)
+ std::cout << i << '\n';
+ return 0;
+}
--- /dev/null
+// Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+// As a special exception, you may use this file as part of a free software
+// library without restriction. Specifically, if other files instantiate
+// templates or use macros or inline functions from this file, or you compile
+// this file and link it with other files to produce an executable, this
+// file does not by itself cause the resulting executable to be covered by
+// the GNU General Public License. This exception does not however
+// invalidate any other reasons why the executable file might be covered by
+// the GNU General Public License.
+
+#include <fstream>
+
+// libstdc++/10672
+int main()
+{
+ using namespace std;
+ fstream s("tmp_perf_seek", ios::binary | ios::in | ios::out | ios::trunc);
+ if (s.good())
+ {
+ for (int i = 0; i < 300000; i++)
+ {
+ s.seekp(0);
+ s.write((char *) & i, sizeof(int));
+ s.seekp(sizeof(int));
+ s.write((char *) & i, sizeof(int));
+ }
+ }
+ return 0;
+}
--- /dev/null
+// Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+// As a special exception, you may use this file as part of a free software
+// library without restriction. Specifically, if other files instantiate
+// templates or use macros or inline functions from this file, or you compile
+// this file and link it with other files to produce an executable, this
+// file does not by itself cause the resulting executable to be covered by
+// the GNU General Public License. This exception does not however
+// invalidate any other reasons why the executable file might be covered by
+// the GNU General Public License.
+
+#include <fstream>
+
+// libstdc++/5001 (100,000 line input file)
+int main ()
+{
+ using namespace std;
+ const char* name = "/usr/share/dict/linux.words";
+
+ ifstream in(name);
+ char buffer[BUFSIZ];
+ while(!in.eof())
+ {
+ in.getline(buffer, BUFSIZ);
+ }
+}
--- /dev/null
+// 2003-03-01 gp dot bolton at computer dot org
+
+// Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+#include <map>
+#include <testsuite_hooks.h>
+
+static bool test = true;
+
+static void create_and_fill(const unsigned int n)
+{
+ typedef std::map<int, int> Map;
+ Map m;
+
+ for (unsigned int i = 0; i < n; ++i)
+ m[i] = i;
+ VERIFY ( m.size() == n );
+}
+
+void test01()
+{
+ const unsigned n = 10000000;
+
+ for (unsigned int i = 0; i < n; ++i)
+ create_and_fill( 0 );
+}
+
+// http://gcc.gnu.org/ml/libstdc++/2003-03/msg00000.html
+int main()
+{
+ test01();
+ return 0;
+}
--- /dev/null
+// Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+// As a special exception, you may use this file as part of a free software
+// library without restriction. Specifically, if other files instantiate
+// templates or use macros or inline functions from this file, or you compile
+// this file and link it with other files to produce an executable, this
+// file does not by itself cause the resulting executable to be covered by
+// the GNU General Public License. This exception does not however
+// invalidate any other reasons why the executable file might be covered by
+// the GNU General Public License.
+
+#include <fstream>
+
+// based on libstdc++/8761 poor fstream performance (converted to float)
+int main()
+{
+ std::ofstream out("tmp_perf_float.txt");
+ for (int i = 0; i < 10000000; ++i)
+ {
+ float f = static_cast<float>(i);
+ out << f << "\n";
+ }
+ return 0;
+};
--- /dev/null
+// Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+// As a special exception, you may use this file as part of a free software
+// library without restriction. Specifically, if other files instantiate
+// templates or use macros or inline functions from this file, or you compile
+// this file and link it with other files to produce an executable, this
+// file does not by itself cause the resulting executable to be covered by
+// the GNU General Public License. This exception does not however
+// invalidate any other reasons why the executable file might be covered by
+// the GNU General Public License.
+
+#include <fstream>
+
+// libstdc++/8761 poor fstream performance
+int main()
+{
+ std::ofstream out("tmp_perf_int.txt");
+ for (int i = 0; i < 10000000; ++i)
+ out << i << "\n";
+ return 0;
+};
--- /dev/null
+ // Copyright (C) 2003 Free Software Foundation, Inc.
+//
+// This file is part of the GNU ISO C++ Library. This library is free
+// software; you can redistribute it and/or modify it under the
+// terms of the GNU General Public License as published by the
+// Free Software Foundation; either version 2, or (at your option)
+// any later version.
+
+// This library is distributed in the hope that it will be useful,
+// but WITHOUT ANY WARRANTY; without even the implied warranty of
+// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+// GNU General Public License for more details.
+
+// You should have received a copy of the GNU General Public License along
+// with this library; see the file COPYING. If not, write to the Free
+// Software Foundation, 59 Temple Place - Suite 330, Boston, MA 02111-1307,
+// USA.
+
+// As a special exception, you may use this file as part of a free software
+// library without restriction. Specifically, if other files instantiate
+// templates or use macros or inline functions from this file, or you compile
+// this file and link it with other files to produce an executable, this
+// file does not by itself cause the resulting executable to be covered by
+// the GNU General Public License. This exception does not however
+// invalidate any other reasons why the executable file might be covered by
+// the GNU General Public License.
+
+#include <ctime>
+#include <iostream>
+#include <string>
+
+using namespace std;
+
+void
+test_append_char(int how_much)
+{
+ string buf; // no preallocation
+ for (int i = 0; i < how_much; ++i)
+ buf.append(static_cast<string::size_type>(1) , 'x');
+}
+
+void
+test_append_string(int how_much)
+{
+ string s(static_cast<string::size_type>(1) , 'x');
+ string buf; // no preallocation
+ for (int i = 0; i < how_much; ++i)
+ buf.append(s);
+}
+
+void
+run_benchmark1(int how_much)
+{
+ clock_t t0 = clock();
+ test_append_char(how_much);
+ clock_t t1 = clock();
+ cout << "Execution time of " << how_much
+ << " string::append(char) calls: "
+ << (static_cast<float>(t1 - t0)/CLOCKS_PER_SEC) << " sec."<< endl;
+}
+
+void
+run_benchmark2(int how_much)
+{
+ clock_t t0 = clock();
+ test_append_string(how_much);
+ clock_t t1 = clock();
+ cout << "Execution time of " << how_much
+ << " string::append(const string&) calls: "
+ << (static_cast<float>(t1 - t0)/CLOCKS_PER_SEC) << " sec." << endl;
+}
+
+// libstdc++/5380
+// libstdc++/4960
+int main()
+{
+ run_benchmark1(10000);
+ run_benchmark2(10000);
+ run_benchmark1(100000);
+ run_benchmark2(100000);
+ run_benchmark1(1000000);
+ run_benchmark2(1000000);
+}
projects / gcc.git / commitdiff