+++ /dev/null
-Testing with QMTest
-===================
-
-You can use QMTest to test G++. (In the future, it may be possible to
-test other parts of GCC with QMTest as well, but it is not possible
-yet.)
-
-The use of QMTest to run the G++ tests has not been approved as an
-officially supported testing procedure. Therefore, you must run the
-tests using DejaGNU (with "make check-g++") before committing changes
-that affect G++.
-
-QMTest emulates DejaGNU behavior very closely when running the tests.
-
-QMTest has two output modes: a DejaGNU emulation mode and a native
-QMTest mode.
-
-In the DejaGNU mode, you should receive output that is almost exactly
-the same as the DejaGNU output; in particular, you should see the same
-number of passes, failures, etc. When using the DejaGNU-style output,
-QMTest uses the "xfail" indications in the test cases to determine
-which tests are expected to pass and which are expected to fail, and
-presents that information in the same way as DejaGNU.
-
-In the QMTest mode, the number of passes and failures will be
-different from that obtained when using DejaGNU. The reason is that a
-single source file may contain multiple DejaGNU tests. In DejaGNU,
-each line where a diagnostic is expected is considered a separate
-test. Testing for successful compilation and testing for successful
-execution of the generated program are considered separate tests. So,
-a single source file "test.C" could contain, say, seven tests; some of
-which might pass and some of which might fail.
-
-In the QMTest mode, each source file is considered a single test. If
-any of the seven sub-tests fail, the entire test is considered to
-fail. However, QMTest does present information about *why* the test
-failed, so the same information is effectively available.
-
-In the QMTest mode, whether or not a test is expected to fail is
-determined not by an indication in the test, but rather by comparing
-the new results to the results of a previous run. Testing for whether
-a change caused a regression is very simple: run the tests before
-making the change, run them again after making the change, and let
-QMTest compare the results.
-
-The mode chosen only affects the output from QMTest, not how it runs
-the tests or how it stores the data. Therefore, if you choose to run
-in the QMTest mode and later want to get the DejaGNU style output, or
-vice versa, you can do that as described below.
-
-Setting Up
-==========
-
-You must download and install the following software:
-
-- Python 2.2 (or greater)
-
- See http://www.python.org.
-
- You may already have Python on your system; in particular, many
- GNU/Linux systems ship with Python installed.
-
- Installation instructions are available on the web-site.
-
-- A current version of QMTest. No released version provides all of
- the functionality required, so you must obtain QMTest from CVS.
-
- To do that, follow the instructions at:
-
- http://www.codesourcery.com/qmtest
-
- Installation instructions are available in the file called README
- after you check out QMTest.
-
-- The "qmtc" and "qmtest_gcc" QMTest support packages. These are
- available from the same CVS repository as QMTest. For example, to
- check out "qmtc", do:
-
- cvs -d :pserver:anoncvs@cvs.codesourcery.com:/home/qm/Repository \
- co qmtc
-
- You do not have to install these packages; you need only check them
- out.
-
-Running the Tests
-=================
-
-First, you must set QMTEST_CLASS_PATH so that it can find the qmtc and
-qmtest_gcc support packages:
-
- export QMTEST_CLASS_PATH=/path/to/qmtc:/path/to/qmtest_gcc
-
-The, run "make qmtest-g++" in the gcc directory of your build tree.
-
-Here are some more advanced usage instructions:
-
-1. To run a particular set of tests (rather than all of the tests),
- use the make variable "QMTEST_GPP_TESTS". For example,
-
- make QMTEST_GPP_TESTS="g++.dg" qmtest-g++
-
- will run only the tests in the g++.dg subdirectory, and:
-
- make QMTEST_GPP_TESTS="g++.dg/special/conpr-1.C \
- g++.old-deja/g++.other/access2.C"
- qmtest-g++
-
- will run only the two tests indicated.
-
-2. To run qmtest with particular flags, use the make variables
- "QMTESTFLAGS" and "QMTESTRUNFLAGS". For example:
-
- make QMTESTFLAGS="-v" QMTESTRUNFLAGS="-f full" qmtest-g++
-
- will run qmtest like this:
-
- qmtest -v run -f full ...
-
- (The "-f full" mode will provide detailed information about each
- test as it runs.)
-
-3. To run the compiler with particular flags, use QMTESTRUNFLAGS to
- set the QMTest context variable "CompilerTable.cplusplus_options",
- like this:
-
- make \
- QMTESTRUNFLAGS='-c CompilerTable.cplusplus_options="-funroll-loops"' \
- qmtest-g++
-
- The compiler will then use the "-funroll-loops" switch when
- compiling.
-
-4. If qmtest is not in your path, you can indicate the full path to
- QMTest by using the make variable "QMTEST_PATH", like this:
-
- make QMTEST_PATH=/path/to/qmtest qmtest-g++
-
-5. To start the QMTest GUI, use:
-
- make qmtest-gui
-
- (Note that this will run the program called "mozilla" in your path.
- If you want to use another browser, you must configure qmtest as
- described in its manual.)
-
- Bear in mind that the QMTest GUI is insecure; malicious users with
- access to your machine may be able to run commands as if they were
- you. The QMTest GUI only binds to the loopback IP addresss, which
- provides a measure of security, but not enough for use in untrusted
- environments.
-
-6. If you have a multiprocessor, you can run the tests in parallel by
- passing the "-j" option to qmtest:
-
- make QMTESTRUNFLAGS="-j 4" qmtest-g++
-
- will run tests in four threads. (It is also possible to run tests
- across multiple machines; for more information see the QMTest
- manual.)
-
-7. If a test (say "g++.dg/abi/bitfield1.C") fails, and you want to get
- more detailed information, you can do:
-
- cd qmtestsuite
- qmtest summarize g++.qmr g++.dg/abi/bitfield1.C
-
- to get more information about the commands that were run and the
- output produced.
-
-\f
-Copyright (C) 2002, 2003, 2004 Free Software Foundation, Inc.
-
-Copying and distribution of this file, with or without modification,
-are permitted in any medium without royalty provided the copyright
-notice and this notice are preserved.
projects / gcc.git / commitdiff