CMake/Testing With CTest: Difference between revisions

From KitwarePublic
Jump to navigationJump to search
(Initial writeup about CTest)
 
(Replace content with link to new CMake community wiki)
 
(42 intermediate revisions by 14 users not shown)
Line 1: Line 1:
==Simple Testing==
{{CMake/Template/Moved}}


CMake has support for adding tests in the project. To do that, insert the following command in the CMakeLists.txt file:
This page has moved [https://gitlab.kitware.com/cmake/community/wikis/doc/ctest/Testing-With-CTest here].
 
ENABLE_TESTING()
 
From that point on, you can add tests in the project using ADD_TEST command:
 
ADD_TEST(SimpleTest ${EXECUTABLE_OUTPUT_PATH}/SimpleTest Hello)
 
After building the project, you should be able to do:
 
1. Unix
 
make test
 
2. On GUI development environments such as Visual Studio rebuild the target RUN_TESTS
 
For more information about ENABLE_TESTING and ADD_TEST, Look at [http://www.cmake.org/HTML/Documentation.html CMake Documentation] or run:
 
cmake --help
cmake --help-full
cmake --help-command-list
 
or
 
cmake --help-command ENABLE_TESTING
cmake --help-command ADD_TEST
 
==Dashboards==
 
Testing Dashboards are web pages that display overview of the project testing. The testing clients configure and build the
project, as well as run some number of tests. The results of these operations are then submitted to the central server, which
prepares the overview pages. Examples of Testing Dashboards are:
[http://www.vtk.org/Testing/Dashboard/MostRecentResults-Nightly/Dashboard.html VTK], [http://www.itk.org/Testing/Dashboard/MostRecentResults-Nightly/Dashboard.html ITK], and [http://www.paraview.org/Testing/Dashboard/MostRecentResults-Nightly/Dashboard.html ParaView].
 
To enable submission to Dart Testing Dashboard, include the following in CMakeLists.txt:
 
INCLUDE(Dart)
 
By default the settings will be to submit to [http://www.kitware.com Kitware's] [http://public.kitware.com/Public/Dashboard/MostRecentResults-Nightly/Dashboard.html Public Dashboard]. In order to submit
to some other dashboard, create file "DartConfig.cmake" in the toplevel source directory and set the dashboard preferences.
 
Example of this file is:
 
<pre>
# Dashboard is opened for submissions for a 24 hour period starting at
# the specified NIGHLY_START_TIME. Time is specified in 24 hour format.
SET (NIGHTLY_START_TIME "23:00:00 EDT")
 
# Dart server to submit results (used by client)
IF(DROP_METHOD MATCHES http)
  SET (DROP_SITE "public.kitware.com")
  SET (DROP_LOCATION "/cgi-bin/HTTPUploadDartFile.cgi")
ELSE(DROP_METHOD MATCHES http)
  SET (DROP_SITE "public.kitware.com")
  SET (DROP_LOCATION "/incoming")
  SET (DROP_SITE_USER "ftpuser")
  SET (DROP_SITE_PASSWORD "public")
ENDIF(DROP_METHOD MATCHES http)
 
SET (TRIGGER_SITE
      "http://${DROP_SITE}/cgi-bin/Submit-vtk-TestingResults.pl")
</pre>
 
This will submit testing results to the [http://www.vtk.org/Testing/Dashboard/MostRecentResults-Nightly/Dashboard.html VTK dashboard].
 
===CTest - Client for Dart Dashboard Server===
 
CTest is a testing program distributed as a part of CMake. It can perform simple task of running a set of tests but it can also generate
and submit Dart compatible Dashboard results. The good thing about CTest is that if you use CMake, you already have CTest. Detailed description
of CTest option can be seen by running:
 
ctest --help
or
 
ctest --help-full
 
To convert existing Dart Client run from the project, find lines like:
 
<pre>
cd ProjectNightlyBuildDirectory
tclsh /location/of/Dart/Source/Client/DashboardManager.tcl DartConfiguration.tcl \
Nightly Start Update Configure Build Test Submit
</pre>
 
and convert them to CTest style:
 
cd ProjectNightlyBuildDirectory
ctest -D Nightly
 
Dashboard can be also generated in stages. This way partial testing results can be submitted and seen before long operations are completed:
 
cd ProjectNightlyBuildDirectory
ctest -D NightlyStart
ctest -D NightlyUpdate
ctest -D NightlyConfigure
ctest -D NightlyBuild
ctest -D NightlySubmit
ctest -D NightlyTest
ctest -D NightlyCoverage
ctest -D NightlySubmit
ctest -D NightlyMemCheck
ctest -D NightlySubmit
 
Finally CTest extends functionality of Dart by adding:
 
# HTTP submission support
# Customization of the testing by providing:
#* Custom build error/warning regular expressions
#* Ability to suppress some tests from being tested or memory checked and ability to run subset of tests
#* Ability to run commands before and after tests are run
# Ability to run whole testing process described in a single script
 
For an example of how CTest can run the whole testing process described in a single script, look at how CMake dashboards are created with the [http://www.cmake.org/HTML/TestingSetup.html ctest -S script].
 
More information about CTest can be found in [http://www.kitware.com/products/cmakebook.html Mastering CMake].

Latest revision as of 15:40, 30 April 2018


The CMake community Wiki has moved to the Kitware GitLab Instance.

This page has moved here.