TubeTK/Midas Unit Tests: Difference between revisions

From KitwarePublic
Jump to navigationJump to search
No edit summary
 
(4 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Introduction =
= Introduction =
Tubetk has adopted [http://midas3.kitware.com/midas/ MIDAS] to store testing and algorithm validation dataset. For this purpose, we have created a dedicated [http://midas3.kitware.com/midas/community/7 tubetk MIDAS community]
TubeTK has adopted [http://midas3.kitware.com/midas/ Midas] to store testing and algorithm validation dataset. For this purpose, we have created a dedicated [http://midas3.kitware.com/midas/community/7 TubeTK Midas community]


In the tubetk community, we have two main types of folders
In the TubeTK community, we have two main types of folders


#'''Input data folder''': Where all input data for testing will be stored
#'''Input data folder''': Where all input data for testing will be stored
#'''Baseline dataset folder''': These are individual folders that will be used to store the baseline images for regression tests in tubetk
#'''Baseline dataset folder''': These are individual folders that will be used to store the baseline images for regression tests in TubeTK


= How to add a new tubetk test =
= How to add a new TubeTK test =


#If the input data is not already in Midas,  
#If the input data is not already in Midas,  
##Upload the data to the "Data" folder
##Upload the data to the "Data" folder
##Download the key file (*.md5) and store it in the MIDAS_KEYS directory at the top level of tubetk source code tree  
##Download the key file (*.md5) and store it in the MIDAS_KEYS directory at the top level of TubeTK source code tree  
#Add baseline dataset for the new test ( if your test is a regression testing )
#Add baseline dataset for the new test ( if your test is a regression testing )
##Upload the dataset to the appropriate CLI or module folder. If your test is part of a new "module" or CLI, create a new a folder in MIDAS and then upload the dataset
##Upload the dataset to the appropriate CLI or module folder. If your test is part of a new "module" or CLI, create a new a folder in Midas and then upload the dataset
##Download the key file  (*.md5) for the baseline image and store it in the MIDAS_KEYS directory.
##Download the key file  (*.md5) for the baseline image and store it in the MIDAS_KEYS directory.
#Modify cmake list files in the Testing directory to "add" unit tests to use the data from MIDAS.  
#Modify cmake list files in the Testing directory to "add" unit tests to use the data from Midas.  
##Use MIDAS_FETCH_ONLY macro, if you would like to fetch the dataset but not use it directly in the test
##Use MIDAS_FETCH_ONLY macro, if you would like to fetch the dataset but not use it directly in the test
##Use MIDAS_DIRECTORY, if you would like to specify a whole directory as an input command line argument to your test
##Use MIDAS_DIRECTORY, if you would like to specify a whole directory as an input command line argument to your test
Line 21: Line 21:
Examples:  
Examples:  


Test using a data store in MIDAS
Test using a data store in Midas


  MIDAS_ADD_TEST( ${PROJECT_NAME}MIDAS-Test4
  MIDAS_ADD_TEST( ${PROJECT_NAME}MIDAS-Test4
Line 54: Line 54:
                       APPEND PROPERTY DEPENDS ${PROJECT_NAME}-TestSphereNoReg )
                       APPEND PROPERTY DEPENDS ${PROJECT_NAME}-TestSphereNoReg )


= MIDAS macro =
= Midas macro =
The following are the essential midas macros for setting up a test using data stored in midas.
The following are the essential midas macros for setting up a test using data stored in midas.
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#How_to_create_a_test_that_will_download_and_use_MIDAS-hosted_data MIDAS_ADD_TEST]
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#How_to_create_a_test_that_will_download_and_use_MIDAS-hosted_data MIDAS_ADD_TEST]
Line 60: Line 60:
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#MIDAS_FETCH_ONLY_option MIDAS_FETCH ONLY]
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#MIDAS_FETCH_ONLY_option MIDAS_FETCH ONLY]
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#MIDAS_DIRECTORY_option MIDAS_DIRECTORY]
#[http://www.kitware.com/midaswiki/index.php/MIDAS%2BCTest#MIDAS_DIRECTORY_option MIDAS_DIRECTORY]
[[Category:TubeTK|Midas Unit Tests]]

Latest revision as of 18:39, 26 July 2013

Introduction

TubeTK has adopted Midas to store testing and algorithm validation dataset. For this purpose, we have created a dedicated TubeTK Midas community

In the TubeTK community, we have two main types of folders

  1. Input data folder: Where all input data for testing will be stored
  2. Baseline dataset folder: These are individual folders that will be used to store the baseline images for regression tests in TubeTK

How to add a new TubeTK test

  1. If the input data is not already in Midas,
    1. Upload the data to the "Data" folder
    2. Download the key file (*.md5) and store it in the MIDAS_KEYS directory at the top level of TubeTK source code tree
  2. Add baseline dataset for the new test ( if your test is a regression testing )
    1. Upload the dataset to the appropriate CLI or module folder. If your test is part of a new "module" or CLI, create a new a folder in Midas and then upload the dataset
    2. Download the key file (*.md5) for the baseline image and store it in the MIDAS_KEYS directory.
  3. Modify cmake list files in the Testing directory to "add" unit tests to use the data from Midas.
    1. Use MIDAS_FETCH_ONLY macro, if you would like to fetch the dataset but not use it directly in the test
    2. Use MIDAS_DIRECTORY, if you would like to specify a whole directory as an input command line argument to your test

Examples:

Test using a data store in Midas

MIDAS_ADD_TEST( ${PROJECT_NAME}MIDAS-Test4
           ${PROJ_EXE}
            MIDAS{SyntheticTubePlusSurfaceImage_oriented.mha.md5}
            ${TEMP}/SyntheticTubePlusSurfaceEEDFilterResultImage_oriented.mha )

Test with a directory as input argument: DICOMSeries

MIDAS_ADD_TEST( ${PROJECT_NAME}MIDAS-Test1
            ${PROJ_EXE}
              -i
              -r
              -a Test1 001
              -w
              -o ${TEMP}
              -b DSC_
              MIDAS_DIRECTORY{DICOMSeries} )

Test that demonstrates fetch only option: tubeEdgeEnhancingAnisotropicDiffusionMIDAS-Test1

MIDAS_ADD_TEST( ${PROJECT_NAME}MIDAS-Test1
           ${PROJ_EXE}
              MIDAS{CroppedWholeLungCTScan.mhd.md5}
              ${TEMP}/itkAnisotropicEdgeEnhancingDiffusionImageFilterTest.mha
              MIDAS_FETCH_ONLY{CroppedWholeLungCTScan.raw.md5}
             )

When setting up test dependencies in your CMakeLists.txt files describing the tests, use something like:

 set_property(TEST ${PROJECT_NAME}-TestSphereNoReg-Compare
                     APPEND PROPERTY DEPENDS ${PROJECT_NAME}-TestSphereNoReg )

Midas macro

The following are the essential midas macros for setting up a test using data stored in midas.

  1. MIDAS_ADD_TEST
  2. MIDAS_TGZ
  3. MIDAS_FETCH ONLY
  4. MIDAS_DIRECTORY