<div dir="ltr">Wow! Really nice data... thanks.<br><br>Maybe you could send me Test.xml off the list to see if there's anything funky about it. Or try uploading it to a random location (where you have write access) using curl directly to see if the same problem exists through raw curl...?<br>
<br>Thanks,<br>David<br><br><br><div class="gmail_quote">On Wed, Aug 27, 2008 at 10:46 AM, Kevin H. Hobbs <span dir="ltr"><<a href="mailto:hobbsk@ohiou.edu">hobbsk@ohiou.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="Ih2E3d">On Wednesday 27 August 2008 10:05:44 am David Cole wrote:<br>
> Can you get a call stack for this hang?<br>
<br>
</div>There are two ctest processes:<br>
$ ps -A | grep ctest<br>
1156 ? 00:00:08 ctest<br>
28150 ? 00:00:00 ctest<br>
<br>
The back trace from gdb attached to 1156 is:<br>
<br>
#0 0x0000003ae7ccbd2f in __poll (fds=0x7fff604117b0, nfds=1, timeout=1000) at ../sysdeps/unix/sysv/linux/poll.c:83<br>
#1 0x0000003144a30556 in Curl_socket_ready () from /usr/lib64/libcurl.so.4<br>
#2 0x0000003144a268f1 in Curl_perform () from /usr/lib64/libcurl.so.4<br>
#3 0x00000000004cb1d3 in cmCTestSubmitHandler::SubmitUsingHTTP (this=0xc69780, localprefix=@0x7fff604129d0,<br>
files=<value optimized out>, remoteprefix=@0x7fff604129c0, url=@0x7fff604129b0)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestSubmitHandler.cxx:366<br>
#4 0x00000000004cfc99 in cmCTestSubmitHandler::ProcessHandler (this=0xc69780)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestSubmitHandler.cxx:1006<br>
#5 0x000000000048514e in cmCTest::ProcessTests (this=0x7fff60413450)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:911<br>
#6 0x0000000000488735 in cmCTest::Run (this=0x7fff60413450, args=@0x7fff604138f0, output=0x7fff60413910)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:1988<br>
#7 0x0000000000474458 in main (argc=5, argv=0x7fff60413950) at /home/kevin/kitware/cmake-2.6.1/Source/ctest.cxx:274<br>
<br>
The back trace from gdb attached to 28150 is:<br>
<br>
#0 0x0000003ae7ccdda3 in __select_nocancel () from /lib64/libc.so.6<br>
#1 0x00000000006a3c83 in cmsysProcess_WaitForData (cp=0x239ebe0, data=0x7fff72817ab0, length=0x7fff72817acc,<br>
userTimeout=0x0) at /home/kevin/kitware/cmake-2.6.1/Source/kwsys/ProcessUNIX.c:1120<br>
#2 0x0000000000567d68 in cmSystemTools::RunSingleCommand (command=<value optimized out>, output=0x7fff72818190,<br>
retVal=0x7fff72818278, dir=0x238d488 "/home/kevin/kitware/Insight_build", verbose=false, timeout=0)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/cmSystemTools.cxx:627<br>
#3 0x000000000056a1ee in cmSystemTools::RunSingleCommand (command=<value optimized out>, output=0x7fff72818190,<br>
retVal=0x7fff72818278, dir=0x238d488 "/home/kevin/kitware/Insight_build", verbose=false, timeout=0)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/cmSystemTools.cxx:738<br>
#4 0x00000000004c0655 in cmCTestScriptHandler::RunConfigurationDashboard (this=0x23568d0)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:898<br>
#5 0x00000000004c166b in cmCTestScriptHandler::RunCurrentScript (this=0x23568d0)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:618<br>
#6 0x00000000004c2938 in cmCTestScriptHandler::RunConfigurationScript (this=0x23568d0, total_script_arg=@0x7fff72818540,<br>
pscope=<value optimized out>) at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:552<br>
#7 0x00000000004c2b8e in cmCTestScriptHandler::ProcessHandler (this=0x23568d0)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/CTest/cmCTestScriptHandler.cxx:191<br>
#8 0x000000000048840a in cmCTest::Run (this=0x7fff72818880, args=@0x7fff72818d20, output=0x7fff72818d40)<br>
at /home/kevin/kitware/cmake-2.6.1/Source/cmCTest.cxx:1965<br>
#9 0x0000000000474458 in main (argc=3, argv=0x7fff72818d80) at /home/kevin/kitware/cmake-2.6.1/Source/ctest.cxx:274<br>
<br>
Is that what you want?<br>
<div class="Ih2E3d"><br>
> Or tell us what CMake is trying to<br>
> do when it happens?<br>
<br>
</div>lsof says they have the following open files:<br>
<br>
$ /usr/sbin/lsof | grep ctest<br>
ctest 1156 kevin cwd DIR 9,3 4096 41844737 /home/kevin/kitware/Insight_build<br>
ctest 1156 kevin rtd DIR 9,2 4096 2 /<br>
ctest 1156 kevin txt REG 9,3 35211646 51807747 /home/kevin/kitware/cmake-2.6.1_Build/bin/ctest<br>
ctest 1156 kevin mem REG 9,2 53816 6730460 /lib64/<a href="http://libnss_files-2.7.so" target="_blank">libnss_files-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 23496 6730458 /lib64/<a href="http://libnss_dns-2.7.so" target="_blank">libnss_dns-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 175280 1894831 /usr/lib64/libsmime3.so<br>
ctest 1156 kevin mem REG 9,2 118440 1880925 /usr/lib64/libnssutil3.so<br>
ctest 1156 kevin mem REG 9,2 1317984 1892726 /usr/lib64/libnss3.so<br>
ctest 1156 kevin mem REG 9,2 196840 1894620 /usr/lib64/libssl3.so<br>
ctest 1156 kevin mem REG 9,2 267464 8462089 /usr/lib64/libcurl.so.4.1.0<br>
ctest 1156 kevin mem REG 9,2 11824 6730608 /lib64/libcom_err.so.2.1<br>
ctest 1156 kevin mem REG 9,2 36216 1869427 /usr/lib64/libkrb5support.so.0.1<br>
ctest 1156 kevin mem REG 9,2 191400 1871936 /usr/lib64/libgssapi_krb5.so.2.2<br>
ctest 1156 kevin mem REG 9,2 153664 1871652 /usr/lib64/libk5crypto.so.3.1<br>
ctest 1156 kevin mem REG 9,2 602376 1871808 /usr/lib64/libkrb5.so.3.3<br>
ctest 1156 kevin mem REG 9,2 133616 6730743 /lib64/<a href="http://ld-2.7.so" target="_blank">ld-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 53464 6730753 /lib64/<a href="http://librt-2.7.so" target="_blank">librt-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 1717544 6730744 /lib64/<a href="http://libc-2.7.so" target="_blank">libc-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 611096 6730745 /lib64/<a href="http://libm-2.7.so" target="_blank">libm-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 23288 6730746 /lib64/<a href="http://libdl-2.7.so" target="_blank">libdl-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 143336 6730752 /lib64/<a href="http://libpthread-2.7.so" target="_blank">libpthread-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 85904 6730750 /lib64/libz.so.1.2.3<br>
ctest 1156 kevin mem REG 9,2 92816 6730759 /lib64/<a href="http://libresolv-2.7.so" target="_blank">libresolv-2.7.so</a><br>
ctest 1156 kevin mem REG 9,2 58600 6730767 /lib64/libgcc_s-4.1.2-20070925.so.1<br>
ctest 1156 kevin mem REG 9,2 9728 6730758 /lib64/<a href="http://libkeyutils-1.2.so" target="_blank">libkeyutils-1.2.so</a><br>
ctest 1156 kevin mem REG 9,2 976512 1892647 /usr/lib64/libstdc++.so.6.0.8<br>
ctest 1156 kevin mem REG 9,2 108064 6730452 /lib64/libselinux.so.1<br>
ctest 1156 kevin mem REG 9,2 208848 1876559 /usr/lib64/libidn.so.11.5.28<br>
ctest 1156 kevin mem REG 9,2 14064 1892811 /usr/lib64/libplds4.so<br>
ctest 1156 kevin mem REG 9,2 18064 1892815 /usr/lib64/libplc4.so<br>
ctest 1156 kevin mem REG 9,2 229280 1892809 /usr/lib64/libnspr4.so<br>
ctest 1156 kevin 0r FIFO 0,6 2136348 pipe<br>
ctest 1156 kevin 1w FIFO 0,6 2282007 pipe<br>
ctest 1156 kevin 2w FIFO 0,6 2282006 pipe<br>
ctest 1156 kevin 3w REG 9,3 776 59179715 /home/kevin/kitware/Insight_build/Testing/Temporary/LastSubmit_20080827-0100.log.tmp<br>
ctest 1156 kevin 4r REG 9,3 7425867 59637761 /home/kevin/kitware/Insight_build/Testing/20080827-0100/Test.xml<br>
ctest 1156 kevin 5u IPv4 2619763 TCP gargon.hooperlab:40824->66-194-253-26.static.twtelecom.net:http (ESTABLISHED)<br>
ctest 28150 kevin cwd DIR 9,3 4096 41058388 /home/kevin/kitware<br>
ctest 28150 kevin rtd DIR 9,2 4096 2 /<br>
ctest 28150 kevin txt REG 9,3 35211646 51807747 /home/kevin/kitware/cmake-2.6.1_Build/bin/ctest<br>
ctest 28150 kevin mem REG 9,2 175280 1894831 /usr/lib64/libsmime3.so<br>
ctest 28150 kevin mem REG 9,2 118440 1880925 /usr/lib64/libnssutil3.so<br>
ctest 28150 kevin mem REG 9,2 1317984 1892726 /usr/lib64/libnss3.so<br>
ctest 28150 kevin mem REG 9,2 196840 1894620 /usr/lib64/libssl3.so<br>
ctest 28150 kevin mem REG 9,2 267464 8462089 /usr/lib64/libcurl.so.4.1.0<br>
ctest 28150 kevin mem REG 9,2 11824 6730608 /lib64/libcom_err.so.2.1<br>
ctest 28150 kevin mem REG 9,2 36216 1869427 /usr/lib64/libkrb5support.so.0.1<br>
ctest 28150 kevin mem REG 9,2 191400 1871936 /usr/lib64/libgssapi_krb5.so.2.2<br>
ctest 28150 kevin mem REG 9,2 153664 1871652 /usr/lib64/libk5crypto.so.3.1<br>
ctest 28150 kevin mem REG 9,2 602376 1871808 /usr/lib64/libkrb5.so.3.3<br>
ctest 28150 kevin mem REG 9,2 133616 6730743 /lib64/<a href="http://ld-2.7.so" target="_blank">ld-2.7.so</a><br>
ctest 28150 kevin mem REG 9,2 53464 6730753 /lib64/<a href="http://librt-2.7.so" target="_blank">librt-2.7.so</a><br>
ctest 28150 kevin mem REG 9,2 1717544 6730744 /lib64/<a href="http://libc-2.7.so" target="_blank">libc-2.7.so</a><br>
ctest 28150 kevin mem REG 9,2 611096 6730745 /lib64/<a href="http://libm-2.7.so" target="_blank">libm-2.7.so</a><br>
ctest 28150 kevin mem REG 9,2 23288 6730746 /lib64/<a href="http://libdl-2.7.so" target="_blank">libdl-2.7.so</a><br>
ctest 28150 kevin mem REG 9,2 143336 6730752 /lib64/<a href="http://libpthread-2.7.so" target="_blank">libpthread-2.7.so</a><br>
ctest 28150 kevin mem REG 9,2 85904 6730750 /lib64/libz.so.1.2.3<br>
ctest 28150 kevin mem REG 9,2 92816 6730759 /lib64/<a href="http://libresolv-2.7.so" target="_blank">libresolv-2.7.so</a><br>
ctest 28150 kevin mem REG 9,2 58600 6730767 /lib64/libgcc_s-4.1.2-20070925.so.1<br>
ctest 28150 kevin mem REG 9,2 9728 6730758 /lib64/<a href="http://libkeyutils-1.2.so" target="_blank">libkeyutils-1.2.so</a><br>
ctest 28150 kevin mem REG 9,2 976512 1892647 /usr/lib64/libstdc++.so.6.0.8<br>
ctest 28150 kevin mem REG 9,2 108064 6730452 /lib64/libselinux.so.1<br>
ctest 28150 kevin mem REG 9,2 208848 1876559 /usr/lib64/libidn.so.11.5.28<br>
ctest 28150 kevin mem REG 9,2 14064 1892811 /usr/lib64/libplds4.so<br>
ctest 28150 kevin mem REG 9,2 18064 1892815 /usr/lib64/libplc4.so<br>
ctest 28150 kevin mem REG 9,2 229280 1892809 /usr/lib64/libnspr4.so<br>
ctest 28150 kevin 0r FIFO 0,6 2136348 pipe<br>
ctest 28150 kevin 1w FIFO 0,6 2136349 pipe<br>
ctest 28150 kevin 2w FIFO 0,6 2136349 pipe<br>
ctest 28150 kevin 3r FIFO 0,6 2282005 pipe<br>
ctest 28150 kevin 4w FIFO 0,6 2282005 pipe<br>
ctest 28150 kevin 5r FIFO 0,6 2282006 pipe<br>
ctest 28150 kevin 7r FIFO 0,6 2282007 pipe<br>
<div class="Ih2E3d"><br>
> Or the last thing it did before the hang...?<br>
<br>
</div>The last thing it wrote to Testing/Temporary/LastSubmit_20080827-0100.log.tmp was:<br>
<br>
Upload files:<br>
0 Build.xml<br>
1 Configure.xml<br>
2 Notes.xml<br>
3 Test.xml<br>
4 Update.xml<br>
Using drop method: HTTP<br>
Upload file: /home/kevin/kitware/Insight_build/Testing/20080827-0100/Build.xml to<br>
gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Build.xml<br>
Upload file: /home/kevin/kitware/Insight_build/Testing/20080827-0100/Configure.xml to<br>
gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Configure.xml<br>
Upload file: /home/kevin/kitware/Insight_build/Testing/20080827-0100/Notes.xml to<br>
gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Notes.xml<br>
Upload file: /home/kevin/kitware/Insight_build/Testing/20080827-0100/Test.xml to<br>
gargon.hooperlab___Linux-gcc-4.1.2-x86_64___20080827-0100-Nightly___XML___Test.xml<br>
<br>
<br>
The last thing it wrote to Testing/20080827-0100/Test.xml was:<br>
<br>
<Measurement><br>
<Value>MaximumRatioDecisionRule: The index of the chosen = 1<br>
</Value><br>
</Measurement><br>
</Results><br>
</Test><br>
<EndDateTime>Aug 27 02:14 EDT</EndDateTime><br>
<EndTestTime>1219817640</EndTestTime><br>
<ElapsedMinutes>26.4</ElapsedMinutes></Testing><br>
</Site><br>
<br>
Is there anything else I can do with the running processes?<br>
</blockquote></div><br></div>