[Insight-users] Image offset is giving bad pointer for large datasets (7Gb)
Michael Jackson
mike.jackson at bluequartz.net
Fri Jul 10 13:21:44 EDT 2009
"contributing" was being nice ;-). Sorry about that, had a long week.
Actually you just need to create an image that is 2^32 + 1 in size. I
tried to get something like this done in VTK but it ended up being a
true API change which never happened.
Let me know how the test goes. If you need another machine I have a
6GB Mac Pro that I can setup to run a few tests if needed.
Thanks for the quick feedback. I think these changes, while possibly
being a bit disruptive _now_ will save lots of bugs/issues in the
future.
Cheers.
_________________________________________________________
Mike Jackson mike.jackson at bluequartz.net
On Jul 10, 2009, at 1:01 PM, Luis Ibanez wrote:
>
> Hi Michael,
>
>
> Thanks for contributing to this topic.
>
>
> I'm running an experimental build by using "size_t" in
> replacement for "long". (on a Windows 64-bits machine).
>
> This, most likely will trigger a cascade of changes
> in other places where replacing "long" with "size_t"
> will be required to provide consistency.
>
>
> To be completely sure, we should add a test that is
> equivalent to what promted Kana to post his question
> to the list. E.g. creating an image of 8Gb or larger.
>
>
>
> Luis
>
>
>
> -----------------------
> Michael Jackson wrote:
>> <rant>
>> I am not a C++ expert by any stretch but with my limited 4 years
>> of experience I have learned one thing. "Long" is the "root of
>> all evil" ;-)
>> IMNSHO, ITK (and VTK by extension) should absolutely BAN the use
>> of "long" in their projects. Period. Too many avoidable bugs come
>> up from its use. If you want a 32 bit integer use int or the
>> standard ansi int type, if you want a 64 bit integer then use
>> "long long int" or the standard ansi 64 bit integer type.
>> "long int" is just a mess waiting to happen. Maybe a rule should
>> be put into the KWStyle project to look for and flag the use of
>> 'long'?
>> </rant>
>> A few years back I spent the better part of 2 days tracking down
>> an issue with VTK that turned out to be the same root problem. The
>> use of "long" when "long long" was really needed. Every time I see
>> something declared 'long' I then have to rethink the ramifications
>> of using those classes or variables as I flip-flop between 32 and
>> 64 bit compiles regularly.
>> Just my thoughts on this friday. Take 'em or leave 'em.
>> I vote for Luis Solution below. I _seems_ like it would solve the
>> problem ..
>> _________________________________________________________
>> Mike Jackson mike.jackson at bluequartz.net
>> On Jul 10, 2009, at 12:30 PM, Luis Ibanez wrote:
>>> Hi Kana,
>>>
>>> Thanks a lot for looking into this and sharing your findings.
>>>
>>> I just confirmed that in Windows 64bits, the "long" type is
>>> only 4 bytes.
>>>
>>> Here is the program I used:
>>>
>>> #include <iostream>
>>> #include "itkOffset.h"
>>> #include "itkNumericTraits.h"
>>>
>>> int main()
>>> {
>>> unsigned long tt;
>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>> tt = -1;
>>> std::cout << "tt = " << tt << std::endl;
>>>
>>> typedef itk::Offset<3> OffsetType;
>>> typedef OffsetType::OffsetValueType OffsetValueType;
>>>
>>> OffsetValueType offsetValue;
>>>
>>> std::cout << "sizeof(offsetValue) = " << sizeof( offsetValue )
>>> << std::endl;
>>>
>>> offsetValue = itk::NumericTraits< OffsetValueType >::max();
>>>
>>> std::cout << "OffsetValueType max() = " << offsetValue <<
>>> std::endl;
>>>
>>> return EXIT_SUCCESS;
>>> }
>>>
>>>
>>> with this CMakeLists.txt file
>>>
>>>
>>> CMAKE_MINIMUM_REQUIRED(VERSION 2.4)
>>> IF(COMMAND CMAKE_POLICY)
>>> CMAKE_POLICY(SET CMP0003 NEW)
>>> ENDIF(COMMAND CMAKE_POLICY)
>>>
>>>
>>> PROJECT(64bitsTest)
>>>
>>> FIND_PACKAGE(ITK REQUIRED)
>>> INCLUDE(${ITK_USE_FILE})
>>>
>>> ADD_EXECUTABLE(typesTest typesTest.cxx )
>>>
>>> TARGET_LINK_LIBRARIES(typesTest ITKCommon)
>>>
>>>
>>>
>>> ---------
>>>
>>>
>>> So, it seems that we have to introduce a new ITK type,
>>> that will be the longest int available in the platform.
>>>
>>>
>>> We currently have the following declarations in the file:
>>>
>>> Insight/Code/Common/itkIntTypes.h
>>>
>>>
>>> /** Convenient and more descriptive integer types. */
>>> typedef char ITK_INT8;
>>> typedef int ITK_INT32;
>>>
>>> #ifndef _WIN32
>>> typedef long long ITK_INT64;
>>> #endif
>>>
>>> #ifdef _WIN32
>>> typedef long ITK_INT64;
>>> #endif
>>>
>>> typedef unsigned char ITK_UINT8;
>>> typedef unsigned short ITK_UINT16;
>>> typedef unsigned ITK_UINT32;
>>>
>>> #ifndef _WIN32
>>> typedef unsigned long long ITK_UINT64;
>>> #endif
>>>
>>> #ifdef _WIN32
>>> typedef unsigned long ITK_UINT64;
>>> #endif
>>>
>>> typedef int ITK_INTPTR;
>>> typedef unsigned ITK_UINTPTR;
>>>
>>> #ifdef __cplusplus
>>> }
>>> #endif
>>>
>>>
>>>
>>> One option that comes to mind is that we
>>> should simply use
>>>
>>>
>>> size_t
>>> or std::size_t
>>>
>>> Which is the type expected by the "mem"
>>> methods, {memcpy,memdup...}, and the
>>> allocation "new" method.
>>>
>>> I have verified that "size_t" will have 8 bits
>>> on Windows 64.
>>>
>>> std::cout << "sizeof( size_t ) = " << sizeof( size_t ) << std::endl;
>>>
>>>
>>>
>>> Unless there are any objections, I'll suggest
>>> that we replace the type of all integer variables
>>> related to image offsets and image size, with
>>> the type "size_t".
>>>
>>>
>>> BTW: note that there is also "size_type"
>>>
>>> It seems that "size_type" will be the type used by STL
>>> containers.
>>>
>>>
>>> Any comments ?
>>>
>>>
>>> Thanks
>>>
>>>
>>> Luis
>>>
>>>
>>> ------------------------------------------------------------------------------------------------------------
>>> On Thu, Jul 9, 2009 at 2:53 AM, Arunachalam Kana <Kana.Arunachalam at fh-wels.at
>>> > wrote:
>>> Hi Luis,
>>>
>>> Thank you for your response. I ran the test program and the given
>>> the
>>> results are given below along with detailed
>>> System information and what option i used to compile itk.
>>>
>>> System Information
>>> ------------------
>>> Time of this report: 7/8/2009, 17:16:29
>>> Machine name: CT-DELL
>>> Operating System: Windows XP Professional x64 Edition (5.2, Build
>>> 3790) Service Pack 2 (3790.srv03_sp2_gdr.090319-1204)
>>> Language: English (Regional Setting: German)
>>> System Manufacturer: Dell Inc.
>>> System Model: Precision WorkStation T7400
>>> BIOS: Default System BIOS
>>> Processor: Intel(R) Pentium(R) III Xeon-Prozessor (8 CPUs),
>>> ~3.2GHz
>>> Memory: 65534MB RAM
>>> Page File: 717MB used, 65267MB available
>>> Windows Dir: C:\WINDOWS
>>> DirectX Version: DirectX 9.0c (4.09.0000.0904)
>>> DX Setup Parameters: Not found
>>> DxDiag Version: 5.03.3790.3959 32bit Unicode
>>>
>>>
>>> Itk compilation
>>> ---------------------
>>> Itk compiled using Microsoft visual studio 2009 x64 (option).
>>>
>>> Microsoft visual studio 2008
>>> ---------------------
>>> Configuration manager details:
>>> Active Solution Configuration: Debug
>>> Active Solution Platform: x64
>>>
>>> Test run details:
>>> 1. Program:
>>> unsigned long tt;
>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>> tt = -1;
>>> std::cout << "tt = " << tt << std::endl;
>>> output: size = 4; tt = 4294967295
>>>
>>> 2. Program:
>>> unsigned long long tt;
>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>> tt = -1;
>>> std::cout << "tt = " << tt << std::endl;
>>> output: size = 8; tt = 18446744073709551615
>>>
>>> I changed the configuration details:
>>> --------------------------------------
>>> Active Solution Configuration: Debug
>>> Active Solution Platform: Win32
>>>
>>> Test run details:
>>> 1. Program:
>>> unsigned long tt;
>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>> tt = -1;
>>> std::cout << "tt = " << tt << std::endl;
>>> output: size = 4; tt = 4294967295
>>>
>>> 2. Program:
>>> unsigned long long tt;
>>> std::cout << "size = " << sizeof(tt) << std::endl;
>>> tt = -1;
>>> std::cout << "tt = " << tt << std::endl;
>>> output: size = 8; tt = 18446744073709551615
>>>
>>> For both Win32 and x64 i get the same result.
>>> Unsigned long is 4 byte and unsigned long long is 8 byte.
>>>
>>> I was a little confused after the result, so i searched for data
>>> type
>>> ranges in msdn. Link below:
>>>
>>> http://msdn.microsoft.com/en-us/library/s3f49ktz.aspx
>>>
>>> From the information given in above link, i understand that for 32
>>> bit
>>> and 64 bit compiler unsigned long is 4 byte and unsigned long
>>> long is 8
>>> byte.
>>>
>>> I checked in the wiki for 64 bit machines (link below)
>>>
>>> http://en.wikipedia.org/wiki/64-bit
>>>
>>> In the link above, under topic "scientific data model": It is
>>> given that
>>> Microsoft win64 uses the LLP data model. From the above links, I
>>> understand the following:
>>>
>>> 1. A 64bit machine with Microsoft Windows x64 Operating
>>> system has
>>> 4 byte for long and 8 byte for long long.
>>> 2. Microsoft visual studio 2009 has long as 4 byte and long
>>> long as
>>> 8 bytes for both win32 and x64 compiler option.
>>>
>>> What's next?
>>>
>>> Thank you,
>>> Regards,
>>> Kana
>>>
More information about the Insight-users
mailing list