[Insight-developers] Unicode filenames, VS6 and fdstream (was:Unicode filenames and borland issue)

[Bill Lorensen]

Tom,

I also tried it with my VS7 compiler and it compiles and runs fine.

Bill

On Wed, Nov 11, 2009 at 11:18 AM, Bill Lorensen <bill.lorensen at gmail.com> wrote:
> Tom,
>
> Compiles and runs fine on VS6. I did have to change the test to use
> append instead of push_back. VS6's string does not have push_back.
>
> I've attached my changed test.
>
> Bill
>
>
> On Wed, Nov 11, 2009 at 8:49 AM, Tom Vercauteren
> <tom.vercauteren at m4x.org> wrote:
>> Hi Bill,
>>
>> I have updated the test on the bug tracker.
>> http://public.kitware.com/Bug/file_download.php?file_id=2649&type=bug
>>
>> I tested it on the same four compilers:
>> * visual c++ 2008 express
>> * mingw with gcc 3.4
>> * borland 5.5
>> * cygwin 1.7 with gcc 4.3
>> but tried to accommodate the code for VS 6.0 and VS 7.x according to:
>> * the information you gave me
>> * http://msdn.microsoft.com/en-us/library/ae1k9a9f%28VS.71%29.aspx and
>> http://msdn.microsoft.com/en-us/library/70bb7saf.aspx
>> * http://www.cdash.org/CDash/testDetails.php?test=36176221&build=469256
>>
>> Could you give it a try on VS6?
>>
>> Thanks a lot,
>> Tom
>>
>> On Wed, Nov 11, 2009 at 11:23, Tom Vercauteren <tom.vercauteren at m4x.org> wrote:
>>> All right, the easiest would then be to not allow unicode filenames on VS6.0.
>>>
>>> I'll try an get something for tomorrow or Friday.
>>>
>>> Tom
>>>
>>> On Wed, Nov 11, 2009 at 06:01, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>>> VS6 does not have a wide string constructor for (i/o)fstream. I'm not
>>>> sure in what VS version they added thw wide string constructor.
>>>>
>>>> Bill
>>>>
>>>>
>>>> On Tue, Nov 10, 2009 at 8:41 PM, Tom Vercauteren
>>>> <tom.vercauteren at m4x.org> wrote:
>>>>> Hi Bill,
>>>>>
>>>>> I just finished implementing your suggestion. I created some stream
>>>>> classes that inherits from (i/o)fstream on all systems except for
>>>>> mingw and borland where
>>>>> 1) they inherit from (i/o)stream
>>>>> 2) they implement some functions that are only in (i/o)fstream to get
>>>>> a uniform API
>>>>> 3) use the c-style open and code from fdstream to allow unicode filenames
>>>>>
>>>>> On MSVC, the new implementation takes advantage of the MSVC-specific
>>>>> wide string constructor of (i/o)fstream [1]. This could thus be easier
>>>>> for VS6.
>>>>>
>>>>> The file is here:
>>>>> http://www.itk.org/Bug/file_download.php?file_id=2646&type=bug
>>>>>
>>>>> I tested it on windows xp with
>>>>> * visual c++ 2008 express
>>>>> * mingw with gcc 3.4
>>>>> * borland 5.5
>>>>> * cygwin 1.7 with gcc 4.3
>>>>>
>>>>> Could you give it a try on VS6?
>>>>>
>>>>> Thanks,
>>>>> Tom
>>>>>
>>>>> [1] This constructor is not available on both borland and mingw which
>>>>> implies that stdio_filebuf is not sufficient and we need a more
>>>>> portable solution such as fdstream.
>>>>>
>>>>> On Tue, Nov 10, 2009 at 22:25, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>>>>> fdstream is the problem on vs6 I think. We might be able to modify the
>>>>>> fdstream header with ifdef's for VS6. Really make fdstream look like
>>>>>> (i/o)stream. Maybe with (uggh!) #defines to redefine fd(i/o)stream to
>>>>>> be (i/o)stream only on VS6 (and maybe VS7).
>>>>>>
>>>>>> On Tue, Nov 10, 2009 at 4:11 PM, Tom Vercauteren
>>>>>> <tom.vercauteren at m4x.org> wrote:
>>>>>>> Hi Bill,
>>>>>>>
>>>>>>> Thanks for taking a look at this!
>>>>>>>
>>>>>>> Are the problems you found related to the use of _wfopen and _wopen or
>>>>>>> to the use of fdstream? This is an important distinction for the
>>>>>>> unicode filename porting because the plan I had in mind relies on
>>>>>>> fdstream being used in all cases (ITK_USE_REVIEW_UTF8_STRINGS on or
>>>>>>> off).
>>>>>>>
>>>>>>> Indeed here is what I found (VS6 not taken into account here):
>>>>>>>
>>>>>>> 1) c-style file IO can be ported relatively easily to utf8 on windows
>>>>>>> (msvc, borland, mingw). Just replace fopen by _wfopen and open by
>>>>>>> _wopen together with some string conversion
>>>>>>>
>>>>>>> 2) c++-style IO is more involved. There is a wide string version of
>>>>>>> ofstream.open() and ifstream.open() on msvc but not on mingw (did not
>>>>>>> try with on borland).
>>>>>>>
>>>>>>> Since the fstreams cannot be opened with a unicode filename on mingw,
>>>>>>> the workaround is to open the file using c-style functions and then
>>>>>>> create a stream from the result of the c-style open.
>>>>>>>
>>>>>>> A gcc specific extension exists for that (stdio_filebuf) but it is
>>>>>>> obviously non-portable. fdstream does the same thing but appeared to
>>>>>>> be portable.
>>>>>>>
>>>>>>> So to get unicode filename enabled streams, we need to replace code like
>>>>>>>
>>>>>>> A)
>>>>>>> std::ifstream readstream(filename, std::ios::binary | std::ios::in)
>>>>>>>
>>>>>>> by code like
>>>>>>>
>>>>>>> B)
>>>>>>> int fd = _wopen(filename, _O_RDONLY);
>>>>>>> boost::fdistream readstream(fd);
>>>>>>> [...]
>>>>>>> close(fd)
>>>>>>>
>>>>>>> We might of course pepper the code with ifdefs each time a stream is
>>>>>>> used (as is already done to workaround a sgi bug in metaImage.cxx
>>>>>>> http://public.kitware.com/cgi-bin/viewcvs.cgi/Utilities/MetaIO/metaImage.cxx?root=Insight&view=markup
>>>>>>> ) but I wanted to avoid that by restricting these ifdefs to live in a
>>>>>>> specified file/module. Hence my plan to wrap only the c-style (f)open
>>>>>>> functions and use fdstreams everywhere.
>>>>>>>
>>>>>>> Note that since we need to close the file descriptor and because
>>>>>>> fd(i/o)streams do not inherit from (i/o)fstream but only from
>>>>>>> (i/o)stream, it is not obvious to have a function that uses either the
>>>>>>> A mode or B mode above.
>>>>>>>
>>>>>>> Sorry for this long email,
>>>>>>> Tom
>>>>>>>
>>>>>>> P.S.: some information is also available on the bug tracker:
>>>>>>> http://public.kitware.com/Bug/view.php?id=9623
>>>>>>>
>>>>>>> On Tue, Nov 10, 2009 at 21:11, Bill Lorensen <bill.lorensen at gmail.com> wrote:
>>>>>>>> My first try with VS6 is not encouraging. I suggest that we instrument
>>>>>>>> the code so that VS6 ITK cannot use Unicode.
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>