Problem in a unicode project

[Kincorn]

I'm a new user, and Crypto++ is really wonderful and powerful,
haha ...

When using FileSink with path including non-ascii chars (e.g.
chinese), I've got the exception:
"FileSink: error opening file for writing: D:\Project\Crypto\测试
\privkey.txt"
But it's well run in ascii environment.

And then, I found out that something's wrong in function wbstowcs_s()
in STL class fstream as the default local setting is C-Locale. So I
can call function setlocale(LC_ALL, szCodePage) to resolve it, but
it's really troubled, isn't it? And the key-problem is, when I use a
special CodePage, it cann't work in another CodePage environment!

And my project is unicode-based, all the code like "std::string, char
[], ..." in Crypto++ may result unpredictable issues.

Has CryptoPP got the unicode version? If not, how can I fix the
problem?
Thanks for your advice...


Thanks,
Kincorn

Development environment:
PC x86
Windows XP SP2
Visual Studio 2005

[Eugene Zolenko]

That's not CryptoPP problem, but rather broken standard C++ unicode
support.

On windows you can use non-standard extention to fstreams -- they can
be created from FILE handles, and FILEs can be created using _wfopen.
And FileSink/Source can be created using fstreams.

std::wstring path = L"D:\bla.txt";
FILE* handle = _wfopen(path, L"rb");
std::ifstream file(handle);
CryptoPP::FileSource(file, ...);

On Linux you will have to convert you std::wstring into UTF8 encoded
std::string first. (Using your favorite UTF8 conversion libraries :)).

Note that using UTF8 converted string on windows doesn't seem to work.

[Kincorn]

Thank you, Eugene, I've just tried the code, and it works exactly.
It's a nice solution.
Is there a better one?
Maybe someone has made a Unicode version, replacing std::string by
std::wstring in the lib-code CryptoPP.

> > Visual Studio 2005- Hide quoted text -
>
> - Show quoted text -

[Eugene Zolenko]

Actually somebody did create Source/Sink based on CFile (MFC). If you
only use windows and can use MFC, that would work for you.

Look here:
http://www.codeproject.com/KB/MFC/CryptoPPCFileMFC.aspx

[Kincorn]

Thanks, the class is useful for my project.

> >>> Visual Studio 2005- Hide quoted text -- Hide quoted text -

[Wei Dai]

Eugene Zolenko wrote:
> On windows you can use non-standard extention to fstreams -- they can
> be created from FILE handles, and FILEs can be created using _wfopen.
> And FileSink/Source can be created using fstreams.
>
> std::wstring path = L"D:\bla.txt";
> FILE* handle = _wfopen(path, L"rb");
> std::ifstream file(handle);
> CryptoPP::FileSource(file, ...);

It turns out that ifstream::open in MSVC 2005 and later can handle a Unicode
filename directly.

> On Linux you will have to convert you std::wstring into UTF8 encoded
> std::string first. (Using your favorite UTF8 conversion libraries :)).

wcstombs() can be used for this, and it seems to be a standard Unix
function.

I've checked in changes to enable FileSource, FileSink, and FileStore to
handle Unicode filenames on Unix and MSVC 2005 and later using the above
methods. Please take a look at
http://cryptopp.svn.sourceforge.net/viewvc/cryptopp?view=rev&revision=472 if
you'd like to review the code.

[Eugene Zolenko]

I would streamline the logic of StoreInitialize like this, removing
goto and half of ifdefs. A goto for name selection is overkill :).

(I didn't compile this though)

void FileStore::StoreInitialize(const NameValuePairs &parameters)
{
m_waiting = false;
m_stream = NULL;
m_file.release();

const char* narrowName = NULL;
const wchar_t* wideName = NULL;
bool useWideName = false;

if(parameters.GetValue(Name::InputFileName(), narrowName))
useWideName = false;
#if defined(CRYPTOPP_UNIX_AVAILABLE) || _MSC_VER >= 1400
else if (parameters.GetValue(Name::InputFileNameWide(), wideName))
useWideName = true;
#endif
else
{
parameters.GetValue(Name::InputStreamPointer(), m_stream);
return;
}

ios::openmode binary = parameters.GetValueWithDefault
(Name::InputBinaryMode(), true) ? ios::binary : ios::openmode(0);
m_file.reset(new std::ifstream);

if(useWideName)
{
#if _MSC_VER >= 1400
m_file->open(wideName, ios::in | binary);
#else
m_file->open(StringNarrow(wideName).c_str(), ios::in | binary);
#endif
}
else
m_file->open(narrowName, ios::in | binary);

if (!*m_file)
throw OpenErr(fileName);
m_stream = m_file.get();

> methods. Please take a look athttp://cryptopp.svn.sourceforge.net/viewvc/cryptopp?view=rev&revision...if

[Wei Dai]

Eugene Zolenko wrote:
> I would streamline the logic of StoreInitialize like this, removing
> goto and half of ifdefs. A goto for name selection is overkill :).

I was trying to keep the preexisting logic as much as possible to minimize
the possibility of introducing new errors. But you're right, the result is a
bit too convoluted. I've adopted your logic and improved it a bit (your code
doesn't compile as written). The new revision is at
http://cryptopp.svn.sourceforge.net/viewvc/cryptopp?view=rev&revision=473.