Protocol buffers and UTF-8?

It may be overkill, but the ICU libraries will do everything you need and you can use them on both Windows and Linux.

It may be overkill, but the ICU libraries will do everything you need and you can use them on both Windows and Linux. However, if you are only wanting conversion, then under Windows, a simple call to MultiByteToWideChar and WideCharToMultiByte can do the conversion between UTF-8 and UTF-16. For example: // utf-8 to utf-16 MultiByteToWideChar( CP_UTF8, 0, myUtf8String, -1, myUtf16Buf, lengthOfUtf16Buf ); With Linux, libidn might do what you need.It can convert between UTF-8 and UCS, which I think is equivalent to UTF-32 at some level.

For example: // utf-8 to UCS ucsStr = stringprep_utf8_to_ucs4( "asdf", 4, &items ); However, in Linux I think you might be best simply working with UTF-8. Unless you have an existing library for UTF-16, I am not sure there is a compelling reason to use it in Linux.

I did stumble on this earlier, but at a second glance it looks more favourable -- especially since the license is not restrictive which I assumed it would be. I will hold out for a while with accepting to see if someone posts a code snippet :P. – Hassan Syed Jan 26 '10 at 14:06.

The Boost Serialization library contains a UTF-8 codecvt facet that you can use to convert unicode to UTF-8 and back. There even is an example in the documentation doing exactly that.

Hmm that looks nice, it would certainly be a weaker dependency than ICU. – Hassan Syed Jan 26 '10 at 14:09.

Take a look at UTF8-CPP: // converts a utf-8 encoded std::string s to utf-16 wstring ws utf8to16(s.begin(), s.end(), back_inserter(ws)).

On Linux it's trivial: each wchar_t is one Unicode codepoint, and with trivial bitops you can find the corresponding UTF-8 byte(s). On Windows it isn't much harder, as there is an API for it: WideCharToMultiByte(CP_UTF8, 0, input. C_str(), input.size(), &out0, out.size(), 0,0).

I cant really gove you an answer,but what I can give you is a way to a solution, that is you have to find the anglde that you relate to or peaks your interest. A good paper is one that people get drawn into because it reaches them ln some way.As for me WW11 to me, I think of the holocaust and the effect it had on the survivors, their families and those who stood by and did nothing until it was too late.