Problem with Russian characters #609

New issue

Closed

opened 2016-04-25 08:34:40 +00:00 by AkumuRu · 5 comments

AkumuRu commented

2016-04-25 08:34:40 +00:00

(Migrated from github.com)

Hello. I'm having troubles displaying Russian characters.

ImFontConfig font_config; 
font_config.OversampleH = 1; //or 2 is the same
font_config.OversampleV = 1; 
font_config.PixelSnapH = 1;

static const ImWchar ranges[] =
{
    0x0020, 0x00FF, // Basic Latin + Latin Supplement
    0x0400, 0x044F, // Cyrillic
    0,
};

io.Fonts->AddFontFromFileTTF("C:\\Windows\\Fonts\\Tahoma.ttf", 16.0f, &font_config, ranges);


ImGui::SetNextWindowSize(ImVec2(200,100), ImGuiSetCond_FirstUseEver);
ImGui::Begin("test window", &show_another_window);

ImGui::Text("Abc Привет");
ImGui::End();

How it looks:
http://dl1.joxi.net/drive/2016/04/25/0001/3646/101950/50/41ddce7ebf.jpg

Atlas texture:
http://dl2.joxi.net/drive/2016/04/25/0001/3646/101950/50/00ef6316db.jpg

Any suggestions?

Hello. I'm having troubles displaying Russian characters. ``` ImFontConfig font_config; font_config.OversampleH = 1; //or 2 is the same font_config.OversampleV = 1; font_config.PixelSnapH = 1; static const ImWchar ranges[] = { 0x0020, 0x00FF, // Basic Latin + Latin Supplement 0x0400, 0x044F, // Cyrillic 0, }; io.Fonts->AddFontFromFileTTF("C:\\Windows\\Fonts\\Tahoma.ttf", 16.0f, &font_config, ranges); ``` ``` ImGui::SetNextWindowSize(ImVec2(200,100), ImGuiSetCond_FirstUseEver); ImGui::Begin("test window", &show_another_window); ImGui::Text("Abc Привет"); ImGui::End(); ``` How it looks: http://dl1.joxi.net/drive/2016/04/25/0001/3646/101950/50/41ddce7ebf.jpg Atlas texture: http://dl2.joxi.net/drive/2016/04/25/0001/3646/101950/50/00ef6316db.jpg Any suggestions?

AkumuRu commented

2016-04-25 08:40:08 +00:00

(Migrated from github.com)

Omar, I used your code from an issue raised back in 2015 to make some tests:

const char* p = "Abc Привет";
while (*p)
{
    Utils::PrintDebug("%02X", (unsigned char)*p);
    p++;
}

I get following output:

41 62 63 20 CF F0 E8 E2 E5 F2

Omar, I used your code from an issue raised back in 2015 to make some tests: ``` const char* p = "Abc Привет"; while (*p) { Utils::PrintDebug("%02X", (unsigned char)*p); p++; } ``` I get following output: > 41 62 63 20 CF F0 E8 E2 E5 F2

ocornut commented

2016-04-25 08:49:36 +00:00

(Migrated from github.com)

Hello,

That's not UTF-8, this is using a local russian/cyrillic codepage.
Unfortunately C++ doesn't make it easy to specify UTF-8 literal.

You need to either save your sources as UTF-8 and some compiler MIGHT emit UTF-8 literals.
In C++11 you can use u8"xxx" synntax to generate UTF-8? As seen here https://github.com/juliettef/IconFontCppHeaders/blob/master/IconsFontAwesome.h
Convert your string to UTF-8 using some utility.

Another solution would be to allow creating glyph redirection in ImFont so you can use your local codepage, but it becomes harder to share code (without UTF-8 non-cyrillic user will see garbage in your literals in the source code). I could add a helper to do so, basically in your ImFont data you need to:

font->IndexLookup[0xCF] = font->IndexLookup[0x043F]; // 043F is codepoint for П
font->IndexXAdvance[0xCF] = font->IndexXAdvance[0x043F];

So you can create a mapping table {{ 0xCF, 0x043F },....} for cyrillic and apply this code over all entries (watch out to check that 0x043F < font->IndexLookup.size();
I could add a helper to remap characters.

Hello, That's not UTF-8, this is using a local russian/cyrillic codepage. Unfortunately C++ doesn't make it easy to specify UTF-8 literal. - You need to either save your sources as UTF-8 and some compiler MIGHT emit UTF-8 literals. - In C++11 you can use u8"xxx" synntax to generate UTF-8? As seen here https://github.com/juliettef/IconFontCppHeaders/blob/master/IconsFontAwesome.h - Convert your string to UTF-8 using some utility. Another solution would be to allow creating glyph redirection in ImFont so you can use your local codepage, but it becomes harder to share code (without UTF-8 non-cyrillic user will see garbage in your literals in the source code). I could add a helper to do so, basically in your ImFont data you need to: font->IndexLookup[0xCF] = font->IndexLookup[0x043F]; // 043F is codepoint for П font->IndexXAdvance[0xCF] = font->IndexXAdvance[0x043F]; So you can create a mapping table {{ 0xCF, 0x043F },....} for cyrillic and apply this code over all entries (watch out to check that 0x043F < font->IndexLookup.size(); I could add a helper to remap characters.

AkumuRu commented

2016-04-25 08:59:41 +00:00

(Migrated from github.com)

Thanks for your reply.
I managed to get it working by storing string as wide chars and then using WideCharToMultiByte function converting to CP_UTF8.
Not sure if this is a correct way of doing it but it works for me.

Thanks for your reply. I managed to get it working by storing string as wide chars and then using `WideCharToMultiByte` function converting to `CP_UTF8`. Not sure if this is a correct way of doing it but it works for me.

ocornut commented

2016-04-25 09:37:02 +00:00

(Migrated from github.com)

I will introduce the helper to add character remapping in the font shortly, it was already in my todo list and would be helpful for "international" users.

👍 5

ocornut commented

2016-04-26 08:12:10 +00:00

(Migrated from github.com)

I have added the ImFont::AddRemapChar() function now.

font->AddRemapChar(0xCF, 0x043F)

The problem with it is that currently it can only be used AFTER the font has been built, aka after the ImFontAtlas::GetTexData* functions has been called. Looking at my notes this was the reason I didn't add this function earlier. So I have added the function and made it assert if the font hasn't been built and documented it as such. Will fix/improve later.

You can borrow one of those table:
CP1251.TXT
https://en.wikipedia.org/wiki/Windows-1251

And then use local codepage in your code.

I have added the ImFont::AddRemapChar() function now. `font->AddRemapChar(0xCF, 0x043F)` The problem with it is that currently it can only be used AFTER the font has been built, aka after the ImFontAtlas::GetTexData\* functions has been called. Looking at my notes this was the reason I didn't add this function earlier. So I have added the function and made it assert if the font hasn't been built and documented it as such. Will fix/improve later. You can borrow one of those table: [CP1251.TXT](https://github.com/ocornut/imgui/files/236161/CP1251.TXT) https://en.wikipedia.org/wiki/Windows-1251 And then use local codepage in your code.

👍 1