Member Avatar for realmasa77

Hi,

I have a problem with the tolower function. I use Netbeans 6.5 with GCC C++ on Ubuntu 64bit. The problem is, that I use nordic letters like å, ä and ö and the tolower function does not affect them at all. I have tried almost everything. I have tried to replace char by char, like this:

for(unsigned int i = 0; i < strToConvert.length(); i++)
    {
        if (strToConvert[i] == 'Ö')
            strToConvert[i] = 'ö';
        else
            strToConvert[i] = tolower(strToConvert[i]);
    }
    return strToConvert;//return the converted string

This won't work! I have also tried to use wstring and transform but no luck. Setting the locale does nothing. I'm prepared to write a custom function, I just need some ideas on how to get around this.

Also, if I print the codes for the letters with:

string str = "ÅÄÖRNS" //an example
for (int i = 0; i < str.length(); i++) {
     cout << (int) str[i] << endl;
}

'r' will print 114, 'a' 97 as expected. But 'Ö' for example will print two number: -61 and -106, 'Ä' prints -61 and -124. What's up with that?


Thanks,
Mattias

  • 3 Contributors
  • 4 Replies
  • 241 Views
  • 7 Hours Discussion Span
  • Latest Post Latest Post by realmasa77

Dani AI

Generated

Short answer: the string contains UTF-8 multibyte bytes, and tolower from <cctype> works on single bytes only. That is why std::string indexing shows two values for one visible letter (e.g. U+00D6 is 0xC3 0x96 -> prints as -61 and -106 when char is signed). Byte-level tolower will not perform Unicode case mapping.

A robust approach is to convert the UTF-8 std::string into a wide string (wide code points), perform wide-character case conversion, then convert back. Example (illustrates the idea; not the same code posted earlier):

#include <clocale>
#include <codecvt>
#include <locale>
#include <string>
#include <cwctype>

std::string to_lower_utf8(const std::string &in) {
    std::setlocale(LC_CTYPE, ""); // pick up a UTF-8 locale from the environment
    std::wstring_convert<std::codecvt_utf8<wchar_t>> conv;
    std::wstring ws = conv.from_bytes(in);
    for (wchar_t &wc : ws) wc = std::towlower(wc);
    return conv.to_bytes(ws);
}

Notes and troubleshooting:

  • Make sure your process locale is actually UTF-8 (check locale in a shell; LANG should be like en_US.UTF-8 or sv_SE.UTF-8). std::setlocale(LC_CTYPE, "") uses the environment setting.
  • Special-casing a few letters (as suggested) or manual byte fiddling (the OP’s find/erase trick) will work only for a few characters and is brittle for other Unicode text.
  • For production-grade Unicode case folding (full language rules, mappings beyond BMP, Turkic rules, etc.) prefer a Unicode-aware library such as Boost.Locale or ICU instead of ad-hoc conversions. std::codecvt/wstring_convert is convenient but has been deprecated in recent C++ standards; consider libraries above for long-term code.

Credit: for the original example, for pointing out special-case pitfalls, and for linking to further examples.

Recommended Answers

All 4 Replies

Member Avatar for Topi Ojala

tolower() doesnt work on Ä , Ö or Å. I think you have to do an extra check if the letter is one of those ( I'd say use switch-case )

EDIT: sorry, you already did that :D

Anyways make a function which you call for each character individually, dont just run the for-loop for the string.

Member Avatar for realmasa77

tolower() doesnt work on Ä , Ö or Å. I think you have to do an extra check if the letter is one of those ( I'd say use switch-case )

EDIT: sorry, you already did that :D

Anyways make a function which you call for each character individually, dont just run the for-loop for the string.

I have tried that. This:

if (ch == 'Ö') //309
        letter= 'ö'; //310

give compiler warnings:

func.cpp:309:15: warning: multi-character character constant
func.cpp: In function ‘char checkLetter(char)’:
func.cpp:310: warning: overflow in implicit constant conversion

Any idea?

The best so far is:

string str = "ö";
    int index = strToConvert.find_first_of('Ö');
    cout << "index: " << index << endl;
    if (index > -1) {
        string::iterator it;
        it = strToConvert.begin() + index;
        strToConvert.erase(it);
        strToConvert.insert(index, str);
    }

Which prints out this: �ö
I don't know where question mark comes from.

/Mattias

Member Avatar for realmasa77

Solved!

string str = "ö";
    int index = strToConvert.find_first_of('Ö');
    if (index > -1) {
        cout << "index: " << index << endl;
        string::iterator it;
        it = strToConvert.begin() + index;
        strToConvert.erase(it);
        strToConvert.insert(index, str);
        it = strToConvert.begin() + index - 1;
        strToConvert.erase(it);
    }
Be a part of the DaniWeb community

We're a friendly, industry-focused community of developers, IT pros, digital marketers, and technology enthusiasts meeting, networking, learning, and sharing knowledge.