PHP：将多字节字符串（字）拆分为不同的字符

20

尝试与 'U' 选项正则表达式，例如

$chars = preg_split('//u', $string, -1, PREG_SPLIT_NO_EMPTY);

来源

2010-03-31 21:56:49 user187291

+0

这只会使用UTF-8编码。 – 2014-04-09 15:31:52

+0

同意Petr。我用BIG5试过了，它不起作用！ – 2014-11-28 05:39:17

9

丑陋的方式来做到这一点是：

mb_internal_encoding("UTF-8"); // this IS A MUST!! PHP has trouble with multibyte 
           // when no internal encoding is set! 
$string = "....."; 
$chars = array(); 
for ($i = 0; $i < mb_strlen($string); $i++) { 
    $chars[] = mb_substr($string, $i, 1); // only one char to go to the array 
}

你也应该尝试用自己的方式与mb_split与之前设置internal_encoding。

来源

2010-03-31 20:46:26 bisko

+0

'mb_internal_encoding（“UTF-8”）;'这帮了我很多。 – ivkremer 2013-12-27 15:47:11

+0

我喜欢这个答案，因为我努力寻找简单的UTF-8安全的方式来分裂文本在同样长的部分。不仅仅是为了制造角色而是部分。只需编辑$ i + 5和5作为最后的mb_substr参数，并将我的文本分为5个utf8字符串。 Thnx多。 – 2016-07-07 20:45:16

+0

很好的解决方案:) – clarkk 2017-01-17 20:14:52

3

可以使用字形函数（PHP 5.3或国际1.0）和IntlBreakIterator（PHP 5.5或国际3.0）。以下代码显示了intl和mbstring和PCRE函数之间的区别。

// http://www.php.net/manual/function.grapheme-strlen.php 
$string = "a\xCC\x8A" // 'LATIN SMALL LETTER A WITH RING ABOVE' (U+00E5) 
     ."o\xCC\x88"; // 'LATIN SMALL LETTER O WITH DIAERESIS' (U+00F6) 

$expected = ["a\xCC\x8A", "o\xCC\x88"]; 
$expected2 = ["a", "\xCC\x8A", "o", "\xCC\x88"]; 

var_dump(
    $expected === str_to_array($string), 
    $expected === str_to_array2($string), 
    $expected2 === str_to_array3($string), 
    $expected2 === str_to_array4($string), 
    $expected2 === str_to_array5($string) 
); 

function str_to_array($string) 
{ 
    $length = grapheme_strlen($string); 
    $ret = []; 

    for ($i = 0; $i < $length; $i += 1) { 
     $ret[] = grapheme_substr($string, $i, 1); 
    } 

    return $ret; 
} 

function str_to_array2($string) 
{ 
    $it = IntlBreakIterator::createCharacterInstance('en_US'); 
    $it->setText($string); 

    $ret = []; 
    $prev = 0; 

    foreach ($it as $pos) { 

     $char = substr($string, $prev, $pos - $prev); 

     if ('' !== $char) { 
      $ret[] = $char; 
     } 

     $prev = $pos; 
    } 

    return $ret; 
} 

function str_to_array3($string) 
{ 
    $it = IntlBreakIterator::createCodePointInstance(); 
    $it->setText($string); 

    $ret = []; 
    $prev = 0; 

    foreach ($it as $pos) { 

     $char = substr($string, $prev, $pos - $prev); 

     if ('' !== $char) { 
      $ret[] = $char; 
     } 

     $prev = $pos; 
    } 

    return $ret; 
} 

function str_to_array4($string) 
{ 
    $length = mb_strlen($string, "UTF-8"); 
    $ret = []; 

    for ($i = 0; $i < $length; $i += 1) { 
     $ret[] = mb_substr($string, $i, 1, "UTF-8"); 
    } 

    return $ret; 
} 

function str_to_array5($string) { 
    return preg_split('//u', $string, -1, PREG_SPLIT_NO_EMPTY); 
}

在生产环境中工作时，您需要更换与替代字符无效字节序列，因为几乎所有的字形和MBSTRING功能无法处理无效的字节序列。如果您有兴趣，请参阅我过去的回答：https://stackoverflow.com/a/13695364/531320

如果您不考虑性能，可以使用htmlspecialchars和htmlspecialchars_decode。这种方式的优点是支持UTF-8以外的各种编码。

function str_to_array6($string, $encoding = 'UTF-8') 
{ 
    $ret = []; 
    str_replace_callback($string, function($char, $index) use (&$ret) { $ret[] = $char; return ''; }, $encoding); 
    return $ret; 
} 

function str_replace_callback($string, $callable, $encoding = 'UTF-8') 
{ 
    $str_size = strlen($string); 
    $string = str_scrub($string, $encoding); 

    $ret = ''; 
    $char = ''; 
    $index = 0; 

    for ($pos = 0; $pos < $str_size; ++$pos) { 

     $char .= $string[$pos]; 

     if (str_check_encoding($char, $encoding)) { 

      $ret .= $callable($char, $index); 
      $char = ''; 
      ++$index; 
     } 

    } 

    return $ret; 
} 

function str_check_encoding($string, $encoding = 'UTF-8') 
{ 
    $string = (string) $string; 
    return $string === htmlspecialchars_decode(htmlspecialchars($string, ENT_QUOTES, $encoding)); 
} 

function str_scrub($string, $encoding = 'UTF-8') 
{ 
    return htmlspecialchars_decode(htmlspecialchars($string, ENT_SUBSTITUTE, $encoding)); 
}

如果你想了解UTF-8的规范，字节操作是一种很好的练习方式。

function str_to_array6($string) 
{ 
    // REPLACEMENT CHARACTER (U+FFFD) 
    $substitute = "\xEF\xBF\xBD"; 
    $size = strlen($string); 
    $ret = []; 

    for ($i = 0; $i < $size; $i += 1) { 

     if ($string[$i] <= "\x7F") { 

      $ret[] = $string[$i]; 

     } elseif ("\xC2" <= $string[$i] && $string[$i] <= "\xDF") { 

      if (!isset($string[$i+1])) { 

       $ret[] = $substitute; 
       return $ret; 

      } elseif ($string[$i+1] < "\x80" || "\xBF" < $string[$i+1]) { 

       $ret[] = $substitute; 

      } else { 

       $ret[] = substr($string, $i, 2); 
       $i += 1; 

      } 

     } elseif ("\xE0" <= $string[$i] && $string[$i] <= "\xEF") { 

      $left = "\xE0" === $string[$i] ? "\xA0" : "\x80"; 
      $right = "\xED" === $string[$i] ? "\x9F" : "\xBF"; 

      if (!isset($string[$i+1])) { 

       $ret[] = $substitute; 
       return $ret; 

      } elseif ($string[$i+1] < $left || $right < $string[$i+1]) { 

       $ret[] = $substitute; 

      } else { 

       if (!isset($string[$i+2])) { 

        $ret[] = $substitute; 
        return $ret; 

       } elseif ($string[$i+2] < "\x80" || "\xBF" < $string[$i+2]) { 

        $ret[] = $substitute; 
        $i += 1; 

       } else { 

        $ret[] = substr($string, $i, 3); 
        $i += 2; 

       } 

      } 

     } elseif ("\xF0" <= $string[$i] && $string[$i] <= "\xF4") { 

      $left = "\xF0" === $string[$i] ? "\x90" : "\x80"; 
      $right = "\xF4" === $string[$i] ? "\x8F" : "\xBF"; 

      if (!isset($string[$i+1])) { 

       $ret[] = $substitute; 
       return $ret; 

      } elseif ($string[$i+1] < $left || $right < $string[$i+1]) { 

       $ret[] = $substitute; 

      } else { 

       if (!isset($string[$i+2])) { 

        $ret[] = $substitute; 
        return $ret; 

       } elseif ($string[$i+2] < "\x80" || "\xBF" < $string[$i+2]) { 

        $ret[] = $substitute; 
        $i += 1; 

       } else { 

        if (!isset($string[$i+3])) { 

         $ret[] = $substitute; 
         return $ret; 

        } elseif ($string[$i+3] < "\x80" || "\xBF" < $string[$i+3]) { 

         $ret[] = $substitute; 
         $i += 2; 

        } else { 

         $ret[] = substr($string, $i, 4); 
         $i += 3; 

        } 

       } 

      } 

     } else { 

      $ret[] = $substitute; 

     } 

    } 

    return $ret; 

}

这些函数之间的基准测试结果就在这里。

grapheme 
0.12967610359192 
IntlBreakIterator::createCharacterInstance 
0.17032408714294 
IntlBreakIterator::createCodePointInstance 
0.079245090484619 
mbstring 
0.081080913543701 
preg_split 
0.043133974075317 
htmlspecialchars 
0.25599694252014 
byte maniplulation 
0.13132810592651

基准代码在这里。

$string = '主楼怎么走'; 

foreach (timer([ 
    'grapheme' => 'str_to_array', 
    'IntlBreakIterator::createCharacterInstance' => 'str_to_array2', 
    'IntlBreakIterator::createCodePointInstance' => 'str_to_array3', 
    'mbstring' => 'str_to_array4', 
    'preg_split' => 'str_to_array5', 
    'htmlspecialchars' => 'str_to_array6', 
    'byte maniplulation' => 'str_to_array7' 
], 
[$string]) as $desc => $time) { 

    echo $desc, PHP_EOL, 
     $time, PHP_EOL; 
} 

function timer(array $callables, array $arguments, $repeat = 10000) { 

    $ret = []; 
    $save = $repeat; 

    foreach ($callables as $key => $callable) { 

     $start = microtime(true); 

     do { 

      array_map($callable, $arguments); 

     } while($repeat -= 1); 

     $stop = microtime(true); 
     $ret[$key] = $stop - $start; 
     $repeat = $save; 

    } 

    return $ret; 
}

来源

2013-05-30 13:53:36 masakielastic

PHP：将多字节字符串（字）拆分为不同的字符

回答

相关问题