Magento: “Wrong charset, conversion from `UTF-16BE’ to `UTF-8′ is not allowed”

We have noticed this issue cropping up with Magento. Whilst we originally assumed it was a system level fault, further testing has revealed there is an issue with the implementation of iconv() in Magento.

There was a problem with Magento converting strings between character sets, specifically when creating a PDF invoice.

Initial testing with a simple PHP script was positive.

<?php
print_r ( iconv_get_encoding());
print_r(get_loaded_extensions ());
print_r(get_defined_constants(true));
printf(" %s n",ICONV_IMPL);
printf(" %s n",ICONV_VERSION);
echo iconv('UTF-8', 'ASCII//TRANSLIT', "test"); echo "n";
echo iconv('ASCII', 'UTF-8//TRANSLIT', "test"); echo "n";
echo iconv('UTF-8', 'ISO-8859-1//TRANSLIT', "test"); echo "n";
echo iconv('ISO-8859-1', 'UTF-8//TRANSLIT', "test"); echo "n";
echo iconv('UTF-8', 'ISO-8859-1', "test"); echo "n";
echo iconv('ISO-8859-1', 'UTF-8', "test"); echo "n";
echo iconv('UTF-16BE', 'UTF-8', "test"); echo "n";
echo iconv('UTF-8', 'UTF-16BE', "test"); echo "n";
?>

There obviously was not an issue with our implemental of glibc or iconv libs. However, when used in Magento’s code, it threw an error.

We do not know what is causing this issue, however, we have submitted a bug report to Varien, in the mean time there is a temporary resolution.

1) Make the local dirs and copy the relevant files

cd app/code
mkdir local/Mage
mkdir local/Mage/Sales
mkdir local/Mage/Sales/Model
mkdir local/Mage/Sales/Model/Order
mkdir local/Mage/Sales/Model/Order/Pdf
cd ../../
cp core/Mage/Sales/Model/Order/Pdf/Abstract.php local/Mage/Sales/Model/Order/Pdf/Abstract.php
cp lib/Zend/Pdf/Resource/Font/Type0.php lib/Zend/Pdf/Resource/Font/Type0.php.orig
cp lib/Zend/Pdf/Resource/Font.php lib/Zend/Pdf/Resource/Font.php.orig

2) Make changes to the offending scripts

app/code/local/Mage/Sales/Model/Order/Pdf/Abstract.php

+        # iconv_hack
+        #$drawingString = iconv('UTF-8', 'UTF-16BE//IGNORE', $string);
+        $drawingString = mb_convert_encoding($string, "UTF-16BE", "UTF-8");
-        $drawingString = iconv('UTF-8', 'UTF-16BE//IGNORE', $string);

lib/Zend/Pdf/Resource/Font/Type0.php

+        # iconv_hack
+        #return iconv($charEncoding, 'UTF-16BE', $string);
+        return mb_convert_encoding($string,'UTF-16BE','UTF-8');
-        return iconv($charEncoding, 'UTF-16BE', $string);

lib/Zend/Pdf/Resource/Font.php

if ((! is_null($characterSet)) && ($characterSet != 'UTF-16BE') && PHP_OS != 'AIX') { // AIX knows not this cha$
+          # iconv_hack
+       #$name = iconv('UTF-16BE', $characterSet, $name);
+          $name = mb_convert_encoding($name, $characterSet, 'UTF-16BE');
-       $name = iconv('UTF-16BE', $characterSet, $name);
}

index.php

+ iconv_set_encoding("internal_encoding", "UTF-8");
+ iconv_set_encoding("output_encoding", "ISO-8859-1");