Magento: "Wrong charset, conversion from `UTF-16BE' to `UTF-8' is not allowed"

We have noticed this issue cropping up with Magento. Whilst we originally assumed it was a system level fault, further testing has revealed there is an issue with the implementation of iconv() in Magento. There was a problem with Magento converting strings between character sets, specifically when creating a PDF invoice. Initial testing with a simple PHP script was positive.
<?php
print_r ( iconv_get_encoding());
print_r(get_loaded_extensions ());
print_r(get_defined_constants(true));

printf(" %s n",ICONV_IMPL);
printf(" %s n",ICONV_VERSION);

echo iconv('UTF-8', 'ASCII//TRANSLIT', "test"); echo "n";
echo iconv('ASCII', 'UTF-8//TRANSLIT', "test"); echo "n";

echo iconv('UTF-8', 'ISO-8859-1//TRANSLIT', "test"); echo "n";
echo iconv('ISO-8859-1', 'UTF-8//TRANSLIT', "test"); echo "n";

echo iconv('UTF-8', 'ISO-8859-1', "test"); echo "n";
echo iconv('ISO-8859-1', 'UTF-8', "test"); echo "n";

echo iconv('UTF-16BE', 'UTF-8', "test"); echo "n";
echo iconv('UTF-8', 'UTF-16BE', "test"); echo "n";
?>
There obviously was not an issue with our implemental of glibc or iconv libs. However, when used in Magento's code, it threw an error. We do not know what is causing this issue, however, we have submitted a bug report to Varien, in the mean time there is a temporary resolution. 1) Make the local dirs and copy the relevant files
cd app/code
mkdir local/Mage
mkdir local/Mage/Sales
mkdir local/Mage/Sales/Model
mkdir local/Mage/Sales/Model/Order
mkdir local/Mage/Sales/Model/Order/Pdf
cd ../../
cp core/Mage/Sales/Model/Order/Pdf/Abstract.php local/Mage/Sales/Model/Order/Pdf/Abstract.php
cp lib/Zend/Pdf/Resource/Font/Type0.php lib/Zend/Pdf/Resource/Font/Type0.php.orig
cp lib/Zend/Pdf/Resource/Font.php lib/Zend/Pdf/Resource/Font.php.orig
2) Make changes to the offending scripts app/code/local/Mage/Sales/Model/Order/Pdf/Abstract.php
+        # iconv_hack
+        #$drawingString = iconv('UTF-8', 'UTF-16BE//IGNORE', $string);
+        $drawingString = mb_convert_encoding($string, "UTF-16BE", "UTF-8");
-        $drawingString = iconv('UTF-8', 'UTF-16BE//IGNORE', $string);
lib/Zend/Pdf/Resource/Font/Type0.php
+        # iconv_hack
+        #return iconv($charEncoding, 'UTF-16BE', $string);
+        return mb_convert_encoding($string,'UTF-16BE','UTF-8');
-        return iconv($charEncoding, 'UTF-16BE', $string);
lib/Zend/Pdf/Resource/Font.php
if ((! is_null($characterSet)) && ($characterSet != 'UTF-16BE') && PHP_OS != 'AIX') { // AIX knows not this cha$
+          # iconv_hack
+       #$name = iconv('UTF-16BE', $characterSet, $name);
+          $name = mb_convert_encoding($name, $characterSet, 'UTF-16BE');
-       $name = iconv('UTF-16BE', $characterSet, $name);
}
index.php
+ iconv_set_encoding("internal_encoding", "UTF-8");
+ iconv_set_encoding("output_encoding", "ISO-8859-1");
  • Tim

    Hi, I got something very similar (just using Zend_Pdf). I was running libiconv 1.1.3, downgrading to 1.1.1 fixed the problem. Tim.

  • Tim

    FYI, I tracked down my problem and got it working with the libiconv 1.13 by changing:

    $page->drawText ($text, $x, $y, 'UTF8'); // errors with libiconv 1.13

    to:

    $page->drawText ($text, $x, $y, 'UTF-8'); // okay with libiconv 1.13

    Tim.

    • Julie P

      Hey Tim - where's that line? What file? Thanks!

  • Hi
    I am using Magento 1.4.0.1. I follow all the steps but its not working. any one can help me more about this issue.

    In the above code i use the following lines at the root index.php

    + iconv_set_encoding("internal_encoding", "UTF-8");
    + iconv_set_encoding("output_encoding", "ISO-8859-1");

    Please help me to resolve this issues.

    Thanks in advance!

  • Jeff

    We have been unable to solve this issue in 1.41. Any further news?

    • Hi Jeff,

      The issue doesn't exist in 1.4.x - sounds like your glibc/iconv install isn't quite right.

  • Saurabh

    You saved my life. Many Many thanks to you guys.

    I've been using 1.1.6 and print function never worked. My client was warning me and you saved my life.

  • Pingback: Fixing problems with Magento creating empty invoice PDFs | Sweet-Apple()

  • If you find you are still having problems with generating PDFs, you might want to check that the iconv php function is correctly configured on your webhost. There's a full post detailing the issue and resolution here:

    http://www.sweet-apple.co.uk/fixing-problems-with-magento-creating-empty-invoice-pdfs/

    This simple script will tell you if iconv is working correctly It should output "Converted: Some text". If it does anything else, you need to get your host to sort out the iconv functionality. It most often occurs when your are running Apache chrooted...

    <php
    error_reporting(E_ALL);
    $string = "Some text";
    $newstr = iconv("UTF-8", "UTF-16BE", $string);
    print "Converted: " .$newstr;

    Hope that helps someone else. Took me a couple of hours to get to the bottom of it...