Uploaded image for project: 'Sakai'
  1. Sakai
  2. SAK-1737

MySQL UTF-8 encoding bug - breaks international characters

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: CLOSED
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.0, 1.5.0, 1.5.1, 2.0, 2.0.1, trunk
    • Fix Version/s: 2.1.0
    • Labels:
      None

      Description

      Report from Leif Johansson:

      From: leifj@it.su.se
      Subject: Possible problem with utf8 encoding in BasicSqlService
      Date: August 11, 2005 5:49:14 AM EDT
      To: sakai-user@collab.sakaiproject.org

      While setting up our 2.0 we discovered what looks (to us at least) a problem with utf8 encoding in BasicSqlService. Our environment is
      jdk (1.4.x) + mysql 4.1.* both running in utf8 locale on linux boxes.

      The problem we have is as follows: When we create a resource (say an
      announement) with non-us-ascii in the subject (the corresponding XML
      attribute is not base64-encoded - this is an important point - for
      storage) the resulting XML is truncated by as many bytes as there are 2-byte utf8 characters in the subject. We have verified that this
      truncation happens in sakai, neither in the jdbc driver nor in the
      database.

      This led us to investigate BasicSqlService where we found that in
      prepareStatement PreparedStatement#setCharacterStream is used to set
      the value of string variables in the SQL statment. The problem is that
      the number of chars to transfer to the statement is calculated from
      the length of the String which is locale-dependent.

      We suggest using setBytes instead. Using a stream is pointless in this
      situation anyway since the full data is already in memory.

      Patch against the 2.0 branch is enclosed.

        Gliffy Diagrams

          Zeplin

            Attachments

              Issue Links

                Activity

                  People

                  Assignee:
                  janderse Jon Andersen (Inactive)
                  Reporter:
                  janderse Jon Andersen (Inactive)
                  Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                    Dates

                    Created:
                    Updated:
                    Resolved:

                      Git Integration