Report from Leif Johansson:
Subject: Possible problem with utf8 encoding in BasicSqlService
Date: August 11, 2005 5:49:14 AM EDT
While setting up our 2.0 we discovered what looks (to us at least) a problem with utf8 encoding in BasicSqlService. Our environment is
jdk (1.4.x) + mysql 4.1.* both running in utf8 locale on linux boxes.
The problem we have is as follows: When we create a resource (say an
announement) with non-us-ascii in the subject (the corresponding XML
attribute is not base64-encoded - this is an important point - for
storage) the resulting XML is truncated by as many bytes as there are 2-byte utf8 characters in the subject. We have verified that this
truncation happens in sakai, neither in the jdbc driver nor in the
This led us to investigate BasicSqlService where we found that in
prepareStatement PreparedStatement#setCharacterStream is used to set
the value of string variables in the SQL statment. The problem is that
the number of chars to transfer to the statement is calculated from
the length of the String which is locale-dependent.
We suggest using setBytes instead. Using a stream is pointless in this
situation anyway since the full data is already in memory.
Patch against the 2.0 branch is enclosed.