click here for details... Sakai Executive Director Position Search now open
Issue Details (XML | Word | Printable)

Key: SAK-15909
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Critical Critical
Assignee: Beth Kirschner
Reporter: Erica Ackerman
Votes: 0
Watchers: 2
Operations

If you were logged in you would be able to see more operations.
Sakai

Creating Portfolio Template throws stack trace (with invalid schema)

Created: 20-Mar-2009 12:10   Updated: 29-Sep-2009 15:06
Component/s: OSP: Portfolio Templates
Affects Version/s: 2.6.0
Fix Version/s: 2.6.1, 2.7.0

Time Tracking:
Not Specified

File Attachments: 1. Text File 2009.03.20.portfoliotemplate.txt (9 kB)
2. XML File generic.xsl (7 kB)
3. XML File schema.xsd (2 kB)

Issue Links:
Relate
 

2.6.x Status: Closed
2.5.x Status: None
2.4.x Status: None


 Description  « Hide
To reproduce, upload the attached xsl file, then in the Portfolio Templates tool, select it as the "Basic Template Outline" form. After attaching the xsl and returning to the "Build Template" "Step 2 of 4" page, click continue. The attached stack trace is thrown. The stack trace is thrown with other xsl files, as well. The problem does not happen on 2.5.

 All   Comments   Work Log   Change History   Subversion Commits   git Commits      Sort Order: Ascending order - Click to sort in descending order
Beth Kirschner added a comment - 23-Mar-2009 06:14
This is reproduceable in 2.6.0-beta03, but not in trunk and not in 2.6.x. We should verify this problem does not occur in the 2.6.0-beta04 release when available.

Charles Hedrick added a comment - 05-Jun-2009 07:19
Is there any hope of finding out what fixed it? I replaced all of OSP with the copy from 2.6.x, and the problem still occurs. I don't want to go to 2.6.x for all of Sakai more than once, at final release. Unfortunately this is keeping us from editing any templates on our production server.

Chris Maurer added a comment - 05-Jun-2009 07:28
What server are you trying this on? Looking at the error, it looks more like it's specific to a particular form (with a bad schema) that it's trying to display.

Chris Maurer added a comment - 05-Jun-2009 07:28
I just tried it on qa3-us and received the error.

Chris Maurer added a comment - 05-Jun-2009 08:20
There is likely a form in the system with an invalid schema. The attached error has a form with an element name of "Today'sDate", which is invalid.
On qa3-us, it appears to be from one of two forms: "Form 1", "Form_1" which are both globally published.

Chris Maurer added a comment - 05-Jun-2009 08:21
So, the quick fix is to remove the bad forms. The solution would be to not process forms with invalid schema. Though, it shouldn't have been allowed in the system if it has a bad schema.

Chris Maurer added a comment - 05-Jun-2009 08:22
Don't think this is really fixed since nothing was done to fix it!

Chris Maurer added a comment - 05-Jun-2009 08:25
This schema should allow you to recreate the problem, I think.

Chris Maurer added a comment - 05-Jun-2009 08:29
Well, the bad news is that I can't create or import a form with this schema since it's invalid!

Charles Hedrick added a comment - 05-Jun-2009 09:39
I still don't know what the cause is, but the following will fix up illegal names, and thus will work around the problem. I have no idea whether it will create other problems.

--- metaobj-impl/api-impl/src/java/org/sakaiproject/metaobj/utils/xml/impl/SchemaNodeImpl.java (revision 1484)
+++ metaobj-impl/api-impl/src/java/org/sakaiproject/metaobj/utils/xml/impl/SchemaNodeImpl.java (working copy)
@@ -78,6 +78,12 @@
       this.schemaElement = schemaElement;
       elementName = schemaElement.getAttributeValue("name");
 
+ // construct legal XML name
+ String firstc = elementName.substring(0,1);
+ if (!firstc.matches("[a-zA-Z_:]"))
+ elementName = "_" + elementName.substring(1);
+ elementName = elementName.replaceAll("[^-a-zA-Z0-9_:.]","_");
+
       // try and create an element to test the name
       try {
          new Element(elementName);

Charles Hedrick added a comment - 05-Jun-2009 09:50
I assume error checking has gotten tighter. I think you probably want to do something like my fixup. Either that or write a conversion script that finds all bad forms and fixes them. My approach is probably more practical.

Beth Kirschner added a comment - 11-Jun-2009 08:10
I am able to create a form using the invalid schema in a Sakai 2.5.x build, so it does appear that the error checking has gotten tighter.

Noah Botimer added a comment - 12-Jun-2009 13:10
This is related to SAK-12555, which was marked for merge into 2.5.x but never merged. It introduces the check that is throwing the error reported in the stack trace by performing some validation on the schema elements. This appears to be an interpretation of element name validity by JDOM (the version of which has not changed between branches). The XML specification includes some naming rules and recommendations. Apostrophes (#x27) and single right-quotes (#x2019) are explicitly recommended to be avoided in XML names, but not forbidden. Apparently, JDOM has implemented this recommendation as a requirement. The test schemas in the other tickets are genuinely invalid because an element name starts with a number.

We need to verify if the schema in question (and one with truly invalid element names) results in a usable form on 2.5.x. My suspicion is that it will not. If this is true, then the perceived tightening of the validation is actually just the choking on the schema early instead of later on form instances. If the forms are usable, then we have an issue where we are potentially breaking usable forms in deployments with the additional checking.

In any case, any schemas that result in invalid form documents should be rejected at form definition update time, gracefully, in the UI. We should also provide a utility to scan all existing definitions for invalid schemas and alert implementors.

At this point, I believe we should accept the JDOM definition of valid element names rather than aiming strictly for the requirements of the spec. Apostrophes are likely to be problematic in element names for us, since we use them as identifiers in HTML forms, so this tighter definition actually helps us.

See also:
Naming requirements: http://www.w3.org/TR/REC-xml/#NT-Name
Naming recommendations: http://www.w3.org/TR/REC-xml/#sec-suggested-names

Noah Botimer added a comment - 12-Jun-2009 13:24
I have just verified that the attached schema.xsd does not work properly on 2.5.x. It can be used to create a form definition, and forms of that type may be created but, if the Today'sDate field is submitted with any data, the submission is rejected with the following stack trace. This means that the validation rules have been applied consistently and we should build a scanner/fixer as suggested by Charles above.

org.sakaiproject.portal.api.PortalHandlerException: org.sakaiproject.tool.api.ToolException: Request processing failed; nested exception is org.jdom.IllegalNameException: The name "Today'sDate" is not legal for JDOM/XML elements: XML names cannot contain the character "'".
    at org.sakaiproject.portal.charon.handlers.ToolHandler.doPost(ToolHandler.java:63)
caused by: org.sakaiproject.tool.api.ToolException: Request processing failed; nested exception is org.jdom.IllegalNameException: The name "Today'sDate" is not legal for JDOM/XML elements: XML names cannot contain the character "'".
    at org.sakaiproject.cheftool.ToolServlet.sendToHelper(ToolServlet.java:652)
caused by: org.springframework.web.util.NestedServletException: Request processing failed; nested exception is org.jdom.IllegalNameException: The name "Today'sDate" is not legal for JDOM/XML elements: XML names cannot contain the character "'".
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:440)
caused by: org.jdom.IllegalNameException: The name "Today'sDate" is not legal for JDOM/XML elements: XML names cannot contain the character "'".
    at org.jdom.Element.setName(Element.java:206)
    at org.jdom.Element.<init>(Element.java:140)
    at org.jdom.Element.<init>(Element.java:152)
    at org.sakaiproject.metaobj.shared.mgt.impl.StructuredArtifactValidationServiceImpl.validateChildElement(StructuredArtifactValidationServiceImpl.java:198)
    at org.sakaiproject.metaobj.shared.mgt.impl.StructuredArtifactValidationServiceImpl.validateElement(StructuredArtifactValidationServiceImpl.java:226)
    at org.sakaiproject.metaobj.shared.mgt.impl.StructuredArtifactValidationServiceImpl.validate(StructuredArtifactValidationServiceImpl.java:176)
    at org.sakaiproject.metaobj.shared.mgt.impl.StructuredArtifactValidationServiceImpl.validate(StructuredArtifactValidationServiceImpl.java:60)
    at org.sakaiproject.metaobj.shared.control.XmlValidator.validate(XmlValidator.java:60)
    at org.sakaiproject.metaobj.shared.control.StructuredArtifactValidator.validate(StructuredArtifactValidator.java:64)
    at org.sakaiproject.metaobj.shared.control.AddXmlElementController.handleRequest(AddXmlElementController.java:113)
    at org.sakaiproject.metaobj.utils.mvc.impl.servlet.FormControllerImpl.onSubmit(FormControllerImpl.java:83)
...

Noah Botimer added a comment - 12-Jun-2009 14:12
As another piece of anecdotal evidence, I just validated the attached schema.xsd in Oxygen and it flags an error for the element name. The XML Schema DTD apparently prohibits apostrophes in element names, not just the JDOM implementation.

Beth Kirschner added a comment - 28-Jul-2009 11:48
In order to reproduce this problem, we need either a legacy 2.5 system, upgraded to 2.6 after defining a form with the invalid schema attached to this ticket, or alternately, create a valid form on a 2.6 system, name it 'SAK-15909', and then execute the following SQL to make use of the invalid schema:

mysql> update metaobj_form_def set schemaData=load_file('/tmp/bad-schema.xsd') where description = 'SAK-15909';

Beth Kirschner added a comment - 29-Jul-2009 11:53
Fixed by checking for invalid schema exception and handling the error appropriately upstrream. Portfolio Templates will filter out any invalid form/schemas and log an error message. Any attempt to create a form defined with an invalid schema will log an error and result in a blank screen (for now).

Perhaps an error.xsd could be created long-term that will put up a meaningful error message to the user.

Beth Kirschner added a comment - 30-Jul-2009 10:11
QA Instructions:
- verify existing behavior with valid forms:
   - verify can add valid forms in resources & matrix (or wizard or portfolio)
   - verify can create portfolio templates

- verify new behavior with forms based on invalid schemas (not a simple task -- see above comment regarding database modification or upgraded sakai)
   - verify can add valid forms in resources & matrix (or wizard or portfolio)
   - verify attempts to add invalid forms do not cause crash (blank screen displayed & error logged)
   - verify can create portfolio templates (invalid forms should not appear in list)

Beth Kirschner added a comment - 25-Aug-2009 07:41
Merged to 2.6.x