[SAK-11960] Documentation- Recent d/l of Oracle driver needed Created: 16-Oct-2007  Updated: 27-Oct-2009  Resolved: 26-Feb-2008

Status: CLOSED
Project: Sakai
Component/s: Content, Documentation/Reference (non-Help)
Affects Version/s: 2.5.0
Fix Version/s: 2.6.0

Type: Task Priority: Blocker
Reporter: Peter A. Knoop Assignee: Unassigned
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

On qa3-us, which uses a 2.4 db conversion, a pre-exisitng account is unable to upload a file to Resources in MyWorkspace (see stack trace in comment below.) A newly created account sees an error in Resources in MyWorkspace, "Alert: Can not find this folder."

On qa4-us, which dumps the db between each tag things work fine. They also work fine for both pre-existing and new users on qa2-osp, which uses a db snapshot.



 Comments   
Comment by Peter A. Knoop [ 16-Oct-2007 ]

stack trace from qa3-us:

org.sakaiproject.portal.api.PortalHandlerException: org.sakaiproject.tool.api.ToolException
at org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:889)
caused by: org.sakaiproject.tool.api.ToolException
at org.sakaiproject.cheftool.ToolServlet.doGet(ToolServlet.java:227)
caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
caused by: java.lang.NullPointerException
at org.sakaiproject.content.tool.ResourcesAction.createResources(ResourcesAction.java:1334)
at org.sakaiproject.content.tool.ResourcesAction.finishAction(ResourcesAction.java:7172)
at org.sakaiproject.content.tool.ResourcesAction.buildMainPanelContext(ResourcesAction.java:4745)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at org.sakaiproject.cheftool.VelocityPortletPaneledAction.toolModeDispatch(VelocityPortletPaneledAction.java:392)
at org.sakaiproject.cheftool.ToolServlet.doGet(ToolServlet.java:227)
at org.sakaiproject.cheftool.VelocityPortletPaneledAction.doGet(VelocityPortletPaneledAction.java:1005)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at org.sakaiproject.vm.ComponentServlet.service(ComponentServlet.java:56)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:555)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:691)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:469)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:364)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:301)
at org.sakaiproject.tool.impl.ActiveToolComponent$MyActiveTool.forward(ActiveToolComponent.java:459)
at org.sakaiproject.portal.charon.SkinnableCharonPortal.forwardTool(SkinnableCharonPortal.java:1340)
at org.sakaiproject.portal.charon.handlers.ToolHandler.doTool(ToolHandler.java:163)
at org.sakaiproject.portal.charon.handlers.ToolHandler.doGet(ToolHandler.java:86)
at org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:889)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:592)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:595)

user: 2a9dac51-94bb-4fc2-0019-02e2f64b39ef

usage-session: 0e5e4ffd-6d08-43e0-a2d3-94e2cdb91a3c

time: Oct 16, 2007 15:43:31

Comment by Megan May [ 16-Oct-2007 ]

This appears to be a problem with any new sites. For instance, I created a new course site with an existing user (mmmay) and that site also displayed the error message

Comment by Megan May [ 16-Oct-2007 ]

Bumping up to Blocker priority

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

I have added Andrew and Ian as watchers.

Can we get more info, Andrew? What is meant by "database conversion"? Was the conversion utility run or is this the conversion script?

Also, there are lots of null pointer errors and database read errors in catalina.out (http://qa3-us.sakaiproject.org/logs/qa3/catalina.out) before anything shows up in the UI. They seem to be null results reading from various tables, but especially the CHS tables trying to read the BINARY_ENTITY field. Any ideas, Ian?)

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

The CHS init seems to go smoothly:

INFO: DB Connection passes UTF-8 tests (2007-10-15 16:36:20,127 main_org.sakaiproject.content.impl.DbContentService)
INFO: Migration of data to the Binary format will be performed by this node (2007-10-15 16:36:20,317 main_org.sakaiproject.content.impl.DbContentService)
INFO: init(): site quota: 1048576 body path: /opt/tomcat-qa3/content volumes: vol1, vol2, vol3, (2007-10-15 16:36:20,667 main_org.sakaiproject.content.impl.BaseContentService)
INFO: init(): tables: CONTENT_COLLECTION CONTENT_RESOURCE CONTENT_RESOURCE_BODY_BINARY CONTENT_ENTITY_GROUPS locks-in-db: false bodyPath: /opt/tomcat-qa3/content (2007-10-15 16:36:20,805 main_org.sakaiproject.content.impl.DbContentService)

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

Are properties files posted anywhere for this server?

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

Here's the first exception and stack trace:

INFO: Entity Scan for /content~78453e33-03e2-46ab-920c-af06d475d3fd for /content~78453e33-03e2-46ab-920c-af06d475d3fd (2007-10-16 12:22:18,289 http-8086-Processor21_org.sakaiproject.entity.impl.EntityManagerComponent)
WARN: readResource(): Failed to parse Entity using SAX (2007-10-16 12:22:18,505 http-8086-Processor21_org.sakaiproject.util.BaseDbDualSingleStorage)
WARN: readResource(): (2007-10-16 12:22:18,548 http-8086-Processor21_org.sakaiproject.util.BaseDbDualSingleStorage)
org.sakaiproject.entity.api.serialize.EntityParseException: Failed to parse Entity using SAX
at org.sakaiproject.util.EntityReaderAdapter.parse(EntityReaderAdapter.java:88)
at org.sakaiproject.util.BaseDbDualSingleStorage.readResource(BaseDbDualSingleStorage.java:241)
at org.sakaiproject.util.BaseDbDualSingleStorage$3.readSqlResultRecord(BaseDbDualSingleStorage.java:481)
at org.sakaiproject.db.impl.BasicSqlService.dbRead(BasicSqlService.java:568)
at org.sakaiproject.db.impl.BasicSqlService.dbRead(BasicSqlService.java:468)
at org.sakaiproject.util.BaseDbDualSingleStorage.loadResources(BaseDbDualSingleStorage.java:474)
at org.sakaiproject.util.BaseDbDualSingleStorage.getResource(BaseDbDualSingleStorage.java:315)
at org.sakaiproject.util.BaseDbDualSingleStorage.editResource(BaseDbDualSingleStorage.java:896)
at org.sakaiproject.util.BaseDbDualSingleStorage.putResource(BaseDbDualSingleStorage.java:632)
at org.sakaiproject.content.impl.DbContentService$DbStorage.putCollection(DbContentService.java:1244)
at org.sakaiproject.content.impl.BaseContentService.addValidPermittedCollection(BaseContentService.java:2029)
at org.sakaiproject.content.impl.BaseContentService.enableResources(BaseContentService.java:7420)
at org.sakaiproject.content.impl.BaseContentService.contextCreated(BaseContentService.java:7366)
at org.sakaiproject.site.impl.BaseSiteService.enableRelated(BaseSiteService.java:1783)
at org.sakaiproject.site.impl.BaseSiteService.doSave(BaseSiteService.java:840)
at org.sakaiproject.site.impl.BaseSiteService.getSite(BaseSiteService.java:670)
at org.sakaiproject.site.impl.BaseSiteService.getSiteVisit(BaseSiteService.java:696)
at org.sakaiproject.site.cover.SiteService.getSiteVisit(SiteService.java:130)
at org.sakaiproject.portal.util.PortalSiteHelper.getSiteVisit(PortalSiteHelper.java:559)
at org.sakaiproject.portal.charon.handlers.SiteHandler.doSite(SiteHandler.java:156)
at org.sakaiproject.portal.charon.handlers.SiteHandler.doGet(SiteHandler.java:114)
at org.sakaiproject.portal.charon.SkinnableCharonPortal.doGet(SkinnableCharonPortal.java:889)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:690)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:803)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:269)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.sakaiproject.util.RequestFilter.doFilter(RequestFilter.java:592)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:215)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:188)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:210)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:174)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:117)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:108)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:151)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:870)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:665)
at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:528)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:81)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.lang.NullPointerException
at java.io.StringReader.<init>(StringReader.java:33)
at org.sakaiproject.util.Xml.processString(Xml.java:179)
at org.sakaiproject.util.EntityReaderAdapter.parse(EntityReaderAdapter.java:83)
... 40 more
WARN: putResource(): didn't get a lock! (2007-10-16 12:22:18,574 http-8086-Processor21_org.sakaiproject.util.BaseDbDualSingleStorage)

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

I ran a 2.4.x instance in which I created a few sites with various tools including the Resources tool and DropBox. I shut down sakai, ran the mysql conversion script on the database, and brought up an instance of trunk (which should be pretty much the same as QA tag 009). I have not gotten any exceptions from the resources tool or Content Hosting. I added folders and resources to sites that were created in 2.4.x and in trunk.

Please provide additional information about how to reproduce this problem.

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

Can someone get a list of the columns and indexes that currently exist in the CONTENT_COLLECTION and CONTENT_RESOURCE tables on qa3-us?

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

mysql> show columns from content_resource;
-------------------------------------------------+

Field Type Null Key Default Extra

-------------------------------------------------+

RESOURCE_ID varchar(255) NO PRI    
RESOURCE_UUID varchar(36) YES MUL NULL  
IN_COLLECTION varchar(255) YES MUL NULL  
FILE_PATH varchar(128) YES   NULL  
XML longtext YES   NULL  
BINARY_ENTITY blob YES   NULL  
FILE_SIZE bigint(20) YES   NULL  
CONTEXT varchar(99) YES MUL NULL  
RESOURCE_TYPE_ID varchar(255) YES MUL NULL  

-------------------------------------------------+

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

I think the oracle equivalent of "show columns from content_resource;" might be this:

select * from user_tab_columns where table_name = 'CONTENT_RESOURCE' ;

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

Most of my comments above have focused on the underlying problem (the second stack trace above). An exception is being thrown by EntityParseException.parse() and caught by BaseDbDualSingleStorage.readResource() at line 242 or so. The catch block logs an error message plus a stack trace and then returns null. I have to follow that back, but it seems that the result back up the call stack is that CHS.addResource returns a null entity instead of throwing an exception. I think this is new behavior. We could test for a null entity back in the Resources tool, but I don't think that's the right response. I think we need to deal with the problem that is causing the parse exception and then see if it's possible to recover and create the resource in spite of that and then (if all else fails) get CHS.addResource to throw an appropriate exception instead of returning null.

Comment by Jim Eng (Inactive) [ 16-Oct-2007 ]

Just to clarify, the second stack trace above occurs during an attempt to add a folder for the new site's resources collection as a result of a call to ContextObserver.contextCreated(), where the context-observer is the ContentHostingService. But the same problem is occurring each time we try to add a resource or collection (in a new site?). It occurs just before the problem in the first stack trace above, the one that shows up in the UI. Here's a sample:

WARN: readResource(): Failed to parse Entity using SAX (2007-10-16 15:19:56,240 http-8086-Processor21_org.sakaiproject.util.BaseDbDualSingleStorage)
WARN: readResource(): (2007-10-16 15:19:56,240 http-8086-Processor21_org.sakaiproject.util.BaseDbDualSingleStorage)
org.sakaiproject.entity.api.serialize.EntityParseException: Failed to parse Entity using SAX
at org.sakaiproject.util.EntityReaderAdapter.parse(EntityReaderAdapter.java:88)
at org.sakaiproject.util.BaseDbDualSingleStorage.readResource(BaseDbDualSingleStorage.java:241)
at org.sakaiproject.util.BaseDbDualSingleStorage$3.readSqlResultRecord(BaseDbDualSingleStorage.java:481)
at org.sakaiproject.db.impl.BasicSqlService.dbRead(BasicSqlService.java:568)
at org.sakaiproject.db.impl.BasicSqlService.dbRead(BasicSqlService.java:468)
at org.sakaiproject.util.BaseDbDualSingleStorage.loadResources(BaseDbDualSingleStorage.java:474)
at org.sakaiproject.util.BaseDbDualSingleStorage.getResource(BaseDbDualSingleStorage.java:315)
at org.sakaiproject.util.BaseDbDualSingleStorage.editResource(BaseDbDualSingleStorage.java:896)
at org.sakaiproject.util.BaseDbDualSingleStorage.putResource(BaseDbDualSingleStorage.java:632)
at org.sakaiproject.content.impl.DbContentService$DbStorage.putResource(DbContentService.java:1510)
at org.sakaiproject.content.impl.BaseContentService.addResource(BaseContentService.java:3413)
at org.sakaiproject.content.impl.BaseContentService.addResource(BaseContentService.java:3259)
at org.sakaiproject.content.cover.ContentHostingService.addResource(ContentHostingService.java:1009)
at org.sakaiproject.content.tool.ResourcesAction.createResources(ResourcesAction.java:1322)
at org.sakaiproject.content.tool.ResourcesAction.finishAction(ResourcesAction.java:7172)
at org.sakaiproject.content.tool.ResourcesAction.buildMainPanelContext(ResourcesAction.java:4745)

So the call to CHS.addResource() at line 1322 of ResourcesAction.createResources() returns null instead of throwing an exception. The result of that is a null pointer exception at line 1334 of ResourcesAction.createResources().

Comment by Jim Eng (Inactive) [ 17-Oct-2007 ]

Ian forwarded an email exchange with Karen Tsao from last week. She was having a similar problem in which both the binary_entity and XML field were coming back null using Oracle. After trying a number of other things, Karen replaced the Oracle driver, and that solved the problem. So Ian suggests the next step should be replacing the Oracle driver used in the QA3-US server. So I'm assigning to Andrew asking that we try that. Thanks.

Comment by Andrew Poland (Inactive) [ 17-Oct-2007 ]

I restarted qa3-us with the 11g 11.1.0.6.0 driver.

Comment by Megan May [ 17-Oct-2007 ]

Newly created spaces for users and sites no longer exhibit this problem.

Accounts created under the old driver load the tool without the error, but clicking an action (add folder) just refreshes the page.

Comment by Peter A. Knoop [ 17-Oct-2007 ]

Andrew, can you provide the version for the driver you were using before, which exhibited the problem? Thanks.

Comment by Megan May [ 23-Oct-2007 ]

According to Ian on the weekly call, an oracle driver from the past 6 months should do the trick. Anthony, can you work this into the documentation

Comment by Filter Maintainer (Inactive) [ 21-Feb-2008 ]

[Bulk Comment] The Target Version for this issues is being reset to Unknown, as though this issue currently has 2.5.0 listed as a Target Version, it is not likely that further work will be incorporated into the 2.5.0 release at this point, and this issue is still Unresolved. Please update the Target Version when you have a new estimate of when the issue is likely to be addressed.

Comment by Peter A. Knoop [ 26-Feb-2008 ]

r41650

Comment by Megan May [ 20-Sep-2008 ]

Peter, this info was included 2.5.0, right?

Generated at Wed Oct 16 22:44:02 CDT 2019 using Jira 8.0.3#800011-sha1:073e8b433c2c0e389c609c14a045ffa7abaca10d.