[#KNL-145] Content conversion query is very expensive

Kernel - K1

Content conversion query is very expensive

Created: 02-Apr-2009 03:45 Updated: 05-Mar-2010 02:46

Component/s:

Impl

Affects Version/s:

1.0.4

Fix Version/s:

1.0.12, 1.1.0

Time Tracking:

Not Specified

Issue Links:

Cloners

This issue cloned from:

~~SAK-13100~~ Content conversion query is very expensive

Relate

This issue is related to:

KNL-427 serious performance problem with counting null file_size

1.0.x Status:

Closed

Description

« Hide

The content conversion code is admirably constructed to allow conversion while Sakai is up. This is essential for large installations. However it uses an expensive query to determine when it can switch from using code compatible with the old version of content to code compatible with the new version.

All

Comments

Work Log

Change History

Subversion Commits

git Commits

Sort Order:

[ Permlink | « Hide ]

Peter A. Knoop added a comment - 02-Apr-2009 07:16

Jim, can you work with Anthony, after reviewing the fix attached to ~~SAK-13100~~, to get the same fix (assuming its appropriate) into K1 to address this issue in 2.6 (and beyond)? Thanks!

[ Permlink | « Hide ]

Stephen Marquard added a comment - 29-Jul-2009 22:42

This is a slightly nasty issue for sites with large content stores, as it's triggered the first time sometime tries to upload something. In our case this query takes 3min to run.

[ Permlink | « Hide ]

Stephen Marquard added a comment - 30-Jul-2009 10:21

Committed in r65390:

Based on patch from Carl Hall in ~~SAK-1300~~. Adjusts queries that check for NULL FILE_SIZE values for efficiency for databases which have many content records.

Adds a new sakai.properties option to disable FILE_SIZE checks (for example once the conversion has run successfully), viz.:

content.filesizeColumnReady=true

(note content. prefix added as compared to the patch).

[ Permlink | « Hide ]

Stephen Marquard added a comment - 30-Jul-2009 10:22

Verified on a local mysql trunk build. Should be verified with Oracle and db2 databases (without content.filesizeColumnReady=true) to ensure that the queries execute correctly.

[ Permlink | « Hide ]

David Horwitz added a comment - 01-Aug-2009 09:45

closing based on UCT production experience

[ Permlink | « Hide ]

Stephen Marquard added a comment - 03-Aug-2009 13:23

Does not handle checks correctly when autoddl is false.

[ Permlink | « Hide ]

Stephen Marquard added a comment - 03-Aug-2009 13:50

Fixed for case of autoddl off in r65486 (trunk) and r65487 (1.0.x)

[ Permlink | « Hide ]

Stephen Marquard added a comment - 09-Aug-2009 08:13

Fixed in r65687 (trunk) and r65688 (1.0.x)

[ Permlink | « Hide ]

Anthony Whyte added a comment - 08-Oct-2009 12:14

In 1.0.x branch.