[SAK-36196] Copy to Pool for imported essay question not working Created: 28-Sep-2017  Updated: 17-Apr-2018  Resolved: 08-Feb-2018

Status: CLOSED
Project: Sakai
Component/s: Tests & Quizzes (Samigo)
Affects Version/s: 11.4, 12.0, 19.0
Fix Version/s: 11.5 [Tentative], 12.0, 19.0

Type: Bug Priority: Blocker
Reporter: Sean Horner Assignee: Miguel Pellicer
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File SAM-3264-exception-20180112.txt     Text File SAM-3264-exception.txt     XML File multiplequestions-all1part.xml     Zip Archive multiplequestions-all1part.zip    
12 status: Resolved
11 status: Resolved
Previous Issue Keys: SAM-3264
Test Plan:

A. Create a question pool

  1. In T&Q click the “Question Pools” tab.
  2. Click “Add New Pool”.
  3. Enter “CTP Test” for the “Pool Name”.
  4. Click “Save”.

B. Create the quiz with an essay question

  1. Click the “Assessments” tab.
  2. Enter “CTP Test (Exported)” for the “Assessment Title”.
  3. Click “Create”.
  4. Select “Short Answer/Essay” from the “Insert New Question” dropdown menu.
  5. Enter “1.0” for the “Answer Point Value”.
  6. Enter “Some prompt” for the “Question Text”.
  7. Click “Save”.

C. Export the quiz

  1. Click the “Assessments” tab.
  2. Select “Export” for the “CTP Test (Exported)” assessment.
  3. Either leave the export type to be “QTI v 1.2” or select “Content Packaging”.
  4. Click “Export”. (If you choose “QTI v1.2”, click “Download” in the popup window, and then close the window.)

D. Distinguish the original quiz

  1. Navigate back to the T&Q assessments list.
  2. Select “Settings” for the “CTP Test (Exported)” assessment.
  3. Expand the “About this Assessment” section if it isn’t already.
  4. Edit the “Title” to replace “CTP Test (Exported)” with “CTP Test (Original)”.
  5. Click “Save”.

E. Import the quiz

  1. On the T&Q assessment list view, click “Import”.
  2. Click “Choose File” and select the file exported above.
  3. Click “Import”.

F. Copy to pool

  1. On the T&Q assessment list view, select “Edit” for the  “CTP Test (Exported)” assessment that has just been imported.
  2. Note that essay question was successfully imported.
  3. Click “Copy to Pool” for Part 1.
  4. Click the checkbox for the “CTP Test” pool.
  5. Click “Copy”.
  6. Note the Tomcat catalina.out log for multiple exceptions thrown correlated with the event of clicking “Copy”.
  7. Click the “Question Pools” tab.
  8. Note the “Questions” column for the “CTP Test” pool which unexpectedly reports “0” instead of “1” for the imported essay question in the attempt to copy it to the question pool.

 Description   

The “Copy to Pool” function (associated with a “part”) does not work in 12.x for an imported assessment that contains a Short Answer/Essay question type. The problem occurs if the assessment is exported and then imported, whether via QTI or content packaging.

When the steps in the “Test Plan” are followed on the nightly server for 12.x, the intended question to be copied to a question pool via the “Copy to Pool” link does not copy.

By way of contrast with the “Copy to Pool” method in the “Test Plan”, an imported “Short Answer / Essay” question type can be copied successfully to the pool with the following steps:

  1. Navigate to Tests & Quizzes.
  2. Select "Edit" for the imported assessment that contains a Short Answer / Essay question.
  3. Click the "Edit" link associated with the imported Short Answer / Essay question.
  4. Scroll down to the “Assign to Question Pool” dropdown menu and select an empty question pool.
  5. Click “Save”.
  6. Click the “Question Pools” tab.
  7. Note that the “Questions” column for the previously empty pool now has 1 question therein, as would be expected.


 Comments   
Comment by Matthew Jones [ 24-Oct-2017 ]

Lowering the priority of this to critical as this isn't a critical workflow of the tool.

Comment by Earle R Nietzel [ 24-Oct-2017 ]

Can you provide the exceptions you mention at this step:

6. Note the Tomcat catalina.out log for multiple exceptions thrown correlated with the event of clicking “Copy”.

Comment by Sean Horner [ 24-Oct-2017 ]

For the 12.x instance on nightly, I'm able to reproduce the bug but not the corresponding exception alluded to in step F6. (I was pretty sure that at the time of filing this bug, the exception was being thrown on the nightly server.)

With that said, I observe an exception thrown on a local instance of 12.x, (today' build) which I've attached as SAM-3264-exception.txt. Accordingly, the key snippet of that stack trace is the following:

Caused by: java.lang.NullPointerException
at org.sakaiproject.tool.assessment.facade.QuestionPoolFacadeQueries.copyItemFacade(QuestionPoolFacadeQueries.java:1219)

Comment by Tiffany Stull [ 12-Jan-2018 ]

I'm not sure I understand the issue - does the Copy to Pool for a Part fail if multiple questions are part of an imported test and one of those questions happens to be a short answer/essay?  Do ALL questions fail to copy in this case, or just the short answer/essay questions?  Or does it only fail if a single question is imported?

I think instructors with many questions in a test are much more likely to use the Copy to Pool option if there's more than one question than just a single one.  For example, I'd be likely to do the following:

  1. Create a question pool in a site.
  2. Export a quiz like this one, where there are 25 questions, two of which are short answer/essay: multiplequestions-all1part.zip
  3. Import this quiz to the site.
  4. Edit the quiz and Copy to Pool to get all the questions in a pool.
  5. If issue present: What copies?  Nothing?  All questions except for Short Answer/Essay?
Comment by Sean Horner [ 12-Jan-2018 ]

Tiffany Stull, I wrote the Test Plan to minimize the number of steps it takes to trigger an exception/error, so those steps include only one essay question, which I agree is not the most likely use case. 

What I'm observing is that if you have multiple questions in an imported assessment, the position or order for where the Short Answer / Essay question is listed in the sequence of questions matters. For example, let's say you have two questions in a quiz, one that's True/False and the other that's Short Answer / Essay. If the T/F question is the first question, it would be copied to the pool but the essay question won't. However, if the T/F is listed second, neither question would be copied to the pool. 

I think this is because the event of encountering an essay question during copy to the pool process is triggering an exception at which point any further action for copying to the pool is disrupted.

I've attached another stack trace that I triggered this morning on a 12.x nightly server (https://qa12-mysql.nightly.sakaiproject.org). See SAM-3264-exception-20180112.txt.

Comment by Sean Horner [ 12-Jan-2018 ]

Tiffany Stull, for what it's worth, I've confirmed my observation above by importing multiplequestions-all1part.zip into a 12.x site I have on https://qa12-mysql.nightly.sakaiproject.org. Because the first Short Answer / Essay question is listed as Question 19 out of 25 questions, only the first 18 questions are copied to the pool when the cited bug is present. 

I hope that helps.

 

Comment by Tiffany Stull [ 12-Jan-2018 ]

Interesting.  Because the error affects all questions below the short answer/essay, I think it would not be unreasonable to call this a blocker.  The reason being that depending on the position of the question affected question, and how many questions exist in the pool prior to the copy attempt, the instructor might never notice the silent failure.

I have seen several instructors use Random draw from question pool, and draw ALL questions in the pool, just to get the additional randomization option for a multiple submission test of randomizing the questions on a per-submission basis.  If they don't realize some questions are missing prior to students taking the test, it could be a problem.

Comment by Benjamin Rappleyea [ 02-Feb-2018 ]

I found this to be true and agree with Tiffany Stull this should definitely be considered a blocker as it is a regression from the way the tool is designed and works in previous versions. 

I also found that if you attempt to copy an imported set of questions (containing a short answer/essay question) to a sub-pool the result is a stack trace.

Comment by Benjamin Rappleyea [ 09-Feb-2018 ]

Verified in Sakai 13 Master MySQL. Working great. Good work Miguel!

Generated at Thu Sep 19 00:18:35 CDT 2019 using Jira 8.0.3#800011-sha1:073e8b433c2c0e389c609c14a045ffa7abaca10d.