[SAK-37031] Students can take test even after the instructor has clicked "stop accepting now" Created: 16-Jan-2017  Updated: 17-Apr-2018  Resolved: 15-Dec-2017

Status: Verified
Project: Sakai
Component/s: Tests & Quizzes (Samigo)
Affects Version/s: 11.2, 12.0, 19.0
Fix Version/s: 11.5 [Tentative], 12.0, 19.0

Type: Bug Priority: Blocker
Reporter: Sristi Shrestha Assignee: Sam Ottenhoff
Resolution: Fixed Votes: 0
Labels: Autosubmit, needstesting, timers
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Tested in Sakai 11.2 on MySql, trunk (master) on MySql, windows 10, chrome.


Attachments: PNG File image-2017-12-14-20-44-29-602.png     PNG File image-2017-12-14-20-45-14-239.png     PNG File image-2017-12-14-20-45-35-548.png     PNG File image-2017-12-14-20-46-20-295.png     PNG File image-2017-12-14-20-47-03-554.png     PNG File image-2017-12-14-20-50-14-661.png    
Issue Links:
Depend
is depended on by SAK-37166 autoSubmitAssessments doesn't run bec... Verified
12 status: Resolved
11 status: Resolved
Previous Issue Keys: SAM-3120
Test Plan:
  1. In a site with at least two students, publish an assessment with Due Date in a week or so, other settings default.
  2. Start the test as Student 1, enter at least one answer, Save but DON'T submit.
  3. As instructor, go to Published Copies, ensure that the student's test is in progress.
  4. Edit Settings on the published assessment.
  5. Select the Autosubmit checkbox.
  6. Click Stop accepting now.
  7. Login as Student 2 to see if they can take assessment or not.
  8. Expected behavior is student should NOT be allowed to take assessment after instructor has clicked Stop accepting now.
    • If Issue present: Students can still take assessment.
    • If issue fixed: Student does not see assessment listed in Tests & Quizzes
  9. Log in as admin and trigger the Autosubmit job.
  10. Log in as instructor and go to Published Copies > Scores confirm Student 1's submission is submitted.
  11. Log in as Student 1 and confirm submission displays in submitted list.

 Description   

Stop Accepting Now button does not work correctly to set the Retract (aka latest submission) date and trigger Autosubmit (if Autosubmit enabled).

Note: For one of the testing steps in test plan, Autosubmit needs to be enabled on server.



 Comments   
Comment by Neal Caidin [ 05-Sep-2017 ]

How long after the instructor clicks "Stop accepting now" does Samigo continue accepting assessments? Is it one minute, 10 minutes, indefinite?

Comment by Matthew Jones [ 05-Sep-2017 ]

Do you also confirm that you want to stop accepting (Step after 3)? Does anything happen to the settings after stopping?

Comment by Neal Caidin [ 07-Sep-2017 ]

I can reproduce the problem on mysql master qa server on nightly. Build - Sakai - fe48a19d - Sakai 13-SNAPSHOT - Server ip-172-31-6-159

It does not seem to save the "Stop Accepting Now" parameter, because if you go back into Settings, you can choose that option again. Nothing is changed about the Due date (I assume it would be the way this works?).

Comment by Neal Caidin [ 21-Nov-2017 ]

I'll see if this is fixed (incorporated) since we know other Samigo fixes have happened recently.

 

Comment by Tiffany Stull [ 08-Dec-2017 ]

The issue here is that the Stop Accepting Now button needs to trigger a fill of the retract date to Now (it also displays this date in the late submission date field).  This forces retract, both stopping students from taking the test and autosubmitting any saved in progress tests if autosubmit is enabled.

If Stop Accepting Now does not trigger a retract date change to the moment it was clicked, the button has no effect.

Another question: What should be the behavior for students with date/time exceptions?  Is it expected that Stop Accepting Now will similarly truncate their due date if late submissions are NOT accepted AND due date field in exceptions is left blank?!  Additional testing steps will need to be added once the intended behavior for the date/time exceptions is indicated.

Comment by Andrea Schmidt [ 14-Dec-2017 ]

The second student does not see the quiz after the instructor clicks to stop accepting now.

The first student was able to continue clicking in the test after the instructor clicked stop accepting now and the auto-submit job ran.

Site: ENIF_540K_0613, Sakai - ab7fbaf8 - Sakai 13-SNAPSHOT - Server ip-172-31-6-159

Quiz: 3120-B, Student1: ndrew

Initial due/late accept date: 12/21/2017 12:00 am

Instructor clicked stop accepting now: 12/14/2017 8:25 PM - both the due and late fields updated to "now"

Ran auto-submit:

Instructor view of assessments:

While writing up this up, I left the test open with student1 and just now clicked next again. Student is still able to continue in test:

From logs:
14-Dec-2017 20:26:40.906 WARN [QuartzScheduler_Worker-2] org.sakaiproject.component.app.scheduler.jobs.SpringJobBeanWrapper.execute Non-stateful wrapper used with stateful job: AutoSubmitAssessments You probably wanted to use SpringStatefulJobBeanWrapper for this job.
14-Dec-2017 20:26:41.044 INFO [QuartzScheduler_Worker-2] org.sakaiproject.tool.assessment.services.AutoSubmitAssessmentsJob.execute Start Job: AutoSubmitAssessmentsJob $ Job: Auto Submit Assessments Job Trigger: MT_8g6gbcg9l9lb6 Fire scheduled: Thu Dec 14 20:26:40 EST 2017 Fire actual: Thu Dec 14 20:26:40 EST 2017
14-Dec-2017 20:26:41.080 INFO [QuartzScheduler_Worker-2] org.sakaiproject.tool.assessment.services.AutoSubmitAssessmentsJob.execute End Job: AutoSubmitAssessmentsJob $ Job: Auto Submit Assessments Job Trigger: MT_8g6gbcg9l9lb6 Fire scheduled: Thu Dec 14 20:26:40 EST 2017 Fire actual: Thu Dec 14 20:26:40 EST 2017 (0 failures)

 

Comment by Sam Ottenhoff [ 15-Dec-2017 ]

"The first student was able to continue clicking in the test after the instructor clicked stop accepting now and the auto-submit job ran." => This should be a new issue

Comment by Matthew Jones [ 15-Dec-2017 ]

Sam Ottenhoff That issues sounds a lot like SAM-3134. I feel like to fix we'd need a samigo communicating more with the server and something that could be implemented if/when the UVA timer improvements got in

Comment by Tiffany Stull [ 15-Dec-2017 ]

The most recent issue identified here (in progress test not submitted by Autosubmit and student still able to navigate through multiple pages of test after Stop Accepting is clicked) sounds like a new bug.

Expected behavior is that following the Stop Accepting (Autosubmit trigger), as soon as the student attempts to click a navigation button or Save, they should see a message indicating that their test has already been submitted and they can't continue taking it (this behavior/message exists in Sakai 11 and earlier).  Any data saved prior to the instructor's Stop Accepting click should be autosubmitted.  Anything the student tried to enter/save after the Autosubmit should NOT be saved/submitted.

Comment by Andrea Schmidt [ 15-Dec-2017 ]

I retested this evening, 2 separate tests:

1) only 1 submission allowed

2) unlimited submissions allowed

Student was in the quiz when the instructor clicked to stop accepting

Ran the auto-submit job

When the student clicked save, "Assessment has been submitted" appeared

The problem I saw yesterday is not present today, so no new Jira

Generated at Mon Sep 23 09:49:30 CDT 2019 using Jira 8.0.3#800011-sha1:073e8b433c2c0e389c609c14a045ffa7abaca10d.