|
[
Permlink
| « Hide
]
Owen McGrath added a comment - 05-Sep-2006 11:13
Thanks for the request. We're digging into the history and feasibility of this today. Hope to have a fuller response tomorrow.
Here at Michigan, we would also like to have events enabled in the code for each meaningful user action in the Gradebook.
These events are very useful for Support auditing, user analysis, and other Support and research activities that use the log files and user events. Also, please update the JIRA ticket to include Sakai 2.3, 2.3.1 and 2.4 Thanks!! Thanks for the reminder, Steve.
We're finally starting to appreciate this important needfor usage stats (and security audits). Looking at Nuno's helpful translation table: https://source.sakaiproject.org/contrib//ufp/sitestats/trunk/sitestats-impl/impl/src/bundle/org/sakaiproject/sitestats/impl/bundle/Messages.properties makes it easy to start brainstorming about what some obvious Gradebook events would be. gradebook.newItem - gradebook assignment create gradebook.deleteItem - gradebook assignment delete gradebook.recordScore - assignment score record gradebook.comment - assignment-level comment add gradebook.override - course grade override gradebook.download - download (need two of these actually) gradebook.upload - csv upload gradebook.import - assignment import Probably want to have something for grade/score display choices. Michigan has been good at reminding everyone that the more thoroughly a tool participates, the better our usability will be. The point is well taken. So next steps will be to move through brainstorming to soliciting some community feedback. [Usual disclaimer about not being able to predict currently whether and when this will be realized in code.] One more thing: since we'll undoubtedly want to at some point to be able to study user behavior, it would be nice to have a way to track students checking their grades. Even if there is no nice way to distinguish student views (vs. instructor views), we'll still want a basic "read" event. Something like this would be ideal:
gradebook.gradereport.read Here at IU we'd also find this beneficial.
Also, let me put on my Sakai staff hat and mention that one of the desirable elements listed in the criteria for core tools is that "The tool should generate event codes that are triggered minimally on new, revise, and delete actions on the basic objects created by the tool" so this task falls in line with fulfilling that element. However, I must mention that I'm a bit concerned about where this data is actually stored - nothing to do with this particular request, but something important to note. As Owen knows, we archive the event & session data daily to our data warehouse, but given the shear volume of the information captured, only retain 1 years worth of data. If this is to be used with site stats, we'll run into problems once sites are > a year old. I share Megan's general concern, and more than a bit. An enterprise system needs a better approach to statistics gathering and analysis than dumping raw usage data indefinitely into the production DB and tying it to cluster session maintenance.
As for this particular request.... Besides needing a final list of event code strings, developers need the following: 1) The Sakai "Event" contains an event code, a timestamp, a pointer to a session record (which lets you derive the current user ID so long as the session record is still around), a "read" or "write" flag, and a single string which is meant to describe the single "resource" being acted on. For each of these events, what's the "resource ID" that's needed? 2) What's the desired granularity here? If someone submits a single update of 100 student grades, are 100 new SAKAI_EVENT records created (to go along with the100 new Grading Event log records that already get created)? 3) Should we log events caused by clients to the Gradebook service as well as events caused by the Gradebook application? If so, should we distinguish them or treat them the same? The event codes should identify the tool, so gradebook.* as above.
The resource ID (REF field in the db) should include as a miminum the context, e.g. /site/SITEID/ and other meaningful info depending on the tool. In the GB's context, if there were more than 1 GB possible per site (currently it isn't? is this planned?) it may be meaningful to include that, thereafter possibly a GB item identifier, and where relevant a student identifier. The granularity should match the event codes, though I'm wondering if 100 event records is too much for a single update, esp. as the GB keeps its own grading log anyway. So perhaps gradebook.recordScore - assignment score record gradebook.comment - assignment-level comment add gradebook.override - course grade override could be collapsed to gradebook.update This is really working backwards from the current uses of event data, which are (a) Site Stats tool and (b) data mining applications, e.g. as done by UCT for analysis such as http://issues.sakaiproject.org/confluence/x/f3k and I believe by IU for examining tool use across sites. For Q3 above, I would say that the service should log events (from the app & clients), with the exception possibly of updates to scores that come from clients like Assignments, in which case we wouldn't want double events (asn.grade.submission and gradebook.recordScore for the same thing). Stephen, any chance you'd be able to finish spec'ing this out :-) ?
Oliver, if Stephen's busy, this would be a good task for the folks in the Data Analysis WG to pick-up. You might want to send a targeted request their email list.
Excuse my ignorance...I haven't been involved in logging discussions previously and don't really understand how it works. From a usability perspective knowing the kinds of actions a user takes is important in understanding how often different activities occur. What would add a lot value to a usability analysis is the workflow happening in the tool. Do we just capture single events or can we also capture a flow of events for a particular login for instance?
Hi, Daphne.
The service is about discrete events but ideally you could align those events across time within a user's session. The key word is "ideally" .. I think a lot of people would like this to become easier. And there's room for improvement in other areas as well. Stephanie Teasley has been an advocate for this important potential that event logging has for usability. She and Emilee Rader gave a presentation in Vancouver " Panel: Sakai Project Sites: Who UsesThem and Why" showed talks about what you can and cannot glean about usability using the current event logging system. Another excellent presentation was given in Atlanta by Greg Doyle: " Online lives of students" That said, we do not want the discussion here to turn into a referendum on the event logging service. :) http://issues.sakaiproject.org/confluence/display/SCP/Tool+Status+Requirements;jsessionid=D693ED01722F7A3BBDCC2C2405EE3A05 made an initial check in added support for event logging without really having event logging yet.
checkin @ revision 21942 Thanks for the pointer to the data analysis group, Peter. We need to get this off of our plate ASAP, so Louis has started implementing the below spec. I decided to punt on the client events. Considering the different ways Assignments, T&Q and Message Center interoperate with GB, we'll leave this piece for someone else to think through.
The event names and resource IDs are pretty self explanatory with these possible exceptions *totalNumberUpdated -- record the number of scores or course grades or comments updated per save action *userAuthzLevel -- determine whether user role is equivalent to Instructor or TA 1. gradebook.newItem /gradebook/SITEID/itemID 2. gradebook.deleteItem /gradebook/SITEID/itemID 3. gradebook.updateItemScores /gradebook/SITEID/totalNumberUpdated/userAuthzLevel 4. gradebook.updateCourseGrades - /gradebook/SITEID/totalNumberUpdated/userAuthzLevel 5. gradebook.comment /gradebook/SITEID/totalNumberAddedUpdated/userAuthzLevel 6. gradebook.downloadRoster /gradebook/SITEID/userAuthzLevel 7. gradebook.downloadCourseGrade /gradebook/SITEID/userAuthzLevel 8. gradebook.importItem /gradebook/SITEID/itemID/userAuthzLevel My suggestion is that the userId should be in the REF field whenever possible. This helps tracking user activity for statistics and troubleshooting. It may be useful to know which Instructor has deleted the Gradebook, commented or imported a gradable item (Excel import?) into GB...
From the spec above, I would replace userAuthzLevel with user/userId (for example), and also add the userId reference to 1. and 2. Hope it helps, Nuno Hi Nuno,
Event records have a session ID, and SAKAI_SESSION has a reference to the userid, so why would you want the userid in the REF field? You're right!
An userId reference in the REF field should only be used if the activity action affects a specific user (and none of the above events do that!). My apologies for the confusion checked in event tracking functionality based on the spec above
checked in @ revision 22020 final check in @ revision 22044
Linked with
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
 |
|
Task
Closed
Major
Log events for meaningful user actions
