Uploaded image for project: 'Contrib: Evaluation System'
  1. Contrib: Evaluation System
  2. EVALSYS-789

Sending of email by multiple machines (cluster aware queue drain)


    • Type: Feature Request
    • Status: CLOSED
    • Priority: Critical
    • Resolution: Non-Issue
    • Affects Version/s: 1.2
    • Fix Version/s: 1.2-UM
    • Component/s: Email / Notifications
    • Labels:


      This is a pending change to the U-M branch that is being queued for merging back into trunk.

      This item is also being tracked in the U-M Footprints tickets as CT 942:

      Entered on 05/07/2009 at 14:54:40 by Richard William Ellis:

      The idea is to change the design of email processing to distribute preparing as well as sending email over the cluster, drastically reduce transaction size, and add additional throttling. Some efficiencies might be gained tuning methods used.

      Quartz cron trigger runs one TX to get groups needing notification - this typically about 4 seconds -and write each group and it's email template as a row in EVAL_EMAIL_GROUP.

      An EVAL_LOCK in EVAL_EMAIL_GROUP is taken out by a server so that only one server is working on a group. In one TX read a a group id from EVAL_EMAIL_GROUP and write each student and email template to EVAL_EMAIL_USER. When this is done the row in EVAL_EMAIL_GROUP is deleted and the TX committed. EVAL_EMAIL_USER has a unique constraints on student + email template so that a student may appear with more than one template, but each template for a student will be unique. One email will be sent for each row in EVAL_EMAIL_USER. An EVAL_LOCK is used to make sure only one server claims a row in EVAL_EMAIL_USER. Sending of email can begin as soon as a row appears in EVAL_EMAIL_USER. The email template is filled in for the student with information such as earliest due date for that student's evaluations and then the email is sent. After sending the email a SENT flag in EVAL_EMAIL_USER is set to true. Reading a row in EVAL_EMAIL_USER, filling in the templates, sending email to the student, and marking the row as SENT constitued a TX.

      At the end when the EVAL_EMAIL_GROUP table has been drained and all rows in EVAL_EMAIL_USER have been marked SENT, delete the rows in EVAL_EMAIL_USER.

      If there is a problem with the Quartz job no rows will be saved in EVAL_EMAIL_GROUP. When teh server restarts Quartz and it restarts the job every thing should process as normal. Similarly the TimeTask will look for work when it i srunning and if there are rows to process it will process a row and process users that need email. If the TimerTask is interrupted it will pick up where it left off. Current code will be used to prevents a server from running more that one task at a time.

      It will be possible to pace the processing of rows in EVAL_EMAIL_GROUP and sending email from EVAL_EMAIL_USER, as well as to enable and disable the TimerTask. Metric logging will be used to track progress.

        Gliffy Diagrams





                Unassigned Unassigned
                demonner Sean Michael DeMonner (Inactive)
                0 Vote for this issue
                3 Start watching this issue



                    Git Integration