Uploaded image for project: 'Sakai'
  1. Sakai
  2. SAK-44657

Add negative caching to UserDirectoryProvider

    XMLWordPrintable

    Details

    • 21 Status:
      Resolved
    • 20 status:
      Resolved
    • Test Plan:
      Hide

      Requires developer testing, and the ability to delete a user from your LDAP/AD source.

      1. as the user who will be deleted, create a bunch of content in various sites
        1. ensure those sites have other users enrolled who can access the site after the user above has been deleted from LDAP
      2. delete the user from LDAP
      3. browse through the sites and content created or modified by the deleted user
      4. check the admin memory tool for the negative cache usage and hit rates
      5. alternatively, you could turn on debug logging to verify the negative cache is being used correctly
      Show
      Requires developer testing, and the ability to delete a user from your LDAP/AD source. as the user who will be deleted, create a bunch of content in various sites ensure those sites have other users enrolled who can access the site after the user above has been deleted from LDAP delete the user from LDAP browse through the sites and content created or modified by the deleted user check the admin memory tool for the negative cache usage and hit rates alternatively, you could turn on debug logging to verify the negative cache is being used correctly

      Description

      When a user is deleted in LDAP/AD and not also cleaned up in the Sakai database, Sakai will constantly ask the LDAP provider(s) for the deleted user. It cannot be cached normally, because LDAP doesn't return the user details and thus a User object cannot be created for storage in the cache.

      This can present performance problems on your LDAP server(s) (and consequently may slow down your Sakai nodes) if Sakai is querying constantly for either a lot of deleted users, or the same deleted users repeatedly. In one such instance, our Sakai instance was bombarding our LDAP server with over 75,000 requests per hour.

      One solution to this is that when a user record is deleted in LDAP/AD, it must also be cleaned up in Sakai. However, this may not be an option for some institutions who have data retention policies in effect: they may not be legally allowed to clean up the user data in Sakai for various reasons.

      Another more progressive solution is to implement a negative cache in Sakai for the UserDirectoryProvider. The way a negative cache works is that if the LDAP provider(s) does not return a result for a particular user, we cache this negative return. If the code recognizes that we've queried for the same user and received a negative result 3 or more times from the provider(s), don't ask the provider(s) for this user again until the TTL (time to live) for the cache expires. In effect, LDAP will only be asked 3 times per TTL per user for deleted accounts, rather than every time.

      This code was developed by Sam/Longsight.

      Set the cache properties via sakai.properties. If using UnboundID (20.x, 21.x, 22.x):

      memory.org.sakaiproject.unboundid.UnboundidDirectoryProvider.negativeCache=timeToLiveSeconds=86400,timeToIdleSeconds=0,maxElementsInMemory=100000

      If using JLDAP (20.x):

      memory.edu.amc.sakai.user.JLDAPDirectoryProvider.negativeCache=timeToLiveSeconds=86400,timeToIdleSeconds=0,maxElementsInMemory=100000

        Gliffy Diagrams

          Zeplin

            Attachments

              Activity

                People

                Assignee:
                bjones86 Brian Jones
                Reporter:
                bjones86 Brian Jones
                Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                  Dates

                  Created:
                  Updated:
                  Resolved:

                    Git Integration