Uploaded image for project: 'Sakai'
  1. Sakai
  2. SAK-27723

Replace Legacy Search with an ElasticSearch implementation

    XMLWordPrintable

    Details

    • Type: Task
    • Status: RESOLVED
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 10.0
    • Fix Version/s: 10.0
    • Component/s: Search
    • Labels:

      Description

      Drop in replacement for sakai's legacy search service which uses ElasticSearch (http://elasticsearch.org)

      This service runs embedded in Sakai it does not require a standalone search server. Like the legacy search service docs are indexed by notification sent via the event service. This handoff simply puts the reference and associated metadata into ES. A timer task runs and looks for docs that do not have content, and then digests the content and stores that in the ES index. This task is necessary to off load this work from any user initiated thread so in the case of large files we aren't holding on to user threads for log periods of time.

      To enable search set the following (this is no different than previously):

      search.enable = true

      To expose the ElasticSearch rest services set the following:

      elasticsearch.http.enabled=true
      elasticsearch.http.port=9200

      This is handy for using curl commands to access administrative functions, or for using admin utilities like https://github.com/lukas-vlcek/bigdesk or https://github.com/Aconex/elasticsearch-head.

      Any other elasticsearch properties can be set in the sakai.properties by simply prepending the elasticsearch config name with "elasticsearch." these will then flow into the initialization. See http://www.elasticsearch.org/ for more information on options.

      For seeing elasticsearch queries and other useful stuff in the tomcat logs try:

      log.config.count=2
      log.config.1 = DEBUG.org.sakaiproject.search.elasticsearch
      log.config.2 = DEBUG.org.elasticsearch

      The bulk of this work is simply a backend replacement that fixes most of the indexing/merging problems that have been experienced in large deployments. There are two notable frontend changes. There is now an autosuggest feature that is part of the search box. There is also the ability to search for specific types by using a type followed by a colon and the search term. The type matches the entity content producer name. So for example you can search for just sites, site:"my site", or just chats, chat:dude, or just content:my_stuff.

      For implementation detail and documentation go here: https://confluence.sakaiproject.org/display/~jbush/Elasticsearch

        Gliffy Diagrams

          Zeplin

            Attachments

              Issue Links

                Activity

                  People

                  Assignee:
                  jbush John Bush
                  Reporter:
                  jbush John Bush
                  Votes:
                  0 Vote for this issue
                  Watchers:
                  5 Start watching this issue

                    Dates

                    Created:
                    Updated:
                    Resolved:

                      Git Integration