[SAK-32082] Elastic Search Refactor to allow more than one index Created: 11-Jan-2017  Updated: 22-Jan-2018  Resolved: 14-Mar-2017

Project: Sakai
Component/s: Search
Affects Version/s: 12.0
Fix Version/s: 12.0

Type: Feature Request Priority: Critical
Reporter: Diego del Blanco Orobitg Assignee: Diego del Blanco Orobitg
Resolution: Fixed Votes: 0
Labels: None

Attachments: Zip Archive searchInSamigo.zip    
Issue Links:
is depended on by SAK-34808 Tagging and Search capabilities RESOLVED


Until this moment, elasticsearch has only one index, the one used by the search tool, and due to the way it is implemente, it doesn't allows new indexes.

The actual sakai index is site related, so there are entities that can't be referenced, like questions in question pools, that are not site related... We are working in a project that needs to search in the samigo questions and we want to use ElasticSearch for that, but we can't add the questions to the main index.

For that, after talking with Earle, we arrived to a refactor plan, that will maintain the main index as it is, so no changes will be needed in any other tool to continue using it, but allows new indexes that can be developed in the tool that needs it. In our case, in Samigo. To add a new index won't be needed to touch search code. It will be enough with implement the right classes in the tool and register the index in the elasticsearch service.

The code will contain an updated README that explains all this with detail.

This code is working in a 11.x version and I've applied it to trunk. Once I finish the testing I will generate the PR.

Comment by Matthew Buckett [ 09-Feb-2017 ]

I don't see any use of this refactor, I think you're saying that once these 2 related Jiras are merged you'll show how it's used? However this makes it much harder to see how you plan to use the multiple indexes. Is it possible to have the second part of this codebase available so it's clearer how it will all fit together.

Comment by Diego del Blanco Orobitg [ 09-Feb-2017 ]

Of course!

I've added this file, searchInSamigo.zip as I have still not ready the branch in the public git with the samigo changes

You can view how we have created a new index and the content providers that index all the questions in Samigo. I've added to the zip file the bean that I use to search on the new index so you can view how it is used. As you will view all that search was impossible to happen in the SiteIndex.. for several reasons, first, because questions in question pools doesn't belong to a site, second because the parameters we need for the search are totally different than the ones allowed by the main index... and third, because we really don't want the questions to be in the main index, so we can control the access security better.

All the code from the new index is included in samigo and nothing needs to happen in the search project. So each tool that wants to use elasticsearch to create their own index, from this moment, can do as samigo does and create it, register it and use it.

I hope the sample helps to understand the main PR, and please feel free to ask any question.

Generated at Wed Feb 19 20:52:56 CST 2020 using Jira 8.0.3#800011-sha1:073e8b433c2c0e389c609c14a045ffa7abaca10d.