Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor shared-libs/search to call cht-datasource freetext search apis #9751

Open
jkuester opened this issue Jan 21, 2025 · 0 comments
Open
Assignees
Labels
Type: Improvement Make something better

Comments

@jkuester
Copy link
Contributor

What feature do you want to improve?
The shared-libs/search library currently directly queries the freetext search views when someone performs a freetext search with the library.

In #9586 @sugat009 is adding support for querying the freetext search views to shared-libs/cht-datasource. This is going to be further enhanced by changes from #9541 which will update the cht-datasource code to toggle between querying the offline freetext views vs the server-side lucene indexes via Nouveau.

Describe the improvement you'd like

Instead of duplicating the toggling logic and view/index queries in both cht-datasource and search, we should just refactor search to consume the cht-datasource apis.

A freetext search is triggered in shared-libs/search by passing a search value in the filters object. Depending on whether the type value is set to 'report' or 'contact', this will cause the code to query either the reports_by_freetext or contacts_by_freetext views. Alternatively, when doing a contact search, if the filters.types.selected is also set, the contacts_by_type_freetext view will be the one queried. Each of these views have corresponding cht-datasource api calls that should be used the replace the Couch view queries.

The tricky part is going to be trying to make the switch with the least amount of disruption to the shared-libs/search code. The existing logic leans pretty heavily on this generic array of requests returned from the generate function. These requests represent the various view queries that need to be made as a part of the search and they are dealt with together because of the need to be able to do paging across them. My initial thought is to try and maybe have a wrapper shim around the cht-datasource calls that allows them to mimic normal view requests.

Another complexity to keep in mind when approaching this code is the fact that AFIK Couch view queries have no default limit value. So, if you query a Couch view with no custom limit, then you get back all results for the query from the entire db. This currently comes into play in the multi-request searches because to do the "paging" shared-libs/search actually just gets literally all the results for all the requests, does a big union of them, and then "pages" the union. On the other hand, cht-datasource does have a default limit. For the multi-request searches, we will need to make repeated calls to cht-datasource to collect all the results (or loop with the the Generator functionality to let cht-datasource handle all the paging).

Describe alternatives you've considered

We considered possibly going the other direction and having cht-datasource consume shared-libs/search. However this does not really make sense architecturally since cht-datasource is intended to be the most basic wrapper layer on top of the interactions with Couch/Pouch and shared-libs/search contains some higher-order searching logic.

Another option was to deprecate/remove shared-libs/search in favor of moving everything to call cht-datasource. IMO this is also not the best thing to do (at least at the current time). The freetext search support in cht-datasource is currently very basic and not suitable as a drop-in replacement for shared-libs/search. For example, cht-datasource does not support multi-view searches (where results from multiple view queries can be paged together). This is important for some basic search workflows like searching for strings containing more than one word. There is currently no good way to efficiently page a multi-view query (or multiple queries to the same view). shared-libs/search does it the in-efficient way, but that is not the kind of think we should be exposing on a cht-datasource REST api endpoint....

Additional context

@jkuester jkuester added the Type: Improvement Make something better label Jan 21, 2025
@witash witash self-assigned this Jan 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Type: Improvement Make something better
Projects
Status: No status
Development

No branches or pull requests

2 participants