Skip to content

[BI-3062] Replace simpleSearch usages for searchInternal to support >65k use cases - #578

Open
jloux-brapi wants to merge 6 commits into
epic/BI-2862from
feature/BI-3062
Open

jloux-brapi wants to merge 6 commits into
epic/BI-2862from
feature/BI-3062

Conversation

@jloux-brapi

@jloux-brapi jloux-brapi commented Nov 16, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Story: BI-3062

  • Because simpleSearch could not handle use cases where there was more than 65k records for a given search, and it could not handle async request, searchInternal in BrAPIDAOUtil was modified to fulfill the main requirement for simple search, was to do all of the filtering/sorting/pagination on the BrAPI side, returning the full response from BrAPI to be processed by BI instead of just the list of data. All old callers of this method now point to this new version of this method, but instead of utilizing the BrAPI response the list data is derived from the response so those callers can process the way they did before. The new implementation, searchInternalFullResponse() should satisfy the simpleSearch and old search callers expectations.
  • The search methods previously had a parameter, sendPaging, passed through to the final search method. I found this naming pretty confusing, considering I would expect sendPaging to mean you sent paging info to the server and it responded accordingly. Instead, this boolean was being used to page through all of the data for a given search request so that bi-api could get all of the data from the brapi server in one request. Because of this, the variable has been renamed to pageThru. If pageThru is true, all of the data found in the search request is paged through and returned in full to the BrAPI server.
  • Usages of the BrAPIDAOUtil.get() method were changed to use the searchMethod for BrAPIObservationDAO, BRAPIObservationUnitDAO and BrAPIGermpaslmDAO, as the GET() method does not support requests that return > 1000 rows, and should not be used that way in general. In the cases where I loaded an extraordinary amount of data, these use cases broke. These changes restore them, though at scale, the expectation is that this will be slow.
  • The BrAPIDAOUtil search methods were all updated to accurately reflect the generics required for the new searchInternalFullResponse() method. Comments were added to help with any confusion. Fixing generics for other methods in this class will be completed another time, as needed.

Dependencies

Testing

Regression of the following features:

  • Germplasm import
  • Experiment import
  • Experiment Appends
  • Subentity data set creation and appends
  • BI-BrAPI GET /observations
  • BI-BrAPI GET /observationunits
  • BI-BrAPI GET /germplasm

Checklist:

  • I have performed a self-review of my own code
  • I have tested my code and ensured it meets the acceptance criteria of the story
  • I have create/modified unit and/or integration tests to cover this change or tests are not applicable
  • I have commented my code, particularly in hard-to-understand areas
  • I have either updated the source of truth or arranged for update with product owner if needed https://breedinginsight.atlassian.net/wiki/spaces/BI/pages/1559953409/Source+of+Truth

* In this example, W should be a BrAPIGermplasmSearchRequest
* @return The full BrAPIResponse object from a completed BrAPI search request
*/
private <T extends BrAPIResponse<U>, U extends BrAPIResponseResult<V>, V, W extends BrAPISearchRequestParametersPaging> T searchInternalReturnResponse(Function<W, ApiResponse<Pair<Optional<T>, Optional<BrAPIAcceptedSearchResponse>>>> searchMethod,

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is really where to start with this PR @humsika. Everything that was changed pretty much calls this method ultimately.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant