[BUG] Search request processors receive different inputs than the search API request body

[ohltyler]

Describe the bug

Search request processors offer manipulation on the search request when users execute a search API call. However, the request the processors receive may be different than the request the users send. This is due to pre-processing/translation/validation that OpenSearch performs before the processors have access to the request. As a result, this can lead to runtime failures.

For example, users may write an ML inference search request processor to parse out a field in the search request, and generate an embedding for it. But, if the request JSON changes between the user-supplied request, and the request received at runtime, it can lead to parsing failures, and failures in the processor to find the original intended value in the search request.

Related component

No response

To Reproduce

See example configurations in opensearch-project/dashboards-flow-framework#574

Expected behavior

From my perspective, ideally the search request processors have access to the exact search request that the users supply in the search API. This provides continuity and consistency, and lets users configure transformations and/or data parsing of that request in a consistent manner. After all search request processors have been completed, the original translation/validation can happen, and the API logic can continue as normal.

Additional Details