Add CAT API automation #9042

Naarcha-AWS · 2025-01-09T17:59:06Z

Description

The following PR adds CAT API automation markers. All descriptions have been reviewed by editorial, see opensearch-project/opensearch-api-specification#726.

Checklist

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Archer <[email protected]>

github-actions · 2025-01-09T17:59:17Z

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

Signed-off-by: Archer <[email protected]>

kolchfa-aws

A couple of suggestions for template consistency.

kolchfa-aws · 2025-01-09T18:59:02Z

_api-reference/cat/cat-cluster_manager.md

+columns: Parameter,Type,Description,Default
+include_deprecated: false
+-->
+## Query parameters


There should be a line between the heading and the table: do the spec tags have to go below the heading for it to happen? There should also be an intro sentence here "The following table lists the available query parameters. All query parameters are optional." per the API template.

Agreed. I noticed now that not having the space breaks the table on prod. I'll omit the header for now.

@nhtruong: can we adjust the workflow to add a space after the header for both query parameters and path parameters?

kolchfa-aws · 2025-01-09T18:59:15Z

_api-reference/cat/cat-cluster_manager.md

+`cluster_manager_timeout` | String | A timeout for connection to the cluster manager node. | 
+`format` | String | A short version of the HTTP `Accept` header, such as `json` or `yaml`. | 
+`h` | List | A comma-separated list of column names to display. | 
+`help` | Boolean | Return help information. | `false`


Updating this description in the spec.

kolchfa-aws · 2025-01-09T18:59:47Z

_api-reference/cat/cat-cluster_manager.md

+<!-- spec_insert_end -->
+
+
+## Example request

 ```


There should be a json highlighter specified for the example requests.

kolchfa-aws · 2025-01-09T19:01:49Z

_api-reference/cat/cat-indices.md

+:--- | :--- | :--- | :---
+`bytes` | String | The units used to display byte values. | 
+`cluster_manager_timeout` | String | The amount of time allowed to establish a connection to the cluster manager node. | 
+`expand_wildcards` | List or String | The type of index that wildcard patterns can match. | 


Can we provide valid values for expand_wildcards?

Updating the description in the spec now.

kolchfa-aws · 2025-01-09T19:03:48Z

_api-reference/cat/cat-nodes.md

+`h` | List | A comma-separated list of column names to display. | 
+`help` | Boolean | Return help information. | `false`
+`s` | List | A comma-separated list of column names or column aliases to sort by. | 
+`time` | String | Specifies the time units, for example, `5d` or `7h`. For more information, see [Supported units](https://opensearch.org/docs/latest/api-reference/units/). | 


The previous instance of time did not include examples. I think examples are nice and a link to the units should be provided.

Are all time parameter descriptions coming from different places in the spec? If so, we should create a reference variable in the spec and source all descriptions from that variable.

Let's talk about using $ref for descriptions in another venue. Using $refs for descriptions also affects how other developers contribute new or missing specs, so I think it warrants a larger discussion about how to do it properly.

We talked about this in the API spec repo a while ago opensearch-project/opensearch-api-specification#726 (comment)

Yes we did. When I tried to do this locally it the spec would error out, similar to what's discussed in this issue: opensearch-project/opensearch-api-specification#577. I believe that you would have to create a separate schema which includes only the shared description, as opposed to referencing an already used description to another schema. I'll make an issue the api-specification repo detailing what I tried and encountered.

I think we just need to keep the definitions in the common schema and generate valid values off of the enum (I opened an issue in the api-spec repo opensearch-project/opensearch-api-specification#783) cc: @nhtruong

kolchfa-aws · 2025-01-09T19:05:43Z

_api-reference/cat/cat-pit-segments.md

+<!-- spec_insert_end -->
+
+## Request body fields
+


There should be an intro sentence here per the API template.

kolchfa-aws · 2025-01-09T19:06:08Z

_api-reference/cat/cat-pit-segments.md

+
+```json
+GET /_cat/pit_segments
+


Suggested change

kolchfa-aws · 2025-01-09T19:06:22Z

_api-reference/cat/cat-pit-segments.md

+        "o463QQEPbXktaW5kZXgtMDAwMDAxFkhGN09fMVlPUkVPLXh6MUExZ1hpaEEAFjBGbmVEZHdGU1EtaFhhUFc4ZkR5cWcAAAAAAAAAAAIWaXBPNVJtZEhTZDZXTWFFR05waXdWZwEWSEY3T18xWU9SRU8teHoxQTFnWGloQQAA"
+    ]
+}
+```


Add a copy button here

kolchfa-aws · 2025-01-09T19:08:37Z

_api-reference/cat/cat-segment-replication.md

+`bytes` | String | The units used to display byte values. | 
+`completed_only` | Boolean | When `true`, the response only includes the last-completed segment replication events. | `false`
+`detailed` | Boolean | When `true`, the response includes additional metrics for each stage of a segment replication event. | `false`
+`expand_wildcards` | List or String | Whether to expand the wildcard expression to include concrete indexes that are open, closed, or both. | 


This description is different from one of the above expand_wildcards descriptions. We should make them the same by using a variable in the spec.

kolchfa-aws · 2025-01-09T19:13:36Z

_api-reference/cat/cat-tasks.md

+include_deprecated: false
+-->
+## Query parameters
+Parameter | Type | Description | Default


Can we make this "Data type" to be consistent with the template?

Sure. But I don't think this should be a blocker for this PR since @nhtruong would have to address it in the workflow.

Addendum to my previous. I figured out how to update this and will add it to this PR.

Signed-off-by: Naarcha-AWS <[email protected]>

Signed-off-by: Archer <[email protected]>

Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS · 2025-01-13T18:57:18Z

Going to close this one for now to open a new PR.

Add CAT API automation

f95be66

Signed-off-by: Archer <[email protected]>

Naarcha-AWS requested review from kolchfa-aws, AMoo-Miki, natebower, dlvenable and epugh as code owners January 9, 2025 17:59

Naarcha-AWS self-assigned this Jan 9, 2025

Naarcha-AWS added backport 2.18 PR: Backport label for 2.18 4 - Doc review PR: Doc review in progress labels Jan 9, 2025

Add CAT API reference

ba94920

Signed-off-by: Archer <[email protected]>

kolchfa-aws reviewed Jan 9, 2025

View reviewed changes

Naarcha-AWS and others added 7 commits January 9, 2025 13:58

Update cat-pit-segments.md

7a45d1d

Signed-off-by: Naarcha-AWS <[email protected]>

Fix mustache template

2e56e27

Signed-off-by: Archer <[email protected]>

Fix table spacing

5314b42

Signed-off-by: Archer <[email protected]>

Updated expected output

e6d2233

Signed-off-by: Archer <[email protected]>

Revert back to type

d0b2fa0

Signed-off-by: Archer <[email protected]>

Fix pretty test expected result

cbdd6c1

Signed-off-by: Archer <[email protected]>

Merge branch 'main' into cat-api-automation

f91bbe7

Naarcha-AWS mentioned this pull request Jan 10, 2025

API automation (Phase 1) #9050

Open

26 tasks

Merge branch 'main' into cat-api-automation

a4358fb

Signed-off-by: Naarcha-AWS <[email protected]>

Naarcha-AWS closed this Jan 13, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add CAT API automation #9042

Add CAT API automation #9042

Naarcha-AWS commented Jan 9, 2025

github-actions bot commented Jan 9, 2025

kolchfa-aws left a comment

kolchfa-aws Jan 9, 2025

Naarcha-AWS Jan 9, 2025

kolchfa-aws Jan 9, 2025

Naarcha-AWS Jan 9, 2025

kolchfa-aws Jan 9, 2025

kolchfa-aws Jan 9, 2025

Naarcha-AWS Jan 9, 2025

kolchfa-aws Jan 9, 2025

kolchfa-aws Jan 9, 2025

Naarcha-AWS Jan 9, 2025

kolchfa-aws Jan 9, 2025

Naarcha-AWS Jan 9, 2025

kolchfa-aws Jan 10, 2025

kolchfa-aws Jan 9, 2025

kolchfa-aws Jan 9, 2025

kolchfa-aws Jan 9, 2025

kolchfa-aws Jan 9, 2025

kolchfa-aws Jan 9, 2025

Naarcha-AWS Jan 9, 2025

Naarcha-AWS Jan 9, 2025

Naarcha-AWS commented Jan 13, 2025

	`help` \| Boolean \| Return help information. \| `false`
	`help` \| Boolean \| Returns help information. \| `false`


		```json
		GET /_cat/pit_segments

Add CAT API automation #9042

Add CAT API automation #9042

Conversation

Naarcha-AWS commented Jan 9, 2025

Description

Checklist

github-actions bot commented Jan 9, 2025

kolchfa-aws left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Naarcha-AWS commented Jan 13, 2025