Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[pkg/ottl] Add ParseSeverity function #37280

Draft
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

bacherfl
Copy link
Contributor

@bacherfl bacherfl commented Jan 17, 2025

Description

Still in Progress

This PR adds the ParseSeverity function, as discussed in the linked ticket. I also had to make a minor change to the
internal mapGetter, handling the map literals to return a raw map[string]any, instead of a pcommon.Map. This is because if there is a map literal within a slice, the pcommon.Slice.FromRaw cannot handle the pcommon.Map, as it only works with raw data types.

This change is however transparent, and the behavior to the outside of this package does not change.
EDIT: After merging main with the support for value expressions, introduced in #36883, this would affect the type of values returned by ParseValueExpression - previously this could potentially return map[string]any/[]any, but with the changes introduced in this PR, this would return a pcommon.Map/pcommon.Slice.
Please let me know if I should do this change in a separate PR though.

Link to tracking issue

Fixes #35079

Testing

Added unit and e2e tests

Documentation

Describe new function in the readme

Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
getter, err := p.newGetter(*parsed)
if err != nil {
return nil, err
}

return &ValueExpression[K]{
getter: getter,
getter: &StandardGetSetter[K]{
Copy link
Contributor Author

@bacherfl bacherfl Jan 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@evan-bradley @TylerHelmuth this is unrelated to the issue this PR aims to solve, but I just noticed that and would like to get your opinion on this: Do you think we should modify the ValueExpression to return the equivalent pcommon.Map/Slice types to callers of its API, rather than raw maps/slices?
Due to the change above I had to adapt this section here to make the tests pass again. Probably it's best to do these adaptations in a separate Issue/PR then once we have agreed on an approach, to not make this current PR too large

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm open to other opinions, but I think we should just return the raw types. It's easy for the consumer to adapt these to the pdata equivalents, and it makes value expressions a bit more flexible.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @evan-bradley , that makes sense. In any case, I will create a separate issue for the limitation regarding the map literals, as this is blocking this PR currently, but will keep the raw values returned by the ValueExpression

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it might be a bit risky returning the raw value here, for example, there are a few places in the OTTL contexts that are relying on maps being pcommon.Map, and change that might silently make they fail (It seems we're lacking tests for those use cases).

For example, if I'm not mistaken, the following statement wouldn't work if the raw value is returned here:

 log_statements:
 - context: log
   statements:
       - set(attributes, {"test": "pass"})

That's because the attributes setter relies on val being a pcommon.Map, which wouldn't be the case anymore, as the set function does not manipulate/cast it:

func accessAttributes() ottl.StandardGetSetter[TransformContext] {
	return ottl.StandardGetSetter[TransformContext]{
		Getter: func(_ context.Context, tCtx TransformContext) (any, error) {
			return tCtx.GetLogRecord().Attributes(), nil
		},
		Setter: func(_ context.Context, tCtx TransformContext, val any) error {
			if attrs, ok := val.(pcommon.Map); ok {
				attrs.CopyTo(tCtx.GetLogRecord().Attributes())
			}
			return nil
		},
	}
}

This same casting-ignore pattern can be found in different places, which makes me think what else could be impacted by this change, and how we could ensure it's not breaking the existing behavior.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edmocosta seems you are right - statements like set(attributes["key"], {"test":"pass"} do still work, as the map will converted to a pcommon.Map internally, but set(attributes, {"test": "pass"}) doesn't.

Do you know if there is a reason behind the accessAttributes function only checking for pcommon.Map, whereas in other places like e.g. the SetValue function (https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/contexts/internal/value.go#L63-L66) we check for both the raw and the pcommon equivalent and do the conversion if needed?

Copy link
Contributor Author

@bacherfl bacherfl Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I have found an alternative solution now, which also works with the edge case mentioned above: #37408 - I have also added a test case for setting the attributes to a map literal in that PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know if there is a reason behind the accessAttributes function only checking for pcommon.Map, whereas in other places like e.g. the SetValue function (https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/contexts/internal/value.go#L63-L66) we check for both the raw and the pcommon equivalent and do the conversion if needed?

I'm not pretty sure why, but maybe @TylerHelmuth has some context on that.
Although changing it to raw values would be easier, I think we could also achieve the same results without necessarily doing it. I personally would prefer to keep it consistent, and I think changing the mapGetter to something like that would make it work with only pcommon.Maps values (not fully tested), and wouldn't require the extra changes on the list getter and ValueExpression:

func (m *mapGetter[K]) Get(ctx context.Context, tCtx K) (any, error) {
	result := pcommon.NewMap()
	for k, v := range m.mapValues {
		val, err := v.Get(ctx, tCtx)
		if err != nil {
			return nil, err
		}
		switch val.(type) {
		case pcommon.Map:
			target := result.PutEmpty(k).SetEmptyMap()
			val.(pcommon.Map).CopyTo(target)
		case []any:
			target := result.PutEmpty(k).SetEmptySlice()
			for _, el := range val.([]any) {
				switch el.(type) {
				case pcommon.Map:
					m := target.AppendEmpty().SetEmptyMap()
					el.(pcommon.Map).CopyTo(m)
				default:
					err := target.AppendEmpty().FromRaw(el)
					if err != nil {
						return nil, err
					}
				}
			}
		default:
			err := result.PutEmpty(k).FromRaw(val)
			if err != nil {
				return nil, err
			}
		}
	}
	return result, nil
}

That's only a draft/idea, if folks agree on changing it to raw value, I'd be okay as well, but we would need to verify/test all those pcommon.Map use cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this suggestion @edmocosta, that seems to do the trick as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[pkg/ottl] ParseSeverity function
3 participants