-
Notifications
You must be signed in to change notification settings - Fork 2.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[pkg/ottl] Add ParseSeverity
function
#37280
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
… made to mapGetter Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
getter, err := p.newGetter(*parsed) | ||
if err != nil { | ||
return nil, err | ||
} | ||
|
||
return &ValueExpression[K]{ | ||
getter: getter, | ||
getter: &StandardGetSetter[K]{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@evan-bradley @TylerHelmuth this is unrelated to the issue this PR aims to solve, but I just noticed that and would like to get your opinion on this: Do you think we should modify the ValueExpression
to return the equivalent pcommon.Map/Slice
types to callers of its API, rather than raw maps
/slices
?
Due to the change above I had to adapt this section here to make the tests pass again. Probably it's best to do these adaptations in a separate Issue/PR then once we have agreed on an approach, to not make this current PR too large
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm open to other opinions, but I think we should just return the raw types. It's easy for the consumer to adapt these to the pdata equivalents, and it makes value expressions a bit more flexible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks @evan-bradley , that makes sense. In any case, I will create a separate issue for the limitation regarding the map literals, as this is blocking this PR currently, but will keep the raw values returned by the ValueExpression
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it might be a bit risky returning the raw value here, for example, there are a few places in the OTTL contexts that are relying on maps being pcommon.Map
, and change that might silently make they fail (It seems we're lacking tests for those use cases).
For example, if I'm not mistaken, the following statement wouldn't work if the raw value is returned here:
log_statements:
- context: log
statements:
- set(attributes, {"test": "pass"})
That's because the attributes
setter relies on val
being a pcommon.Map
, which wouldn't be the case anymore, as the set
function does not manipulate/cast it:
func accessAttributes() ottl.StandardGetSetter[TransformContext] {
return ottl.StandardGetSetter[TransformContext]{
Getter: func(_ context.Context, tCtx TransformContext) (any, error) {
return tCtx.GetLogRecord().Attributes(), nil
},
Setter: func(_ context.Context, tCtx TransformContext, val any) error {
if attrs, ok := val.(pcommon.Map); ok {
attrs.CopyTo(tCtx.GetLogRecord().Attributes())
}
return nil
},
}
}
This same casting-ignore pattern can be found in different places, which makes me think what else could be impacted by this change, and how we could ensure it's not breaking the existing behavior.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@edmocosta seems you are right - statements like set(attributes["key"], {"test":"pass"}
do still work, as the map will converted to a pcommon.Map
internally, but set(attributes, {"test": "pass"})
doesn't.
Do you know if there is a reason behind the accessAttributes
function only checking for pcommon.Map
, whereas in other places like e.g. the SetValue
function (https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/contexts/internal/value.go#L63-L66) we check for both the raw and the pcommon
equivalent and do the conversion if needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I have found an alternative solution now, which also works with the edge case mentioned above: #37408 - I have also added a test case for setting the attributes to a map literal in that PR
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you know if there is a reason behind the accessAttributes function only checking for pcommon.Map, whereas in other places like e.g. the SetValue function (https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/ottl/contexts/internal/value.go#L63-L66) we check for both the raw and the pcommon equivalent and do the conversion if needed?
I'm not pretty sure why, but maybe @TylerHelmuth has some context on that.
Although changing it to raw values would be easier, I think we could also achieve the same results without necessarily doing it. I personally would prefer to keep it consistent, and I think changing the mapGetter
to something like that would make it work with only pcommon.Map
s values (not fully tested), and wouldn't require the extra changes on the list getter and ValueExpression:
func (m *mapGetter[K]) Get(ctx context.Context, tCtx K) (any, error) {
result := pcommon.NewMap()
for k, v := range m.mapValues {
val, err := v.Get(ctx, tCtx)
if err != nil {
return nil, err
}
switch val.(type) {
case pcommon.Map:
target := result.PutEmpty(k).SetEmptyMap()
val.(pcommon.Map).CopyTo(target)
case []any:
target := result.PutEmpty(k).SetEmptySlice()
for _, el := range val.([]any) {
switch el.(type) {
case pcommon.Map:
m := target.AppendEmpty().SetEmptyMap()
el.(pcommon.Map).CopyTo(m)
default:
err := target.AppendEmpty().FromRaw(el)
if err != nil {
return nil, err
}
}
}
default:
err := result.PutEmpty(k).FromRaw(val)
if err != nil {
return nil, err
}
}
}
return result, nil
}
That's only a draft/idea, if folks agree on changing it to raw value, I'd be okay as well, but we would need to verify/test all those pcommon.Map
use cases.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for this suggestion @edmocosta, that seems to do the trick as well
Signed-off-by: Florian Bacher <[email protected]>
Signed-off-by: Florian Bacher <[email protected]>
Description
Still in Progress
This PR adds the
ParseSeverity
function, as discussed in the linked ticket. I also had to make a minor change to theinternal
mapGetter
, handling the map literals to return a rawmap[string]any
, instead of apcommon.Map
. This is because if there is a map literal within a slice, thepcommon.Slice.FromRaw
cannot handle thepcommon.Map
, as it only works with raw data types.This change is however transparent, and the behavior to the outside of this package does not change.
EDIT: After merging main with the support for value expressions, introduced in #36883, this would affect the type of values returned by
ParseValueExpression
- previously this could potentially returnmap[string]any
/[]any
, but with the changes introduced in this PR, this would return apcommon.Map
/pcommon.Slice
.Please let me know if I should do this change in a separate PR though.
Link to tracking issue
Fixes #35079
Testing
Added unit and e2e tests
Documentation
Describe new function in the readme