Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[exporter][batching] configuration and config validation for bytes based batching #12154

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sfc-gh-sili
Copy link
Contributor

Description

This PR adds config API that will be used for serialized bytes based batching.

type MinSizeConfig struct {
	MinSizeItems int `mapstructure:"min_size_items"`
	MinSizeBytes int `mapstructure:"min_size_bytes"`
}
type MaxSizeConfig struct {
	MaxSizeItems int `mapstructure:"max_size_items"`
	MaxSizeBytes int `mapstructure:"max_size_bytes"`
}

Link to tracking issue

#3262

Testing

Documentation

@sfc-gh-sili sfc-gh-sili requested a review from a team as a code owner January 22, 2025 03:23
@sfc-gh-sili sfc-gh-sili requested a review from mx-psi January 22, 2025 03:23
Copy link

codecov bot commented Jan 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.73%. Comparing base (9757ead) to head (4edf746).

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #12154   +/-   ##
=======================================
  Coverage   91.73%   91.73%           
=======================================
  Files         463      463           
  Lines       24819    24834   +15     
=======================================
+ Hits        22767    22782   +15     
  Misses       1670     1670           
  Partials      382      382           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Comment on lines 33 to +34
MinSizeItems int `mapstructure:"min_size_items"`
MinSizeBytes int `mapstructure:"min_size_bytes"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we have a "Sizer"("SizerType") as enum with 3 values (request, items, bytes) and the "size" value instead?

Copy link
Contributor Author

@sfc-gh-sili sfc-gh-sili Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. How about something like this:

type BatcherConfig struct {
	...
	minSize SizeConfig `mapstructure:"min_size"`
	maxSize SizeConfig `mapstructure:"max_size"`
}
type SizeConfig struct {
	sizer string `mapstructure:"sizer"`
	size int `mapstructure:"size"`
}
func (c SizeConfig) validate() {
	...
}

Does the option request mean that the request will implement its own sizing method?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we need "sizer" in both min/max, since I don't think we need to accept different sizer for min and max.

Copy link
Contributor Author

@sfc-gh-sili sfc-gh-sili Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need to accept different sizer for min and max.

Good point. How about

type BatcherConfig struct {
	...
	sizer SizeType `mapstructure:",squash"`
	minSize int `mapstructure:"max_size"`
	maxSize int `mapstructure:"max_size"`
}
type SizerType struct {
	sizer string `mapstructure:"sizer"`
}
func (c SizerType) validate() {
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants