Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v2] Benchmark harness improvements & add more service benchmarks #9264

Merged
merged 16 commits into from
Feb 4, 2025

Conversation

aemous
Copy link
Contributor

@aemous aemous commented Jan 30, 2025

Issue #, if available:
N/A

Description of changes:

  • Adds support for specifying file literals in the benchmark definitions JSON.
  • Updated README with documentation on specifying file literals.
  • Added dynamodb benchmarks for get-item, batch-get-item, put-item commands, and a benchmark for the customization ddb put.
  • Aded ec2, ecs, cloudwatch logs, and cloudwatch benchmarks.
  • Added STS benchmarks for AssumeRole, GetCallerIdentity, AssumeRoleWithWebIdentity.
  • Removed S3 benchmarks using the CRT client because they are currently non-functional with a non-trivial fix. Will possibly revisit getting them working later.
  • Fix bug in benchmark summarizer with computing averages.
  • Add support to the benchmark harness for debugging the child process.
  • Redirect child process output to files so that the only output of run-benchmarks script on success is the benchmark results.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@aemous aemous changed the title Benchmarks more services [v2] Benchmark harness improvements & DynamoDB benchmarks Jan 30, 2025
@aemous aemous requested review from aws-sdk-python-automation and a team January 30, 2025 16:14
@aemous aemous marked this pull request as ready for review January 30, 2025 16:14
@aemous aemous changed the title [v2] Benchmark harness improvements & DynamoDB benchmarks [v2] Benchmark harness improvements & add more service benchmarks Jan 30, 2025
@aemous aemous merged commit a721936 into aws:benchmarks Feb 4, 2025
41 of 45 checks passed
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (benchmarks@8951dcd). Learn more about missing BASE report.

Additional details and impacted files
@@             Coverage Diff              @@
##             benchmarks   #9264   +/-   ##
============================================
  Coverage              ?       0           
============================================
  Files                 ?       0           
  Lines                 ?       0           
  Branches              ?       0           
============================================
  Hits                  ?       0           
  Misses                ?       0           
  Partials              ?       0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants