refactor(nemesis): modified compaction strategies related nemeses #9913
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
timtimb0t
force-pushed
the
modify_compaction_strategies
branch
2 times, most recently
from
e8b0b64 to
201fe69
Compare
timtimb0t
changed the title
timtimb0t
force-pushed
the
modify_compaction_strategies
branch
from
201fe69 to
6ab5ec1
Compare
temichus
reviewed
Jan 26, 2025
sdcm/nemesis.py
Outdated
| 'min_threshold': 4, | ||
| 'max_threshold': 32, | ||
| }, | ||
| def calculate_strategy_params(strategy_class): |
There was a problem hiding this comment.
I think. method calculate_strategy_params and "Dict strategies" are unnecessary here
you can use "list strategies" as previously, just extend it with lambda and random
| if (current_size // 24) > 2: | ||
| current_unit = "DAYS" | ||
| current_size = 3 | ||
| compaction = settings["compaction"] |
There was a problem hiding this comment.
unnecessary change value "compaction" is not used rather than the next 2 lines
sdcm/nemesis.py
Outdated
| "gc": 864000, | ||
| "dttl": 86400} | ||
|
|
||
| """ Adjust window unit and size with random increments in acceptable borders, |
There was a problem hiding this comment.
That is too short comment for such function
sdcm/nemesis.py
Outdated
| "DAYS": 365, | ||
| } | ||
|
|
||
| random_escalation_sizes = { |
sdcm/nemesis.py
Outdated
| "DAYS": (2, 14), | ||
| } | ||
|
|
||
| def escalate_unit(unit: str) -> (str, int): |
There was a problem hiding this comment.
can be just Dict instead of functions with if-elif-else statement
| current_size = int(settings["compaction"].get("compaction_window_size") or 1) # default value | ||
| multiplier = 3600 if current_unit in ["DAYS", "HOURS"] else 60 | ||
| self.log.debug("Initial twcs settings are: %s", settings) | ||
| MAX_TTL = 4_300_000 # ~49 days in seconds | ||
| expected_sstable_number = 35 |
There was a problem hiding this comment.
Can't understand why previous logic has hardcoded the number 35.
sdcm/nemesis.py
Outdated
| if proposed_ttl > MAX_TTL: | ||
| # Recalculate a smaller window size so we do not exceed MAX_TTL | ||
| # integer division to keep it safe | ||
| max_size = MAX_TTL // (multiplier * expected_sstable_number) |
There was a problem hiding this comment.
current_size = MAX_TTL // (multiplier * expected_sstable_number)
if max_size < 1:
current_size = 1
sdcm/nemesis.py
Outdated
| settings["gc"] = proposed_ttl // 2 | ||
| settings["dttl"] = proposed_ttl | ||
|
|
||
| compaction["compaction_window_unit"] = current_unit |
timtimb0t
force-pushed
the
modify_compaction_strategies
branch
5 times, most recently
from
461a353 to
c639b95
Compare
modify_table_twcs_window_size nemesis to set twcs borders in wider range. modified modify_table_compaction nemesis to set compaction strategies params in wider range to adjust test coverage. refs: 9361, 9362
timtimb0t
force-pushed
the
modify_compaction_strategies
branch
from
c639b95 to
18f775d
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
modify_table_twcs_window_size nemesis modified to test twcs strategy in wider borders
modify_table_compaction nemesis modified for the same purposes.
Both collected in single commit due to modify_table_twcs_window_size nemesis depends on modify_table_compaction (ie modify_table_compaction nemesis adds randomness that should affect modify_table_twcs_window_size)
refs: 9361, 9362
fixes: #9361, #9362
Testing
https://jenkins.scylladb.com/view/staging/job/scylla-staging/job/eugene_test_folder/job/twcs_cluster_without_changes/59/
PR pre-checks (self review)
backportlabelsReminders
sdcm/sct_config.py)unit-test/folder)