Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store events in a Redis stream instead of a list #8

Merged
merged 1 commit into from
Jan 21, 2025

Conversation

jjmerchante
Copy link
Collaborator

@jjmerchante jjmerchante commented Jul 11, 2024

This PR updates the method of storing events in Redis for later consumption.
Events are now added to a Redis Stream, allowing different types of consumers to process the items for various purposes. The trade-off is that the stream has a fixed length, causing older items to be deleted when new items are added.

@jjmerchante jjmerchante force-pushed the redis-stream branch 2 times, most recently from 153f64b to 6e5e0d9 Compare December 4, 2024 09:18
@jjmerchante jjmerchante marked this pull request as ready for review December 4, 2024 09:19
@jjmerchante jjmerchante force-pushed the redis-stream branch 4 times, most recently from a962b7d to 7cb5cee Compare December 5, 2024 10:42
Copy link
Member

@sduenas sduenas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if you can reduce the PR just to the scope of writing data to the events stream. I think we can add the tasks that process data later.

@jjmerchante
Copy link
Collaborator Author

I updated the PR to include only the change that inserts data in the stream. I also removed the file grimoirelab-dev that was not being used because the contents were moved to runner/commands/run/

Copy link
Member

@sduenas sduenas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good in general. Check my comments, though.

src/grimoirelab/core/runner/commands/run.py Show resolved Hide resolved
src/grimoirelab/core/config/settings.py Outdated Show resolved Hide resolved
src/grimoirelab/core/config/settings.py Outdated Show resolved Hide resolved
@jjmerchante jjmerchante force-pushed the redis-stream branch 4 times, most recently from d9c3e88 to bbd89c7 Compare January 20, 2025 16:08
Copy link
Member

@sduenas sduenas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

This commit updates the method of storing events in
Redis for later consumption.
Events are now added to a Redis Stream, allowing
different types of consumers to process the items for
various purposes. The trade-off is that the stream has a
fixed length, causing older items to be deleted when
new items are added.

Signed-off-by: Jose Javier Merchante <[email protected]>
@sduenas sduenas merged commit 2e6fd19 into chaoss:main Jan 21, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants