Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add use of zstd compression on compute services #336

Merged
merged 23 commits into from
Jan 24, 2025
Merged
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
67f7964
Add zstd compression to set_task_result in compute service
ianmkenney Nov 25, 2024
41ad174
Use request and manually process payload
ianmkenney Dec 2, 2024
54cd184
Add compression module and send send bytes from API path as latin-1
ianmkenney Dec 2, 2024
f2da1fc
Use compression module in objectstore
ianmkenney Dec 11, 2024
18131f1
Attempt to decompress object store objects in a try block
ianmkenney Dec 13, 2024
8ee5349
Use fixed filename for compressed object
ianmkenney Dec 18, 2024
b32f62d
Implement test_set_task_result_legacy
ianmkenney Dec 26, 2024
6f3ce3c
Separate executing tasks and pushing results in TestClient
ianmkenney Dec 30, 2024
63f9982
Clear leftover state before testing legacy PDR pull
ianmkenney Dec 31, 2024
6985ca7
Parameterize test_get_transformation_and_network_results
ianmkenney Dec 31, 2024
32bef06
Merge branch 'main' into feature/220-zstd-compression-compute-services
ianmkenney Dec 31, 2024
cda3541
Small docstring adjustment
dotsdl Jan 23, 2025
664cce4
Merge branch 'main' into feature/220-zstd-compression-compute-services
dotsdl Jan 23, 2025
0a15138
Merge fixes
dotsdl Jan 23, 2025
8b65c72
Removed need to decompress protocoldagresult in S3ObjectStore.push_pr…
dotsdl Jan 24, 2025
7d32286
Merge branch 'main' into feature/220-zstd-compression-compute-services
dotsdl Jan 24, 2025
1187666
Some clarity edits
dotsdl Jan 24, 2025
7d3c86c
Black!
dotsdl Jan 24, 2025
e149d09
CI fixes, other edits from review
dotsdl Jan 24, 2025
8473729
Black!
dotsdl Jan 24, 2025
d61d133
Assign value to protocoldagresult
ianmkenney Jan 24, 2025
443e193
Simplification
dotsdl Jan 24, 2025
321f5d7
Revert "simplification"
dotsdl Jan 24, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Black!
dotsdl committed Jan 24, 2025

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
commit 7d3c86c72cc26a5c4123221d3a2f662dee00617a
2 changes: 1 addition & 1 deletion alchemiscale/compute/api.py
Original file line number Diff line number Diff line change
@@ -348,7 +348,7 @@ async def set_task_result(
protocoldagresult_ok=pdr.ok(),
protocoldagresult_gufekey=pdr.key,
transformation=tf_sk,
creator=compute_service_id
creator=compute_service_id,
)

# push the reference to the state store