package job fails with error: "CPU too busy" #151
Comments
|
If the packager would fail, we'd never see the "Uploading to" log, packaging did succeed locally. I suspect it's B2 throwing that error and has nothing to do with your local CPU. Found something alike on https://xcp-ng.org/forum/topic/5397/delta-backups-failing-aws-s3-uploadpartcopy-cpu-too-busy Syncing from and to S3 is handled by https://www.npmjs.com/package/s3-sync-client, I've been thinking of writing our own syncing logic (on top of S3 JS SDK) that properly handles retries / is more robust. It'll take more time but I'll see if I can write a quick proof of concept (tomorrow evening perhaps), atleast we'd be able to test our hypothesis whether it's related to B2. If you can build from source, feel free to set |
|
@insanity54, I've pushed a commit (9e868ac) where you can specify
{
assetId: "...", // Your asset id returned from the transcode job.
public: false, // This is an opt-in, for now, I default it to "true" for backwards compatibility.
concurrency: 1, // Instruct the package job to upload each segment one after another instead of parallel, I suspect this lowers the burden on B2 which causes the "CPU too busy" error message.
}Since you got the source up and running, I didn't take the time to deploy this to the Docker |
|
Testing 9e868ac, I got some successes and some intermittent failures. /package test 1 (success)input{
assetId: "cea2db20-1d89-4f8b-855f-d1c2e6ae2302"
concurrency: 5
name: "hls"
public: true
step: 2
}output {
assetId: "cea2db20-1d89-4f8b-855f-d1c2e6ae2302"
}logs/package test 2 (failure)input {
assetId: "cea2db20-1d89-4f8b-855f-d1c2e6ae2302"
concurrency: 1
name: "test"
public: false
}output(none) logs/package test 3 (success)Input {
assetId: "cea2db20-1d89-4f8b-855f-d1c2e6ae2302"
concurrency: 1
name: "test2"
public: false
step: 2
}Output {
assetId: "cea2db20-1d89-4f8b-855f-d1c2e6ae2302"
}Logs |
|
What surprised me is that you got successful for both "public" being "false" and "true", while earlier we would get the ACL mismatch error (mentioned in the previous issue). This makes me suspect you're not in the latest Meanwhile, I've deployed the latest main branch to the Docker' If your B2 bucket is set private, you must set Both errors, "CPU too busy" and "no tomes available", seem to come from the Backblaze B2 API.
|
Describe the bug
When running a pipeline job, transcode succeeds, then package fails with an error. "CPU too busy"
To Reproduce
POST
/pipelinewith following input{ "inputs": [ { "type": "video", "path": "s3://2024-06-04.mp4" }, { "type": "audio", "path": "s3://2024-06-04.mp4", "language": "eng" } ], "streams": [ { "type": "video", "codec": "h264", "height": 360 }, { "type": "video", "codec": "h264", "height": 144 }, { "type": "audio", "codec": "aac" } ], "tag": "test" }I re-tried 3 times and got other errors too.
On the second try, I got an error during the ffmpeg(video,360) Job.
Unable to connect. Is the computer able to access the url?followed by anUnknownErroron ffmpeg(video,144). These two errors coincided with my (Starlink) internet cutting out as it does a few times a day, so I think those were unrelated.On the third try, I got another error.
This one was again during a temporary internet outage, so I think it's unrelated too.
Fourth try, this time using
POST /packageinstead ofPOST /pipeline, another CPU too busy error during package Job.Fifth try using
POST /pipeline, transcode was successful, but package errored. New error.no tomes available. This is a backblaze error I've seen before. Apparently it happens more commonly at night when people are using B2 to backup their infrastructure.This leads me to think, could the
CPU too busyalso be an error from Backblaze? I did a search for that string in superstreamer's codebase and I couldn't find a match, so at this point I think that could be a plausible explanation.I'll try again during the daytime to see if I can narrow down the problem.
Expected behavior
I was hoping to see the package Job succeed. I think it's ironic that the transcode Job completes successfully because I would think transcoding would be the more CPU intensive task. But then again, software is complex and doesn't always do what we want 😆
Screenshots
Desktop (please complete the following information):
Smartphone (please complete the following information):
n/a
Additional context
The last artisan log before the Job failed
The text was updated successfully, but these errors were encountered: