Avoid starving system task workers#1204
Draft
v1r3n wants to merge 3 commits into
Draft
Conversation
…-queue semaphores
Replace scheduled-executor polling with a tight loop per queue: poll immediately when
tasks are available, sleep only on empty poll. All non-isolated queues share one thread
pool so threads aren't wasted on idle task types, while each queue gets its own semaphore
so one busy type cannot starve others.
Adds `conductor.app.taskWorkerConfigs.<TaskType>.{threadCount,permitCount}` for per-type
overrides: a non-zero threadCount gives the type a dedicated pool; permitCount caps
in-flight concurrency independently of thread count.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…-queue semaphores
Replace scheduled-executor polling with a tight loop per queue: poll immediately when tasks are available, sleep only on empty poll. All non-isolated queues share one thread pool so threads aren't wasted on idle task types, while each queue gets its own semaphore so one busy type cannot starve others.
Adds
conductor.app.taskWorkerConfigs.<TaskType>.{threadCount,permitCount}for per-type overrides: a non-zero threadCount gives the type a dedicated pool; permitCount caps in-flight concurrency independently of thread count.Pull Request type
NOTE: Please remember to run
./gradlew spotlessApplyto fix any format violations.Changes in this PR
Describe the new behavior from this PR, and why it's needed
Issue #
Alternatives considered
Describe alternative implementation you have considered