fix(bthread): refactor sharded priority queue with lock-free MPSC inb…

[yannan-wyn]

What problem does this PR solve?

Problem Summary:
The original priority queue implementation misuses WorkStealingQueue — multiple producers call push() concurrently, but push() is designed for single-owner use only. This leads to potential data races under contention.
See also: #2819, #3055, #3078, #3096

What is changed and the side effects?

Changed:

• Replace the single WorkStealingQueue per tag with a sharded-mode design (which is PriorityShard), each shard containing:
  • butil::MPSCQueue inbound (lock-free) for external producers
  • WorkStealingQueue for owner flush/pop and non-owner steal
• Add owner lifecycle management (bind/unbind/draining) tied to TaskGroup creation/destruction
• Add owner-preferred shard selection (thread-local round-robin, skip ownerless/draining shards)
• Add fallback path: clear BTHREAD_GLOBAL_PRIORITY and re-enqueue to normal _remote_rq during shard teardown
• Add _priority_shard_index field to TaskGroup for O(1) owner shard lookup
• Add unit tests (correctness, concurrent producers, owner dynamic changes, stress) and microbenchmarks

Side effects:

• Performance effects:

  • Owner hot path: ~32ns (same as baseline WSQ, zero overhead)
  • Multi-producer inbound (8 threads): ~260ns vs baseline _remote_rq ~1050ns (3-4x faster)
  • Full pipeline (8 producers): ~155ns vs baseline ~1050ns (5-10x faster)

• Breaking backward compatibility:

  • None. Gated by FLAGS_enable_bthread_priority_queue (default false).

[Copilot]

Pull request overview

Refactors the bthread “global priority” scheduling path to avoid unsafe multi-producer use of WorkStealingQueue by introducing a sharded design with an inbound lock-free queue per shard, plus adds new unit tests and a microbenchmark.

Changes:

• Introduces PriorityShard per tag and routes priority submissions via shard inbound queues with owner bind/unbind lifecycle.
• Updates scheduling/stealing logic to prefer owner shards, steal across shards, and provide fallback behavior during teardown.
• Adds unit tests for correctness/concurrency/owner changes and a microbenchmark suite for the new primitives.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/bthread_priority_queue_unittest.cpp	Adds correctness + concurrent-producer tests for priority queue behavior.
test/bthread_priority_queue_owner_unittest.cpp	Adds tests around dynamic worker/owner binding behavior under load.
test/bthread_priority_queue_benchmark.cpp	Adds microbenchmarks for inbound enqueue/dequeue, flush, steal, and baseline comparisons.
src/bthread/task_group.h	Adds _priority_shard_index for O(1) lookup of a TaskGroup’s owned shard.
src/bthread/task_control.h	Adds PriorityShard definition and TaskControl helper APIs for shard management.
src/bthread/task_control.cpp	Implements sharded priority queue logic, owner lifecycle, flushing, stealing, and fallback enqueue behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[yannan-wyn]

@wwbmmm
comments from copolit replied, please review again

[wwbmmm]

LGTM