[FLINK-39404][runtime] HardwareDescription reports incorrect CPU cores in containerized environments with fractional CPU limits

[Dennis-Mircea]

What is the purpose of the change

This PR fixes inaccurate CPU core reporting in containerized environments (Kubernetes, YARN) where fractional CPU limits are used (e.g. 0.5 cores).

Currently, Hardware.getNumberCPUCores() relies on Runtime.getRuntime().availableProcessors(), which returns an int, ceiling-rounding fractional container CPU limits (0.5 → 1). This causes:

1. Misleading Web UI: The Task Manager pages display "1 CPU core" when only 0.5 is allocated.
2. Thread pool over-provisioning: ClusterEntrypointUtils computes 4 * ceil(0.5) = 4 threads instead of ceil(4 * 0.5) = 2.

Brief change log

• Hardware.java:

  • Added getContainerCpuLimit() - reads the actual container CPU limit from Linux cgroup files (v2: /sys/fs/cgroup/cpu.max, v1: cpu.cfs_quota_us / cpu.cfs_period_us). Returns the fractional value (e.g. 0.5) or -1 if not in a container / no limit set.
  • Added getNumberCPUCoresAsDouble() - returns the fractional CPU core count (container limit with fallback to availableProcessors()). Use this when fractional precision matters (display, arithmetic before rounding).

• HardwareDescription.java:

  • Changed numberOfCPUCores from int to double (field, constructor, getter, equals, toString).
  • extractFromSystem() now uses Hardware.getNumberCPUCoresAsDouble().
  • The REST API JSON field cpuCores now emits fractional values (e.g. 0.5 instead of 1).

• ClusterEntrypointUtils.java:

  • Changed 4 * Hardware.getNumberCPUCores() to (int) Math.ceil(4 * Hardware.getNumberCPUCoresAsDouble()) so the multiplication happens before ceiling, avoiding thread over-provisioning (0.5 CPU → 2 threads instead of 4).

Verifying this change

• On a non-container environment (macOS, bare-metal Linux without cgroup limits): getContainerCpuLimit() returns -1, getNumberCPUCores() falls back to availableProcessors(), representing no behavioral change.
• On a container with fractional CPU (e.g. Kubernetes with cpu: 0.5):
  • getContainerCpuLimit() returns 0.5
  • getNumberCPUCoresAsDouble() returns 0.5
  • getNumberCPUCores() returns 1 (ceiling)
  • Web UI displays 0.5 instead of 1
  • IO executor pool: ceil(4 * 0.5) = 2 threads instead of 4

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

[flinkbot]

CI report:

• a647772 Azure: FAILURE

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[Dennis-Mircea]

@flinkbot run azure re-run the last Azure build