Skip to content

Refresh component versions and offline examples for 26.11 - #1428

Draft
dholt wants to merge 8 commits into
masterfrom
dholt/26.11-version-bumps
Draft

dholt wants to merge 8 commits into
masterfrom
dholt/26.11-version-bumps

Conversation

@dholt

@dholt dholt commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Summary

Refresh published component versions and keep offline preparation aligned with role defaults:

  • Slurm 26.05.5; GPU Operator v26.7.2; device plugin and GPU Feature Discovery 0.20.1; MIG Manager 0.15.1.
  • Prometheus 3.15.0; Alertmanager 0.34.1; Grafana 13.2.4; node-exporter 1.12.1.
  • Distribution registry 3.1.2; registry proxy 0.7.0; Network Operator 26.7.0.
  • Retain kube-prometheus-stack 88.6.5; major-chart migration is outside this change.
  • Align offline chart, image and archive examples; add regression tests for pin consistency.
  • Preserve the Operator-managed driver pin at 580.126.20 rather than changing driver branches with the chart update.

Validation

On head a1e1f4cbb7562d5dabf000baaf6445e379240a75:

  • Independent code review approved.
  • Role lint passed for 341 files; 12/12 playbook syntax checks and 134/134 local validation tests passed.
  • Offline pin checks, MIG package mappings and both monitoring example renders passed.
  • Single-node Kubernetes, GPU Operator and a GPU pod running nvidia-smi passed with an A40 and preinstalled driver 580.178.04. This does not validate the Operator-managed driver pin.
  • Nonpersistent monitoring deployed successfully; Prometheus collected GPU, node and Kubernetes metrics. Grafana loaded 30 dashboards, and sampled panel queries returned data.

Remaining validation

This PR is intentionally draft, not release-ready:

  • Slurm 26.05.5 build and GPU job smoke tests were not run.
  • Persistent monitoring, upgrade paths, live MIG reconfiguration and Network Operator secondary-network traffic are not established by these results.
  • Some existing GPU Nodes host/node panels returned no data; the GPU metrics queries were checked separately. This is not a claim that every panel is validated.
  • Network Operator 26.7.0 requires Kubernetes 1.32 or newer; older cluster-version overrides need separate consideration.
  • No new operating-system, firmware or hardware support is claimed.

dholt and others added 8 commits October 5, 2026 08:25
Update monitoring, registry proxy and Slurm pins from public upstream
releases. Keep the registry-cache example aligned with its role default.

Leave Network Operator and MIG Manager unchanged pending verification.

Signed-off-by: Doug Holt <dholt@nvidia.com>
Keep Slurm archives and monitoring pull, save, mirror and override
examples aligned with the selected 26.11 component defaults.

Signed-off-by: Doug Holt <dholt@nvidia.com>
Default kube-prometheus-stack to 88.6.5 so monitoring meets the
minimum version required by Run:ai platform mode.

Signed-off-by: Doug Holt <dholt@nvidia.com>
Use published stable releases and align offline download examples.
Select Network Operator 26.7.0, the newest chart currently published
in the configured repository, rather than its newer source-only release.
Retain the monitoring chart floor pending major-upgrade review.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Signed-off-by: Doug Holt <dholt@nvidia.com>
Preserve the version-refresh commits while integrating current master.
Retain Slurm 26.05.5 alongside the upstream opt-in Lua build setting.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Signed-off-by: Doug Holt <dholt@nvidia.com>
Mirror GPU Operator v26.7.2 and registry 3.1.2 consistently across
connected-side preparation, transfer records, and offline imports.
Add regression checks against role defaults, including the explicit
driver override. Retain R580 to avoid an unvalidated driver-branch
upgrade as part of the chart refresh.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Signed-off-by: Doug Holt <dholt@nvidia.com>
Preserve the version refresh while incorporating the latest upstream changes.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Include the driver role update and monitoring failure handling while
preserving the reviewed version-refresh commits.

Signed-off-by: Doug Holt <dholt@nvidia.com>
Co-Authored-By: Paperclip <noreply@paperclip.ing>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant