Skip to content

bug(k8s): sandbox create shows 'Image pulled' for the bootstrap image and hides the pull failure reason #4415

Closed

Description

@rhuss

Summary

With the Kubernetes driver, openshell sandbox create --from <image> reports ✓ Image pulled before the sandbox image has been pulled at all, then falls back to Pulling image... Error: ImagePullBackOff without the reason Kubernetes already reports.
A simple typo in the image reference looks like a flaky pull instead of a wrong reference.

Reproduction (v0.1.3, Kubernetes driver on OpenShift)

openshell sandbox create --from docker.io/openshell-demo:latest   # namespace missing, should be docker.io/<user>/openshell-demo

CLI output:

✓ Sandbox allocated (2s)
✓ Image pulled (3s)
⠹ Pulling image... Error: ImagePullBackOff (15s)

Pod events for the same sandbox:

Pulled   spec.initContainers{openshell-sandbox-bootstrap}  Container image "ghcr.io/nvidia/openshell/sandbox:0.1.3" already present on machine
Pulling  spec.initContainers{workspace-init}               Pulling image "docker.io/openshell-demo:latest"
Failed   ...  reading manifest latest in docker.io/library/openshell-demo: requested access to the resource is denied

The same happens with a localhost/<name> reference (an image that only exists in the local engine): Kubernetes reports pinging container registry localhost: ... connection refused, the CLI shows only ImagePullBackOff.

Cause

attach_kube_progress_metadata in crates/openshell-driver-kubernetes/src/driver.rs maps every Pulled event to "pulling image complete" and every Pulling event to "pulling image active", without looking at involvedObject.fieldPath.
The bootstrap init container's Pulled event (runtime image already on the node) therefore completes the step, and the later Pulling event of the sandbox image reactivates it.
Failed events are not mapped at all, so their message never reaches the progress output.

Expected

  1. Only events for containers that run the sandbox image (workspace-init, the agent container) drive the "Pulling image" step; the bootstrap image is either ignored or shown as its own step.
  2. On Failed / ErrImagePull / ImagePullBackOff, the CLI shows the event message, ideally with a hint for the two common cases:
    • docker.io/library/... access denied: the reference has no namespace, Docker Hub expanded it to library/.
    • registry localhost unreachable: the image exists only in a local engine and the gateway runs on a cluster; push it to a registry the nodes can reach.

Related: #2675 (stale resource after ImagePullFailed).

Activity

  1. rhuss commented on Oct 12, 2026

    @rhuss
    ContributorAuthor

    Closing in favor of #2675, which covers the same provisioning path for failed image pulls. I moved the details there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions