Repository navigation
feat: e2e tests - #1501
feat: e2e tests#1501
Conversation
|
@MrMuzyk is attempting to deploy a commit to the Callstack Team on Vercel. A member of the Team first needs to authorize it. |
|
# Conflicts: # apps/tester-federation-v2/package.json # pnpm-lock.yaml
There was a problem hiding this comment.
Thanks for your work on this. I ran the suite locally at 4feac606 on a dedicated API 35 emulator and an iOS 26.4 simulator: 3/3 on Android and 3/3 on iOS, with --workers 1 --retries 0, in a separate run per platform. So the tests themselves work.
Inline comments cover the details. The main points:
- Trigger model: I'd switch from
pull_request_targettopull_request. See the comment onon:. - The root cause from #1499 is still there.
LogBox.ignoreAllLogs(true)hides the toast, but the MF2 shared versions still resolve tocatalog:testers. WithoutE2E=true, the Gallery test still fails locally. - Determinism: pin the device, set
retries: 0explicitly, and turn telemetry off.
Also:
e2elabel: the repo doesn't have it yet (gh label listreturns nothing). Without it, labelled fork runs are skipped silently. The description already lists this as a step before merge, so I'm noting it here so it doesn't get missed.- No CI run at this head. The green run linked in the description is a
workflow_dispatchonc33c7bb3. The only difference from4feac606is thatsrc/host/index.jsloses the comment explaining the LogBox gating. If the gating stays, please keep that comment. - Running locally with Xcode 27: the iOS build needs
IPHONEOS_DEPLOYMENT_TARGET=15.1passed toxcodebuild, because AsyncStorage's pod targets 13.0. This doesn't affect CI onmacos-15/Xcode 16.4.
|
@dannyhw Ive addressed the comments and moved away from the approach with 'e2e' label as implied by the comments in the review. It's ready for another round of review ;) |
|
Added manifest check to bot jobs; Added gathering ios device logs; Added a new helper for configs; Ready for re-review :) |
dannyhw
left a comment
There was a problem hiding this comment.
Nice work.
Thanks for your contribution 🙏
Summary
Adds an e2e lane that runs the MF2 tester app on a real emulator/simulator on CI, so Re.Pack bundling/runtime regressions are caught before merge instead of in a consumer app.
.github/workflows/e2e.yml— Android on an API 35 emulator, iOS on an iPhone 16 simulator. Each job starts both Re.Pack dev servers (host app :8081, MiniApp :8082), warms both platform compilers, builds the Debug app, then runs the suite against it. Android runs on PRs; iOS runs on merges tomain(and dispatch), since it can't fit inside a PR's deadline yet — Phase 2 moves it onto a beefier runner.apps/tester-federation-v2/e2e/smoke.e2e.ts— 3 deterministic tests (app boots, Home → Detail → back, Home → Gallery loads the MiniApp over the MF remote), viatester-army/e2e+@e2e-dev/mobile. No model credential needed.retries: 0andworkers: 1are set explicitly: a flaked first attempt must read as a failure, notflaky, and the device is pinned (E2E_DEVICE) so a local run never lands on someone else's booted emulator.pull_requestwithpaths:filters (app, packages, lockfile, the workflow itself),contents: read, no secrets, and the default merge-ref checkout — so GitHub's fork-workflow approval is the gate, no label needed, and PR edits toe2e.ymlare self-tested before merge. Apushtomainnever cancels, so the "main is healthy" signal survives a back-to-back merge. Telemetry off at workflow level (E2E_TELEMETRY_DISABLED: '1').Notes
e2e@0.19.0/@e2e-dev/mobile@0.11.0declareengines.node: ^22.22.3 || >=24.8.0while the repo's rootenginesallows>=24, and there's noengine-strict— on Node 24.0–24.7 a contributor only gets a warning and then a confusing failure. CI pins 24.18.0.tester-federation(v1) carries the samecatalog:bug in its configs (incl.async-storage) — deliberately untouched here; worth a follow-up PR.IPHONEOS_DEPLOYMENT_TARGET=15.1forxcodebuild(AsyncStorage's pod targets 13.0). CI onmacos-15/Xcode 16.4 is unaffected.Test plan
CI is the evidence — with the
pull_requesttrigger, the Android lane runs on this PR's head; iOS can be proven per-PR viaworkflow_dispatchand runs on every merge tomain. On failure, thee2e-android/e2e-iosartifacts carry junit + screenshots + logcat + both dev server logs.Locally (needs Node ≥ 24.8 or 22.22.3; CI pins 24.18.0):
pnpm android/pnpm ioswith the app's own scripts).cd apps/tester-federation-v2, start both servers:pnpm start:hostappandpnpm start:miniapp(no flag needed anymore).E2E_DEVICE=<name or serial> pnpm exec e2e run --target android(or--target ios). With several devices booted, the pin keeps the run off the ones other work is using; unset,@e2e-dev/mobilepicks up every booted device.