How to Migrate from Firebase Test Lab to Marathon Cloud

MarathonLabs ·

Firebase Test Lab works until it doesn’t. The 45-minute hard timeout, suite-level reruns on flaky tests, and GCloud CLI setup complexity are manageable when you’re running 20 tests nightly. They become blockers when you’re running 500+ tests on every PR.

If you’ve hit any of these, you’re not alone:

  • Tests hang until the 45-min wall — common with Flutter integration tests (flutter/flutter#172290, flutter/flutter#183935)
  • --num-flaky-test-attempts reruns the entire suite, not just the failed test — so a single flake costs you another full run
  • Billing surprises — teams have reported $70K and $100K single-day Firebase bills with no automatic shutoff
  • Setup requires GCloud CLI + Firebase Console + Flank just to get basic sharding working

This guide walks through replacing FTL with Marathon Cloud. The migration is straightforward — your test code doesn’t change, only the infrastructure that runs it.

What changes

                        Firebase Test Lab          Marathon Cloud
                        ─────────────────          ──────────────
CLI ................... GCloud CLI + Flank         Single binary
Timeout ............... 45-min hard limit          5-min per test (configurable)
Retries ............... Reruns entire suite        Only the failed test
Sharding .............. Manual (via Flank)         Automatic (under 15 min)
Flake detection ....... None                       Built-in + analytics
iOS pricing ........... $5/hr (physical)           $3/hr (simulators)
Android pricing ....... $1/hr (virtual)            $2/hr (virtual)

Your Espresso tests, XCUITest suites, and CI pipeline structure stay the same. You’re swapping the runner, not the tests.

Step 1: Install the CLI

Homebrew (macOS/Linux):

brew tap malinskiy/tap
brew install malinskiy/tap/marathon-cloud

Binary download (Linux/macOS/Windows):

Grab the latest binary from the releases page.

Verify the install:

marathon-cloud --version

Step 2: Get an API key

  1. Sign up or log in at cloud.marathonlabs.io
  2. Go to Settings → Tokens
  3. Generate a new token
  4. Export it:
export MARATHON_CLOUD_API_KEY=your_token_here

You get 50 free testing hours to start.

Step 3: Run your tests

Android

Before (FTL + Flank):

# Upload and run with GCloud CLI
gcloud firebase test android run \
  --type instrumentation \
  --app app.apk \
  --test appTest.apk \
  --num-uniform-shards=10 \
  --timeout 45m \
  --results-bucket your-gcs-bucket

# Or with Flank (flank.yml + separate config)
flank firebase test android run

After (Marathon Cloud):

marathon-cloud run android \
  --application app.apk \
  --test-application appTest.apk

That’s it. Sharding, retries, and device provisioning are automatic.

iOS

Before (FTL):

gcloud firebase test ios run \
  --test path/to/Tests.zip \
  --device model=iphone14pro,version=16.6 \
  --timeout 45m

After (Marathon Cloud):

marathon-cloud run ios \
  --application sample.zip \
  --test-application sampleUITests-Runner.zip

Step 4: Update your CI

GitHub Actions

Before (FTL):

- name: Run tests on Firebase Test Lab
  uses: google-github-actions/auth@v2
  with:
    credentials_json: ${{ secrets.GCP_SA_KEY }}

- run: |
    gcloud firebase test android run \
      --type instrumentation \
      --app app.apk \
      --test appTest.apk \
      --num-uniform-shards=10 \
      --timeout 45m

After (Marathon Cloud):

- name: Run tests on Marathon Cloud
  uses: MarathonLabs/action-test@master
  with:
    apiKey: ${{ secrets.MARATHON_CLOUD_API_KEY }}
    application: app.apk
    testApplication: appTest.apk
    platform: android

Or call the CLI directly:

- name: Run tests on Marathon Cloud
  env:
    MARATHON_CLOUD_API_KEY: ${{ secrets.MARATHON_CLOUD_API_KEY }}
  run: |
    marathon-cloud run android \
      --application app.apk \
      --test-application appTest.apk

Marathon Cloud also has integrations for Bitrise, CircleCI, and Docker. See the CI/CD docs for setup instructions.

What you get that FTL doesn’t

  • No suite-level timeout — individual tests have a 5-min default batch timeout, but there’s no hard cap on total suite duration
  • Per-test retries — only the failed test reruns, not the entire suite
  • Automatic flake detection — Marathon flags unstable tests and tracks flakiness over time
  • Automatic sharding — you don’t choose the shard count; Marathon scales devices to hit a 15-minute target
  • Video per test — not just a device-level recording, but per-test video with logs and stack traces
  • Predictable billing — usage-based pricing with no surprise spikes; you pay for actual test execution time

Next steps