Peloton: Firebase Test Lab couldn't keep up with Android TV and wearables

Anton Malinski ·

The problem

Peloton was expanding to Android TV and wearables. Firebase Test Lab broke in two ways:

Wearables: FTL’s physical wearable device worked for 3-4 months, then Peloton’s app needed a GMS library the device didn’t support. Google’s response: “we might add that someday.” Peloton fell back to local testing — one phone, one wearable, no sharding. Test suite runtime: 1.5 hours. Triggered daily, before/after PR merges, and every release candidate.

Android TV: FTL’s TV devices were stuck on API 28. Slow. Frequent “Test Instrumentation Fail” errors. CI/CD was unreliable.

Both problems hit during high development velocity on the wearable product. Physical device costs were climbing.

What changed

Peloton switched to Marathon Cloud. Virtual devices, automatic sharding, PR-level test triggers.

Wearable results:

  • 1.5 hours → ~10 minutes
  • No physical device dependency
  • Tests run on every PR without budget concerns

Android TV results:

  • No more “Test Instrumentation Fail”
  • No more outdated API levels
  • Stable CI/CD pipeline

By the numbers

Wearable test time ..... 1.5 hours  →  ~10 min
Android TV stability ... frequent failures → stable
Release time saved ..... 3-5 weeks/year
Physical device costs .. eliminated

Why FTL failed here

Firebase Test Lab is designed around phones. Their wearable and TV device support is an afterthought — limited physical inventory, slow to update, no guarantees on GMS library availability. When Google tells you “we might add that someday,” you need a different solution.

Marathon Cloud runs virtual devices. No physical device inventory constraints. No waiting for Google to update hardware.