Comparing accuracy of the GPS-based running apps

Radiant Tap is hard at work on new GPS-based tracking engine for our running apps, which will first appear in our best-selling Go Couch to 5k – in version 2.0. One major upgrade we plan to do is to significantly improve the accuracy of distance tracking. As part of this work, we have tested many competing apps, to see where we stand and how precise others are.

How we tested

On iOS, Core Location is the framework responsible for finding and tracking one’s current location. It uses several different technologies to do that, but the most precise one is based on GPS. This is available since iPhone 3G, although our tests show that GPS is usable for walking/running starting with 3GS and on. Current generation – iPhone 4S – goes one step better then its predecessors since it includes the support for GLONASS, which is similar system developed and maintained by Russia. All together, this makes iPhone 4S the perfect test phone.

We used a straight path in the middle of New Belgrade (Serbia’s capitol), exactly 400m long.

Before testing each app, iPhone was reset, to eliminate GPS cache influence. Then we went walking the path two times, one test walk after another. Why like this? Well, as you will see, tests and experience tell us that Core Location updates are wildly inaccurate at the very start, while iPhone is still trying to triangulate user position. The theory says that for a good match, you need to receive position info from at least 3 different satellites. On iOS though, we have no idea when this happens – Core Location handles this transparently and all you get is an object saying: this is your lat/long position and accuracy is +/– 10m or 30m, like that.

So lets see how competition fares. We tested:

  • Nike+GPS (currently best-selling app of its kind on App Store, uses MotionX engine)
  • Active.com Couch-to-5k (a Top 10 app at the moment)
  • MapMyRUN
  • RunKeeper
  • RunMonster
  • Runmeter

All of these are almost always in Top 50 of the Healthcare and Fitness category on the US App Store.

First test walk results

Here are test results for each of these apps on cold-start, after iPhone was restarted. For each one, we waited until they reported GPS signal as maximum and then started the walk test.

Nike+GPS

RunKeeper

Runmeter

MapMyRUN

Run Monster

As you can see, both from the measured distance and the maps, each of these has significantly missed the target 400m range, reporting values from 1.25km (Nike+GPS app) all the way to almost 3km – 3-7x the actual length! The one exception is Active.com Couch-to-5K app which was nearly perfect:

Active.com c25k

Our kudos to their developers, they did a really good job in filtering out the crazy outlier positions.

This test walk was around 4 minutes and for each app, immediately after we got to the end of the test path, we would start another test walk in the opposite direction.

Second test walk results

Nike+GPS

RunKeeper

Runmeter

MapMyRUN

Run Monster

As you can see, all the apps are very good to perfect. We did test Active.com app as well, but it basically repeated the results from the previous run, thus no need for a screenshot.

What this means is that iPhone 4S probably needs about 2-3minutes to fix the device position properly; when that happens, CoreLocation API call distanceFromLocation: is returning usable values (we suspect that most devs are using this value to calculate covered distance).

How Go Couch to 5k fairs..?

Our current live version (1.7) is acting the same as majority of the tested apps. Our new engine – which is still in development – is near the perfect results. We are currently filtering bad position updates so strictly that we calculate the distance under the 400m :) – this is the cold start test walk:

You can see from the screenshots that even though map shows crazy zig-zag line, our measured distance is very close to actual path length.

We intend to get this perfect for the 2.0 release. Stay tuned.

Published on · Couch to 5k, apps