Shutting down the Fitness Data Importer

[Jason Levine]

After a lot of thought, I've decided that I'm shutting down the Fitness Data Importer, at least for right now. The bottom line is that I take strong issue with the Health Graph terms of service -- the terms to which all RunKeeper-related apps have to agree -- and I can't in good conscience continue running an application that helps users move their data into the Health Graph platform.

Over the past three days, I've had extensive discussions with the folks behind the Health Graph, and they have clarified that the terms of service disallow any applications which provide users the ability to export their own data. Their perspective is that the Health Graph is being provided primarily for applications, websites and devices to feed data *into* their system, and that the only way users will have to get their data *out* will be via mechanisms that are eventually built out on the RunKeeper website. But given that the Fitness Data Importer's purpose has been to allow users to export their data on their own terms from other platforms (like Nike+), it's dishonest for me not to expect FitnessKeeper to permit users to do the same with via the Health Graph.

I feel strongly that users are the owners and stewards of their own health and fitness data. When I go for a run and start RunKeeper, I'm pushing *my* run data into the Health Graph; when someone uses WeightConnect, they're sending *their* weight data into the Health Graph. From the day I brought the Fitness Data Importer online, I did so in order to help users get control of their data -- I had been evangelizing RunKeeper to plenty of people whose data was locked into other platforms, and I knew that I could help them make the move to a better system. But I was never interested in helping people get the data into another platform that locked it behind a wall -- I saw the Health Graph as just the first of what might someday be many data stores that allowed users to upload their health information, and my goal was to build a tool that would let users to make ongoing choices about where their data lived.

RunKeeper's current native data export facilities are limited to a single comma-separated-value with 11 columns of data about fitness activities -- no speed, pacing or heart rate timepoint data, and no data about the non-fitness activities supported by the Health Graph (like weights, blood pressures, calorie counts, blood glucose measurements, and sleep data) -- and the individual GPX files that can be exported on a one-by-one basis from activity pages. As such, Health Graph applications are currently the only option users could have to get the majority of their data out. If a user tracks blood glucoses or blood pressures in the Health Graph, there's literally *no* current way for them to get that data to their doctor without maintaining a separate, parallel record of it. The API makes it easy to get that data, but any apps that do so are forbidden by the terms of service.

So to this end, I no longer feel comfortable maintaining an application that allows users to import their data into the Health Graph from other services. I didn't come to this decision lightly, nor do I think that this is a done deal -- I have hope that the terms of service change over time and allow applications to give users the choice to export their own data in meaningful formats, even if that means exporting it in a way that lets them jump ship to another platform. In the end, for me this is about preventing platform lock-in, and when there's a way for Health Graph applications to provide users with their own data, then I'll reconsider.

[Ron Heerkens Jr]

I am really saddened it has come to this decision, I have enjoyed the interaction I have had with you and the ability to bring my Nike+ workouts into RK. Now I am left with no way to do so, and that makes me even sadder. Hope the best for you, and I hope this can come back in some way or form sometime soon.

-Ron

[Ron Heerkens Jr]

[craig wickesser]

I concur, it's a shame to see this go. Perhaps RunKeeper will loosen their terms or someone else will provide a tool to at least get Nike+ into RK, that's my current path as well.

-craig

[Jason Levine]

Ron and Craig, I'm obviously glad that the data importer helped you guys, and I assume you know that I too wish that it was possible to keep it up and running. But if the Health Graph is happy for people to use tools like this to export their data out of other ecosystems, but not happy to provide that same level of export from their own ecosystem, then it's just a matter of conscience on my part.

I've been thinking through whether there are other, non-Health Graph specific options -- like giving users the ability to get their Nike+ data into a common format like GPX or TCX or whatever, which would be very easy to do. But you should know that that doesn't help you fully on a platform like the RK website -- it's happy to get your GPS data from files like that, but will ignore the other data (like heart rate), and you have no ability to do anything bulk.

In any event, like I said, this is a work in progress. I think that if this saddens you, then it's worth you not only saying so here, but telling the RunKeeper/Health Graph folks that via their forums or support email. I agree with you, but my best understanding of their perspective is that they will provide all the user-facing export functionality forever, and hearing that this is a detriment to their users is likely a worthwhile message.

Jason

[Ron Heerkens Jr]

I originally used EagerFeet to grab my Nike+ and get the GPX. I was also able to use TrailRunner for Mac (until they updated the program to no longer work on my computer) and it imported the HR data, and then I would export the TCX which kept the HR data in tow. You'rs was just a simpler one stop solution.

[Reed Lilley]

I am also very saddened to see this go. I understand your criteria for ending the service, but I hate that now I won't be able to import my Nike + data into Runkeeper. It seems like it should be such an obvious task for them to allow. Ugh. 

Thank you for all of your hard work. I hope to see it back online again someday. 

[Dan Lundmark]

Thanks for this, I hope other end users will complain to RunKeeper as well and petition for a change in policy. You're right: our data is our data to import and export at our will as an end user. I expect RunKeeper to adapt their policy to this reality.

[craig wickesser]

Not to play devils advocate to much b/c I certainly like to have the ability to export "my" data from services, but when you sign up for the service (or use the API) you agree to these sorts of horrible things.

@reed -- ". It seems like it should be such an obvious task for them to allow" ....they do allow data to be imported via their API, which the importer was doing. So RunKeeper isn't stopping that, what they aren't allowing is bulk export which is (at least part of) the reason why the importer was shutdown (by the creator of it). However, it would be nice if RunKeeper would have a "import my nike+ data" feature, but from what I've read/seen so far, the Nike+ API isn't exactly that good/great and would probably distract the RunKeeper folks from doing other interesting things with their service. Instead, they'd prefer someone to use their API to get data imported into their site, which is what Jason had done with the importer.

@jason - do you have links or other information about the Nike+ API that you could share, perhaps someone could look into it and maybe get the Nike+ import into RunKeeper back online, at least for now until a better solution can be provided?

Thanks!

[Jason Levine]

Craig, there really is no link that details the Nike+ API -- they actually don't make the API public or document it in any way. I spent a lot of time looking at other code that interacts with the various Nike+ JSON and XML methods, like EagerFeet (https://github.com/eagereyes/eagerfeet) to figure out how to get the data. But with that said, since it's not a public API and not maintained for use and consumption by anything but Nike's own website, there are a *lot* of bugs and quirks that I needed to work around (to which Ron can attest!).

Here's the list of sites I bookmarked along the way that might have been helpful...

http://pinboard.in/u:delfuego/t:nike%252B/

Jason

[Reed Lilley]

Anything new about this? Does it seem like it's a done deal and those of us with Nike+ are just going to have to enter our runs manually? Or is there a bright light at the end of a long tunnel? 

[Jason Levine]

Nothing new at my end -- there've been no changes to the terms of service for the Health Graph API, so the importer remains shuttered. Have you followed up with the Health Graph API/RunKeeper folks about this? This isn't a forum of theirs, and I assume they don't read anything here...

Jason

[craig w]

reed -- i'm playing around with possibly putting something up to import data from nike+ to runkeeper, as time permits. Although I might make it open source so other people could host it / change it themselves. Jason's application is something he did himself and paid to have hosted, I may not incur the cost of hosting the application myself which is why I'd put the source code out and probably instructions on how to get hosted somewhere (there are a few free options).

As I make progress I'll post something here.

-craig

[Ron Heerkens Jr]

Craig, keep me updated. I was a constant user of Jason's project and fully supported it(I even offered donations). As you get it going let me know, as I know we kept running into several Nike+ side glitches during many of my imports. Thanks!

[Ron Heerkens Jr]

I have also come to a crossroads. Personally I liked the Nike+ watch at first, but its limitations (AND SITE) are killing me. I inherited a Garmin 110 last night for free, and even that has me more pleased in 5 minutes than the 4 months I have had the nike+. I most likely will be getting rid of my Nike+ and moving into a Garmin 610 sometime in the near future.

[Jason Levine]

That's awesome, Ron -- Nike+ has always seemed like a totally closed ecosystem to me, which is why I wrote my app to help folks escape it in the first place.

J

[rc178]

What we need is an Open Health Graph.  It's sad that each fitness company has their own proprietary method to store fitness data.  Maybe we can lobby RunKeeper to make their Health Graph open source.  

[Chris Lukic]

I've been working on solving this problem for the last year. The result is smashrun.com. You can see my running stats here for an example. I whole heatedly believe that individuals should own the data that they collect about themselves. The very idea that your running data should be locked up in a silo and held hostage just seems contrary to the ethos of sharing and support that runners the world over exhibit. I think if you compare how sites like DailyMile have crafted their terms of service (ie grab what you need but try not to bring our site down) versus how sites like Runkeeper (your data is our property) it's an illustration of the often contradictory demands of building a billion dollar company versus supporting a community you care about.

We currently import data from Nike+, but I'm working as fast as I can to offer more options. Although Smashrun is more of an aggregator then an originator of running data, users do have the ability to add and and amend runs manually, so high on my list is providing ways for users to get their data out as well.

Information is the currency of the modern age, and I think as more people begin to realize that they're giving up title to their own words, photos, and even intimate health data by sharing them with the wrong company their will be a severe backlash against these closed ecosystems.

Happy running,

Chris

PS

Sorry, I'm a bit late to this thread... 

[Chopsicles]

Hey Chris,

I checked out your smashrun. It looks pretty cool and imports my old (very old, 2008) nike+ data just fine. I'm just wondering how/if there will be an option to export them so runkeeper will understand - or to import from runkeeper (mapmyrun, sanoodi, etc etc). I was sure I would stick with runkeeper, now I'm not so sure haha... well it looks great anyways, so keep up the good work :)

Chops

[craig w]

Chris,

The app looks great! I can't wait to use it to import my Nike+ data. I was also inspired to create something, however, I never got through it. I had started fetching my Nike+ data but stopped there, so I'm glad to see someone else take it on.

Just curious, looks like you're using PHP, HTML/CSS/JavaScript, jQuery and maybe 960 grid for layout? is that about right? I'm a developer myself and I really appreciate the layout and design of the site, so I was digging around.

Thanks,

Craig

[Reedly]

Just wanted to chime in and say that I've been using Smashrun for a few months now and it's a blast! Site looks great, fun to use. Thanks!!

Reed

[Chris Lukic]

There will definitely be an option to export data in a variety of standard formats, as far as import goes were looking to partner with running iPhone apps who have an open data policy.

Runkeeper unfortunately has a very onerous terms of service similar to that of Facebook. If we import their data, we could be sued for providing functionality for free that they charge for. Additionally we'd also have to prevent users from exporting that data from our site. They're trying to be the "Facebook of Health" and I imagine a large part of the value in that is being able to mine and sell their users data. If you let people export it for free than you undermine that value.

[Chris Lukic]

Oh man, I don't blame you for not finishing it. The work involved is a nightmare, at least now I'm on the other side of a lot of the hard bits. Err...I hope.

Our stack is unusual, but it's worked out really well. SQL Server / C#  (just about zero ASP.NET) / JQuery and some heavily customized ajax libraries I developed while building my last site NewYorkonTap.com I used 960.gs on the home page and it was awesome. If I had it to do over again I'd have build the HTML on it from the ground up. 

[Chris Lukic]

Small world of techy runners huh? I've been recommending that site you found to our users who need Garmin import, and it's been working out great while I try and sort it out naively.

 http://awsmithson.com/tcx2nikeplus/ 

[craig w]

Chris,

My Nike+ account was based off my Facebook info, meaning I never setup a password. I just set a password on Nike+ but smashrun still says "invalid login".

Could smashrun just use my nike+ id instead? That's the approach I was taking (just like the Fiteness importer did) which didn't require knowing the users login credentials for Nike+.

-craig

[Chris Lukic]

You can register on Nike+ with Facebook without creating a login? Uh oh.

I could use the Nike Id, but initially decided not to because I didn't want to open up the possibility of someone accidentally (or less likely on purpose) importing someone else's runs,  and because I felt that Nike's process for setting your data to public might be too confusing for some people.

[craig w]

Yea, I think you can register using Facebook w/o setting a username/password.

Anyways, I set a password on Nike+ and now smashrun is working.

Thanks,

craig

[Reedly]

Yeah I've been using that site religiously to upload runs from Garmin to Nike - then from Nike to Smashrun, Dailymile, and Facebook. Hopefully someday all of the big running/workout sites can come together and allow sharing of data with each other. It's fun to use one site to geek out on, but it's even more fun to use multiple sites! Especially since some sites are based on the social aspect, while some are all about the data.

It's great to watch and learn as you guys discuss these things. Thanks for keeping some of the tech talk public!! 

Reed

[Jarrid Rollins]

This is a shame that it comes to this.

[PJ]

Maybe I'm lost, but it seems to me that Runkeeper actually made the >right< decision here. I don't >want< other apps to have access to my data in the Health Graph. It's way too simple to "click and forget" on services requiring access to a "Graph", which seem convenient at first, but later turn out to be spam data collectors. Only a few days ago I found out Facebook is allowing third party websites to access my data - via my friends social graph - and immediately turned off that function.. I agree the export function from Runkeeper should be as complete as possible and I don't understand the lag in making this happen. For that matter, I don't understand why Runkeeper wouldn't build their own import tool...!?

I think you jumped the gun on this one and would love to have this import option available

-- PJ

[Jason Levine]

PJ, you're essentially saying that (a) you're glad that apps don't have access to your RunKeeper data, and (b) you wish RunKeeper had access to OTHER fitness tracking apps' data so you could import it. It's hard to square those two things, which is why I brought the importer down -- it's hard to justify RK benefitting from the ability to get data out of other trackers if RK doesn't likewise allow other trackers to get data out of it.

And to make sure it's said, there's a *huge* difference between the Facebook issue you're concerned with -- FB having *default* settings which allow access to your data by third parties -- and what we're talking about here. In the case of RK and the Health Graph, we're talking about you giving explicit, opt-in permission to a single third-party app which has known functionality to you. The choice is entirely yours, not left in the hands of default settings you're unaware of.

In any event, the great thing about public APIs is that they provide anyone with the ability to provide the functionality they seek. I'm not ruling out the Importer coming back to life someday, but in the mean time if you're keen on seeing import functionality, you're always able to hook into the Health Graph yourself and provide it.

Jason

[RonJ]

Jason, 

Just out of curiosity, have you looked to see how RK data is stored on the various devices they support?  One reason it was possible that you could take Nike+ data and import it into RK (or elsewhere) is because the Nike+ data files are saved in plain text/XML on the various devices.  On a Nano, you could get to these files just by opening a Terminal window.  iPhones and Touches have made it more difficult, but if you root them it's still possible.  I can't help but think that RK data (generated by an iPhone or Android) must also be stored on the device somewhere.  Have you looked?  Are the data stored as binary files?  I'm not advocating this, but is it possible to get to and read RK data files?  This procedure would only work for run data generated on the device, and not manually entered on the RK site, but it's a start.  

I applaud your decision to no longer support the data import service if RK isn't willing to provide a full-featured export service/API.  But I've reached the last straw with Nike+, and need to move on to a new run monitoring tool.  I want whatever I switch to to be around for several years, and I'm afraid RK is the company most likely to do that.  It will be sad in 10 years when iNewRunMonitoringCompany comes online and we still can't get our data out of RK, but I'll worry about that then.

-R

On Sunday, October 9, 2011 12:30:21 PM UTC-7, Jason Levine wrote:

After a lot of thought, I've decided that I'm shutting down the Fitness Data Importer, at least for right now. The bottom line is that I take strong issue with the Health Graph terms of service -- the terms to which all RunKeeper-related apps have to agree -- and I can't in good conscience continue running an application that helps users move their data into the Health Graph platform.

Over the past three days, I've had extensive discussions with the folks behind the Health Graph, and they have clarified that the terms of service disallow any applications which provide users the ability to export their own data. Their perspective is that the Health Graph is being provided primarily for applications, websites and devices to feed data *into* their system, and that the only way users will have to get their data *out* will be via mechanisms that are eventually built out on the RunKeeper website. But given that the Fitness Data Importer's purpose has been to allow users to export their data on their own terms from other platforms (like Nike+), it's dishonest for me not to expect FitnessKeeper to permit users to do the same with via the Health Graph.

I feel strongly that users are the owners and stewards of their own health and fitness data. When I go for a run and start RunKeeper, I'm pushing *my* run data into the Health Graph; when someone uses WeightConnect, they're sending *their* weight data into the Health Graph. From the day I brought the Fitness Data Importer online, I did so in order to help users get control of their data -- I had been evangelizing RunKeeper to plenty of people whose data was locked into other platforms, and I knew that I could help them make the move to a better system. But I was never interested in helping people get the data into another platform that locked it behind a wall -- I saw the Health Graph as just the first of what might someday be many data stores that allowed users to upload their health information, and my goal was to build a tool that would let users to make ongoing choices about where their data lived.

RunKeeper's current native data export facilities are limited to a single comma-separated-value with 11 columns of data about fitness activities -- no speed, pacing or heart rate timepoint data, and no data about the non-fitness activities supported by the Health Graph (like weights, blood pressures, calorie counts, blood glucose measurements, and sleep data) -- and the individual GPX files that can be exported on a one-by-one basis from activity pages. As such, Health Graph applications are currently the only option users could have to get the majority of their data out. If a user tracks blood glucoses or blood pressures in the Health Graph, there's literally *no* current way for them to get that data to their doctor without maintaining a separate, parallel record of it. The API makes it easy to get that data, but any apps that do so are forbidden by the terms of service.

So to this end, I no longer feel comfortable maintaining an application that allows users to import their data into the Health Graph from other services. I didn't come to this decision lightly, nor do I think that this is a done deal -- I have hope that the terms of service change over time and allow applications to give users the choice to export their own data in meaningful formats, even if that means exporting it in a way that lets them jump ship to another platform. In the end, for me this is about preventing platform lock-in, and when there's a way for Health Graph applications to provide users with their own data, then I'll reconsider.

[Jason Levine]

Ron, there's really no way to get the data off of a phone -- at least not in a way that most (if any) users could pull off. I'm not interested in only supporting jailbroken phones, etc. And to be clear, when the Importer was running, it used the Nike+ API (an unpublished one, but the same API that is being used by Nike itself to get and send data to the service).

Jason

[Steve Smith]

Just use http;//eagerfeet.org to export your data from Nike+ to RK. I understand the author's choice to support this app or not but in the end it's the end user's decision to upload to health graph with or without the ability to export their data. Thanks for looking out for us end users, but it's my decision, so I just use eagerfeet.org.