Training on a course with Computrainer in free mode

[Ron George]

Hi guys,

Great work with the new beta 3.6. So far, its stable on my system and working fine. 

I'm only trying this for the first time on an old Computrainer (which works) so bear with me. I have a .gpx file for the Luz Ardiden climb and I want to be able to ride this in "free mode" on the trainer, kind of like in Zwift, wherein it simulates a power based on variables I have set in GC's new Virtual Bicycle Specifications and the gradient within the .gpx file. The file has no power or speed data, just distance, lat,long and gradient information.

What steps do I need to take? Or am I misunderstanding the purpose of this feature?

If there's a simpler way I can achieve what I need, please provide guidance.

Best regards,

RG

[Ale Martinez (Please don't email or cc me)]

Just use Import Workout or Scan Workouts from Tools or hamburger menu to import the .gpx and select it as workout on Train Sidebar, you can visualize the elevation profile adding a Workout chart using View > Add Chart... There is a "Working with Train Mode" tutorial on http://www.goldencheetah.org/#section-tutorials and a FAQs section on the wiki which may help.

[Ron George]

Ale,

Thanks.

1. I dont have my trainer connected right now. But I did the View > Add Chart and I see the elevation profile presented in gray shading in terms of Watts on the Y axis over Distance (km) on the X axis. Watts is in the 1000s. Is this alright? 

2. What does Virtual Bicycle Specifications feature do, any documentation? I thought it would be handy to code my bike right in.

I'll try in the morning. Awaiting your responses.

Regards,

Ron

[Ale Martinez (Please don't email or cc me)]

El lunes, 31 de agosto de 2020, 18:54:06 (UTC-3), Ron George escribió:

Ale,

Thanks.

1. I dont have my trainer connected right now. But I did the View > Add Chart and I see the elevation profile presented in gray shading in terms of Watts on the Y axis over Distance (km) on the X axis. Watts is in the 1000s. Is this alright? 

When a distance/slope workout s.t. GPX is selected the Workout chart (but not the Workout Editor) will show the elevation profile instead of watts over time, which is intended for structured time/power workouts.

You can use the Robot device to play with workouts without the need to step on the trainer.

Having a synch video makes that experience more enjoyable, current version includes Live Map, see https://github.com/GoldenCheetah/GoldenCheetah/wiki/FAQ-TRAIN#how-can-i-have-synchronized-video-in-my-training-session

 

2. What does Virtual Bicycle Specifications feature do, any documentation? I thought it would be handy to code my bike right in.

Each parameter has a tooltip but I don't know about more extensive documentation, this is the PR https://github.com/GoldenCheetah/GoldenCheetah/pull/3263

[Ron George]

Hello Ale,

I tried your suggested approach this morning and it was so choppy in terms of the load I was getting on the trainer that it was practically non-rideable. I don't know why it makes the trainer behave that way, but I could see some rapid fluctuations on the "slope" telemetry, i.e it didn't seem I was riding a steady slope. 

Steps I took :

1) Added computrainer in list of devices

2) Added .gpx file in the workout

3) Added video 

4) Added a few items of telemetry

5) Hit start and then the play button

6) Video plays successfully with telemetry, too much choppiness in load felt at trainer. 

If someone from dev team wants to try this on their trainer, I've attached the file here. It'll be good to know whether there's something wrong with the file or if this slope mode feature actually works as intended when there is no power data available in the file. Thx.

Ron

[Ron George]

Just wanted to add : I also used Training > Preferences > Use Simulated Speed in Slope Mode. The parameters were set in Training > Virtual Bicycle Specifications. 

-Ron

[Ale Martinez (Please don't email or cc me)]

El martes, 1 de septiembre de 2020 a la(s) 16:04:50 UTC-3, ron.r....@gmail.com escribió:

Hello Ale,

I tried your suggested approach this morning and it was so choppy in terms of the load I was getting on the trainer that it was practically non-rideable. I don't know why it makes the trainer behave that way, but I could see some rapid fluctuations on the "slope" telemetry, i.e it didn't seem I was riding a steady slope. 

Steps I took :

1) Added computrainer in list of devices

2) Added .gpx file in the workout

3) Added video 

4) Added a few items of telemetry

5) Hit start and then the play button

6) Video plays successfully with telemetry, too much choppiness in load felt at trainer. 

Slope can be too jumpy with GPX files, you can import the GPX and use Edit > Fix GPS to smooth altitude, when you are happy with the result, export as GPX or better as JSON and import as workout 

[Ale Martinez (Please don't email or cc me)]

* Import as an activity 

[Ron George]

Thanks, I thought that might be the problem. Would you suggest B-spline 2 pass smoothing for this file within Settings for Fix GPS? 

[Ale Martinez (Please don't email or cc me)]

Yes

[Eric Christoffersen]

Hi Ron,

Please let us know if you don't like any of this. Love feedback. I'd like to hear if you think the gps smoothing interface is usable. The smoothing has two phases, first is altitude, second is location smoothing. The altitude is first and separate because satellite data seems to need a pretty heavy hand before its ridable.

There is a "LiveMap Chart" change pending, when its in you'll be able to see a scrolling map centered on where you are. Its almost as good as video :).

Pics here:

https://github.com/GoldenCheetah/GoldenCheetah/pull/3487 

 

[Ron George]

Hi Eric,

Thanks for the email and your development efforts.

 

I applied the altitude smoothing and have yet to ride it , but I'll give it a shot today. I also split the original .gpx to just the climb segment and think my video and climb should align well now.

As for feedback, sure I can help you out. Here are a few :

1. It should be easier for someone to move telemetry charts up, down, sideways. Its inflexible now. I see that it can be re-sized but I was looking to shift things around easily. 

2. Have you used Perf Pro studio? If you need some inspiration for how "compact" telemetries should look like, download a demo and have a look. I really like that F1 style interface look. :) Not sure how much coding this would take, but I can't imagine it'd be more complicated than the things you have already done. You could get rid of all these rectangles that way. 

3. I'm having to scroll down the Train view page to get to the video, and scroll back up to see the numbers. Too much mouse clicking while riding. Can the video open up in a "separate" instance or window, so if someone were to use two screens, they could move that window over to the monitor? 

Sorry I should really offer this feedback over on github, but this is easier for me. 

I'll give the slope mode trial a shot. But can anyone confirm to me that my virtual speed will be impacted by the virtual bicycle specifications I set if I check the box in device settings? Hope I'm using it as intended.

Regards,

Ron

[Ron George]

Hi Eric,

*Update** 

Slope mode with GPS corrected file is still way too choppy for my enjoyment. Too many rapid load changes. I'd rather do it in erg mode and manually change the slope. 

Please have a look at the file attached from your end in train mode. 

-Ron

[Eric Christoffersen]

Ron,

Thanks Much for the smoothing feedback. I have been trying for a while to come up with a way to make the smoothing more intuitive to use, but haven't found something that works better (for me.) I suspect there are changes that could be made in the controls that would make this much easier to use. I'd love more feedback or ideas.

I loaded your file and played with it. Slope in your file was computed from altitude. The slope is noisy because of noise in the altitude data. The smoothing acts on the altitude data, so should be able to achieve whatever you desire. Almost certainly the problem is with the controls and ui, so I am going to explain here.

I am just going to discuss altitude smoothing but everything is the same with route smoothing, except altitude data is 2d (distance, alt) and route smoothing is a 4d spline (distance, lat, lon, alt)

The altitude smoothing works in two passes. First it finds and discards outliers, second it rewrites all data so they live on a smooth line.

The first pass works by building 'a smooth line' with your data and then throwing away all points in your file that are 'too far' from the smooth line.

The second pass draws another smooth line and simply sets all the altitude data so it lives on it.

This means you can use a big wide smooth line for the first pass, throw out the outliers, then use a less smooth line to better fit the non-outlier data.

If you don't like the concept of outliers you can prevent any from being thrown out by increasing your criteria % to something huge like 10000, or set your first pass smoothing degree to something small like 3.

Here is the default smoothing menu:

The first thing to look at is the "Step Deviation" statistic. Roughly speaking, that is the distance between your activity's samples.

The 'pass 2 altitude smoothing degree' is the size of the window (in data points) that will be used to build the smoothing line for your data.

For example, a 'degree' of 100 with a step deviation of 2.25 means the smoothing window size will be about 225 meters. That means that points on our activity that are more than 225/2 == 112.5 meters away from a point will have no bearing on the smoothing of a point. Generally a larger degree means more points act to smooth and the resulting graph will be smoother. You can increase or decrease degree to make line smoother or less smooth.

The test button and the stats are there to help you pick good criteria. Usually I use the test button and iterate the parameters to get something close to what I want, then I apply and view the slope. If it doesn't look good I revert the edit and go back to the gps smoothing menu to try, try again.

Here is the original slope:

Close up of some spikes:

I tried the default settings, 500/100%/200, in the ui, pressed 'test' and go this:

See that the max slope is 36.3? That is much too high.

I tried 100/3%/200:

Max slope is 19.5, still too high.

I played more and ended up with 500/1%/500.

Can see the spikes in above closeup are gone:

If you want it even smoother try 500/0.5%/500.

Fundamentally smoothing is subjective so you must be the judge.

Issues I found when I was playing with the smoothing:

- there's a small region at the start (from 0-5 meters) where it looks like I'm getting 'ringing' from how the smoothing works at the start boundary. These artifacts are throwing off the stats. This is a bug and I need to fix it.

- % standard deviation isn't a great way to specify outlier criteria.

- window size in points isn't great either. I'd love to use meters but the problem is sample distance can be extremely variable which means window size is also variable in terms of meters.

Eric

 

[Ron George]

Great explanation and guidance Eric!  I'll need to cut-paste what you wrote into a Word doc and save for future! Infact, this can be edited somewhat and even loaded into Github under help documentation. 

 

I now understand what all those numbers mean.  The ringing you describe at the beginning was very unpleasant especially on my knees and I got pretty frustrated! 

So looks like this is an iterative process, until I have the right window size to get a good avg slope with slope outliers eliminated. I'll also need to throw in a slope chart to see what effects it has on my sample after the smoothing is done. That was something I didn't have in my graphs. 

I'll give this another shot tomorrow. 

Ron

[Eric Christoffersen]

I believe what I call ringing is an artifact of the smoothing. Its just in the first 3 meters, so you shouldn't really feel it. Its real damage is that it causes the min/max statistics to show much worse numbers. The smoothing approach should not be showing this startup artifact so there's a math bug there.

Something in what you wrote above, note that the 'average' slope should not change very much at all because the peak and trough won't change much. You should be focused on reducing the max and min slope. Almost certainly the crazy stuff in your legs is because the slope was still super steep and changing quickly. Needed more smoothing! :)

When I smooth my rides I start by viewing the graph and characterizing the noise, write down how big and steep are the outliers? After I eyeball that I can derive smoothing parameters that work pretty well. Ultimately though, if you know the place you'll know that this spot is really steep, and that other spot on your ride shouldn't see big jumps. That is where your knowledge is needed to direct the smoothing.

I wish I could snap my fingers and create a graph view of the smoothing so you could see the input, outlier smoothing line and the output graph. The picture would be much better than numbers. Problem is that its a bunch of pretty boring typing to make that happen.

Was also thinking it might be better to define outlier in meters, instead of by % of deviation from the outlier spline. Benefit of deviation approach is that it allows for natural 'float' in noisy regions, this helps to preserve altitude peak and troughs but it might be what people really want is to follow the smoothing line with a crisp cutoff.

I'd really like to hear input on how to make the smoothing easier to use.

[Ron George]

Eric, 

I'm curious what you would do with a file for another climb. See attached .gpx file. 

The ground reality is that this climb have a max slope of around 11.7% only, not 12.2 you get with the default smoothing parameters. The min slope is only around -6, not -13.9 testing with the default smoothing parameters. Avg slope is ok. 

I iterated on a set of parameters I thought gave me a reasonable avg slope, max slope and min slope which was (300,4%,200). 

 

I've attached a pre-smoothing and a post-smoothing image showing the differences in slope in the first 0.5-0.6kms of the segment. What do you reckon? Would I need a new kneecap? :) 

Original .gpx file is attached for your review.

Thanks,

-RG

[Ron George]

Eric/Ale,

Following my previous post, next thing I did was to export this smoothed file as another GPX, imported it into my Train view as a workout and connected a Robot device operating on one of the trainer curves listed. 

When I connect and hit play, I notice that the slope telemetry wildly swings around despite the smoothing that was done. The slope fluctuations will continue to be a problem for pedaling smoothness. I believe one of you could try what I have done, I've attached the .gpx file again. Is it "really" reading the file that I just smoothed, if not what's is it reading?

-RG

[Ron George]

Eric/Ale,

*Update**

I utilized a Kalman smoother from an external 3rd party over this file with Process noise = 0.001, Derivative = 0, Polynomial = 3 and the smoothing was much nicer. See attached file. 

I input that file into the Train workouts and this has virtually eliminated some of the quick jitters that the GC smoothed file was still facing. This could be much more enjoyable to ride.

I think one of you need to have a review of my troubles mentioned in the last two emails. The questions really are :

1. Please review the smoothing I have done on the file previously and would there be a way to make it better.

2. Despite smoothing the .gpx in GC, Train view file via "Import workout from file" still face same jitteriness in the slope making the course virtually unrideable (atleast for me). 

3. Can additional filtering tools be provided for fixing GPX errors to give user a wider flexibility in options?  

Thx

RG

[Eric Christoffersen]

Hi Ron,

Wow, what an amazing place to ride. I sure wish I had access to the accompanying video. I bet the food is ok too.

I debugged the smoothing on your file and think I found the issue. Remember before I was saying it looked like a math error around the end points of the route? In this case when I went to use reasonable parameters the outlier removal was removing the route endpoint which left the spline with too much freedom. In this case the final 30m or so of the routes points found themselves outside of the region of the spline so the altitude query failed. This aborts the smoothing pass and no change was made. I was able to get some other parameters to complete but they weren't great.

Wondering is it possible you were re-riding unsmoothed rides?

You can view the raw data for your activity under 'details' / 'raw data'. For example here are the two points at the start that are responsible for the initial spike.

Anyway. I fixed the smoothing so it will no longer remove the route start and end and the smoothing works now.

Here is the original:

Here is way oversmoothed with 500/1%/300 (too many outliers removed):

Here is 500/5%/300:

Here is with 300/5%/300 (same outlier criteria but smoother outlier spline):

200/5%/200 (I like this one: just gritty enough for me):

I tried this 200/5%/200 on my wahoo kickr and I didn't experience anything that I thought was bad. I've attached the json file to this response so you can try if you'd like.

FWIW: here is the kalman activity you shared:

Kalmann is absolutely the gold standard for removing noise. It is provably optimal. One of my first attempts to implement smoothing was a kalman filter. What I found is that it for our general use it depended too heavily on having a correct probability model. It is provably optimal when the model is correct, but if you don't tune it correctly I found it will confidently deliver a giant mess with all sorts of magical thinking. I couldn't find a sane way to parameterize the probability matrix to account for all the different data sources we have, its very much a science project to come up with a correct model for kalmann. I finally gave up because I was literally unable to fix the issues that I had with my own ride files without resorting to machine learning. And then for my very next sample file (from a different vendor, etc) my learned model created all sorts of nonsensical artifacts. You can see in your kalmann smoothed file above there's a big slope drop at about 11.5km. Why do you think the kalmann smoothing thought that slope dip was reasonable?

I'll try and get this fix ready soon. Hopefully you can try it out. If you really think you want a different smoothing then I'm happy to look at doing it - probably I'd need a lot of convincing before I'd do kalmann again.

Eric

[Eric Christoffersen]

https://github.com/GoldenCheetah/GoldenCheetah/pull/3599 

[Ron George]

Thanks Eric,

So 200,5%,200 was the secret sauce, I wasn't too far off. Curious how this will work. Thanks for your explanations into the smoothing issue removing route start and end and the bug fixes. Will get you some feedback.

Ron

[Eric Christoffersen]

When I was debugging I realized that I got rid of the stddev % based outlier system a long time ago, but I never changed the ui. The actual outlier spec now is in centimeters. So 100 means anything outside 1m from spline is an outlier, and 5 means 5 cm. The change above updates the ui so it describes the code. I have no idea why it is cm but that is what it means, figured better to change the ui than the behavior.

I don't know if 200/5/200 is the best, I'd really like to hear what you think. Maybe with use we'll see patterns and come up with a better way to parameterize the smoothing. The difference in smoothing really matters when you have video synced with the ride. To me a ride feels a much better when a virtual ride's slopes are crisply aligned with the terrain you see.

Eric

[Ron George]

Eric,

General feedback with new smoothed file : the graphics and the real time slope dont align. jitteriness is apparently gone but tested only a few bits in the beginning. 

I'd be happy to upload the full HD video somewhere online, file size = 3.16 GB. I'll see if I can post it later this evening.

RG

[Eric Christoffersen]

First step is to verify that the live map location matches the video. That is super easy to do now that there is a real-time map. If you click the FF button it'll jump you ahead 1km, so you can pretty quickly verify important sync points around the ride, like cross streets, switchbacks and passes. That at least can help you set your start point correctly. So long as you made video and gps record at the same time they will have time sync which means location sync - so long as start is synced correctly.

The slope is purely derived from gps location. I've found gps noise is a pretty difficult thing because it is impacted by the environment, there are places on my rides that consistently skew in one direction, and other places where its just noisy.

The other thing... export your edited activity as JSON, not GPX. The GC GPX export rounds the altitude data to nearest 10cm so will undo your nice smoothing.

Eric

[Ron George]

Eric,

Trying to breathe (or re-breathe) some fire into this thread. 

Not sure if you've had the chance to look into Real Grand Tours (RGT cycling). Its an app like Zwift (albeit with a bit more realism) and has a feature called Magic Roads where you can submit a smoothed/pruned GPX file to the RGT team and they get back to you with virtual road that you can ride within the app. 

It looks like people are using tools like GPXmagic to do the cleanup. Quality of data will ofcourse vary with the experience of the user doing the cleanup. There's a profit-free user database featuring several routes made using Magic Roads check em out : https://magicroads.org

I'm wondering if you've had the chance to check out GPXmagic. I'm curious what you think of the smoothing approaches : http://stepwiserefinement.co.uk.s3-website-eu-west-1.amazonaws.com/GPXmagic/index.html

My thought is - if one took a raw file off Strava and threw it at GC and GPXmagic separately, which one does justice to the actual slope of in-real-life climb?

I spoke to the creator of this tool Peter Ward and he wrote this about his approach : 

My "approach" is to bundle a collection of simple tools and let the user play with them, with rapid feedback in a 3D visualization 

With spiky gradients, the relevant tools are:

a. Smooth gradient - replaces the spiky section with spikiness reduced by a factor you set, down to completely uniform.

b. Nudge - move individual track points or ranges up and down by up to 5m a go, as much as you like.

c. "Simplify" - removes track points that don't contribute much to the route (actually not good for smoothing).

d. Smooth - applies simple centre-weighted average, which can be effective on noise.

e. "Chamfer", which reduces awkward gradient transitions simply by replacing one track point with two, each with half the transition.

f. (finally), you can get a segment from Strava and paste it over your track. These often have quite good elevation data.

Just sharing this here for whatever it is worth. 

I'm hoping to play around with GC and GPXMagic side-by-side. If I can only get GC to run. Stable version 3.5 just pops up the loading screen and disappears. :( 

Ron 

[Eric Christoffersen]

The gpxmagic sounds terrific. Sort of like a photoshop for routes. What could do better than to give users tools to gracefully fix the stuff they think is broken. User can spend as much time as they like to make it right, then share and use the file for years. Thumbs up, that is what I'd want if I was making virtual routes.

Gc's gpsfix approach is more "what reasonable filtering can be done almost automatically". It handles the case where you have 50 recorded rides from the same device and want to smooth them all so you can run power estimation on them. I think it is healthy to draw the line at whole route smoothing, if you need something finer probably gc isn't the right tool. In the face of skewed noise I'm skeptical that any automatic passes can make a route 'right', it is more like a gloss to make it feel 'not bad'. gpxmagic gets a big thumbs up from me!

PS: WRT 3.5. That seems really old to me, I think it doesn't even have the bspline gpsfix. 

PPS: Someone gave me a computrainer so I got to play with it. I had big problems with machine's slope limits so there is pending work to support 'virtual gears'. This lets you do a ride with high wheel speed (shift into high gear), then use virtual gears to shift back down to ease force. This will let you climb steeps on computrainer without overheat or tire slippage. I'd love to hear your feedback.

Artifact here:

GoldenCheetah ci.3574 - AppVeyor 

 

Eric

[App 4G]

This is all very interesting. Although I need to admit that the "fix gps" settings is really something that scares me off. 

WHile GPXMagic:  http://stepwiserefinement.co.uk.s3-website-eu-west-1.amazonaws.com/GPXmagic/index.html (thanks for the link) is definitely interesting in that there's something to see I don't really know how to utilise it.

All in all, this is a great conversation thread for learning. The live maps is a nice touch as well. Didn't know such exists.