pandas 1.3 upgrade

[Poncho]

Hey there

I'm having issues with pandas after updating to pandas 1.3

I'm using the following code:

> import pandas as pd
>
> rd = GC.activity()
> ix = range(0, len(rd['seconds']))
> df = pd.DataFrame(rd, columns=rd.keys(), index=ix)
>
> print(df)


With pandas 1.2, I get:

> Python Console (3.9.5 (default, Jul 4 2021, 10:35:01) [GCC 10.3.0])
>>>> seconds cadence cadenced ... POWER_Kickr POWER_Target TRAIN_TARGET
> 0 0.0 70.0 0.0 ... 100.0 92.0 NaN
> 1 1.0 70.0 0.0 ... 100.0 92.0 92.0
> 2 2.0 70.0 0.0 ... 95.0 92.0 92.0
> 3 3.0 69.0 0.0 ... 95.0 92.0 92.0
> 4 4.0 71.0 2.0 ... 99.0 92.0 92.0
> ... ... ... ... ... ... ... ...
> 5398 5398.0 76.0 1.0 ... 91.0 92.0 92.0
> 5399 5399.0 75.0 0.0 ... 91.0 92.0 92.0
> 5400 5400.0 77.0 2.0 ... 0.0 92.0 92.0
> 5401 5401.0 77.0 0.0 ... 0.0 92.0 92.0
> 5402 5402.0 78.0 1.0 ... 0.0 92.0 92.0
>
> [5403 rows x 26 columns]


But with pandas 1.3:

> Python Console (3.9.5 (default, Jul 4 2021, 10:35:01) [GCC 10.3.0])
>>>> seconds ... TRAIN_TARGET
> 0 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> 1 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> 2 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> 3 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> 4 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> ... ... ... ...
> 5398 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> 5399 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> 5400 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> 5401 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
> 5402 [0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, ... ... [nan, 92.0, 92.0, 92.0, 92.0, 92.0, 92.0, 93.0...
>
> [5403 rows x 26 columns]


I don't understand the structure of GC.activity() and whether this is a
regression in pandas or an intended change in behavior.

https://pandas.pydata.org/pandas-docs/version/1.3.0/whatsnew/v1.3.0.html
https://pandas.pydata.org/pandas-docs/version/1.3.1/whatsnew/v1.3.1.html

I'd would appreciate some help to create a test case not involving
goldencheetah to report this to the pandas developers.

Kind regards,
Poncho

[Ale Martinez]

See https://github.com/GoldenCheetah/GoldenCheetah/wiki/UG_Special-Topics_Working-with-Python#gcactivityactivitynone, it’s a dictionary with series names as keys and an array like object as values.

The array like objects are iterable and they can be easily converted to standard types using list(), np.asarray(), etc.

[Poncho]

On 26.07.21 14:14, Ale Martinez wrote:
>> I don't understand the structure of GC.activity()
>
> See
> https://github.com/GoldenCheetah/GoldenCheetah/wiki/UG_Special-Topics_Working-with-Python#gcactivityactivitynone,
> it’s a dictionary with series names as keys and an array like object as
> values.

Thanks for the pointer.

I think it is:
BUG: list-like objects are broadcast to each row (1.3 regression)
https://github.com/pandas-dev/pandas/issues/42549


Kind regards,
Poncho