sqlalchemy serialization bug (TypeError: list indices must be integers or slices, not str)

[Evgenii]

Hello!
We found bug in SA version `1.4.38` during deserialization.
How to reproduce:

Serialize any ORM object with pickle using environment with any SA compatible version `1.4.n` except `1.4.38` in , ex. `1.4.31`:

```
pickle.dumps(obj)
```

Deserialize result using environment with SA version == `1.4.38`:

```
obj_loaded = pickle.loads(b'\x...
```
Got:
```
Traceback (most recent call last):
File "/home/user/anaconda3/envs/python375/lib/python3.7/code.py", line 90, in runcode
exec(code, self.locals)
File "<input>", line 1, in <module>
File "/home/user/anaconda3/envs/python375/lib/python3.7/site-packages/sqlalchemy/orm/state.py", line 577, in __setstate__
state_dict["manager"](self, inst, state_dict)
File "/home/user/anaconda3/envs/python375/lib/python3.7/site-packages/sqlalchemy/orm/instrumentation.py", line 506, in __call__
manager.dispatch.unpickle(state, state_dict)
File "/home/user/anaconda3/envs/python375/lib/python3.7/site-packages/sqlalchemy/event/attr.py", line 343, in __call__
fn(*args, **kw)
File "/home/user/anaconda3/envs/python375/lib/python3.7/site-packages/sqlalchemy/ext/mutable.py", line 505, in unpickle
for val in state_dict["ext.mutable.values"][key]:
TypeError: list indices must be integers or slices, not str
```

Seems that backward compatibility is lost.

[Mike Bayer]

hey there -

yes this is unfortunately the case as we had to fix a bug in the mutable extension, issue 8133 doc'ed at https://docs.sqlalchemy.org/en/14/changelog/changelog_14.html#change-f8b03063d70397a8f275287ed2c8f2e6 .   this seemed to be enough of an edge case that I didn't attempt to reconcile the old broken format.

--

SQLAlchemy -

The Python SQL Toolkit and Object Relational Mapper

 

http://www.sqlalchemy.org/

 

To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.

---

You received this message because you are subscribed to the Google Groups "sqlalchemy" group.

To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/f1ef4019-5ce4-4d14-938e-7a58f17db807n%40googlegroups.com.

[Mike Bayer]

if you can test this patch and confirm it reads your old pickle format, I can merge it and release 1.4.39

diff --git a/lib/sqlalchemy/ext/mutable.py b/lib/sqlalchemy/ext/mutable.py

index 934ac37a05..cbec06a31f 100644

--- a/lib/sqlalchemy/ext/mutable.py

+++ b/lib/sqlalchemy/ext/mutable.py

@@ -502,8 +502,14 @@ class MutableBase(object):

 

         def unpickle(state, state_dict):

             if "ext.mutable.values" in state_dict:

-                for val in state_dict["ext.mutable.values"][key]:

-                    val._parents[state] = key

+                collection = state_dict["ext.mutable.values"]

+                if isinstance(collection, list):

+                    # legacy format

+                    for val in collection:

+                        val._parents[state] = key

+                else:

+                    for val in state_dict["ext.mutable.values"][key]:

+                        val._parents[state] = key

 

         event.listen(parent_cls, "load", load, raw=True, propagate=True)

         event.listen(

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/be8f330d-d0ab-4b40-a753-df3f02a87cc7%40www.fastmail.com.

[Evgenii]

Sure, I can

пятница, 24 июня 2022 г. в 17:06:39 UTC+3, Mike Bayer:

[Evgenii]

Yes, it works

пятница, 24 июня 2022 г. в 17:58:46 UTC+3, Evgenii:

[Mike Bayer]

1.4.39 is released

To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/d9e1b086-2187-45ef-a1dc-410d8d5d58f1n%40googlegroups.com.