Insecure string Pickle with web2py 2.15.2

[José Leite]

My previously stable application is giving me the following error in windows environment (without SSL):

Error ticket for "HolidayExperience"

Ticket ID

127.0.0.1.2017-08-03.16-32-21.275e696d-3a11-4a38-966d-71e266f71b04

<type 'exceptions.ValueError'> insecure string pickle

Versão

web2py™	Version 2.15.2-stable+timestamp.2017.07.19.12.18.41
Python	Python 2.7.11: C:\Python27\python.exe (prefix: C:\Python27)

Traceback

1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25.

Traceback (most recent call last): File "C:\web2py\gluon\restricted.py", line 219, in restricted exec(ccode, environment) File "C:/web2py/applications/HolidayExperience/models/db.py", line 59, in <module> auth.define_tables(username=True, signature=False) File "C:\web2py\gluon\tools.py", line 2215, in define_tables super(Auth, self).define_tables(username, signature, migrate, fake_migrate)._table_signature_list File "C:\web2py\gluon\authapi.py", line 320, in define_tables format='%(username)s')) File "C:\web2py\gluon\packages\dal\pydal\base.py", line 587, in define_table table = self.lazy_define_table(tablename, *fields, **args) File "C:\web2py\gluon\packages\dal\pydal\base.py", line 621, in lazy_define_table polymodel=polymodel) File "C:\web2py\gluon\packages\dal\pydal\adapters\base.py", line 795, in create_table return self.migrator.create_table(*args, **kwargs) File "C:\web2py\gluon\packages\dal\pydal\migrator.py", line 295, in create_table sql_fields_old = pickle.load(tfile) File "C:\Python27\lib\pickle.py", line 1384, in load return Unpickler(file).load() File "C:\Python27\lib\pickle.py", line 864, in load dispatch[key](self) File "C:\Python27\lib\pickle.py", line 972, in load_string raise ValueError, "insecure string pickle" ValueError: insecure string pickle

Error snapshot 

<type 'exceptions.ValueError'>(insecure string pickle)

[Leonel Câmara]

It seems your .table files are corrupted. Do a fake migration to rebuild them.

[José Leite]

Hello Leonel,

I've already done a fake migration and nothing changed.

db = DAL('postgres://postgres:%s@localhost:5432/HolidayExperience' % DEV_DB_PWD, fake_migrate_all=True)

[Leonel Câmara]

Are you using table files done on another machine or something like that?

[José Leite]

But you are right the .table files are corrupted since I can not access them from the Admin interface also.

Shall I delete them all? Or what shall I do?

This never happened with installation of previous web2py versions.

:-(

[Leonel Câmara]

Try just one more thing.

Where you have auth.define_tables(username=True, signature=False)

Put:

auth.define_tables(username=True, signature=False, migrate=True, fake_migrate=True)

[José Leite]

The same error appears!

Doesn't solve it...

Error ticket for "HolidayExperience"

Ticket ID

192.168.8.100.2017-08-04.16-58-01.7faba99b-7fcd-401e-9153-e8b071880ca4

) File "C:/web2py/applications/HolidayExperience/models/db.py", line 60, in <module> auth.define_tables(username=True, signature=False, migrate=True, fake_migrate=True)

File "C:\web2py\gluon\tools.py", line 2215, in define_tables super(Auth, self).define_tables(username, signature, migrate, fake_migrate)._table_signature_list File "C:\web2py\gluon\authapi.py", line 320, in define_tables format='%(username)s')) File "C:\web2py\gluon\packages\dal\pydal\base.py", line 587, in define_table table = self.lazy_define_table(tablename, *fields, **args) File "C:\web2py\gluon\packages\dal\pydal\base.py", line 621, in lazy_define_table polymodel=polymodel) File "C:\web2py\gluon\packages\dal\pydal\adapters\base.py", line 795, in create_table return self.migrator.create_table(*args, **kwargs) File "C:\web2py\gluon\packages\dal\pydal\migrator.py", line 295, in create_table sql_fields_old = pickle.load(tfile) File "C:\Python27\lib\pickle.py", line 1384, in load return Unpickler(file).load() File "C:\Python27\lib\pickle.py", line 864, in load dispatch[key](self) File "C:\Python27\lib\pickle.py", line 972, in load_string raise ValueError, "insecure string pickle" ValueError: insecure string pickle

[Leonel Câmara]

This is weird I would just move all table files to a backup migration and do a fake migration to recreate them and see if it solves the problem.

[José Leite]

It solved the problem ... thanks again Leonel

[Seth J]

So, the data in those tables is permanently gone?!

[Dave S]

On Monday, August 7, 2017 at 1:08:47 PM UTC-7, Seth J wrote:

So, the data in those tables is permanently gone?!

No, no ... "table files" describes the METADATA that PyDal uses to interface with the Data Base Engine.  Unless you're using Sqlite3, the actual data isn't even in the application directory, and (for production) not even on the same machine.
 
/dps

[Seth J]

It is Sqlite3 and after following directions in this post to solve the "pickle issue" I get the following error with username table.  Am I missing something?

<class 'sqlite3.OperationalError'> no such column: auth_user.username

Version

web2py™	Version 2.15.3-stable+timestamp.2017.08.07.12.51.45
Python	Python 2.7.10: C:\Python27\python.exe (prefix: C:\Python27)

Traceback

26. 27. 28. 29. 30.

Traceback (most recent call last

): File ".\gluon\restricted.py", line 219, in restricted exec(ccode, environment) File "C:/inetpub/wwwroot/applications/PEDchangeLog/controllers/default.py", line 90, in <module> File ".\gluon\globals.py", line 409, in <lambda> self._caller = lambda f: f() File "C:/inetpub/wwwroot/applications/PEDchangeLog/controllers/default.py", line 41, in user return dict(form=auth()) File ".\gluon\tools.py", line 1925, in __call__ return getattr(self, args[0])() File ".\gluon\tools.py", line 2704, in login user = table_user(**{username: entered_username}) File "C:\inetpub\wwwroot\gluon\packages\dal\pydal\objects.py", line 566, in __call__ orderby_on_limitby=False).first() File "C:\inetpub\wwwroot\gluon\packages\dal\pydal\objects.py", line 2211, in select return adapter.select(self.query, fields, attributes) File "C:\inetpub\wwwroot\gluon\packages\dal\pydal\adapters\sqlite.py", line 82, in select return super(SQLite, self).select(query, fields, attributes) File "C:\inetpub\wwwroot\gluon\packages\dal\pydal\adapters\base.py", line 762, in select return self._select_aux(sql, fields, attributes, colnames) File "C:\inetpub\wwwroot\gluon\packages\dal\pydal\adapters\base.py", line 718, in _select_aux rows = self._select_aux_execute(sql) File "C:\inetpub\wwwroot\gluon\packages\dal\pydal\adapters\base.py", line 712, in _select_aux_execute self.execute(sql) File "C:\inetpub\wwwroot\gluon\packages\dal\pydal\adapters\__init__.py", line 67, in wrap return f(*args, **kwargs) File "C:\inetpub\wwwroot\gluon\packages\dal\pydal\adapters\base.py", line 412, in execute rv = self.cursor.execute(command, *args[1:], **kwargs) OperationalError: no such column: auth_user.username

Error snapshot 

<class 'sqlite3.OperationalError'>(no such column: auth_user.username)

[Leonel Câmara]

Seth it seems like you did a fake_migrate after setting username=True in auth.define_tables. So the username column is not there and the .table file says it is.  

  

Set username=False. Do a fake_migrate. Turn fake_migrate=False. Set username=True. This will create the column.

[Mirek Zvolský]

2.15.3-2017.08.07

I had same here, with SQLite too.

I confirm it can be solved so:

1) rename databases/ to databases_corrupted/

2) create empty folder databases/

3) navigate to appadmin or main page -> database is now empty

4) copy databases_corrupted/sqlite.db back to databases/sqlite.db

or so:

1) remove databases/*.table metadata

2) in models in db=DAL() temporary add db=DAL(..., fake_migrate_all=True)

3) navigate to appadmin or main page -> database is now empty

4) remove fake_migrate_all=True from source code

However in my case this is a localhost copy (localhost backup) of production database.

There is no problem in production, but when I copy sqlite.db from production to localhost, I will receive "insecure string pickle" again.

Can it be some cross-versions problem of some library, between production and localhost?

Dne pátek 4. srpna 2017 18:15:29 UTC+2 Leonel Câmara napsal(a):