SQLAlchemy 1.4 / 2.0 distribution

1,033 views
Skip to first unread message

Marat Sharafutdinov

unread,
Jun 9, 2021, 4:15:28 PM6/9/21
to sqlalchemy
Currently I'm on SQLAlchemy 1.3 and there is a lot of work I have to do to migrate to 1.4 / 2.0. I think it's good idea to distribute 1.4 / 2.0 versions not only as "SQLAlchemy" project but as additional separate "SQLAlchemy2" project too with initial 1.4 version and then 2.0. This will give opportunity to have 1.3 and earlier version of SQLAlchemy which is already in use and latest SQLAlchemy2 version to migrate gradually step by step, in case of Flask for example, - one API method by another. Just one additional way to migrate painlessly.

Mike Bayer

unread,
Jun 9, 2021, 7:10:48 PM6/9/21
to noreply-spamdigest via sqlalchemy
hi there -

having a separate project name on pypi doesn't solve any issue that isn't already solved by using version pinning - the "sqlalchemy" name on pypi is already at 1.4.  the only way to install 1.3 is by requesting "pip install sqlalchemy < 1.4".

As you've probably seen, SQLAlchemy 1.4 /2.0 includes a very specific upgrade path with step-by-step instructions at https://docs.sqlalchemy.org/en/14/changelog/migration_20.html .



On Wed, Jun 9, 2021, at 4:15 PM, Marat Sharafutdinov wrote:
Currently I'm on SQLAlchemy 1.3 and there is a lot of work I have to do to migrate to 1.4 / 2.0. I think it's good idea to distribute 1.4 / 2.0 versions not only as "SQLAlchemy" project but as additional separate "SQLAlchemy2" project too with initial 1.4 version and then 2.0. This will give opportunity to have 1.3 and earlier version of SQLAlchemy which is already in use and latest SQLAlchemy2 version to migrate gradually step by step, in case of Flask for example, - one API method by another. Just one additional way to migrate painlessly.


--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
 
 
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

Marat Sharafutdinov

unread,
Jun 9, 2021, 7:21:37 PM6/9/21
to sqlalchemy
The problem is that currently only the entire codebase can be migrated from 1.3 to 1.4, even though it can be extremely difficult or too long to do.

I suggest the following way of pinning dependencies:
sqlalchemy==1.3
sqlalchemy2==1.4 or sqlalchemy2==2.0

Then current codebase will continue to work because "sqlalchemy" package stays at version 1.3 but it will be possible to use SQLAlchemy 1.4 / 2.0 additionally to migrate not whole codebase but part by part by importing "sqlalchemy2" package.

Federico Caselli

unread,
Jun 10, 2021, 7:39:47 AM6/10/21
to sqlalchemy
Hi,

> The problem is that currently only the entire codebase can be migrated from 1.3 to 1.4, even though it can be extremely difficult or too long to do.

I don't agree with this assessment. From my experience the main incompatibility is the url that has become immutable. Other than this most thing will work on 1.4  without changes. They will raise warnings that can be tackled right away or not, but it will keep working.

Do you have some examples that prevent running 1.4 on a 1.3 codebase that make it long to adapt?

Moving to the 2.x syntax can then be done gradually.

I think having two package name will make the migration worse 

Mike Bayer

unread,
Jun 10, 2021, 8:57:08 AM6/10/21
to noreply-spamdigest via sqlalchemy


On Wed, Jun 9, 2021, at 7:21 PM, Marat Sharafutdinov wrote:
The problem is that currently only the entire codebase can be migrated from 1.3 to 1.4, even though it can be extremely difficult or too long to do.

I suggest the following way of pinning dependencies:
sqlalchemy==1.3
sqlalchemy2==1.4 or sqlalchemy2==2.0

Then current codebase will continue to work because "sqlalchemy" package stays at version 1.3 but it will be possible to use SQLAlchemy 1.4 / 2.0 additionally to migrate not whole codebase but part by part by importing "sqlalchemy2" package.

the entire migration process was designed around not having to do it like this, and I spent months thinking about it.   It would not be feasible for two versions of sqlalchemy to exist at the same time because then the entire import space would have to become "sqlalchemy2" and this is too drastic of a change which would heavily impede migration.

I think your assessment that "only the entire codebase can be migrated from 1.3 to 1.4" is not true, most applications that run on 1.3 will run on 1.4 with little to no changes and hundreds of users have already done so.   The 1.3-> 1.4 change is roughly comparable to the change that it has been for other releases, like 1.0->1.1, 1.1->1.2 etc.



On Thursday, June 10, 2021 at 2:10:48 AM UTC+3 Mike Bayer wrote:

hi there -

having a separate project name on pypi doesn't solve any issue that isn't already solved by using version pinning - the "sqlalchemy" name on pypi is already at 1.4.  the only way to install 1.3 is by requesting "pip install sqlalchemy < 1.4".

As you've probably seen, SQLAlchemy 1.4 /2.0 includes a very specific upgrade path with step-by-step instructions at https://docs.sqlalchemy.org/en/14/changelog/migration_20.html .



On Wed, Jun 9, 2021, at 4:15 PM, Marat Sharafutdinov wrote:
Currently I'm on SQLAlchemy 1.3 and there is a lot of work I have to do to migrate to 1.4 / 2.0. I think it's good idea to distribute 1.4 / 2.0 versions not only as "SQLAlchemy" project but as additional separate "SQLAlchemy2" project too with initial 1.4 version and then 2.0. This will give opportunity to have 1.3 and earlier version of SQLAlchemy which is already in use and latest SQLAlchemy2 version to migrate gradually step by step, in case of Flask for example, - one API method by another. Just one additional way to migrate painlessly.


--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
 
 
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.


--
SQLAlchemy -
The Python SQL Toolkit and Object Relational Mapper
 
 
To post example code, please provide an MCVE: Minimal, Complete, and Verifiable Example. See http://stackoverflow.com/help/mcve for a full description.
---
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+...@googlegroups.com.

Marat Sharafutdinov

unread,
Jul 14, 2021, 11:10:35 AM7/14/21
to sqlalchemy
The problem is the part of my code base depends on third party library (aiopg) which should be compatible with SQLAlchemy 1.4 too but there some problems with it ( https://github.com/aio-libs/aiopg/issues/798#issuecomment-879815102 ) which make migration from 1.3 to 1.4 impossible for now =/

Federico

unread,
Jul 14, 2021, 1:09:45 PM7/14/21
to sqlal...@googlegroups.com
Sqlalchemy 1.4 has asyncio support for postgresql using asyncpg. Not sure how feasible it is to replace that library for you.

You received this message because you are subscribed to a topic in the Google Groups "sqlalchemy" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/sqlalchemy/BZPzqwnqwnU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to sqlalchemy+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sqlalchemy/203f42c9-29e2-4aed-bdbd-25abd717b1f3n%40googlegroups.com.

Mike Bayer

unread,
Jul 14, 2021, 1:59:16 PM7/14/21
to noreply-spamdigest via sqlalchemy
As I mentioned on the github issue, aiopg's approach is one that has always alarmed me, which is that they have taken large tracts of SQLAlchemy's internals, copied them into their own source base, and simply altered them to fit their needs.     I would never recommend this approach as it has non-maintainability and constant regressions built in by design.    The "backwards incompatible" changes you've referred towards in this thread have nothing to do with SQLAlchemy's public API.  aiopg makes deep and highly modifying use of SQLAlchemys *private* internals, and that's where they are having problems, as the statement execution and result set internals in SQLAlchemy 1.4 are *completely* different.  

The practice of lifting and copying source code with modifications throughout is commonly known as "forking", and while aiopg is not exactly a "fork" as it still maintains SQLAlchemy as a dependency, it's pretty close in spirit to one.   There are no open source projects that support partial forks of their source code, as this is inherently unmaintainable.   The purpose of internal, private API is that all of it may change at any time due to refactoring.   As long as the public API and behaviors are maintained, this does not impact backwards compatibility in any way provided the community does their part by using only public APIs.

Now that SQLAlchemy has published a technical approach to adapting to asyncio that does not require rewriting the internals, aiopg should look to maintaining their public facing API, but removing all the copied internal source code stuff and instead use SQLAlchemy's asyncio approach so that they no longer are essentially re-implementing the library from the ground up.     A library that is doing this now is Gino https://python-gino.org/, where they are reworking their homegrown approach to asyncio adaptation and migrating to SQLAlchemy 1.4's style in their 2.x branch.
Reply all
Reply to author
Forward
0 new messages