Old-fashioned async programming with callbacks

[A. Jesse Jiryu Davis]

Hi friends. I'm preparing version 2.0 of Motor, my async Python driver for MongoDB. Version 2.0 is an opportunity for me to delete a lot of deprecated Motor APIs and clean up the internal code.

One of Motor's messiest problems is how to support Futures alongside its original callback-based API:

# Old-fashioned:

def callback(result, error):

    if error:

        print(error)

    else:

        print(result)

def func():

    motor_client.db.collection.insert_one({'_id': 1}, callback=callback)

# Modern:

@gen.coroutine

def func():

    result = yield motor_client.db.collection.insert_one({'_id': 1})

# Contemporary:

async def func():

    result = await motor_client.db.collection.insert_one({'_id': 1})

If I remove the callback API and I only support Futures, then the Modern and Contemporary styles will still work but the Old-fashioned code will break. Users who want callbacks could use Future.add_done_callback, but their code would be very different from today, and slower.

What's the opinion on this list? Are callbacks completely outdated, or do you think some async programmers still rely on them?

[Ben Darnell]

My opinion is probably obvious given Tornado 5.1/6.0, but just for the record:

If you were starting today, I don't think there's any question that you'd go entirely with the `async def` style and wouldn't worry about callbacks (except as provided through `Future.add_done_callback` for callers who really want them). So this is really a question about how you feel about maintaining or breaking backwards compatibility. 

I tend to think that for async code in python, the benefits of moving to Python 3.5 and native coroutines are very compelling. There are certainly some users committed to the callback-based mode, so I'm tying the removal of callbacks in Tornado to dropping Python 2, which is also going to leave some users behind. For those users, older versions remain available so they won't lose anything, but they won't get new features or support going forward. 

The appropriate decision varies depending on what kind of library you're building. I'm planning to continue to support Python 2 in the `cockroachdb` package after I've dropped it from Tornado, for example. For `motor`, I could see continuing Python 2 and callback support by the same logic, since the goal is to enable asynchronous access to MongoDB, and you want the largest audience possible (and you don't want users on older versions to have trouble accessing new database features). 

-Ben

--
You received this message because you are subscribed to the Google Groups "Tornado Web Server" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python-tornad...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Grégoire]

Just my data point: I run tornado and motor in production (since around 2014) and never used the callback-based APIs

[A. Jesse Jiryu Davis]

Thanks everyone. I'm going to delete the callbacks. Users have had gen.coroutine and "yield" on Python 2 since 2013 or so, and "async/await" on Python 3 since 2015. Even if they still want callbacks they can shim their old code with Future.add_done_callback. If none of those is an option for callback users, they can stay with Motor 1.x until they need MongoDB transactions or another Motor 2.0 feature.

[TaoBeier]

Thanks for your working. 

For my usage scenario, I never used the callback-based APIs.

在 2018年6月26日星期二 UTC+8下午9:14:11，A. Jesse Jiryu Davis写道：