Skip to first unread message
whycrying
Aug 28, 2018, 1:50:22 AM8/28/18
to python-cn(华蟒用户组,CPyUG 邮件列表)
Victor Stinner 两天前在 Twitter 上透露了新的项目：
https://twitter.com/VictorStinner/status/1033484627174989824
Design a new better C API for Python
https://pythoncapi.readthedocs.io/
起因：
Keeping Python competitive
https://lwn.net/Articles/723752/#723949
> Victor Stinner sees a need to improve Python performance in order to keep it competitive with other languages. He brought up some ideas for doing that in a 2017 Python Language Summit session. No solid conclusions were reached, but there is a seemingly growing segment of the core developers who are interested in pushing Python's performance much further, possibly breaking the existing C API in the process.
他之前写的 PEP 草稿：
[Python-ideas] PEP: Hide implementation details in the C API
https://mail.python.org/pipermail/python-ideas/2017-July/046399.html
> Abstract
> ========
>
> Modify the C API to remove implementation details. Add an opt-in option
> to compile C extensions to get the old full API with implementation
> details.
>
> The modified C API allows to more easily experiment new optimizations:
>
> * Indirect Reference Counting
> * Remove Reference Counting, New Garbage Collector
> * Remove the GIL
> * Tagged pointers
>
> Reference counting may be emulated in a future implementation for
> backward compatibility.
他之前发布的博客文章：
A New C API for CPython
https://vstinner.github.io/new-python-c-api.html
> Tracing garbage collector without reference counting
> ...
> It even seems possible to implement a tracing garbage collector and use reference counting. But I'm not an expert in this area, need to dig the topic.
> ...
> Gilectomy
> Abstracting the ABI allows to customize the runtime for Gilectomy needs, to be able to reemove the GIL.
>
> Removing reference counting would make Gilectomy much simpler.
https://twitter.com/VictorStinner/status/1033484627174989824
Design a new better C API for Python
https://pythoncapi.readthedocs.io/
起因：
Keeping Python competitive
https://lwn.net/Articles/723752/#723949
> Victor Stinner sees a need to improve Python performance in order to keep it competitive with other languages. He brought up some ideas for doing that in a 2017 Python Language Summit session. No solid conclusions were reached, but there is a seemingly growing segment of the core developers who are interested in pushing Python's performance much further, possibly breaking the existing C API in the process.
他之前写的 PEP 草稿：
[Python-ideas] PEP: Hide implementation details in the C API
https://mail.python.org/pipermail/python-ideas/2017-July/046399.html
> Abstract
> ========
>
> Modify the C API to remove implementation details. Add an opt-in option
> to compile C extensions to get the old full API with implementation
> details.
>
> The modified C API allows to more easily experiment new optimizations:
>
> * Indirect Reference Counting
> * Remove Reference Counting, New Garbage Collector
> * Remove the GIL
> * Tagged pointers
>
> Reference counting may be emulated in a future implementation for
> backward compatibility.
他之前发布的博客文章：
A New C API for CPython
https://vstinner.github.io/new-python-c-api.html
> Tracing garbage collector without reference counting
> ...
> It even seems possible to implement a tracing garbage collector and use reference counting. But I'm not an expert in this area, need to dig the topic.
> ...
> Gilectomy
> Abstracting the ABI allows to customize the runtime for Gilectomy needs, to be able to reemove the GIL.
>
> Removing reference counting would make Gilectomy much simpler.
Shell Xu
Aug 28, 2018, 2:21:06 AM8/28/18
to CUPG
我没看所有细节文档，但是从介绍来看，这个动作后果很难说。关键是remove GIL这步上有没有太多的性能问题。如果又砸出一个性能问题来，社区估计就不买账了。
如果没有额外的性能问题，这个PEP大概在两可之间。抛弃固有的东西都是有一定开销的，就看收益是不是大于开销了。
--
邮件来自: `CPyUG`华蟒用户组(中文Python技术邮件列表)
规则: http://code.google.com/p/cpyug/wiki/PythonCn
详情: http://code.google.com/p/cpyug/wiki/CpyUg
严正: 理解列表! 智慧提问! http://wiki.woodpecker.org.cn/moin/AskForHelp
---
您收到此邮件是因为您订阅了Google网上论坛上的“python-cn(华蟒用户组,CPyUG 邮件列表)”群组。
要退订此群组并停止接收此群组的电子邮件，请发送电子邮件到python-cn+...@googlegroups.com。
要发帖到此群组，请发送电子邮件至pyth...@googlegroups.com。
要查看更多选项，请访问https://groups.google.com/d/optout。
彼節者有間，而刀刃者無厚；以無厚入有間，恢恢乎其於游刃必有餘地矣。
blog: http://shell909090.org/
blog: http://shell909090.org/
yu fan (fy)
Aug 28, 2018, 10:39:52 AM8/28/18
to pyth...@googlegroups.com
很棒 虽然我也没有看细节，但也早就觉得api需要革新了。
另外我希望新的API可以搞个PyInterpreter对象，不要再全局变量了。
发送自 Windows 10 版邮件应用
发件人: whycrying
发送时间: 2018年8月28日 13:50
收件人: python-cn(华蟒用户组,CPyUG 邮件列表)
主题: [CPyUG] 新的 C API 或许将给 Python 带来巨变？
--
whycrying
Aug 28, 2018, 12:06:22 PM8/28/18
to pyth...@googlegroups.com
remove GIL 这步应该是可期的：
Progress on the Gilectomy
https://lwn.net/Articles/723514/
> More performance
> ...
> The benchmark that he always uses is a "really bad recursive Fibonacci". He showed graphs of how various versions of Gilectomy fare versus CPython. Gilectomy is getting better, but is still well shy of CPython speed in terms of CPU time. But that is not what he is shooting for; when looking at wall time, the latest incarnation of Gilectomy is getting quite close to CPython's graph line. The "next breakthrough" may show Gilectomy as faster than CPython, he said.
>
> Next breakthrough
>
> He has some ideas for ways to get that next breakthrough. For one, he could go to a fully per-thread object-allocation scheme. Thomas Wouters suggested looking at Thread-Caching Malloc[1] (TCMalloc), but Hastings was a bit skeptical. The small-block allocator in Python is well tuned for the language, he said. But Wouters said that tests have been done and TCMalloc is no worse than Python's existing allocator, but has better fragmentation performance and is multi-threaded friendly. Hastings concluded that it was "worth considering" TCMalloc going forward.
> He is thinking that storing the reference count separate from the object might be an improvement performance-wise. Changing object locking might also improve things, since most objects never leave the thread they are created in. Objects could be "pre-locked" to the thread they are created in and a mechanism for threads to register their interest in other threads' objects might make sense.
>
> The handbook that he looked in to find buffered reference counts says little about reference counting; it is mostly focused on tracing garbage collection. So one thought he has had is to do a "crazy rewrite" of the Python garbage collector. That would be a major pain and break the C API, but he has ideas on how to fix that as well.
> ...
> But Van Rossum is concerned that all of the C-based Python extensions will be broken in Gilectomy. Hastings thinks that overstates things and has some ideas on how to make things better. Someone had suggested only allowing one thread into a C extension at a time (so, a limited GIL, in effect), which might help.
>
> The adoption of PyPy "has not been swift", Hastings said; he thinks that since CPython is the reference implementation of Python, it will be the winner. He does not know how far he can take Gilectomy, but he is sticking with it; he asked Van Rossum to "let me know if you switch to PyPy". But Van Rossum said that he is happy with CPython as it is. On the other hand, Wouters pointed out one good reason to stick with experimenting with CPython; since the implementation is similar to what the core developers are already knowledgeable about, they will be able to offer thoughts and suggestions.
[1] Thread-Caching Malloc : http://goog-perftools.sourceforge.net/doc/tcmalloc.html
Gilectomy 的性能越来越好，已经接近使用 GIL 的 CPython 了，下个突破都可能比使用 GIL 的 CPython 快了。
Progress on the Gilectomy
https://lwn.net/Articles/723514/
> More performance
> ...
> The benchmark that he always uses is a "really bad recursive Fibonacci". He showed graphs of how various versions of Gilectomy fare versus CPython. Gilectomy is getting better, but is still well shy of CPython speed in terms of CPU time. But that is not what he is shooting for; when looking at wall time, the latest incarnation of Gilectomy is getting quite close to CPython's graph line. The "next breakthrough" may show Gilectomy as faster than CPython, he said.
>
> Next breakthrough
>
> He has some ideas for ways to get that next breakthrough. For one, he could go to a fully per-thread object-allocation scheme. Thomas Wouters suggested looking at Thread-Caching Malloc[1] (TCMalloc), but Hastings was a bit skeptical. The small-block allocator in Python is well tuned for the language, he said. But Wouters said that tests have been done and TCMalloc is no worse than Python's existing allocator, but has better fragmentation performance and is multi-threaded friendly. Hastings concluded that it was "worth considering" TCMalloc going forward.
> He is thinking that storing the reference count separate from the object might be an improvement performance-wise. Changing object locking might also improve things, since most objects never leave the thread they are created in. Objects could be "pre-locked" to the thread they are created in and a mechanism for threads to register their interest in other threads' objects might make sense.
>
> The handbook that he looked in to find buffered reference counts says little about reference counting; it is mostly focused on tracing garbage collection. So one thought he has had is to do a "crazy rewrite" of the Python garbage collector. That would be a major pain and break the C API, but he has ideas on how to fix that as well.
> ...
> But Van Rossum is concerned that all of the C-based Python extensions will be broken in Gilectomy. Hastings thinks that overstates things and has some ideas on how to make things better. Someone had suggested only allowing one thread into a C extension at a time (so, a limited GIL, in effect), which might help.
>
> The adoption of PyPy "has not been swift", Hastings said; he thinks that since CPython is the reference implementation of Python, it will be the winner. He does not know how far he can take Gilectomy, but he is sticking with it; he asked Van Rossum to "let me know if you switch to PyPy". But Van Rossum said that he is happy with CPython as it is. On the other hand, Wouters pointed out one good reason to stick with experimenting with CPython; since the implementation is similar to what the core developers are already knowledgeable about, they will be able to offer thoughts and suggestions.
[1] Thread-Caching Malloc : http://goog-perftools.sourceforge.net/doc/tcmalloc.html
Gilectomy 的性能越来越好，已经接近使用 GIL 的 CPython 了，下个突破都可能比使用 GIL 的 CPython 快了。
...
--彼節者有間，而刀刃者無厚；以無厚入有間，恢恢乎其於游刃必有餘地矣。
blog: http://shell909090.org/
...
Leo Jay
Aug 28, 2018, 3:01:26 PM8/28/18
to python-cn:CPyUG
我非常同意 Python C API 有很多可以改进的地方。很多 Python 改进项目就是死在了 C API 兼容性这个坑里。
以至于官方的态度都是，如果你要写扩展，可以考虑用 Cython [1]
如果这个 PEP 在 Python 3 出现之前提出，我举双手双脚支持。
但是现在提出，我就呵呵了。这个提议绝对死在半路上。
2008 年发布的 Python 3，采用率到现在才勉强赶上 Python 2，再搞一个完全不兼容的 C API，不得至少要等到 2030 年才能被采用啊？
[1] https://docs.python.org/3/howto/cporting.html#other-options
以至于官方的态度都是，如果你要写扩展，可以考虑用 Cython [1]
如果这个 PEP 在 Python 3 出现之前提出，我举双手双脚支持。
但是现在提出，我就呵呵了。这个提议绝对死在半路上。
2008 年发布的 Python 3，采用率到现在才勉强赶上 Python 2，再搞一个完全不兼容的 C API，不得至少要等到 2030 年才能被采用啊？
[1] https://docs.python.org/3/howto/cporting.html#other-options
laike9m
Aug 28, 2018, 9:49:51 PM8/28/18
to python-cn(华蟒用户组,CPyUG 邮件列表)
您收到此邮件是因为您订阅了 Google 网上论坛的“python-cn(华蟒用户组,CPyUG 邮件列表)”群组。
要退订此群组并停止接收此群组的电子邮件，请发送电子邮件到python-cn+...@googlegroups.com。
要向此群组发帖，请发送电子邮件至 pyth...@googlegroups.com。
要查看更多选项，请访问 https://groups.google.com/d/optout。
TaoBeier
Aug 29, 2018, 4:08:59 AM8/29/18
to python-cn(华蟒用户组,CPyUG 邮件列表)
大致看了下， 想法是好的，也许完成后，真的能带来很大收益。 但是 注意是在完成后，在目前，主要是要界定，到底哪些是好的哪些是不好， 不好的要如何修改或者移除之类的。 而这个过程， 路很长。 另一方面，有众多的软件依赖原有 C API， 搞了新的还要考虑兼容， 那也需要有大量的测试之类的。 期待，但也有可能是有生之年系列...
在 2018年8月28日星期二 UTC+8下午1:50:22，whycrying写道：
在 2018年8月28日星期二 UTC+8下午1:50:22，whycrying写道：
Reply all
Reply to author
Forward
0 new messages