Verona 项目:Python 的无畏并发
Project Verona: Fearless Concurrency for Python

原始链接: https://microsoft.github.io/verona/pyrona.html

Verona 项目正在开发 Lungfish,这是一种用于 Python 的新型所有权模型,旨在改进内存管理和并发安全性。他们采用分阶段的方法,首先在 Python 中实现深度不变性,这在 CPython 3.12 的一个分支中进行了原型设计,以便在不发生数据竞争的情况下安全地在线程之间共享类型信息。这涉及使用引用计数管理循环不变垃圾,以实现子解释器兼容性,并与消息传递 (PEP734) 集成以实现并行性。 该项目利用 FrankenScript 这种玩具语言进行快速原型设计并动态地可视化基于区域的所有权概念。这项探索为 Lungfish 的设计提供了信息,将 Rust 等语言中的所有权原则应用于 Python 的动态类型和现有对象图。 虽然最初的重点是消息传递,但未来的工作可能会将区域的概念扩展到锁。Verona 项目的动态所有权研究产生了一些在静态类型语言中不明显的见解,促使人们重新评估静态检查与动态检查。要参与其中,请浏览链接的 GitHub 代码库,提出问题,或加入 [email protected] 讨论列表。

这篇 Hacker News 帖子讨论了微软的 Verona 项目,该项目旨在为 Python 带来“无畏并发”。讨论源于微软裁撤 Faster CPython 团队的新闻。评论者们就 Verona 项目在裁员后的未来展开了辩论,一些人指出,Verona 所属的微软研究院比受影响的 DevDiv 部门更有保障。 随后,讨论转向了人工智能对编程的更广泛影响。一些评论者预测,人工智能最终将直接生成可执行文件,这可能会降低像 Python 这样的三代语言的相关性。另一些人则对这种“人工智能主导的未来”的临近性和影响表示怀疑,他们指出了大型语言模型固有的非确定性,以及需要使用形式化语言来明确地表达程序逻辑。一个共同的主题出现了:虽然人工智能可以提供帮助,但编程语言对于向计算机、人类或大型语言模型传达思想仍然至关重要。人们对向后兼容性和静态类型在人工智能生成代码时代中的价值表示担忧。一些人主张推出具有严格所有权模型的“Python 4”,而另一些人则指出这可能是一种全新的语言。

原文

As part of Project Verona, we have been developing a new ownership model for Python, called Lungfish. This model is designed to provide a safe and efficient way to manage memory and concurrency in Python programs. This page explains our experiments to develop this approach.

Plan for Python

Quick prototyping in FrankenScript

Modifying a production language is a complex task. As an initial step, we have been developing a toy language called FrankenScript that allows us to quickly prototype our ideas for region-based ownership. FrankenScript is a small language that is designed to be easy to modify and extend. It is based on the ideas of ownership and concurrency that we are exploring in Project Verona, but where all the checks are dynamic.

This prototype has given us confidence in the conceptual ideas behind our ownership model, and has allowed us to explore the design space of ownership in a dynamic language. It has also helped us explain our ideas and the different design decisions to people outside the Project Verona team.

FrankenScript generates a series of mermaid diagrams that show the region structure of the program. An example of this is shown below:

mermaidexample

Over the last two years, we have been engaging with the Faster CPython team at Microsoft as a sounding board for our ideas. In May 2025, we will be taking our first steps into the boarder Python community at the Python Language Summit. The presentation that we gave on the Python language summit can be found here. We are seeking feedback from the core developers and broader community behind the language.

Steps to a new ownership model

Building an ownership model for Python is a complex task, and we are taking a step-by-step approach to ensure that we get it right.

The first step is actually to build a concept of deep immutability into Python. This can be split into three parts:

  • Deep Immutability: We are starting with a deep immutability model, we have been drafting a PEP to describe this model. The current prototype is in a PR to a fork of CPython 3.12: https://github.com/mjp41/cpython/pull/51

  • Manage cyclic immutable garbage with reference counting and atomic reference counting of immutable objects. This will enable objects to be moved between sub-interpreters as they will no longer be managed by the interpreter local cycle collector. The current prototype is on the following branch: https://github.com/mjp41/cpython/tree/scc, which is based on top of the PR 51 mentioned above.

  • Integration with message passing between sub-interpreters (PEP734). This will enable us to get parallel performance even though we are technically not losing the GIL.

Deep Immutability is a key part of any ownership model as we need concurrent threads to be able to share type information without risking it being mutated under foot. As types in Python are mutable objects, we need a way to share these objects between threads.

Once the immutability model is in place, then we can start applying the region-based ownership model to Python to permit safe sharing and transfer of mutable state. This involves applying the ideas developed in the FrankenScript prototype to Python. We have made progress on this in our prototyping ideas on CPython, but nothing is ready to try yet.

FAQ

Why Python?

Python is the most popular programming language in the world. With PEP703, aka NoGil or “Free-Threaded” Python, Python is moving to a fully concurrent model. This means a vast quantity of programmers will be potentially exposed to concurrency issues. Bringing an ownership model to Python will help to make it easier to write concurrent programs, and to avoid the pitfalls of concurrency.

This is a perfect opportunity to influence the future of Python, and to help make programming experience better for everyone.

Why not threads and locks?

We are initially exploring using message passing between sub-interpreters. However, we could also apply the region ideas to locks, and provide a lock that gives temporary access to a region of memory associated with that lock.

We have sketched out a design for how to integrate with the full PEP 703 work, but have not attempted to implement that yet.

Why not simply use Rust’s ownership model?

Rust’s ownership model is designed for a statically typed language, and restricts the type of object graphs that can be used in the language. There is so much existing code in Python that the restrictions of Rust’s ownership model would be too limiting. We are designing a new ownership model based on regions that is designed to work with Python’s dynamic typing and existing object graphs.

Our approach draws heavily from the experience of languages with ownership models like Rust, Cyclone Encore, and Pony, but is based completely on dynamic checks. This completely alters the kind of things that can be checked in comparison to a statically typed approach.

What has Project Verona learnt from this work?

Performing ownership research in a dynamically typed language has been a massive learning experience for the Project Verona team. It has challenged every assumption we have made about ownership and concurrency in a statically typed language. Moreover, it has shown us certain programming patterns that are really easy to check in a dynamic region system, but are really hard to check in a statically typed language. We are currently re-evaluating the balance between the static and dynamic checks in our ownership model, and how to best combine them. Hopefully, we will be able to bring some of these ideas back into the Project Verona research language.

Where can I find out more?

We have a detailed list of publications related to Project Verona on our publications page. The most relevant paper to this work is Dynamic Region Ownership for Concurrency Safety.

You can try our toy language, FrankenScript. We also provide a docker container to test FrankenScript quickly.

We are also working on several forks of Python that implement our ideas, these are currently on GitHub.

How can I get involved? Who can I talk to?

We have listed a few places where you can find parts of the project that are available. If you want to find out more about those, raise issues on the appropriate GitHub repo.

If it is more general, please raise an issue on the Project Verona Github repo, or you can join our discussions list [email protected].

联系我们 contact @ memedata.com