Real-time AST modification

[Raed Abdullah]

Hello everyone,

I'm building a visual scripting editor, that reads AST and then the user will be able to change the structure, and then be able to save back to dart code.

As I read here, analyzer package is meant to be used as read-only, but for such a use case I'll need to modify an in memory data structure, so either that or a custom mutable structure.

And if it's possible to refer me to ways to modify the structure, is it with setProperty? Or I have to venture to the files in the src folder?

Thanks in advance,

Raed Abdallah

[Brian Wilkerson]

As I read here, analyzer package is meant to be used as read-only, but for such a use case I'll need to modify an in memory data structure, so either that or a custom mutable structure.

Yes, the AST structures in the `analyzer` package are intended to be read-only, and I don't expect that to change.

Many people assume that the best way to write code editing applications is to build an AST, modify the AST, then write the AST out as source code. But after many decades of experience writing such tools we have come to realize that it's actually much easier to modify the source code directly and then rebuild the AST from source in order to continue editing. The biggest difficulty you face when modifying the AST is keeping the AST consistent with what the analyzer would have produced. The more complex the language the more true this is, and Dart is fairly complex from an analysis perspective because of flow analysis, type inference, and other such non-local features. And if your modifications produce an AST that's different than what the analyzer would have produced then you're effectively having to support two subtly different AST representations, which increases your implementation and maintenance burdens.

The approach of modifying the source directly and rebuilding the ASTs is what the `analysis_server` uses to power our IDE support. The class `ChangeBuilder`, in the `analyzer_plugin` package, contains support code (used by server) to make it easier to handle some of the more subtle editing situations correctly, such as keeping imports correct when adding references to types or other top-level declarations, and you might find that support useful if you decide to explore using this approach.

And if it's possible to refer me to ways to modify the structure, is it with setProperty?

The `getProperty` / `setProperty` pair of methods are there to allow clients to associate additional (client-specific) information with the AST structure. They can't be used to either access or modify the `analyzer` managed properties.

Or I have to venture to the files in the src folder?

We make no guarantees about code in the `src` folder. In particular, we don't guarantee that we won't make breaking changes in that code without bumping the major version number of the package.

While we would prefer that you didn't use the internal implementation details at all, we obviously can't enforce that, so if you do decide to use them the only way you can protect yourself against breakages is to use a version constraint for the `analyzer` package that allows for a single version. That will prevent you and your users (assuming you publish your package) from getting bug fixes, but will also protect against unexpected breakage.

--
You received this message because you are subscribed to the Google Groups "Dart Analyzer Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to analyzer-discu...@dartlang.org.
To view this discussion on the web visit https://groups.google.com/a/dartlang.org/d/msgid/analyzer-discuss/f0c07330-f1cd-4e90-9306-e7ba54896ebdn%40dartlang.org.

[Raed Abdullah]

Thanks for the helpful reply.

So the recommended process is to analyze the code, then modify it directly, then reanalyze?

I mean even if it would be easier to implement, it would be much more inefficient than modifying the AST directly. The system requires modifications to work within 16.6 ms, building AST or writing to source can take time tho.

[Brian Wilkerson]

That's a very tight time constraint. Even code completion can reasonably take longer than that. It's certainly true that the code modification operations we support don't have anywhere near that time constraint and it does seem unlikely that you'll be able to write and re-resolve most files in that amount of time. I don't know whether the `analyzer` package will be able to provide the performance that you need, but I wish you the best of fortune. I'd love to know whether and how well it works for you.