Using LSP indexers for Chromium development

[Dominik Röttsches]

Hi,

is anyone using an LSP enabled editor (such as VSCode, Emacs, and yes..vim) to connect to an indexing service such as ccls, cquery or clangd?

I've managed to get ccls working on Linux, and integrated with Emacs through Eglot. This is working quite well and I am enjoying the powerful symbol lookup tooling. 

I've tried getting cquery to work on Windows but I ran into problems with receiving errors from the indexer, which does not seem to be able to understand the compilation command line syntax and drops out of indexing.  Does anyone have a working LSP setup for Chromium on Windows?

What is the status of clangd? Would it be feasible to build clangd and make it part of our custom clang binaries on each platform, including Windows? Then I could perhaps try to use clangd on Windows. So far I have not attempted to build clangd myself.

Dominik

[Asanka Herath]

On Tue, Sep 25, 2018 at 5:12 AM Dominik Röttsches <dr...@chromium.org> wrote:

Hi,

is anyone using an LSP enabled editor (such as VSCode, Emacs, and yes..vim) to connect to an indexing service such as ccls, cquery or clangd?

I haven't tried ccls, but have tried cquery and clangd. It's been a while since I tried, but neither was responsive enough for regular development. cquery took a pretty long time to warm up, and clangd was quite sluggish when presented with Chromium's huge compilation DB.

If you are using clangd, make sure you are using one which includes https://reviews.llvm.org/D51314 . It's supposed to bring down compilation DB load times from ~30s to under a second for Chromium.

Also, if you are only working on a smaller target and aren't really interested in xrefs spanning the entire project, then you can use something like this to extract a target specific compile DB from ninja. That should make all of the LSPs you mention much more responsive even after a fresh start.

 

I've managed to get ccls working on Linux, and integrated with Emacs through Eglot. This is working quite well and I am enjoying the powerful symbol lookup tooling. 

I've tried getting cquery to work on Windows but I ran into problems with receiving errors from the indexer, which does not seem to be able to understand the compilation command line syntax and drops out of indexing.  Does anyone have a working LSP setup for Chromium on Windows?

What is the status of clangd? Would it be feasible to build clangd and make it part of our custom clang binaries on each platform, including Windows? Then I could perhaps try to use clangd on Windows. So far I have not attempted to build clangd myself.

Dominik

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAN6muBtmuFwkMGCDK%2Bt_kPB2VSfJ_2iZBOUhLZMvKY7gCE5Acw%40mail.gmail.com.

[James Cook]

I use VSCode + cquery on Linux. It works pretty well for me.

I use a generated compile_commands.json file. It takes ~20 seconds to load that file when I launch VSCode, but I only launch VSCode once a day, so that doesn't bother me. Indexing takes a while, but the indexer prioritizes the file I have open, so I'm not blocked on waiting for indexing to complete.

I don't work on code in //blink or //third_party, so I blacklist those directories. This saves some indexing time.

I have not tried clangd or ccls.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAM%3DuaOGTpd6rnvcBzV8PhXkkgzKAoKKGDwN-8-7fivD_13M5Aw%40mail.gmail.com.

[Jacob Dufault]

FWIW, I'm the author of cquery. As jamescook@ said, with some filtering (mainly blink and third_party) I think it works pretty well. Indexing should take ~20-30 minutes (the only comparable tool, Eclipse, takes hours). Reloading a project should take 1-2 minutes, potentially longer if you have synced since code needs to be re-indexed. If your workflow is to constantly close and reopen your editor, then likely cquery will not work well, but if you have your editor open all day the initial load time should be bearable.

For completion I use lsp-filter + clangd; the clangd team have added a bunch of heuristics for completion quality that make it significantly better and faster. clangd has extremely responsive completion, I recommend trying it out again.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAKLzqjGCLzzy1tv%2ByLbNVQtwO5-XP1N-5btxxTKv7AEg70hhdQ%40mail.gmail.com.

[Sunny Sachanandani]

Probably off-topic, but QtCreator has pretty good code completion and code navigation capabilities, and takes only 1-2 mins to index the entire source tree.  You can generate a project with `gn gen out/Debug --ide=qtcreator`, and load the project file in `out/Debug/qtcreator_project/all.creator`.  It's not 100% accurate since it applies the same includes, defines, and cflags to every file, but it works well in practice.  This is similar to Eclipse IIUC.  Lately I've been using clangd (with above performance fixes) on Linux, and it works well too.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAFxOgiV3t2kr7xB%3DyCHtpJQZf-5%2B1aEHvxMQTnL%3DyrTyNigZYw%40mail.gmail.com.

[Jacob Dufault]

I'd say QtCreator and Eclipse are in a different class, since QtCreator doesn't provide cross references/type hierarchy/etc at a usable performance for Chrome. QtCreator uses a custom parser which works well for global symbol search (the custom parser is why it can index Chrome so quickly). For completion it uses libclang, so clangd should provide nicer results.

[Asanka Herath]

Resurfacing some of drott@'s questions that I think didn't get addressed yet:

I've tried getting cquery to work on Windows but I ran into problems with receiving errors from the indexer, which does not seem to be able to understand the compilation command line syntax and drops out of indexing.  Does anyone have a working LSP setup for Chromium on Windows? 

 

What is the status of clangd? Would it be feasible to build clangd and make it part of our custom clang binaries on each platform, including Windows? Then I could perhaps try to use clangd on Windows. So far I have not attempted to build clangd myself.

[Olivier Li Shing Tat-Dupuis]

My previous contained a URL that is not externally visible. Apologies for that. Here is a github gist : https://gist.github.com/OlivierLi/2c8f2a5e85bc1f3884b696e6fcf3bc59.

[Olivier Li Shing Tat-Dupuis]

I just now realize that I answered privately. Here is my answer concerning cquery and windows :

Hello,

During a Windows development stint I was able to get pretty good indexing using cquery. Setting up the editor integration is explained  pretty clearly here. I used it with vim mostly but vscode integration was great.

The hard thing like you figured out is to have the compile_commands.json be in an understandable format for the indexer. This needs some pretty heavy massaging. I put up a script that generates and then modifies the json file here. Please note that the current version of this script sets up a pretty restrictive filter at line 16 because indexing all of chromium takes a very long time.

I hope that is enough to get you started!

PS: I'm also curious about clangd.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/CAM%3DuaOHCaxwLm-2R1jVc2amqMzgWUEv4DG51BHFBV1W8idi2Fw%40mail.gmail.com.

--

Olivier Li Shing Tat-Dupuis |

Software developer |

oliv...@google.com |

1-415-761-1995

[Sunny Sachanandani]

It seems that, at least for VS Code's C++ intellisense, the crucial bit is using escaped backslashes \\ instead of forward slashes / for file paths in compile_commands.json. You can configure the compiler path separately in VS Code to override what's in the json so that part of the script is not needed, and neither is removing the extra cflags.

None of gn's escaping modes performs this substitution, and ninja only performs minimal JSON escaping when writing the file.

gn's implementation is here: https://gn.googlesource.com/gn/+/master/tools/gn/compile_commands_writer.cc#200

ninja's implementation is here: https://github.com/ninja-build/ninja/blob/ca041d88f4d610332aa48c801342edfafb622ccb/src/ninja.cc#L693

+gn-dev

[Ray Song]

To accelerate project loading, ccls provides an initialization option index.initialBlacklist, as documented on ccls/wiki/Initialization-options (there are a few others for tuning the indexer)

    Files that should not be loaded on initialization, but will still be indexed after you open them.

For example, I use index.initialBlacklist: ["/(clang|lld|llvm)/(test|unittests)/" "/llvm/(lib|include|tools/[a-bd-z]|utils)" "/StaticAnalyzer/"] to exclude some llvm/clang files.

I believe cquery's index.blacklist works similarly but has a slightly different semantic.

BTW, ccls's wiki pages describe some not well-known but useful cross reference features, and semantic navigation.

clangd leverages some latency reducing technique, which ccls also uses (blatantly copied), e.g. SkipFunctionBodies StoreInMemory WriteCommentListToPCH

clangd has more sophiticated completion system and an interesting global completion feature, but to me what ccls currently does seems to suffice.

clangd-indexer can probably do a decent job now but I worry much about their memory footprint and its pipeline (the component that tracks dependencies, dispatch index requests and apply updates to the global index)

  Regarding pipeline, cquery did a great job, in this regards.

To try out multiple language servers for the best of both worlds, lsp-filter whould be a good choice. AFAIK, it is still tricky (unrealistic in the near term) to support multiple language servers in one project, for eglot lsp-mode and LanguageClient-neovim.

[shp...@gmail.com]

Hello, 

I am also trying Emacs on Linux with ccls to navigate chromium source code.

Yet, I failed to index source codes and it seems to quit indexing at some point and every time I re-open the Emacs, it starts over indexing again and again.

I am already using ccls with quite large code bases (e.g. Linux kernel, LLVM, gecko), however I cannot use it for Chromium

Could you share your experiences indexing chromium source with ccls?

Which (1) build command and (2) compile_commands.json generation command did you use?

And did you apply any custom settings for ccls?

Thanks!

Seonghyun

[Erik Jensen]

I have don't use ccls or Emacs, but for generating the compilation database, `gn gen --export-compile-commands` seems to do the trick.

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/chromium-dev/2908684b-56fd-4f72-8cdc-d36d856989cd%40chromium.org.

[Yuchen Wang]

Hey,

I am using vim for develop chromium, I think neovim-language client + clanged works fine for me.

Thanks,

Yuchen

<shp...@gmail.com> 于2019年4月25日周四 上午12:26写道：

--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
---
You received this message because you are subscribed to the Google Groups "Chromium-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to chromium-dev...@chromium.org.