Questions about the Inspect native memory project in Dart DevTools

[Nourhan Hasan]

Hi,
I hope you're all well.

I'm Nourhan, and I'm interested in working on the (Inspect native memory in Dart DevTools) project as part of GSoC26.

After some research, I found that this problem has already occurred in other languages ​​and has been addressed, as GDB/LLDB does for C/C++ pointers.

And these are some notes and questions, if you don't mind:

The Dart VM Service runs inside the debuggee process, which means an invalid pointer dereference during debugging would crash the app being debugged. LLDB avoids this entirely by doing memory reads from a separate lldb-server process using ptrace(PTRACE_PEEKDATA). If the address is invalid, the OS returns an error to lldb-server without crashing the target.

1. Is there any existing separation in the Dart debugger infrastructure between the debuggee process and a debug-server component, similar to the lldb / lldb-server split? Or does the VM Service always run in the same process as the application being debugged? If it's always in-process, has anyone explored whether process_vm_readv on Linux (which reads another process's memory from outside) could be used by having the VM Service talk to a small helper process for the actual memory read, rather than relying on signal handlers?

2. Would you suggest some issues related to the project that I can work on, which will help me understand the project more deeply?

Thanks for your time,
Nourhan Hasan

[Daco Harkes]

Hi Nourhan,

Starting a new process sounds like a much more robust idea than relying on signalhandlers indeed! Great idea!

I believe the VM service always runs in the same process. So, exploring whether we can change that would be interesting. cc @Ben Konyi

You can look at https://github.com/dart-lang/sdk/issues?q=is%3Aissue%20state%3Aopen%20label%3Alibrary-ffi, especially the ones marked with contributions welcome. For VM service bugs to work on cc @Ben Konyi 

Kind regards,

 •  Daco Harkes
 •  Software Engineer
 •  dacoh...@google.com 

--
You received this message because you are subscribed to the Google Groups "dart-gsoc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dart-gsoc+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/dart-gsoc/8c7b5980-d04b-4001-aa33-3ddf1e6f88acn%40googlegroups.com.

[Ben Konyi]

Hi Nourhan,

Daco's correct in stating that the VM service always runs in the same process. We won't be able to change this because it's tightly integrated with the runtime. However, we have a separate service, called the Dart Development Service (commonly known as DDS), that acts as middleware between the VM service and developer tool clients. This always runs outside of the process being debugged, so it's possible that DDS could be a place to implement something like this.

I don't have any particular issues to suggest, but you can look at our issue backlog here and see if there's anything you'd like to take a swing at: https://github.com/dart-lang/sdk/issues?q=is%3Aissue%20state%3Aopen%20(label%3Apkg-vm-service%20OR%20label%3Apkg-dds%20OR%20label%3Avm-service)

Here's some links to code and documentation that might be useful while you explore:

• Dart Development Service implementation
  
• package:vm_service, which contains most of the tests for the VM service implementation
  
• The VM service protocol specification
  
• The C++ implementation of much of the VM service protocol
  
• The Dart/Flutter DevTools project
  
• VM developer tool implementations, which may provide some inspiration and examples of how DevTools interacts with various Dart object representations (see the object_inspector in particular)
  

- Ben

[Nourhan Hasan]

Hi Daco and Bin,
I hope you're both doing well.

Based on my understanding and analysis of the codebase, I think the solution can be implemented as follows. Please correct me if anything is inaccurate.

The VM already exposes the process PID in the getVM response  runtime/vm/service.cc#L5625, so no changes are needed there. Since DDS already calls getVM at startup, we can simply read and store the PID field  pkg/dds/lib/src/dds_impl.dart.
The main VM change needed is in Pointer::PrintJSONImpl  runtime/vm/object_service.cc#L1865  , which currently just delegates to the generic Instance serializer, emitting kind: "PlainInstance" with empty fields and no address. We would modify it to emit kind: "FfiPointer", the raw memory address, and the struct class name.
We would also add a new getFfiStructLayout RPC to the VM that, given a struct class, returns its field names, types, and byte offsets. DDS needs this to interpret the raw bytes it reads from memory.
On the DDS side, we would register a getObject handler  pkg/dds/lib/src/client.dart  that intercepts responses where kind == "FfiPointer", calls getFfiStructLayout to get the field layout, then reads the memory at the pointer's address using the stored PID.

Since DDS is written in Dart and Dart cannot call OS APIs without dart:ffi bindings, we would need to write FFI bindings for the cross-process memory read APIs on each platform (process_vm_readv on Linux, vm_read_overwrite on macOS, ReadProcessMemory on Windows). If the address is invalid, the OS returns an error code to DDS and the VM process is unaffected. DDS then interprets the bytes using the field layout and returns the enriched response to the client.

Do you think this approach is reasonable? And for the dart:ffi memory read bindings, do you find this suitable? Or do you have another suggestion?

Sorry if you received this message repeatedly. It happened by mistake. I am resending it here so that it is visible to everyone in the Dart community.

Thank you for your time, 
Nourhan

[Gurleen Kaur]

Hi all,

I've been working on this project as well. While studying the FFI transformation pipeline in definitions.dart....where I recently contributed a fix... I noticed that the compiler pass already adds a @pragma('vm:ffi:struct-fields') annotation to every struct class as a ListConstant. This is readable via getObject() RPC at zero cost and works in AOT-compiled code too, so type recovery may not require a new RPC.

I agree with Ben that DDS is the right coordination layer for safe memory reads since it already runs outside the debuggee process.

Looking forward to the discussion!

Gurleen-kansray

[Ben Konyi]

Thanks for the high-level details Nourhan. I think that sounds reasonable, although I would also like to hear Daco's opinion as our dart:ffi expert :-) My main concerns would be around the DDS integration. 

Right now, DDS is platform agnostic and works everywhere Dart can run. Introducing use of dart:ffi into DDS would need to be done in such a way that platforms that don't support the bindings can still work. We also need to ensure that DDS has the right permissions to access memory in the target VM process, which may involve putting this particular functionality behind a flag for security reasons. It's also possible that DDS isn't running on the same device as the target VM. For example, when working with Flutter on Android, the VM service is running on the Android device and DDS is running on the developer's host machine. It's also possible for DDS to be running on the developer's host machine when the target VM is somewhere else on the network.

I wouldn't say any of the above issues are too hard to overcome, but they definitely need to be kept in mind. Ideally we have a solution that works everywhere, all the time, but obviously that may not be possible when working with a feature like dart:ffi that breaks through the Dart layer and pokes at memory directly :-)

[Daco Harkes]

Hey!

Yes it makes sense that we can't offer reading arbitrary memory if the DDS process runs on a different machine. And yes permissions will be a thing, we basically want our DDS process to have the same permissions as a debugger process would have then.

We can probably just use a conditional import for ensuring DDS works for non-native contexts. I don't see any issues using system calls with dart:ffi.

Give it a spin and let us know if it works turning the DDS process into a debugger. 🚀

Kind regards,

 •  Daco Harkes
 •  Software Engineer
 •  dacoh...@google.com 

To view this discussion visit https://groups.google.com/d/msgid/dart-gsoc/aa4dee88-13ae-45d6-b141-a1b4c6217379n%40googlegroups.com.

[Nourhan Hasan]

Thank you for the clarification. I will take your feedback into consideration. I had assumed DDS and the VM always run on the same machine. This means the out-of-process memory read approach I proposed would only work for local debugging. For remote debugging, the VM is the only component on the target machine, so the memory read would have to happen there. The standard solution, used in every major debugging system, is a co-located debug agent that sits outside the target process on the target machine, similar to how GDB uses gdbserver, LLDB uses lldb-server, and Java uses a JDWP agent. I am not sure whether introducing such an agent fits Dart's infrastructure, given Ben's point. On Linux/Android specifically, a lighter alternative would be for the VM to read its own memory via /proc/self/mem, reading from an invalid address through that file descriptor returns EIO instead of delivering SIGSEGV to the process, so the app stays alive without requiring a new process. Sources: LLDB remote debugging, gdbserver, Java Platform Debugger Architecture. This is what I have in mind regarding an out-of-process approach, given that DDS and the VM don't always run on the same machine. If there are other options I've missed, I'm open to any suggestions. Is remote debugging within the scope of the GSoC26 project? Before taking any action, I just want to confirm my understanding of your earlier comments (for local debugging): * DDS would need debugger-like permissions to access the target VM's memory. * To keep DDS working in non-native environments, we could use conditional imports so that the FFI-based implementation is only used where it's supported.
* Using system calls through dart:ffi itself shouldn't be an issue.

1. Please let me know if my understanding is correct or if I've missed anything.
   In the meantime, I will try turning the DDS process into a debugger and report back. 🚀 Thank you for your time, Nourhan

[Shanu Kumawat]

Hi Nourhan, Gurleen, Ben, and Daco,

This is a great discussion.I have also been doing some research and taking a deep dive into the architecture for this project, specifically looking at how we can achieve LLDB-style safety without breaking remote Flutter debugging.

Nourhan, your point about reading memory from the outside to prevent a SIGSEGV crashing the VM is exactly the right safety model. However, as Ben pointed out, relying on DDS to do the reading introduces a major topological issue: in remote Flutter debugging (e.g., debugging an Android device from a Mac), DDS runs on the host machine, while the target memory lives on the mobile device. DDS physically cannot execute OS-level memory reads across a network boundary. Furthermore, an external DDS process reading the VM's memory on macOS would likely be blocked by System Integrity Protection (SIP) without elevated privileges.

However, I think we can get the exact out-of-process safety you described while staying entirely in-process inside the VM Service.

A process is allowed to use OS-level safe-read APIs to read its own memory. If the VM Service routes all DAP ReadMemory requests through these APIs—targeting its own PID—the OS kernel will validate the page tables. If the pointer is dangling or unmapped, the kernel simply aborts the copy and returns an error (like EFAULT), and no hardware MMU trap or SIGSEGV is ever generated. The VM doesn't crash.

The cross-platform implementations for the VM Service would look like this:

• Linux/Android: process_vm_readv (targeting its own PID) or reading from /proc/self/mem.
• Windows: ReadProcessMemory(GetCurrentProcess(),...)
• macOS/iOS: mach_vm_read_overwrite(mach_task_self(),...) (This completely bypasses SIP issues).

If we implement this safe-read wrapper inside the VM Service, we solve the remote debugging topology problem, bypass all cross-process security permissions, and completely protect the Dart VM from segmentation faults when a developer inspects a bad Pointer.

I am currently looking into how we can pair this safe-read mechanism with the layout metadata to serve standard DAP ReadMemoryRequest payloads.

Would love to hear your thoughts on this approach!

[Daco Harkes]

Hi Shanu,

Interesting! Yes, if in-process works, it would simplify the solution! I think the best way forward is to simply try adding it for the various OSes and see what happens.

From what I can gather these system calls are rather slow, but that doesn't matter for our use case.

Kind regards,

 •  Daco Harkes
 •  Software Engineer
 •  dacoh...@google.com 

To view this discussion visit https://groups.google.com/d/msgid/dart-gsoc/2239420b-5dd6-4d5e-ad56-0d98140c0d13n%40googlegroups.com.

[Shanu Kumawat]

Thanks for the suggestion. I've conducted some tests by implementing a small prototype under Linux using `process_vm_readv(getpid())` to understand its behavior when it attempts to read valid as well as invalid addresses within the same process.

When it comes to valid addresses, it works as expected. For invalid or unmapped addresses, it returns `-1` with `errno` being `EFAULT`. The important point is that no `SIGSEGV` is sent, indicating that the kernel is rejecting invalid access in a stable manner when using this syscall.

Next, I'm going to perform a couple of tests that cover some practical debugger use cases, including reading from freed (dangling) pointers, reading from clearly unmapped high addresses, as well as reading across page boundaries. I'll also check the equivalent APIs on macOS and Windows to confirm their behavior there

Thanks,
Shanu

[Gurleen Kaur]

Hi all,

Since Shanu mentioned looking into the Windows equivalent, I wanted to share results I already have from testing `ReadProcessMemory(GetCurrentProcess(), ...)` on Windows via Dart FFI.

- Valid address: returned 1 (success), correctly read the expected value with no issues.
- Invalid address: returned 0 (failure) with no crash and no exception — the process remained stable.

This matches what Shanu observed on Linux with `process_vm_readv` — the OS rejects invalid addresses cleanly without destabilizing the process, which confirms the in-process approach is viable on Windows as well.

I'll follow up with edge case tests similar to Shanu's (dangling pointers, unmapped high addresses, cross-page-boundary reads).

Thanks,
Gurleen-kansray