JSPromise::Resolve and builtin ResolvePromise can pick different contexts to enqueue onto

[Eric Rannaud]

Hi,

I was hoping to get a discussion started on issue https://issues.chromium.org/issues/441679231 that manifests itself in Node when multiple microtask queues are in use (by package node:vm).

In a nutshell, there are a couple of questions for V8 developers:

- Currently, the C++ and Torque code paths for promise resolutions are subtly different. I believe the Torque code is more in line with the spec.

- However, to solve the issue faced by Node, the C++ code is actually closer to what we need, I think. After evaluating an inner SourceTextModule with its own microtask queue, the execution control flow can get "lost"  when returning to the surrounding context with a different microtask queue. At issue is the logic to pick the context associated with the resolution, where the thenable job task will be enqueued, and the shortcut taken by the implementations to avoid the "then"  property lookup when the promise object is "native".

There are more details and a proposed patch in the issue tracker, linked above.

Thanks,
Eric

[Leszek Swirski]

Hi Eric,

Thanks for reaching out, and for the very detailed analysis. I think you haven't seen much interaction on the bug because of a combination of August holidays, bystander syndrome, and general complexity of all the promise stuff.

It sounds like you've found a bug in our C++ implementation which is incompatible with the spec; I'd have to read into the exact details but it sounds plausible. This is great, and we should fix this, however your subsequent comments sound like they want the broken behaviour? In general, for us, spec is non-negotiable (the few times we do break spec is for web compatibility or some serious performance concern), so if you think a break from spec behaviour would be useful for Node, then you'd want to make that a spec proposal.

The nested microtask queues Node is using sound a bit broken, since it sounds like the microtask queue is only drained once? If I understand the issue correctly, then the right fix would be for Node to always drain the inner microtask queue after something executes in the inner realm, or, I guess, patch the native context after draining it to refer to the outer microtask queue.

- Leszek

--
--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/v8-dev/e87cecc7-bcf3-4898-a5fc-4d991bc8b632n%40googlegroups.com.

[Eric Rannaud]

Hi Leszek,

On Fri, Sep 5, 2025 at 7:36 AM Leszek Swirski <les...@chromium.org> wrote:

It sounds like you've found a bug in our C++ implementation which is incompatible with the spec; I'd have to read into the exact details but it sounds plausible. This is great, and we should fix this, however your subsequent comments sound like they want the broken behaviour?

Something in between. While the C++ code is closer, I think we'd want a variant: we want the C++ behavior only if the resolution is an already-fulfilled promise, otherwise we want the Torque behavior.

 

In general, for us, spec is non-negotiable (the few times we do break spec is for web compatibility or some serious performance concern), so if you think a break from spec behaviour would be useful for Node, then you'd want to make that a spec proposal.

Understood.

The nested microtask queues Node is using sound a bit broken, since it sounds like the microtask queue is only drained once? If I understand the issue correctly, then the right fix would be for Node to always drain the inner microtask queue after something executes in the inner realm, or, I guess, patch the native context after draining it to refer to the outer microtask queue.

I did consider patching the context to point to the outer queue after evaluate(), or just set the queue to null, but in typical node:vm usage this context object may be shared with other (inner) modules that may be evaluated later, so I don't think we can actually modify it.

Every time some code awaits on the already-fulfilled promise returned by evaluate(), a new (trivial) task will appear on the inner microtask queue, otherwise (long) inactive.

The issue for Node is that there is no way of knowing that the inner microtask queue should be drained. This (trivial) task gets enqueued after inner_module.evaluate() returns, at a point where we are done with inner_module, and I'm not sure where there would be a reasonable place to check "what if the queue for this other context we're not using anymore needs draining?"

Another way to describe the issue is that with two contexts A and B, with queues QA and QB, context B is effectively waiting on something to happen in context A (the trivial thenable job task), but while there is a task enqueued on QA, there are no tasks enqueued on QB to indicate that the (outer) module B has more work to do, leading to the evaluation of module B to just terminate early. The lack of dependency tracking between contexts B and A is what is broken in the spec, I think.

Whenever context B enqueues something onto QA, it is possible (but not guaranteed) that A will in turn enqueue something back onto QB. So, context B should be aware of that possibility and consider itself waiting on a QA checkpoint. It's only after a QA checkpoint is performed that context B can check if its queue is effectively empty or not.


Possibly, In Node or in v8::SourceTextModule, after finding that the current queue is empty, before terminating the evaluation of the current module, we could check if any other "related" microtask queues are non-empty and drain them, before checking if doing this added any tasks to our current queue.


Or, after all, it may be worth considering another option I included in comment #3: "In V8, add a callback to the public API MicrotaskQueue to be notified, with policy kExplicit: either when a task is enqueued, or more efficiently, when there are enqueued tasks in strategic places (maybe in the same places where, with policy kAuto, a checkpoint may be performed). This would let Node know that the qeueue associated with an already-evaluated module needs to be drained again. I have not tried this, but it seems very clunky."

This callback could be associated with the queue: OnMicrotaskEnqueue(microtask_queue). In Node, we would maintain enough state to know if we've entered a context associated with this queue. In the callback: if the queue is in an entered context, we know that the queue will get drained later, we have nothing to do; but if this queue is not associated with an entered context, we immediately drain the queue.

Or this callback could be associated with the native context: OnMicrotaskEnqueue(native_context). The Node implementation of this callback would be simpler, as we only need to check if native_context matches the isolate current native context.

Thanks,
Eric

[Eric Rannaud]

Hi,

I will submit the following patch to V8 (option B from comment #2), which aligns the C++ code with the Torque version.

diff --git a/deps/v8/src/objects/objects.cc b/deps/v8/src/objects/objects.cc
index 3e8fa8d3368f..7b40af964558 100644
--- a/deps/v8/src/objects/objects.cc
+++ b/deps/v8/src/objects/objects.cc
@@ -5166,13 +5166,13 @@ MaybeHandle<Object> JSPromise::Resolve(DirectHandle<JSPromise> promise,
   MaybeDirectHandle<Object> then;
 
   // Make sure a lookup of "then" on any JSPromise whose [[Prototype]] is the
-  // initial %PromisePrototype% yields the initial method. In addition this
-  // protector also guards the negative lookup of "then" on the intrinsic
-  // %ObjectPrototype%, meaning that such lookups are guaranteed to yield
-  // undefined without triggering any side-effects.
+  // initial %PromisePrototype% (in the current native context) yields the
+  // initial method. In addition this protector also guards the negative lookup
+  // of "then" on the intrinsic %ObjectPrototype%, meaning that such lookups are
+  // guaranteed to yield undefined without triggering any side-effects.
   if (IsJSPromise(*resolution_recv) &&
-      resolution_recv->map()->prototype()->map()->instance_type() ==
-          JS_PROMISE_PROTOTYPE_TYPE &&
+      resolution_recv->map()->prototype()->SafeEquals(
+          *isolate->promise_prototype()) &&
       Protectors::IsPromiseThenLookupChainIntact(isolate)) {
     // We can skip the "then" lookup on {resolution} if its [[Prototype]]
     // is the (initial) Promise.prototype and the Promise#then protector

I have a possible workaround on the Node side: the inner module registers a callback with the outer microtask queue, and whenever the outer queue completes a checkpoint, the callback tries to checkpoint the inner queue too, just in case. This is a little silly as there will typically be nothing to drain on the inner queue. At least it's not an expensive operation to invoke PerformCheckpoint() on an empty queue.

Thanks,
Eric

[Eric Rannaud]

Hi,

I have a possible workaround on the Node side: the inner module registers a callback with the outer microtask queue, and whenever the outer queue completes a checkpoint, the callback tries to checkpoint the inner queue too, just in case. This is a little silly as there will typically be nothing to drain on the inner queue. At least it's not an expensive operation to invoke PerformCheckpoint() on an empty queue.

As explained in the Node commit https://github.com/nodejs/node/pull/59801/commits/9f3b583e7abdbbd3c29a7c5eca04682b2b66993c, the workaround is only partial:

"This workaround is not foolproof. While it should work with multiple
nested contexts (not tested), it will not work if, in the default
context A, we have two contexts B and C, each evaluated from A, and we
create promises in B and pass them into context C, where C resolves
them. To handle this more complicated setup, I think we would likely
need the help of V8: we would want to be notified when a task is
enqueued onto the queue of a different context than the current one."

To flesh this out a little bit, EnqueueMicrotask(handlerContext, task) would, in pseudo code:

if (handler_native_context != current_native_context

    && handler_native_context_queue != current_native_context_queue) {
  invoke EnqueudFromOtherContextCallback on handler_native_context_queue

}

In Node, this callback would be used to checkpoint the queue in handlerContext, either right away or at an opportune time.

I'd imagine the torque version of EnqueueMicrotask() would call into C++ in the same branch, rather than invoke the callbacks itself.

Is this new API something that V8 would even consider?

Thanks,
Eric

[Leszek Swirski]

Hi Eric,

We took a closer look at how this works in V8+Chromium, and long story short we only have one microtask queue shared between different contexts that can interact with each other. I think this is the only real reasonable solution, if I'm honest, multiple microtask queues interleaving with each other are asking for trouble. If the node module has any dependencies on promises from the outer context, I expect you'll hit the same issue in reverse, and I bet that you could interleave chaining promises from inside and outside to cause this problem up to an arbitrary depth. Could the inner source module post a full event loop task after it finishes its execution, and resolve the execute promise in that? Then any microtasks started by the execute are guaranteed to complete before that task executes.

- Leszek

--

--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/v8-dev/CA%2BzRj8W8U7t_fvHgpMROdmO31OTiOddLM9gfi6JNN6m9wkV1EA%40mail.gmail.com.

[Eric Rannaud]

Hi Leszek,

On Fri, Sep 12, 2025 at 2:18 AM Leszek Swirski <les...@chromium.org> wrote:

We took a closer look at how this works in V8+Chromium, and long story short we only have one microtask queue shared between different contexts that can interact with each other. I think this is the only real reasonable solution, if I'm honest, multiple microtask queues interleaving with each other are asking for trouble. If the node module has any dependencies on promises from the outer context, I expect you'll hit the same issue in reverse, and I bet that you could interleave chaining promises from inside and outside to cause this problem up to an arbitrary depth. Could the inner source module post a full event loop task after it finishes its execution, and resolve the execute promise in that? Then any microtasks started by the execute are guaranteed to complete before that task executes.

Yes, I agree, the interactions between contexts can get really complicated, with promises shared back and forth, or transitively across N contexts with N queues.

The following example is indeed broken, even with our tentative fix in node:vm. Waiting on `outer_promise` lets the execution flow fall through.

```js
import * as vm from "node:vm";

const context = vm.createContext({
  console,
  outer_promise: Promise.resolve(),
}, {
  microtaskMode: "afterEvaluate",
});

const m = new vm.SourceTextModule(
  // OUPS, we enqueued a task on the global queue:
  'await outer_promise;',
  {context},
);

await m.link(() => null);
await m.evaluate();

// The global queue will be drained, but we would then need to drain the
// inner queue again, and we've already given up on our chance to do that.

console.log("NOT PRINTED");
```

I now wonder a little bit if the standards folks threw in the concept of multiple microtask queues without a clear use case and it's not really workable as it is.

The tentative fix in Node, an idea of @addaleax, is narrow and aligns with what you're suggesting I think: instead of returning a Promise built in the inner context from `module.evaluate()`, we return a promise built in the outer context. This is done within the node:vm library, by resolving an outer Promise with the return value of v8::SourceTextModule::Evaluate(), which was built in the inner context, and then we checkpoint the inner context microtask queue once. This at least makes it possible to write:

```

    await module.evaluate();

```
without blowing up the execution flow. The `module` object is meant to mimic the  SourceTextModule() API, so returning an outer-context Promise is a slight departure from the API, but it is necessary to make it actually usable with a separate queue.

BTW, the use of multiple queues in node:vm is optional; it is needed when the user wants to constrain the inner module with a timeout or SIGINT. I'm not sure there is a way to implement a timeout feature for the inner context without a separate queue.

We're considering adding a section to the node:vm documentation warning users of the (remaining) consequences of sharing promises between different contexts, when using multiple microtask queues. It's not pretty, see the end of this email. I'm not even sure that this new section is scary enough!

With that in mind, I still think it might be a good idea for v8 to offer a notification mechanism on MicrotaskQueue. I do get your point: foreign-context enqueuing can happen in any direction between contexts. Since the global queue uses kAuto, however, only the inner queues need to be "monitored". If I simulate such a notification mechanism with a setInterval() in the example at the top of this email, the code can be made to behave sanely.

Right now, while user-code can try to anticipate the need to manually drain the inner queue, by setting up a callback with setInterval(), say, it is not clear when it is safe to *stop* doing so. You'd need to detect that you've made enough progress both inside the inner module and within the global context, and call clearInterval(). It can get complicated. There is also no way to simply *detect* that you've hit this strange behavior.

```js
module.onPendingExternalMicrotasks(() => {
  // Here can be added timeout logic, or simply print a warning that some unexpected
  // promise sharing has happened.
  module.evaluate();

});
```

Thanks,
Eric

### When `microtaskMode` is `'afterEvaluate'`, beware sharing Promises between Contexts

In `'afterEvaluate'` mode, the `Context` has its own microtask queue, separate
from the global microtask queue used by the outer (main) context. While this
mode is necessary to enforce `timeout` and enable `breakOnSigint` with
asynchronous tasks, it also makes sharing promises between contexts challenging.

In the example below, a promise is created in the inner context and shared with
the outer context. When the outer context `await` on the promise, the execution
flow of the outer context is disrupted in a surprising way: the log statement
is never executed.

```mjs
import * as vm from 'node:vm';

const inner_context = vm.createContext({}, { microtaskMode: 'afterEvaluate' });

// runInContext() returns a Promise created in the inner context.
const inner_promise = vm.runInContext(
  'Promise.resolve()',
  context,
);

// As part of performing `await`, the JavaScript runtime must enqueue a task
// on the microtask queue of the context where `inner_promise` was created.
// A task is added on the inner microtask queue, but **it will not be run
// automatically**: this task will remain pending indefinitely.
//
// Since the outer microtask queue is empty, execution in the outer module
// falls through, and the log statement below is never executed.
await inner_promise;

console.log('this will NOT be printed');
```

To successfully share promises between contexts with different microtask queues,
it is necessary to ensure that tasks on the inner microtask queue will be run
**whenever** the outer context enqueues a task on the inner microtask queue.

The tasks on the microtask queue of a given context are run whenever
`runInContext()` or `SourceTextModule.evaluate()` are invoked on a script or
module using this context. In our example, the normal execution flow can be
restored by scheduling a second call to `runInContext()` _before_ `await
inner_promise`.

```mjs
// Schedule `runInContext()` to manually drain the inner context microtask
// queue; it will run after the `await` statement below.
setImmediate(() => {
  vm.runInContext('', context);
});

await inner_promise;

console.log('OK');
```

[Leszek Swirski]

I'm sorry that this will be a short reply to a long and well thought through email, but really the discussion needs to fork at:

I now wonder a little bit if the standards folks threw in the concept of multiple microtask queues without a clear use case and it's not really workable as it is.

The standard _doesn't_ have a concept of multiple microtask queues (it actually doesn't have a concept of microtasks at all, just "Jobs"). In particular, there is a note in https://tc39.es/ecma262/#sec-hostenqueuepromisejob that "Jobs must run in the same order as the HostEnqueuePromiseJob invocations that scheduled them"; there is no qualifier here that this is per-realm behaviour. Interleaving two microtask queues is only allowed in V8 because of a separate concept of "agents" (https://tc39.es/ecma262/#agent) which are not able to communicate with each other and therefore the different agent's microtask queues are not able to depend on each other -- in Chromium+V8, each Agent (WindowAgent) has a single microtask queue (https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/core/execution_context/window_agent.cc;l=14;drc=c1fb77438b37eda722f5708c2277f25438af0bc3).

Node's behaviour here, with nested interleaving microtask queues that can post to each other, is therefore violating spec, and the consequences of this are, well, visible in this issue. I don't think we're comfortable making this even more complicated on the API level to break further from spec --  honestly, it's a bit unfortunate that the implementation is as flexible as it is already.

I'm not sure there is a way to implement a timeout feature for the inner context without a separate queue.

 

 What prevents Node from using the same timeout mechanism when spinning the main queue? You can trigger an isolate->TerminateExecution or isolate->RequestInterrupt from any thread.

- Leszek

--
--
v8-dev mailing list
v8-...@googlegroups.com
http://groups.google.com/group/v8-dev
---
You received this message because you are subscribed to the Google Groups "v8-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+un...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/v8-dev/CA%2BzRj8UjT7v%2BywhTHBzjbVGs%2BSXtGY5C%2Bome%3DBX408fFZS2DjA%40mail.gmail.com.