Implement reading STDIN to an ArrayBuffer. Precedent: writeFile("/proc/self/fd/1")

[guest271314]

d8's readline() is useless for reading STDIN where standard input is not UTF-8, e.g., readline() will stop reading when the initial input is a Uint32Array denoting the size of the following message.

While ECMA-262 does not specify reading stadard input or writing to standard output there is precedent in d8 for writing to STDOUT: writeFile("/proc/self/fd/1"). SpiderMonkey's js does read past the initial Uint32Array with readline(), though like d8, expects a newline character.

Circa 2024 JavaScript is a general programming language - that lacks the basic capability to read STDIN and write to STDOUT in an engine and runtime agnostic way, using an (resizable) ArrayBuffer to store variable standard input. For the non-browser environments we can use Web IDL to indicate when tyo expose or not expose the capability, just like certain interfaces are only available or not available in certain contexts; whether that be no fetch() in an AudioWorklet, or TCPSocketServer only in Isolated Web Apps.

Something like this https://github.com/guest271314/NativeMessagingHosts/blob/main/nm_host.js#L45-L69 should be standardized for JavaScript as a whole so we can use the same code in different engines and runtimes, instead of having to use different standard input and standard output code for each engine and runtime. 

Thanks for your consideration.

async function* getMessage() {
  let messageLength = 0;
  let readOffset = 0;
  for await (let message of readable) {
    if (buffer.byteLength === 0) {
      buffer.resize(4);
      for (let i = 0; i < 4; i++) {
        view.setUint8(i, message[i]);
      }
      messageLength = view.getUint32(0, true);
      message = message.subarray(4);
      buffer.resize(0);
    }
    buffer.resize(buffer.byteLength + message.length);
    for (let i = 0; i < message.length; i++, readOffset++) {
      view.setUint8(readOffset, message[i]);
    }
    if (buffer.byteLength === messageLength) {
      yield new Uint8Array(buffer);
      messageLength = 0;
      readOffset = 0;
      buffer.resize(0);
    }
  }
}

try {
  await sendMessage(encodeMessage([{ dirname, filename, url }, ...args]));
  for await (const message of getMessage()) {
    await sendMessage(message);
  }
} catch (e) {
  exit();
}

[Jakob Kummerow]

d8's readline() is useless for reading STDIN where standard input is not UTF-8

That's OK. As I told you before, d8 isn't trying to be a general-purpose shell; it's a tool for developing V8. If you want a V8-embedding shell that does more than d8 offers, find one or write your own. In the other thread, you said that Node.js serves your needs -- great, problem solved.

Something like [snip...] should be standardized for JavaScript

This mailing list isn't the right place to propose or discuss possible JS language additions.

[guest271314]

Node.js does not serve the needs of what I'm doing in d8. 

I'm talking about d8 specifically. 

I'm developing V8 and pointing out an omission that can be addressed by providing an option to remove UTF-8 checks for readline() or by adjusting read("path", "binary") to not check for file length, thus providingthe capability to read from STDIN to an ArrayBuffer.

As to this mailing list, I've read all kinds of proposals here, from Float16Array to others that wind up in TC-39.

I don't understand the resistance to developer feedback from the field. 

Nothing horrible can happen by providing a means to read STDIN in d8.

I'm not an adversary here. I'm diving in the JavaScript and sharing my feedback.

Anyway, thanks. Have a great day.

--
--
v8-users mailing list
v8-u...@googlegroups.com
http://groups.google.com/group/v8-users
---
You received this message because you are subscribed to a topic in the Google Groups "v8-users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/v8-users/NsnStT6bx3Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to v8-users+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/v8-users/CAKSzg3RbUBoK%2B-SYoommqzFsMFt4vX3q0qp8N7JqCjNBvXS%3Dag%40mail.gmail.com.

[guest271314]

For anyone who happens to have interest in this domain, what I wound up doing is getting the PID of the current d8 process, then reading the STDIN to d8 using dd.

nm_d8.js

#!/usr/bin/env -S /home/user/.jsvu/engines/v8/v8 --enable-os-system
// d8 Native Messaging host
// guest271314 7-7-2023, 6-20-2024

function encodeMessage(str) {
  return new Uint8Array([...str].map((s) => s.codePointAt()));
}

function getMessage(pid) {
  try {
    // readline() doesn't read past the Uint32Array equivalent message length
    // V8 authors are not interested in reading STDIN to an ArrayBuffer in d8
    // https://groups.google.com/g/v8-users/c/NsnStT6bx3Y/m/Yr_Z1FwgAQAJ
    // Use dd to get message length and return message from Bash script
    const stdin = (os.system("./getNativeMessage.sh", [pid])).trim();
    return encodeMessage(stdin);
  } catch (e) {
    writeFile("getMessageError.txt", encodeMessage(e.message));
  }
}

function sendMessage(message) {
  const header = new Uint32Array([message.length]);
  writeFile("/proc/self/fd/1", header);
  writeFile("/proc/self/fd/1", message);
}

function main() {
  // Get PID of current process
  const pid = (os.system("pgrep", ["-n", "-f", os.d8Path])).replace(/\D+/g, "");
  while (true) {
    const message = getMessage(pid);
    sendMessage(message);
  }
}

try {
  main();
} catch (e) {
  writeFile("mainError.txt", encodeMessage(e.message));
  quit();
}

getNativeMessage.sh

#!/bin/bash
# Bash Native Messaging host
# Read STDIN for V8's d8 shell, return message to d8
# guest271314 2024

set -x
set -o posix

getNativeMessage() {
  # https://lists.gnu.org/archive/html/help-bash/2023-06/msg00036.html
  length=$(dd iflag=fullblock bs=4 count=1 if=/proc/$@/fd/0 | od -An -td4)
  message=$(dd iflag=fullblock bs=$((length)) count=1 if=/proc/$@/fd/0)
  echo "$message"
}

getNativeMessage "$1"

in an MV3 browser extension ServiceWorker (background.js); or via "web_accessible_resources" and "externally_connectable" from any arbitrary Web page; or an offscreen document

globalThis.name = chrome.runtime.getManifest().short_name;

globalThis.port = chrome.runtime.connectNative(globalThis.name);
port.onMessage.addListener((message) => console.log(message));
port.onDisconnect.addListener((p) => console.log(chrome.runtime.lastError));
port.postMessage(new Array(209715));

chrome.runtime.onInstalled.addListener((reason) => {
  console.log(reason);
});