mg-dbx-napi blocking issue

75 views
Skip to first unread message

Sorin Petrescu

unread,
Mar 15, 2025, 2:42:07 PMMar 15
to Enterprise Web Developer Community
Hello,

I have a simple NGINX router setup, where:

  • A GET request triggers a function in InterSystems Cache, which contains a HANG 10 (deliberate 10-second delay).
  • A POST request triggers another function in Cache, which does nothing (executes instantly)

Since I am using mg-dbx-napi for database calls, I expected the POST request to execute independently of the GET request. 

  • If I first send the GET request (which contains HANG 10)
  • And immediately send the POST request,
  • The POST request waits until the GET request finishes (i.e., after 10 seconds), instead of executing independently.

The function called within the router is:

function(command) {

    return new Promise((resolve, reject) => {

      this.openTCP().then(() => {

        this.db.function(command, (err, result) => {
       
          if (err) {
            reject(err);
          } else {     
            resolve(result)
          }
          this.close();
        });
      }).catch((error) => {

        reject(error);
      });
    });
  }

openTCP() {
    return new Promise((resolve, reject) => {
      this.db.open({
        type: "Cache",
        host: "127.0.0.1",
        tcp_port: 7041,
        username: "_SYSTEM",
        password: "SYS",
        namespace: "%SYS"
      }, (err, res) => {
        if (err) {
          reject(err);
        } else {

          resolve(true);
        }
      });
    });
  }

The issue is not caused by the router, it works as expected when excluding the db.function calls:
router.get('/long-process', (req, res) => {
    console.log('Started long-running process...');
   
    const dbConn = new DB();

    dbConn.function('test1^AzTest').then((result) => {

        console.log(result);

    }).catch((error) => {

        console.log(error)
    });

    longRunningTask(() => {
        console.log('Finished long-running process');
        res.json({ message: 'Long process completed after 10 seconds' });
    });
});

router.post('/submit', (req, res) => {
    console.log('Received POST request');

    const dbConn = new DB();

    dbConn.function('test2^AzTest').then((result) => {

        console.log(result);

    }).catch((error) => {

        console.log(error)
    });

    res.json({ message: 'POST request processed' });
});

 How can I ensure that multiple database calls (from different requests) execute independently? Thank you.


rtweed

unread,
Mar 15, 2025, 2:52:49 PMMar 15
to Enterprise Web Developer Community
I would recommend you take a look at how NGINX, mg-web, mgweb-server and mg-dbx-napi are used in our mg-showcase repository.  There are tutorials and worked examples on how to achieve exactly what you're trying to achieve.

If, as your example suggests, your logic is all in M / ObjectScript code, there's little point in going through a JavaScript routing layer.  Use mgweb-server to provide the equivalent routing and  bi-directionalJSON mapping you'll need.

Make sure you configure NGINX to use a pool of workers,eg 8 or more.  That will ensure that any back-end handlers that hang or take time won't block other incoming requests.

Sorin Petrescu

unread,
Mar 22, 2025, 4:37:59 AMMar 22
to Enterprise Web Developer Community

Thank you very much for the detailed response.


I have set up a simple test system based on the documentation from https://github.com/chrisemunt/mg-dbx-napi to test the asynchronous execution of functions in Caché.

The Node.js router executes a long-running process via the GET method, by using the hang function in Cache. The POST method executes a simple method in Caché.

The connection to Caché is established via IP and 'do start^%zmgsi(0)'.

Given the following setup, I would expect that when calling in the order:

  1. GET
  2. POST

The POST method would return its result first. However, what happens is that the POST method is only executed after the GET method has finished.


import { server } from 'mg-dbx-napi';
import express from 'express';

const app = express();
const port = 8080;

var router = express.Router();

app.use(express.json());

app.use('/', router);

router.get('/long-process', (req, res) => {

    console.log('Started long-running process...');
   
    let db = new server();

    db.open({
        type: "Cache",
        host: "127.0.0.1",
        tcp_port: 7041,
        username: "_SYSTEM",
        password: "SYS",
        namespace: "%SYS"
      }, (err, result) => {

        if (!err) {

            db.function("get^AzNodeObject", (err, result) => {

                db.close();

                res.status(200).json(result);
            });
        }
      });
});

router.post('/submit', (req, res) => {

    console.log('Started short-running process...');
   
    let db = new server();

    db.open({
        type: "Cache",
        host: "127.0.0.1",
        tcp_port: 7041,
        username: "_SYSTEM",
        password: "SYS",
        namespace: "%SYS"
      }, (err, result) => {

        if (!err) {

            db.function("post^AzNodeObject", (err, result) => {

                db.close();

                res.status(200).json(result);
            });
        }
      });
});

// Start the server
app.listen(port, () => {
    console.log(`Server running at http://localhost:${port}`);
});


The functions in Cache are:

get()

H 10

Q "result from get"

post()

Q "result from post"

rtweed

unread,
Mar 22, 2025, 5:47:41 AMMar 22
to Enterprise Web Developer Community
Node.js (and all server-side JS runtimes) is a single-threaded process.  You're trying to integrate with Cache using mg-dbx-napi in completely the wrong way: it isn't going to work the way you expect or want.

Rather than trying to explain in detail why this is the case, please save yourself a lot of time and effort and use my QOper8p-cp (child process) module which will give you exactly what you need.  As I said in my previous response, this is all covered in detail in the mg-showcase repository so please study the documentation and tutorials there. 

I would recommend you use Fastify rather than Express and then you can use my Fastify integration of QOper8 instead of trying to build it all yourself from scratch.  Start here:


If you want further detailed technical assistance, then contact me about a support contract.

Sorin Petrescu

unread,
Mar 26, 2025, 1:06:12 PMMar 26
to Enterprise Web Developer Community

The example with Fastify and qoper8-fastify does not seem to work on Windows, with node v18.17.0. Does it also work with Cache?  What are the conditions or requirements for support?

"package.json"

{
  "name": "fastify",
  "version": "1.0.0",
  "description": "",
  "main": "fastify.js",
  "scripts": {
    "start": "node fastify.mjs",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "",
  "dependencies": {
    "fastify": "^4.29.0",
    "qoper8-fastify": "^1.4.0"
  }
}


And this is the output:

{"level":30,"time":1743005153761,"pid":15512,"hostname":"WIN-LK41IH4IP6C","reqId":"req-1","req":{"method":"GET","url":"/helloworld","hostname":"localhost:3000","remoteAddress":"127.0.0.1","remotePort":63889},"msg":"incoming request"}
1743005153765: try processing queue: length 1
1743005153765: no available workers
1743005153766: starting new worker
1743005153985: Unable to load onStartup customisation module mgdbx-worker-startup.mjs
1743005153992: {"stack":"Error [ERR_UNSUPPORTED_ESM_URL_SCHEME]: Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'\n    at new NodeError (node:internal/errors:405:5)\n    at throwIfUnsupportedURLScheme (node:internal/modules/esm/load:131:11)\n    at defaultLoad (node:internal/modules/esm/load:82:3)\n    at nextLoad (node:internal/modules/esm/loader:163:28)\n    at ESMLoader.load (node:internal/modules/esm/loader:603:26)\n    at ESMLoader.moduleProvider (node:internal/modules/esm/loader:457:22)\n    at new ModuleJob (node:internal/modules/esm/module_job:64:26)\n    at #createModuleJob (node:internal/modules/esm/loader:480:17)\n    at ESMLoader.getModuleJob (node:internal/modules/esm/loader:434:34)\n    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)","message":"Only URLs with a scheme in: file, data, and node are supported by the default ESM loader. On Windows, absolute paths must be valid file:// URLs. Received protocol 'd:'","code":"ERR_UNSUPPORTED_ESM_URL_SCHEME"}
1743005153997: response received from Worker: 0

rtweed

unread,
Mar 27, 2025, 12:07:47 PMMar 27
to Enterprise Web Developer Community
For details on support, contact me by email - my address is on our web site

Rob

Sorin Petrescu

unread,
Apr 22, 2025, 6:07:01 AMApr 22
to enterprise-web-de...@googlegroups.com

Hi Rob,

I’ve sent the request  to the support email on March 30th. Is the email address correct rtweed @ mgateway.com? Do you think there’s a chance to get this up and running for Cache and Windows?

thank you.


--
You received this message because you are subscribed to a topic in the Google Groups "Enterprise Web Developer Community" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/enterprise-web-developer-community/mPmD3PgFM4Y/unsubscribe.
To unsubscribe from this group and all its topics, send an email to enterprise-web-develope...@googlegroups.com.
To view this discussion, visit https://groups.google.com/d/msgid/enterprise-web-developer-community/9e5f8211-d41c-41bc-b987-7dba0839c045n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages