Question about YottaDB calling node.js

[Kevin Toppenberg]

I have created javascript code (veridigm.js) that is run by node.js which uses puppeteer to retrieve information from our electronic pharmacy prescribing program website, via web scraping.  Getting this working was challenging, but I finally got it (with help). 

This works fine when I run it from the command-line from inside a terminal shell. I am running the puppeteer in a headless mode, such that it spins up chromium, but it doesn't need any display to show to etc. 

Here is my mumps code that is not working:

BATCHMEDS(OUT,PATLIST) ;
  ;"Purpose: call out to node.js / puppeteer and scrape Veridigm for meds for list of patients
  ;"INPUT: OUT -- PASS BY REFERENCE.  AN OUT PARAMETER.  FORMAT:
  ;"
  ;"       PATLIST -- PASS BY REFERENCE.  FORMAT:
  ;"          PATLIST(#)='LNAME^FNAME^DOB^SDT^EDT'  
  ;"                note: SDT is optional start of search date range.  
  ;"                      EDT is optional end of search date range.  
  ;"                      Date format is mm/dd/yyyy (e.g. 5/12/2024)
  ;"          Note: Due to limitations in GTF^%ZISH, the index numbers (#) MUST be
  ;"                sequential integers, e.g. 1,2,3,4.   1,3,5,7 would NOT work
 
  ;"Result: 1^OK or -1^ErrorMessage
  NEW RESULT,CRED SET RESULT=$$GETCRDNTL(.CRED,"Veridigm")
  IF RESULT'>0 GOTO BMDN
  NEW HFSPATH SET HFSPATH=CRED("HFS PATH")
  IF $EXTRACT(HFSPATH,$LENGTH(HFSPATH))'="/" SET HFSPATH=HFSPATH_"/"
  NEW OUTPATH SET OUTPATH=HFSPATH_"output/"
  NEW INFNAME SET INFNAME=$$UNIQUE^%ZISUTL("patients.csv")
  SET RESULT=$$ARR2HFS^TMGIOUT3("PATLIST",HFSPATH,INFNAME)  
  IF RESULT'>0 GOTO BMDN
  ;      
  NEW CMDSTR SET CMDSTR=""
  SET CMDSTR=CMDSTR_"cd "_HFSPATH_" ; "
  SET CMDSTR=CMDSTR_"node verdigm.js "
  SET CMDSTR=CMDSTR_"--headless "
  SET CMDSTR=CMDSTR_"--ptlist="_"./"_INFNAME_" "
  SET CMDSTR=CMDSTR_"--username="_CRED("USER")_" "
  SET CMDSTR=CMDSTR_"--password="_CRED("PASSWORD")_" "
  SET CMDSTR=CMDSTR_"--URL="_CRED("URL")_" "
  SET CMDSTR=CMDSTR_"--debug=2 "
  ;    
  NEW P SET P="TEMP" OPEN P:(COMMAND=CMDSTR:readonly)::"pipe" USE P
  NEW PIPEOUT,X,IDX SET IDX=1
  FOR  QUIT:$ZEOF  DO
  . READ X IF X=""&$ZEOF QUIT  
  . SET PIPEOUT(IDX)=X,IDX=IDX+1
  CLOSE P USE $P
  NEW PIPERESULT SET PIPERESULT=$ZSYSTEM&255  ;"get result of execution. (low byte only)
  ;"QUIT OUT BASED ON PIPERESULT
  ;"Now get back results
 - snip -
 

When I run this, here is what CMDSTR looks like (user name and password changed)

CMDSTR=cd /opt/worldvista/EHR/nodejs/veridigm/ ; node verdigm.js --headless --ptlist=./patients_1849074702-5.csv --username=xxxxxx --password=xxxxxxxxx --URL=https://eprescribe.allscripts.com/Login.aspx --debug=2

When I run this, I am getting back an error as below.

PIPEOUT
}~1 = node:internal/modules/cjs/loader:1189
}~2 = "  throw err;"
}~3 = "  ^"
}~4 = ""
}~5 = Error: Cannot find module '/opt/worldvista/EHR/nodejs/veridigm/verdigm.js'
}~6 = "    at Module._resolveFilename (node:internal/modules/cjs/loader:1186:15)"
}~7 = "    at Module._load (node:internal/modules/cjs/loader:1012:27)"
}~8 = "    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:158:12)"                  
}~9 = "    at node:internal/main/run_main_module:30:49 {"
}~10 = "  code: 'MODULE_NOT_FOUND',"
}~11 = "  requireStack: []"
}~12 = }                  
}~13 = ""                 
}~14 = Node.js v22.2.0

Does anyone know if this has anything to do with running this from a pipe command.  I think I have been told that a pipe is more efficient, because an entire shell doesn't have to be created.   But would this then limit node?

I am pretty new to using node, so any help would be appreciated.

Thanks in advance

Kevin

[Benjamin Irwin]

You might want to try the following.  And use a direct address for the csv file.

CMDSTR=node /opt/worldvista/EHR/nodejs/veridigm/verdigm.js --headless --ptlist=./patients_1849074702-5.csv --username=xxxxxx --password=xxxxxxxxx --URL=https://eprescribe.allscripts.com/Login.aspx --debug=2

[K.S. Bhaskar]

While I know little about JavaScript, two general things to check, Kevin:

• Is there a difference in the current directory between the stand-alone program that works, and the one which fails in the PIPE?
• Is there a difference in the environment variables between when you run the stand-alone program, and the failing one in the PIPE?

Regards

– Bhaskar

[Kevin Toppenberg]

Thank you Ben and Bhaskar,

Ben, I have tried adding the paths, but now am having a problem with the Pipe command due to CMDSTR being > 255 chars.  I'll ask about this over on the Everything Mumps forum.

Bhaskar, those are good thoughts, and I investigate that next.

Kevin

[K.S. Bhaskar]

Try replacing the ";" in the command string with "&&", just in case the cd command is failing.

Also https://gitlab.com/YottaDB/DB/YDB/-/blame/master/sr_unix/ydbenv.mpt#L733 and https://gitlab.com/YottaDB/DB/YDB/-/blame/master/sr_unix/ydbenv.mpt#L398 are examples of starting a shell in a PIPE device and feeding it commands.

Regards

– Bhaskar

[David Blackstone]

One Idea I have. Perhaps if its a character issue, then maybe you can just write a shell script that you call from the pipe that builds the command you want to execute nodejs. Like for example build CMDSTR as

CMDSTR=". /path_to_script.sh"_" "_INFNAME_" "_CRED("USER")_" "_CRED("PASSWORD")"_" "_CRED("URL"). Then just have the shell script add the other details that are hardwritten rather than variable, and execute the command.

I'm not exactly sure how you'd get the output from this but perhaps its doable.

- David

[Kevin Toppenberg]

Thanks everyone for their help.

I'm embarrassed to say I figured out the problem.  It was on this line....

SET CMDSTR=CMDSTR_"node verdigm.js "

I misspelled "veridigm.js"

But then to add insult to injury, I realized that the website is "veradigm" (with an 'a').  So went through and renamed the folder and all the javascript files to veradigm.js.

And now I messed up something in the node modules that must have set something when I installed them with the different directory name....  But that is a bug to stamp out another day.

Thanks again   :-)

Kevin

[David Whitten]

Aren’t computers so fun to use ?

--
--
http://groups.google.com/group/Hardhats
To unsubscribe, send email to Hardhats+unsubscribe@googlegroups.com

---
You received this message because you are subscribed to the Google Groups "Hardhats" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hardhats+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hardhats/1d344618-bb28-4d49-a354-86bff2e678cdn%40googlegroups.com.