Throwing user data over the line printer "wall"

[Jonathan Rosen]

I have a problem that I keep trying to solve that has stumped me at every turn.

I have setup a bash script at the Unix level of our d3/Pick implementation, and am trying to create a system where every print function in PICK can be emailed to the user who initiated the job, by selecting a printer queue that creates PDFs.

The printer is setup as a line printer with a reference to a file path in Unix, that is actually a bash script. The bash script takes the input from the line printer, writes it to a temporary file, and then emails it to the user/copies it to the network.

i.e. 90,90,0,S290,lp.unix,(/tmp/mybashfile)

Through the use of shell variables, I can also send dynamic data to the bash script.... i.e. 90,90,0,S290,lp.unix(/tmp/mybashfile @{USER})

However, when the printer is first started, it "locks in" the value of the session variables as they exist at the time the printer is started.

Does anyone have any clever ideas on how I can refresh these values for the line printer on each subsequent print job? This really only needs to be done on each user login, and obviously a separate queue needs to be assigned to each user.

Does anyone else have any clever strategies for "throwing data over the PICK/Unix" wall for print jobs?

Tony Gravagno had a really old post that mentioned setting up a separate spooler that writes to the file system:https://groups.google.com/forum/#!topic/comp.databases.pick/rP5GHj1YQh0

However, does this require changing the main spooler, or can a secondary spooler be used just for pdfs? Tony mentioned "hooks" regarding hdr:peqs, but didn't elaborate, and that is the whole game, since hdr:peqs describes who sent the print job (thus the "wall" that needs to be crossed).

[Tony Gravagno]

Welcome to the group!

Off the top of my head (involved in other projects at the moment) I'm thinking PrintWizard would be an awesome solution to this. All jobs can go to the same queue with a small header for email purposes. PW will strip the header, generate a PDF, and then email per the details.

One queue for every user? Nah, overkill.

I can offer to put that together as a service. No time right now for a deep solution like this in a forum.

Any help?

Regards,

T

[geneb]

On Thu, 12 Jul 2018, Jonathan Rosen wrote:

> I have a problem that I keep trying to solve that has stumped me at every
> turn.
>
> I have setup a bash script at the Unix level of our d3/Pick implementation,
> and am trying to create a system where every print function in PICK can be
> emailed to the user who initiated the job, by selecting a printer queue
> that creates PDFs.
>

I kind of do what you're after, but I only have a small # of users that
require emailed print jobs, so I've got printer queues dedicated to them.
I can show you how I did that if you're interested.

g.


--
Proud owner of F-15C 80-0007
http://www.f15sim.com - The only one of its kind.
http://www.diy-cockpits.org/coll - Go Collimated or Go Home.
Some people collect things for a hobby. Geeks collect hobbies.

ScarletDME - The red hot Data Management Environment
A Multi-Value database for the masses, not the classes.
http://scarlet.deltasoft.com - Get it _today_!

[Ian McGowan]

I do something like this in Unidata, but the printer definitions have some extra parameters that get passed to the lp command there, so there's an extra "channel" to get some data out of your basic program to the bash script.  Basically you can add "-o i...@simbian.org" and the bash script will get the email address passed thru.  I use "-o OWNER" to mean dynamically look up the current user, but from my vague memories of the Pick spooler, you're SOL for that approach.

https://github.com/ianmcgowan/scripts/blob/master/pdfmail.sh

Can you add "env >> /tmp/spool.log" to your script and see if there are any environment variables that can be leveraged?  If there's something available to the bash script (like a port#) that can be tied back to the user, then you can populate the email address in a work file when the user logs in, and the script can pull from that work file.  Grasping at straws..

What about adding the path of a named pipe as a parameter to be sent to the script?  Before you print, you write some information to the named pipe (e.g. an email address), and then the bash script will read from that named pipe when it gets called and pop it off.  I can see things going badly wrong with multiple processes writing to the named pipe - timing is a problem.

If all of that is too much of a hack to think of, how about printing to a hold entry, and then having a process that picks up that hold file and runs the bash script passing in the path to the file and any other parameters needed?

On Thursday, July 12, 2018 at 12:58:20 PM UTC-7, Jonathan Rosen wrote:

[Jonathan Rosen]

Ian, what you are describing is exactly what I'd like to do. Unfortunately, this seems impossible in D3/PICK. I dumped all the env variables, but didn't see anything promising.

We already have what Tony is describing with PrintWizard, to a degree. The problem is that if you do LIST MYFILE SAMPLING 1 LPTR from TCL, there is no way to frontrun that process. A fair number of our reports have been built with what is effectively the same as running that command from TCL (no way to frontrun the print job with a data header at the top), and therefore it would be tedious at best to add the functionality for data headers in the print jobs.

Assigning queues to individual users also isn't ideal, since some reports that run automatically need to be sent to a user who is different than the one running the report.

The hold file that Ian describes is the closest I've come to thinking through a solution that could solve this problem with current D3/Pick APIs (thus my reference to HDR:PEQS above).

It would look like this:

1) user prints to hold file

2) from unix side, pull down hdr:peqs and peqs data, release print job to printer

3) lookup username from hdr:peqs and email file to user

4) do the same thing again

However, this could potentially be a memory management nightmare, especially given some of the printing volumes we do on occasion. It has the potential to crush system performance and cause a whole host of other issues.

What is frustrating is that it is obviously possible to have this functionality (as Ian said it's a function in Unidata), but Rocket doesn't seem very interested in solving this problem. Effectively, in order to get sane and comprehensive virtual printing, you have to kludge the hell out of your system, which shouldn't be necessary. I can't imagine it would take all that much programming to bake this into D3/PICK's virtualization stack.

[Jonathan Rosen]

I was thinking a little more about Ian's suggestion regarding ports. That could actually work.

The user's active port could be written to the Unix file system from PICK (probably using some pass through function to Unix layer), and then there would have to be some way to log what port the print spooler received the job on. This actually seems doable; not sure how complex it would be though? Any thoughts on PICK/Unix functions that could be used to do this? I can access all Unix functionality from the bash script that receives the print job info from the spooler.

i was also wondering if anyone is familiar with built in functionality for Accuterm to record and send back session variables that exist apart from the server? So for example, log into PICK, write session var to telnet session with user's port information, then when print job is received on port x, tell Unix to read back Accuterm session variable from telnet session. I'm not really sure if this is possible or not...but I'm trying to get creative here.

On Thursday, July 12, 2018 at 1:51:27 PM UTC-7, Ian McGowan wrote:

[Ian McGowan]

I think having two spoolers in the picture is the challenge.  By the time you get to the unix spooler you've lost a bunch of context about who ran the report.  You just have a file and a destination but you need some kind of side-channel to pass extra information to the bash script.

One really klugey idea is to have a printer per port, and then when someone logs on, you write to /tmp/portXX the login id/email address of the person logging in that the bash script can pickup (do you need XX bash wrappers too? ug).  So you have to reserve as many printers as you have ports for that to work.  Then you can have more virtual printers in a higher range (900-999) that are hard coded to go to specific email addresses (say 909 is a distribution list for accounting).

There's so much code with so many assumptions about how the spooler works that I doubt that can easily be changed.  I do remember printing getting so much easier on moving from Ultimate to Unidata.  It is sort of magical for the users when they run a report that used to physically print and it shows up nicely formatted in their inbox, so it's worth the struggle.

--
You received this message because you are subscribed to
the "Pick and MultiValue Databases" group.
To post, email to: mvd...@googlegroups.com
To unsubscribe, email to: mvdbms+un...@googlegroups.com
For more options, visit http://groups.google.com/group/mvdbms

[Wols Lists]

On 13/07/18 03:55, Jonathan Rosen wrote:
> I was thinking a little more about Ian's suggestion regarding ports.
> That could actually work.
>
> The user's active port could be written to the Unix file system from
> PICK (probably using some pass through function to Unix layer), and then
> there would have to be some way to log what port the print spooler
> received the job on. This actually seems doable; not sure how complex it
> would be though? Any thoughts on PICK/Unix functions that could be used
> to do this? I can access all Unix functionality from the bash script
> that receives the print job info from the spooler.

Changing tack somewhat, can the D3 spooler put the print job in a hold
file like UV? And can you specify the name of that file? It's easy in
UV, I know.

Then you just run a script over the hold file, "for each file in hold,
get user from file name, print file to pdf, delete file".

Cheers,
Wol

[Ryengoth]

 The way I solved this in our D3/Linux system decades ago was to create application printer profiles that write to /tmp/ and ran LPR from the printing subroutine based on the printer profile in the application. Print Wizard is your best option to get something solid working for multiple media outputs. Hacking up the D3 spooler setup for one-off routing is not the best idea IMHO.

Glen

[Patrick Payne]

I have always used the assignfq command (http://www.d3ref.com/?token=assignfq) which basically bypasses the real spooler and despools the file directly.   The nice thing with assignfq is it does bypass the spooler, therefore there is no waiting/polling to see if it is done.  You can also extend this and adjust the command on the fly to pass params.   It is also fast, meaning the executing of the assignfq is nearly instant.

We built  two new verbs as it then allowed you to even do tcl reports (SP-ASSIGN-PDF, SP-CLOSE-PDF).  There was a third command show-pdf that would just launch IE and point the url at the pdf (which was in a exposed apache directory).  We went this way instead of emailing.

btw - jBase 5.7 added the same type of feature: https://docs.jbase.com/PN5_60673.  The idea here is to offer a non-mv spooler option and allow you the developer to spool to a file and do whatever with the output you wish. 

SP-ASSIGN-PDF

OPTIONS=''

QUEUE.ID=@USER.NO

PATH='/dbms/tmp/pcmnd/':QUEUE.ID:'.pcl'

EXECUTE '!rm /dbms/tmp/pcmnd/':QUEUE.ID:'.pcl'

OPTIONS<4>='cat > /dbms/tmp/pcmnd/':QUEUE.ID:'.pcl'

* HERE I WOULD ADJUST WHAT FORM QUEUE TO USE.  BEFORE EVERYTHING SWITCHED TO THIS TYPE OF PRINTING

* I WOULD ADD 700 TO THE USER TO GET IT TO BE UNIQUE

* LATER WHEN EVERYTHING WAS USING THIS ADJUSTED IT TO BE THE USERS PORT.  1-1 MATCHING

IF @USER.NO > 700 THEN

    SPOOLER.NO = @USER.NO

END ELSE

    SPOOLER.NO = @USER.NO

END

CMND='ASSIGNFQ ':SPOOLER.NO:',hp-lzriii(cat>>':PATH:')'

PRINT CMND

EXECUTE CMND

EXECUTE 'TERM ,,,,0,,,61' ;* REMOVE LEADING FF AND SET PAGE LENGTH

EXECUTE 'SP-SKIP F':SPOOLER.NO:' 0'

EXECUTE 'SP-ASSIGN F':SPOOLER.NO

EXECUTE 'SP-ASSIGN ?'

SP-CLOSE-PDF

CMND = SENTENCE()

IF INDEX(CMND,'[NOSEND',1) THEN NO.SEND=1 ELSE NO.SEND=0

EXECUTE 'SP-CLOSE'

LINE.NO=@USER.NO

CMND="!perl -i.org -pe 's/\n/\r\n/' /dbms/tmp/pcmnd/":LINE.NO:".pcl"

EXECUTE CMND

* DO WHATEVER YOU WANT WITH THE FILE, PCL2PDF, EMAIL IT, ETC

* which our version we used pcl2pdf from http://www.visual.co.uk/pcl2pdf.html to create the pdf, and then moved

* it to a directory that was exposed via apache.  We would then launch the browser and point the url at the pdf.

* This was faster and cleaner than emailing and worked well with nationwide distributed system.

[Bob Rasmussen]

As I understand it, the key problem is how to do something user-specific
(or tty-port-specific) in printing. Right?

What about taking advantage of d3's passthrough print mechanism? It easily
sends a print job through the user's terminal emulator (thus session
specific), intended to go to that terminal's printer. If you run AnzioWin
(our product) as your terminal emulator, you can use its built-in Print
Wizard feature to do the PDF generation and emailing. Or use Accuterm to
receive the passthrough print job and hand it off to Print Wizard on the
user's PC, where it can similarly handle the PDF generation and emailing.

Regards,
....Bob Rasmussen, President, Rasmussen Software, Inc.

personal e-mail: r...@anzio.com
company e-mail: r...@anzio.com
voice: (US) 503-624-0360 (9:00-6:00 Pacific Time)
fax: (US) 503-624-0760
web: http://www.anzio.com
street address: Rasmussen Software, Inc.
10240 SW Nimbus, Suite L9
Portland, OR 97223 USA

[Ryengoth]

  Native slave printing is an option and easy to do if you only need to send ASCII data. If you need to send PS with embedded binary blobs it could blow up your session.

[Bob Rasmussen]

On Fri, 13 Jul 2018, Ryengoth wrote:

>   Native slave printing is an option and easy to do if you only need to send ASCII data. If you need to send PS with
> embedded binary blobs it could blow up your session.

OTOH, if your printing programs generate PCL reports (instead of
PostScript reports), native slave printing works, and Print Wizard
translates the PCL to PDF.

[Patrick Payne]

When I was going over this with Peter Schellenbach he was involved with the Aux Passthru for D3 which does bypass the spooler.  I believe the assignfq was added to gain access to the same "bypass the spooler" feature.

btw - This process overall worked pretty well.  I had it working just fine on D3 7.3 on Aix.

[Tony Gravagno]

Jonathan Rosen wrote:

>am trying to create a system where every print function in PICK can be emailed to the user who initiated the job

It seems to me that this is being approached synchronously where perhaps it should be async:

1) User generates report

2a) Report gets printed

2b) Report gets emailed

2c) anything else gets done with the report...

OK, the process of generating a report is called Spooling.

The process of doing soething with that report is called De-Spooling.

Or call it EnQueuing and DeQueueing, whatever...

So a user does an SP-ASSIGN and sends spools one or more jobs.

Use SP-ASSIGN HSF20, to Spool the job into a hold file on formqueue 20, and leave it there.

Every print job has the User ID associated, and User email and other info can be put in the Users file or somewhere else application-specific.

That is your link between the job and the person who created it.

Now to De-Spool...

Use a phantom runner to periodically loop through jobs on F20.

SP-ASSIGN F21 or some other queue dedicated to generating a PDF

SP-EDIT #UMS

- # is the job#

- M suppresses further prompts

- S means Spool .. really DeSpool, but D already means Delete...

(Note, I can't take the time right now to set this up to QA, and that may not be the exact/correct sequence.)

So you've now De-Spooled that job to PDF and I believe the formqueue of the job has been reassigned to F21, so it won't get selected again.

You now have a PDF and a User ID, get the email address via lookup, and use whatever tool that you prefer to do the email with the PDF as an attachment.

On completion, DELETE PEQS # to remove the job.

Or SP-EDIT #MD

Or SP-EDIT #MSZ on the initial de-spool and the job will be deleted immediately on completion of the PDF generation

Is that a solution to this challenge?

HTH

T

[Patrick Payne]

Tony this is what I was recommending.  The item to worry about is knowing when the spooler is done.  Once I found the assignfq command this no longer became an issue.  Therefore to your point.

EXECUTE 'assignfq 70,hp-lzriii(cat>>/tmp/70.printer.file)'

EXECUTE 'SP-ASSIGN F70'

EXECUTE 'LIST-RUNTIME-ERRORS'

EXECUTE 'SP-CLOSE'

** Due to it bypassing the spooler this entire process is now async and your /tmp/70.printer.file is filled.

Items to watch for with assignfq

1. You must clean up your files first.  You can see me doing a >> vs a > in the command.  This will make this concatenate jobs correctly.  Therefore you should clean up your entries first

2. There is some confusion needing to retain the concept of a form queue.  You must think this thru.

Letting the spooler get if first will work but I wasn't sure about the timing.  I do know then spooling via startptr to a file you do have a timing issue.  I was never really into the spooler therefore you may have better undertsanding of that.  I know on jBase 5.6 I had to go back to relying on the spooler (and had the feature added to 5.7). 

 I think I modified my command to do something like

cat >>/tmp/70.printer.file; echo "done" to /tmp/70.printer.file.done

This allowed me to know when it was dumped.  I also think my issue was my print jobs had binary data (pcl printer fonts) dumped via the u3f oconv which meant I could not directly play with the jobs in the spooler.

Good luck on your test =)

[Jonathan Rosen]

Thanks for all the help everyone. A lot of really good suggestions here.

I think what Patrick has proposed is the closest I've seen to solving the problem as I've outlined it. The only slight hickup we may have is in the number of available form queues. We have a lot of physical printers already on our system.

Is there a maximum number of form queues available for assignfq, or does the ad-hoc nature of this command literally allow you to assign as many form queues as there are numbers?

[Jonathan Rosen]

Looks like D3 does not care about how many assignfq form queues there are (or at least there are enough that i doesn't matter for our org).

Thanks for your help Patrick! This is a really valuable solution to this problem!

[Jonathan Rosen]

Patrick-

With your SP-ASSIGN-PDF, AND SP-CLOSE-PDF verbs, it seems this requires you to run SP-CLOSE-PDF to generate your files, correct?

Another way to do this would be to point ASSIGNFQ at a bash script, that would then stream out the file to a temporary file in a directory specific to the user/port. The bash script could be copied from a template script to the name of the user's queue  in SP-ASSIGN-PDF.

CMND='ASSIGNFQ ':SPOOLER.NO:',hp-lzriii(/tmp/prnt_':'QUEUEID':')'

This would allow for the generation of the PDFs without running SP-CLOSE-PDF.

Just thinking about how to make this even more automated...

[Patrick Payne]

Yes you could, but be careful as it is not really how pick wants to close out a job.  In my case there was programs on the system that would be executed before print jobs to set paper size, orientation, fonts, etc based on the size of the reports.  Therefore the code would look like this

EXECUTE 'SP-ASSIGN-PDF'

EXECUTE 'SET.PRINTER.SIZE.LEGAL'; * sends the pcl codes to set to legal and landscape, needed this included in the print job.

EXECUTE REPORT.COMMAND

EXECUTE 'SP-CLOSE-PDF'

You can see in the command I used a >> to concatenate jobs which made this all work.  In addition, we were able to go thru the system and basically replace any SP-ASSIGN and SP-CLOSE statement with the two above commands and nearly every printing function worked.  If you have complete control over what you are doing then your idea would work just fine.

With above you can actually do this also from tcl

>SP-ASSIGN-PDF

>SORT MD LPTR

>SP-CLOSE-PDF