JZOS Java CLASSPATH

[Andrew Rowley]

The JCL supplied with the JZOS batch launcher has a section to add jars
to the classpath:

# Add Application required jars to end of CLASSPATH
for i in "${APP_HOME}"/*.jar; do
CLASSPATH="$CLASSPATH":"$i"
done

My understanding is you should be able to use a wildcard instead and the
JVM will add all files ending with jar or JAR to the classpath,

e.g. CLASSPATH="$CLASSPATH":"${APP_HOME}"/*

but it doesn't seem to work. The classes are not found. If I display the
classpath from within the program I can see the directory including the
wildcard. If I copy and paste that entry and use it in "ls" it lists the
jars.

What an I missing? Does the z/OS JVM support the wildcard syntax (under
JZOS)?

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to list...@listserv.ua.edu with the message: INFO IBM-MAIN

[Denis]

Hi Andrew,

your understanding is not correct.
Wildcards do not and have never worked with CLASSPATH.

Hope it helps.
Denis.

[Michael Klaeschen]

Composing the classpath via for loop in STDENV DD is a script run by
/bin/sh and therefore the term "${APP_HOME}"/*.jar is implicitly evaluated
to the correct file names by the shell.
In contrast, just giving the classpath as a variable value does not
evaluate to the files referenced by "*". It just appends the asterisk
symbol to the variable's value -- which does not help you unless your
classes are all contained in the one file named "*" (asterisk symbol as
file name works in z/OS Unix.....however it is something you sure do not
want.).

Cheers
Michael

_________________________
_________________________
____________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
_________________________
_________________________
____________________


Basler Sachversicherungs-AG
Amtsgericht Bad Homburg v.d.H., HRB 9357 | USt-ID-Nr. DE 276021973
Basler Straße 4, 61345 Bad Homburg v.d.H.
Vorstand: Dr. Jürg Schiltknecht - Vorsitzender, Markus Jost, Ralf Stankat,
Dr. Alexander Tourneau, Julia Wiens
Aufsichtsratsvorsitzender: Peter Zutter

Basler Lebensversicherungs-AG |
Amtsgericht Hamburg, HRB 4659 | Ust-ID-Nr. DE 276021973
Ludwig-Erhard-Straße 22, 20459 Hamburg
Vorstand: Dr. Jürg Schiltknecht - Vorsitzender, Markus Jost, Ralf Stankat,
Dr. Alexander Tourneau, Julia Wiens
Aufsichtsratsvorsitzender: Peter Zutter

Basler Versicherung AG Direktion für Deutschland |
Amtsgericht Bad Homburg v.d.H., HRB 1228 | USt-ID-Nr. DE 281452875
Basler Straße 4, 61345 Bad Homburg v.d.H.
Hauptbevollmächtigter für Deutschland: Dr. Jürg Schiltknecht

Basler Leben AG Direktion für Deutschland |
Amtsgericht Bad Homburg v.d.H., HRB 1229 | Ust-ID-Nr. DE 281452875
Basler Straße 4, 61345 Bad Homburg v.d.H.
Hauptbevollmächtigter für Deutschland: Dr. Jürg Schiltknecht

______________________________________________________________________
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
______________________________________________________________________

[Andrew Rowley]

On 26/04/2017 09:19 PM, Denis wrote:
> Hi Andrew,
>
> your understanding is not correct.
> Wildcards do not and have never worked with CLASSPATH.

I was reading:
https://docs.oracle.com/javase/6/docs/technotes/tools/windows/classpath.html
"Understanding class path wildcards"

I thought it was working on another platfom, I will have to go back and
check.

Maybe the IBM JVM is different to the Oracle one? I tried to find IBM
documentation, but it is not so explicit.

[Allan Staller]

<snip>

Maybe the IBM JVM is different to the Oracle one? I tried to find IBM documentation, but it is not so explicit.

</snip>

No it is not! Oracle has a lab in France that is in charge of the porting process.
IBM merely repackages what Oracle provides from the porting lab into PTFs.



::DISCLAIMER::
----------------------------------------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only.
E-mail transmission is not guaranteed to be secure or error-free as information could be intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or may contain viruses in transmission. The e mail and its contents
(with or without referred errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Views or opinions, if any, presented in this email are solely those of the author and may not necessarily reflect the
views or opinions of HCL or its affiliates. Any form of reproduction, dissemination, copying, disclosure, modification,
distribution and / or publication of this message without the prior written consent of authorized representative of
HCL is strictly prohibited. If you have received this email in error please delete it and notify the sender immediately.
Before opening any email and/or attachments, please check them for viruses and other defects.

----------------------------------------------------------------------------------------------------------------------------------------------------

[Denis]

Hi Andrew,

I have been working with Java since Java 1.1. So it seams wildcard work since JDK 6. However *.jar does not work, you need to use dir1/* to get what you wanted with dir1/*.jar.
In addition it will not search in subdirectories, so you have to specify every subdirectory.
I have not tried what is decribed in the document you have referenced.

Denis.


-----Original Message-----
From: Andrew Rowley <and...@BLACKHILLSOFTWARE.COM>
To: IBM-MAIN <IBM-...@LISTSERV.UA.EDU>

[Andrew Rowley]

On 26/04/2017 10:37 PM, Allan Staller wrote:
> <snip>
> Maybe the IBM JVM is different to the Oracle one? I tried to find IBM documentation, but it is not so explicit.
> </snip>
>
> No it is not! Oracle has a lab in France that is in charge of the porting process.
> IBM merely repackages what Oracle provides from the porting lab into PTFs.
>

With more testing today, it looks like the culprit is the JZOS batch
launcher. It seems that it doesn't support the Java 1.6+ classpath
wildcards.

Invoking Java direcly from BPXBATCH worked, the wildcard worked
correctly for the CLASSPATH environment variable as well as the
-classpath command line option.

I was curious to see whether you could get around it by passing the
-classpath as a JVM option, but it seems not. I suspect JZOS takes the
job of setting the classpath away from the JVM entirely and it is a bit
limited in what is supports. I wonder whether this is a deliberate
restriction, or something that wasn't updated for 1.6+ behaviour?

[Peter Hunkeler]

>> No it is not! Oracle has a lab in France that is in charge of the porting process.
>> IBM merely repackages what Oracle provides from the porting lab into PTFs.
>>
>With more testing today, it looks like the culprit is the JZOS batch
>launcher. It seems that it doesn't support the Java 1.6+ classpath
>wildcards.

The input in the JCL where you specify the CLASSPATH (and other variavles) is intrpreted by the shell (sourced). That means the shell will see the * and expand it to the list of files separated by blanks.


You need to escape the * so that the shell will assign it to the environment variable instead of "resolving" it. Try:


CLASSPATH="$CLASSPATH":"${APP_HOME}"/\*


That is "slash" "backslash" "*"




--Peter Hunkeler

[Andrew Rowley]

On 27/04/2017 09:49 PM, Peter Hunkeler wrote:
> You need to escape the * so that the shell will assign it to the environment variable instead of "resolving" it. Try:
>

I don't think so. I have a small Java program to display the CLASSPATH
environement variable and what it resolves to:

import java.net.URL;
import java.net.URLClassLoader;

public class ClassPath {
public static void main (String args[]) {
System.out.println(System.getenv("CLASSPATH"));
ClassLoader cl = ClassLoader.getSystemClassLoader();
URL[] urls = ((URLClassLoader)cl).getURLs();
for(URL url: urls){
System.out.println(url.getFile());
}
}
}

Under JZOS:

JVMJZBL1023N Invoking ClassPath.main()...
JVMJZBL1024N ClassPath.main() completed.
JVMJZBL1021N JZOS batch launcher completed, return code=0
/home/andrewr/java/target:/usr/lpp/java/J8.0/lib:/usr/lpp/java/J8.0/lib/ext:/home/andrewr/java/lib/*:
/home/andrewr/java/target/
/VERSYSB/usr/lpp/java/J8.0/lib/
/VERSYSB/usr/lpp/java/J8.0/lib/ext/
/home/andrewr/java/lib/*
/home/andrewr/

Under BPXBATCH:

java/target:java/lib/*
/home/andrewr/java/target/
/home/andrewr/java/lib/easysmf-je-1.5.2.jar
/home/andrewr/java/lib/easysmf-je-samples-1.5.2.jar
/home/andrewr/java/lib/javax.mail.jar
/home/andrewr/java/lib/slf4j-api-1.7.21.jar
/home/andrewr/java/lib/slf4j-simple-1.7.21.jar
/home/andrewr/java/lib/jsoup-1.10.2.jar

In both cases I can see the asterisk in the CLASSPATH variable. Under
BPXBATCH the jars in that directory are in the classpath as expected.
Under JZOS the asterisk itself ends up as an entry.

[Kirk Wolf]

Yes, this is a feature of the "java" command JVM launcher that is not
present in the JZOS batch launcher and the "java" command launcher.

Both launchers invoke the JVM launcher API and pass in "-Djava.class.path="
as an option using $CLASSPATH, but the "java" command launcher seems to be
enhanced (somewhat recently) to expand an entry with a basename of "*" to
include all of the jar files in that directory:
http://docs.oracle.com/javase/6/docs/technotes/tools/windows/classpath.html

And yet I fail to see why this is an issue since the JZOS sample JCL shows
how to use the shell language to handle this - with more flexibility.

For example, the "java" command launcher can't do this:

for i in "${MYDIR}"/foo*.jar; do
CLASSPATH="$CLASSPATH":"$i"
done



Kirk Wolf
Dovetailed Technologies
http://dovetail.com

[Carmen Vitullo]

Funny I'm looking at both right now, getting ready for a migration of an application from Java on Unix to Java on z/OS, I've been testing the BCD batch process and the JZOS proc, the scripting seems to work as designed, and seeing how it works with a small test I wonder if that's overkill to allow Java to load all the class libraries ?
I'll need some real testing with an application right now using the defaults I set



APP_HOME=$JAVA_HOME <--- change
CLASSPATH=$APP_HOME:"${JAVA_HOME}"/lib:"${JAVA_HOME}"/lib/ext

# Add Application required jars to end of CLASSPATH

for i in "${APP_HOME}"/*.jar; do
CLASSPATH="$CLASSPATH":"$i"
done
export CLASSPATH="$CLASSPATH":


Carmen

----- Original Message -----

From: "Kirk Wolf" <ki...@DOVETAIL.COM>
To: IBM-...@LISTSERV.UA.EDU
Sent: Thursday, April 27, 2017 12:23:27 PM
Subject: Re: JZOS Java CLASSPATH

[Denis]

Hi Carmen,

as the classpath size affects performance, its definitly a good idea to limit the jars and its contents to what is required.
In addition some customers require a certain order of the jar files in some cases and as such, wildcards are not an option, since they produce unpredictable results with the order of the jars.

Denis.

[Carmen Vitullo]

Thanks Denis, this make sense and that's what I was hoping, I'm trying to port the Unix ksh script file to a jzos process, that seems almost straight forward, I'll be meeting with the developers next week and try to setup a real lfe test for the POC.
I'm wondering, since the class libraries are in a zfs file system, one of my colleagues wanted to test loading the Java filesystem into storage using hiper batch, has anyone attempted this? is this a solution?
thanks

Carmen


----- Original Message -----

[Kirk Wolf]

the JZOS batch launcher does work as designed - it allows you to use a z/OS
Unix shell script to configure everything including CLASSPATH.

What it doesn't do is what the new "java" command line launcher for the JVM
does: it expands directory wildcards in CLASSPATH to point to all jars in
that directory (something easy to do in the JZOS shell script). I see it
as a nice convenience since it is a common thing to do and most Java users
don't know their shell :-)

Kirk Wolf
Dovetailed Technologies
http://dovetail.com

[Carmen Vitullo]

Thanks Kirk - it appears its does, not knowng the enviorment myself since it was designed and developed on HP/UX we surly don't know what's needed and I'm not sure the developers know or care to know, we're looking to optimize these java batch processes eventually, first we need to prove it works and is easily portable, so far I see the scripts themselves need to be modified, they're still using ksh
thanks so much for your insight - this will be interesting, nice to see someone wanting to port from a distributed system to Z :)

[Andrew Rowley]

On 28/04/2017 4:34 AM, Denis wrote:
> as the classpath size affects performance, its definitly a good idea to limit the jars and its contents to what is required.
>

I wouldn't expect the classpath size to have a significant effect on
performance, since Java can read and cache the directories from the jar
files. A quick google search suggests this is the consensus.

The bigger problem of course is management of dependencies.

--
Andrew Rowley
Black Hill Software
+61 413 302 386

[Denis]

Hi Andrew,

One example, the jvm needs to scan every java class for specific annotations. The more unneccessary files in the classpath, the longer that takes. And the cache size is not unlimited, I have seen classpaths with more than 100mb in size.
If shared classloader cache is used, common storage is also not unlimited. As soon as a cache is too small, it uses more cpu time. There is plenty of more stuff.
I am not talking about the java applications that have 5mb jars in the classpath, but once started the classpath usually continues to grow.

Denis.


-----Original Message-----
From: Andrew Rowley <and...@BLACKHILLSOFTWARE.COM>
To: IBM-MAIN <IBM-...@LISTSERV.UA.EDU>

Sent: Fri, Apr 28, 2017 02:08 AM
Subject: Re: JZOS Java CLASSPATH

send email to <a href="mailto:list...@listserv.ua.edu">list...@listserv.ua.edu</a> with the message: INFO IBM-MAIN

[Andrew Rowley]

On 28/04/2017 10:50 AM, Denis wrote:
> One example, the jvm needs to scan every java class for specific annotations. The more unneccessary files in the classpath, the longer that takes.

Is that the JVM doing it, or is it actually done by certain frameworks
that use annotations?

My understanding is that there are some frameworks that scan the
classpath for annotations, and yes in that case it would be wise to
minimize it, but in the more general case the class loader will load the
directories but not the classes until they are actually referenced.

--
Andrew Rowley
Black Hill Software
+61 413 302 386

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,

[Carmen Vitullo]

I've been testing the BPXBATCH , JZOS and BCH (bcp) batch process, so far, performance wise and ease of porting the JZOS for us performs better and seems it would be easier to port to Z, still in the initial stages, so we shall see
thanks Andrew

Carmen


----- Original Message -----

From: "Andrew Rowley" <and...@BLACKHILLSOFTWARE.COM>
To: IBM-...@LISTSERV.UA.EDU

Sent: Thursday, April 27, 2017 8:13:03 PM
Subject: Re: JZOS Java CLASSPATH

On 28/04/2017 3:23 AM, Kirk Wolf wrote:
> And yet I fail to see why this is an issue since the JZOS sample JCL shows
> how to use the shell language to handle this - with more flexibility.
>
> For example, the "java" command launcher can't do this:
>
> for i in "${MYDIR}"/foo*.jar; do
> CLASSPATH="$CLASSPATH":"$i"
> done

I love JZOS, except for this shell script. I feel that a tailored shell
script in every Java batch job is a major obstacle, because as you say
"most Java users don't know their shell". Even more so for most z/OS users.

You are just setting the CLASSPATH environment variable, so yes you can
do the same thing using the java command in BPXBATCH if you need to
(apologies for naming my program ClassPath, which is slightly confusing):

//S1 EXEC PGM=BPXBATCH,REGION=512M
//STDENV DD *
CLASSPATH=java/target
//STDPARM DD *
SH for i in java/lib/slf*.jar; do
CLASSPATH="$CLASSPATH":"$i";
done;
export CLASSPATH="$CLASSPATH";
/usr/lpp/java/J8.0/bin/java ClassPath
//STDOUT DD SYSOUT=*
//STDERR DD SYSOUT=*

Output:
java/target:java/lib/slf4j-api-1.7.21.jar:java/lib/slf4j-simple-1.7.21.jar
/home/andrewr/java/target/
/home/andrewr/java/lib/slf4j-api-1.7.21.jar
/home/andrewr/java/lib/slf4j-simple-1.7.21.jar

It's the handling of the CLASSPATH environment variable that's the issue.

I have been playing around with PROCs to try to simplify things so you
can use more traditional JCL for JZOS jobs (hence my other questions
about JCL substitutions in instream data).
e.g.

//JAVA EXEC PROC=JAVA8G,
// JAVACLS='''MyProgram''',
// APPHOME='''java/target''',
// CLASPATH='''java/lib/*:java/easysmf-je-1-5-2/jar/*'''
//INPUT DD DISP=SHR,...

It is difficult to handle the classpath like this if you need to allow
multiple entries and support any sort of wildcard. It would be much
easier if it followed the same rules as the java command.

[Carmen Vitullo]

so, just for grins I modified the COFXDLF1 exit adding my Java 8_31 and 8_64 zfs's to be managed by DLF
my first thought was ZFS does a pretty good job Caching and I'd be shooting myself in the foot doing this, but my initial test results;



0.06 MINUTES EXECUTION TIME with java zfs in DLF
0.55 MINUTES EXECUTION TIME without

Carmen






----- Original Message -----

From: "Kirk Wolf" <ki...@DOVETAIL.COM>
To: IBM-...@LISTSERV.UA.EDU

Sent: Friday, April 28, 2017 9:33:07 AM
Subject: Re: JZOS Java CLASSPATH

>
>

> A good practice would be to have helper shell scripts that do site
standard setup for JZOS and then call or dot them in from STDENV.

Or simple ones:

CLASSPATH="$CLASSPATH":$(expandJars $MYDIR)

If you feel strongly that JZOS is less usable without wildcard jar
expansion of CLASSPATH, then best to submit an RFE.