Graph database on z/OS?

[John McKown]

I am guessing that most z/OS shops which as a true database environment are
most likely running Db2. There really isn't much else out there since
Oracle decided not to upgrade from 10.2 on z/OS (as best as I can tell).

I have been reading up on graph databases. In particular, I've watched a
few videos on Neo4j. Some of the concepts (nodes or vertices and edges or
relations) seem to be similar to the old network database model, to me. But
it is very interesting. What is really interesting (again to me) is that
Neo4j is both open source (parts AGPL, other parts GPL) and it is written
in Java. Being written in Java makes it interesting since "pure Java" code
can run "for free" (no MSU usage) on a zIIP. Naturally, I have downloaded
the source. But I am struggling to actually get it to compile & pass its
tests on my Linux/Intel system. The build system is in Maven, which I don't
know. But I was thinking that it would be quite interesting to see if Neo4j
could be built and run on z/OS.

So I'm curious if anyone else finds this interesting. And please forgive me
for not saving this for our usual "Friday" discussions of "not really z/OS"
topics.

--
I have a theory that it's impossible to prove anything, but I can't prove
it.

Maranatha! <><
John McKown

----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to list...@listserv.ua.edu with the message: INFO IBM-MAIN

[Graham Harris]

John, are you specifically after a 'graph database', or a 'pure java'
database?

H2 is an example of a pure java database, which is >99% zIIP eligible, in
my experience.

Any pure java library (typically in the form of jar file[s]) should 'just
work' on z/OS (i.e. no porting required).

Havent quite worked out though, if Neo4j is actually a java library, or
something that is "binaried" to specific platforms. There seems to be a
jar file available, but is not clear if that is the actual product, or just
some 'helpers' to interface with java.

Neo4j advertises being capable of generating actual graphs from DOT
directed graph notation, and if that is do-able from a pure java
capability, that is something I have been patiently waiting for for many
many years.....so will be investigating that further, for sure.

[John McKown]

On Thu, Mar 22, 2018 at 5:27 PM, Graham Harris <harr...@gmail.com> wrote:

> John, are you specifically after a 'graph database', or a 'pure java'
> database?
>

​My area of curiosity right now is "graph database", not "java".

>
> H2 is an example of a pure java database, which is >99% zIIP eligible, in
> my experience.
>

​I've used the Derby​ database long ago, when my interest was in learning
some java.

>
> Any pure java library (typically in the form of jar file[s]) should 'just
> work' on z/OS (i.e. no porting required).
>

​That has been my experience too. I wrote a java application on Linux
(using Derby) and just binary FTP'd the resultant jar to Windows, Mac OSX,
and z/OS. It ran identically on all four platforms.​

>
> Havent quite worked out though, if Neo4j is actually a java library, or
> something that is "binaried" to specific platforms. There seems to be a
> jar file available, but is not clear if that is the actual product, or just
> some 'helpers' to interface with java.
>

​I have downloaded the Neo4j source to my Linux home system. As far as
"language files" go, all of them that I saw were Java.​

[Scott Chapman]

The default encoding on z/OS occasionally causes problems. Particularly when doing network I/O. Adding option "-Dfile.encoding=ISO8859-1" in my experience takes care of those issues. Of course you have to deal with ASCII files then, but that's a minor issue.

Scott Chapman

On Thu, 22 Mar 2018 22:27:44 +0000, Graham Harris <harr...@GMAIL.COM> wrote:

>Any pure java library (typically in the form of jar file[s]) should 'just
>work' on z/OS (i.e. no porting required).

[John McKown]

On Thu, Mar 22, 2018 at 11:56 AM, John McKown <john.arch...@gmail.com>
wrote:

> I am guessing that most z/OS shops which as a true database environment
> are most likely running Db2. There really isn't much else out there since
> Oracle decided not to upgrade from 10.2 on z/OS (as best as I can tell).
>
> I have been reading up on graph databases. In particular, I've watched a
> few videos on Neo4j. Some of the concepts (nodes or vertices and edges or
> relations) seem to be similar to the old network database model, to me. But
> it is very interesting. What is really interesting (again to me) is that
> Neo4j is both open source (parts AGPL, other parts GPL) and it is written
> in Java. Being written in Java makes it interesting since "pure Java" code
> can run "for free" (no MSU usage) on a zIIP. Naturally, I have downloaded
> the source. But I am struggling to actually get it to compile & pass its
> tests on my Linux/Intel system. The build system is in Maven, which I don't
> know. But I was thinking that it would be quite interesting to see if Neo4j
> could be built and run on z/OS.
>
> So I'm curious if anyone else finds this interesting. And please forgive
> me for not saving this for our usual "Friday" discussions of "not really
> z/OS" topics.
>
>

​Well, my analysis of the files (using the Linux "file") command seems to
bear out that this is a "pure Java" implementation. I could never the the
Maven project to run to completion. Oh, Maven is a "make" for Java​
projects. But I did find a Linux repository which worked on my Linux/Intel
Fedora system (see below). This came with all the files that I needed to
get the database server started. For the Community version, there basically
isn't much security. All you can do is allow access by creating an ID and
password. There is no way to "grant" any authorities, so everybody who is
defined to the system can do what they want with the data. {shudder}.
Another {shudder}, at least for me, is that the design is "one graph
(database) per server". This has been complained about for years now and is
still a restriction. Of course, since you can have a ginormous graph
database, this is more a backup / recovery concern than a size concern.

Maybe more latter.

​[neo4j]
name=Neo4j Yum Repo
baseurl=http://yum.neo4j.org/stable
enabled=1
gpgcheck=1

[David Crayford]

On 26/03/2018 6:58 AM, John McKown wrote:
> ​Well, my analysis of the files (using the Linux "file") command seems to
> bear out that this is a "pure Java" implementation. I could never the the
> Maven project to run to completion. Oh, Maven is a "make" for Java​
> projects.

Maven is a bit more than just a build tool as it's also a package
manager. It's showing its age a bit now as you can probably tell from
the fact is uses XML.
It's quite extraordinary just how massive the Java ecosystem is! There
are 225,964 unique artifacts indexed on Maven central!!

> But I did find a Linux repository which worked on my Linux/Intel
> Fedora system (see below). This came with all the files that I needed to
> get the database server started. For the Community version, there basically
> isn't much security. All you can do is allow access by creating an ID and
> password. There is no way to "grant" any authorities, so everybody who is
> defined to the system can do what they want with the data. {shudder}.

That's pretty much the same security model for all the NoSQL data bases
I've come across. Redis, MongoDB, Aerospike etc. In theory the database
servers should
be behind a firewall and secured but the reality can be quite shocking
https://www.theregister.co.uk/2017/01/09/mongodb/.

> Another {shudder}, at least for me, is that the design is "one graph
> (database) per server". This has been complained about for years now and is
> still a restriction. Of course, since you can have a ginormous graph
> database, this is more a backup / recovery concern than a size concern.

[John McKown]

On Mon, Mar 26, 2018 at 1:49 AM, David Crayford <dcra...@gmail.com> wrote:

> On 26/03/2018 6:58 AM, John McKown wrote:
>
>> ​Well, my analysis of the files (using the Linux "file") command seems to
>> bear out that this is a "pure Java" implementation. I could never the the
>> Maven project to run to completion. Oh, Maven is a "make" for Java​
>> projects.
>>
>
> Maven is a bit more than just a build tool as it's also a package manager.
> It's showing its age a bit now as you can probably tell from the fact is
> uses XML.
> It's quite extraordinary just how massive the Java ecosystem is! There are
> 225,964 unique artifacts indexed on Maven central!!
>
> But I did find a Linux repository which worked on my Linux/Intel
>> Fedora system (see below). This came with all the files that I needed to
>> get the database server started. For the Community version, there
>> basically
>> isn't much security. All you can do is allow access by creating an ID and
>> password. There is no way to "grant" any authorities, so everybody who is
>> defined to the system can do what they want with the data. {shudder}.
>>
>
> That's pretty much the same security model for all the NoSQL data bases
> I've come across. Redis, MongoDB, Aerospike etc. In theory the database
> servers should
> be behind a firewall and secured but the reality can be quite shocking
> https://www.theregister.co.uk/2017/01/09/mongodb/.

​So NoSQL implies NoSecurity as well. And the "webbies" think this is
acceptable? I don't know whether to laugh (as the house implodes) or cry
(as my money disappears).​

>
>
> Another {shudder}, at least for me, is that the design is "one graph
>> (database) per server". This has been complained about for years now and
>> is
>> still a restriction. Of course, since you can have a ginormous graph
>> database, this is more a backup / recovery concern than a size concern.
>>
>
>

--
I have a theory that it's impossible to prove anything, but I can't prove
it.

Maranatha! <><
John McKown

[David Crayford]

On 26/03/2018 9:11 PM, John McKown wrote:
>> That's pretty much the same security model for all the NoSQL data bases
>> I've come across. Redis, MongoDB, Aerospike etc. In theory the database
>> servers should
>> be behind a firewall and secured but the reality can be quite shocking
>> https://www.theregister.co.uk/2017/01/09/mongodb/.
> ​So NoSQL implies NoSecurity as well. And the "webbies" think this is
> acceptable? I don't know whether to laugh (as the house implodes) or cry
> (as my money disappears).​
>

I think the general ROT for those kind of systems is that the network
defines security. All back-end services should be hidden behind firewalls
and not accessible to the outside world. It's a different world these
days where everything seems to run on docker images orchestrated by
something like kuebernetes and
secured by LDAP or whatever. Nobody dishes out userids unless you need
admin.

[Anne & Lynn Wheeler]

dcra...@GMAIL.COM (David Crayford) writes:
> I think the general ROT for those kind of systems is that the network
> defines security. All back-end services should be hidden behind
> firewalls and not accessible to the outside world. It's a different
> world these days where everything seems to run on docker images
> orchestrated by something like kuebernetes and secured by LDAP or
> whatever. Nobody dishes out userids unless you need admin.

Skip containers and do serverless computing instead; Container
technologies like Docker are very powerful, but require talent you can't
get. Serverless computing provides the same benefits -- with talent you
can actually get
https://www.infoworld.com/article/3265457/containers/why-serverless-is-the-better-option-than-containers.html

we had worked with several people at Oracle on cluster scaleup ... part
of getting cluster scaleup being transferred were mainframe DB2
complaining if I was allowed to continue, it would be at least 5yrs
ahead of them. Over a period of a few weeks, cluster scaleup was
transferred, announced as IBM supercomputer (for technical/scientific
*ONLY*) and we were told we couldn't work on anything with more than
four processors. we leave a few months later. past posts
http://www.garlic.com/~lynn/subtopic.html#hacmp

not long later, we are brought in as consultants by two of the (former
Oracle) people we had worked with ... who were then at a small
client/server startup responsible for something called commerce server,
the startup had also invented this technology they called "SSL" they
wanted to use, the result is now frequently called "electronic
commerce".

As webservers got more complex, there was increasing number of
RDBMS-backed servers (compared to flat-file based implementations) that
had significant larger number of exploits. Part of it was RDBMS were
much more complex & corresponding increase in mistakes (along with
rapidly exploding demand for scarce skills). A specific example was they
would disable all outside connections for RDBMS maintenance ... and
during maintenance they would relax various security processes.
Complexity of RDBMS met that increasingly likely they would overrun
maintenance windows, in mad rush to get back online they would
frequently overlook reactivating various security processes.

more recent
https://en.wikipedia.org/wiki/SQL_injection

all of these have web application with access ... and attacks are
typically against the web application (where webserver frontends are
also responsible for access control).

--
virtualization experience starting Jan1968, online at home since Mar1970

[Rob Schramm]

Seems like there is a drift about security and walls.. interesting article
I found about walls when reading Cryptograms...

https://warontherocks.com/2018/02/wall-wall-fortresses-fail/

I am interested to see if you get it working for z/OS.

Rob Schramm



On Tue, Mar 27, 2018 at 12:15 PM Anne & Lynn Wheeler <ly...@garlic.com>
wrote:

--

Rob Schramm

[Anne & Lynn Wheeler]

rob.s...@GMAIL.COM (Rob Schramm) writes:
> Seems like there is a drift about security and walls.. interesting article
> I found about walls when reading Cryptograms...
>
> https://warontherocks.com/2018/02/wall-wall-fortresses-fail/

re:
http://www.garlic.com/~lynn/2018c.html#9 Graph database on z/OS?

possibly more than you ever wanted to know, in part because of doing
electronic commerce, was sucked into financial standards, financial
industry critical infrastructure protection, and other efforts, like
doing some work with these guys (but from 2004)

Electronic Safety and Soundness Securing Finance in a New Age
http://documents.worldbank.org/curated/en/756761468778791728/pdf/284050PAPER0WBWP026.pdf

This monograph presents a four pillar framework for policymakers in
emerging markets to use in designing responses to the challenge of
assuring electronic safety and soundness of their financial systems. As
such, this paper is focused in part on technological solutions, but more
importantly on the incentives of the many parties involved in assuring
the security of critical infrastructures--from telecommunications and
financial sector service providers to the government and even to the
many final consumers of financial or other services.

... snip ...

we had been also brought in to help wordsmith some cal. state
legislation, they were working on electronic signature, data breach
notification, and opt-in privacy. several entities involved in privacy
were involved and had done detailed, in-depth public surveys on privacy
and the #1 issue was identity theft, specifically the form involving
various breaches that resulted in fraudulent financial transactions.

A problem was that little or nothing was being done about these breaches
(except trying to keep them out of the news). A major issue is that
entities take security measures in self protection ... the problem with
the breaches was that the institutions weren't at risk, it was the
public ... so they had little motivation. It was hoped that the
publicity from the data breach notifications might motivate institutions
to take security measures.

that and a combination of other things resulted in doing financial
transaction standard that slightly tweaked the current infrastructure
... and eliminated criminals ability to use information from previous
transactions obtained in breaches for doing fraudulent transactions
(form of replay attack) ... it didn't prevent breaches, but eliminated
risk from (and major motivation for doing) breaches.

two (other) problems: 1) "security proportional to risk": value of
transaction information to merchant can be a few dollars (and a few
cents to transaction processors), the value of the information to
criminals can be the account balance (or credit limit) ... as a result
criminals may be able to outspend by factor of 100 times attacking (than
defenders can afford to spend) and 2) "dual use": transaction
information is used for both authentication and dozens of business
processes at millions of locations around the world ... as a result it
has to be both kept absolutely secure and never divulged and
simultaneously readily available.

for various reasons there are numerous stakeholders with vested
interests in preserving the status quo.

from the law of unintended consequences ... "SSL" for electronic
commerce (worked on earlier) was used to hide financial transaction
information during transmission. the "tweak" eliminates the need to hide
the information ... whether in transmission or "at rest".