Shared Objects And DLL Hell

[lawren...@gmail.com]

I saw some discussion elsewhere on why Microsoft Windows is prone to “DLL Hell”, and whether Linux had a similar problem.

Linux has shareable libraries (“Dynamic Shared Objects” or DSOs, I believe is the official term) in the form of .so files. I think it avoids DLL hell because of two reasons:
* library versioning to cope with backward-incompatible ABI changes, and
* package management

That is, a shareable library comes from a distro-provided package. Other packages that require that library declare a dependency on that package, which can also specify required versions. So the package manager ensures that installed packages form a consistent dependency graph.

Library versioning is done by sticking a version number after the .so extension. For example, just browsing through my own system, I came across the following:

root@theon:~ # ls -l /usr/lib/x86_64-linux-gnu/
...
lrwxrwxrwx 1 root root 20 Jun 23 01:28 libapt-inst.so -> libapt-inst.so.2.0.0
lrwxrwxrwx 1 root root 20 May 23 2015 libapt-inst.so.1.5 -> libapt-inst.so.1.5.0
-rw-r--r-- 1 root root 63360 May 23 2015 libapt-inst.so.1.5.0
lrwxrwxrwx 1 root root 20 Jun 23 01:28 libapt-inst.so.2.0 -> libapt-inst.so.2.0.0
-rw-r--r-- 1 root root 51280 Jun 23 01:30 libapt-inst.so.2.0.0
...

Those files “libapt-inst.so.1.5.0” and “libapt-inst.so.2.0.0” come from two entirely different packages with two different names, not two versions of the same package, which I have installed at the same time:

root@theon:~ # dpkg-query -S /usr/lib/x86_64-linux-gnu/libapt-inst.so.1.5.0
libapt-inst1.5:amd64: /usr/lib/x86_64-linux-gnu/libapt-inst.so.1.5.0
root@theon:~ # dpkg-query -S /usr/lib/x86_64-linux-gnu/libapt-inst.so.2.0.0
libapt-inst2.0:amd64: /usr/lib/x86_64-linux-gnu/libapt-inst.so.2.0.0

This way, other packages that depend on the older ABI can coexist with those that need the newer one.

Note the symlinks: a bugfix to version 2.0.1 could produce a new library “libapt-inst.so.2.0.1", which would be installed under its own name. The symlink “libapt-inst.so.2.0” can then be updated to point to 2.0.1 instead of 2.0.0, and executables (and other libraries) using the “2.0” name will automatically pick up the new version.

Note also the name with no version suffix:

root@theon:~ # dpkg-query -S /usr/lib/x86_64-linux-gnu/libapt-inst.so
libapt-pkg-dev:amd64: /usr/lib/x86_64-linux-gnu/libapt-inst.so

As you can see, this comes from an entirely separate package, with “-dev” in its name.

The versioned libraries come from “run-time” packages, while the unversioned name comes from the corresponding “development” package. The former are all you need to run prebuilt application binaries, while the latter is what you need to build those applications. The linker follows the symlink, and automatically includes the versioned library name in the executable it generates.

Windows has nothing like this. Which is why it has so much trouble with library version conflicts.

[Stephen Hoffman]

On 2016-06-30 06:06:08 +0000, lawren...@gmail.com said:

> I saw some discussion elsewhere on why Microsoft Windows is prone to
> “DLL Hell”, and whether Linux had a similar problem.

DLL Hell is a solved problem. Has been for many years. Apps are
adopting the available solutions, too. Slowly.

Have a look at .NET, and with the way that applications now deploy with
the expected frameworks, and with how applications are increasingly
bundled together and don't need to splatter their giblets around a
system. Or how the applications are prohibited from that.

Dealing with incompatible versions through version-numbered shareable
image filenames or the occasional version-numbered object library is
common on most platforms, including on OpenVMS. DECwindows did this
with version-numbered naming ~twenty years ago, and — for an entire
package — Oracle Rdb has one of the best multi-version implementation
available on OpenVMS.

Where this naming gets in trouble is now further along, around
deploying applications and later with cleaning up any added
dependencies, and with more subtle issues such as code signing and with
ensuring that packages can't hijack each other — is FOOSHARE010.EXE
really FOO V1.0, or is that something else that only seems to be FOO
V1.0? OpenVMS has no concept of signed executable images, much less
signed component objects, and the structures intended for parts of that
work including installation verification (VMSKITBLD.XML, etc) were
never finished.


--
Pure Personal Opinion | HoffmanLabs LLC

[Richard Levitte]

Den torsdag 30 juni 2016 kl. 08:06:10 UTC+2 skrev lawren...@gmail.com:
> I saw some discussion elsewhere on why Microsoft Windows is prone to “DLL Hell”, and whether Linux had a similar problem.
>
> Linux has shareable libraries (“Dynamic Shared Objects” or DSOs, I believe is the official term) in the form of .so files. I think it avoids DLL hell because of two reasons:
> * library versioning to cope with backward-incompatible ABI changes, and
> * package management
>
> That is, a shareable library comes from a distro-provided package. Other packages that require that library declare a dependency on that package, which can also specify required versions. So the package manager ensures that installed packages form a consistent dependency graph.
>
> Library versioning is done by sticking a version number after the .so extension. For example, just browsing through my own system, I came across the following:
>
> root@theon:~ # ls -l /usr/lib/x86_64-linux-gnu/
> ...
> lrwxrwxrwx 1 root root 20 Jun 23 01:28 libapt-inst.so -> libapt-inst.so.2.0.0
> lrwxrwxrwx 1 root root 20 May 23 2015 libapt-inst.so.1.5 -> libapt-inst.so.1.5.0
> -rw-r--r-- 1 root root 63360 May 23 2015 libapt-inst.so.1.5.0
> lrwxrwxrwx 1 root root 20 Jun 23 01:28 libapt-inst.so.2.0 -> libapt-inst.so.2.0.0
> -rw-r--r-- 1 root root 51280 Jun 23 01:30 libapt-inst.so.2.0.0
> ...
>
> Those files “libapt-inst.so.1.5.0” and “libapt-inst.so.2.0.0” come from two entirely different packages with two different names, not two versions of the same package, which I have installed at the same time:
>
> root@theon:~ # dpkg-query -S /usr/lib/x86_64-linux-gnu/libapt-inst.so.1.5.0
> libapt-inst1.5:amd64: /usr/lib/x86_64-linux-gnu/libapt-inst.so.1.5.0
> root@theon:~ # dpkg-query -S /usr/lib/x86_64-linux-gnu/libapt-inst.so.2.0.0
> libapt-inst2.0:amd64: /usr/lib/x86_64-linux-gnu/libapt-inst.so.2.0.0
>
> This way, other packages that depend on the older ABI can coexist with those that need the newer one.
>
> Note the symlinks: a bugfix to version 2.0.1 could produce a new library “libapt-inst.so.2.0.1", which would be installed under its own name. The symlink “libapt-inst.so.2.0” can then be updated to point to 2.0.1 instead of 2.0.0, and executables (and other libraries) using the “2.0” name will automatically pick up the new version.

It's not that simple. Exactly what file name is being written into the application depends on a string called SONAME in the library, which gets copied to the NEEDED strings in the applications linked with that library.

Here's an example, the library libdl:

: ; ls -l /usr/lib/x86_64-linux-gnu/libdl.*
-rw-r--r-- 1 root root 12754 jun 27 02:37 /usr/lib/x86_64-linux-gnu/libdl.a
lrwxrwxrwx 1 root root 32 jun 27 02:37 /usr/lib/x86_64-linux-gnu/libdl.so -> /lib/x86_64-linux-gnu/libdl.so.2
: ; ls -l /lib/x86_64-linux-gnu/libdl.so.2
lrwxrwxrwx 1 root root 13 jun 27 02:36 /lib/x86_64-linux-gnu/libdl.so.2 -> libdl-2.22.so
: ; ls -l /lib/x86_64-linux-gnu/libdl-2.22.so
-rw-r--r-- 1 root root 14640 jun 27 02:37 /lib/x86_64-linux-gnu/libdl-2.22.so

So, when building your application using something like -ldl, the linker will go through that series of symlinks until it gets to libdl-2.22.so. The SONAME is this:

: ; LANG=C readelf -d /lib/x86_64-linux-gnu/libdl-2.22.so | grep SONAME
0x000000000000000e (SONAME) Library soname: [libdl.so.2]

So now, let's have a look at a program that uses libdl:

: ; LANG=C readelf -d /usr/bin/openssl | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libssl.so.1.0.2]
0x0000000000000001 (NEEDED) Shared library: [libcrypto.so.1.0.2]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]

See how the stored name is libdl.so.2? That's the file name, and really the only file name that the dynamic loader will look for when starting the program. No more magic than that. All other file names to the same library are just convenience...

Cheers,
Richard

P.S. LANG=C because I run my terminals with swedish locale. I thought english would be more convenient here ;-)

[lawren...@gmail.com]

On Friday, July 1, 2016 at 3:07:20 AM UTC+12, Stephen Hoffman wrote:

> DLL Hell is a solved problem. Has been for many years. Apps are
> adopting the available solutions, too. Slowly.
>
> Have a look at .NET, and with the way that applications now deploy with
> the expected frameworks, and with how applications are increasingly
> bundled together and don't need to splatter their giblets around a
> system. Or how the applications are prohibited from that.

Windows Dotnet has been around since about 2001, hasn’t it? So “slowly” is a bit of an understatement.

The idea of collecting all your dependencies together into “assemblies”, so that different apps can have different versions of the same dependency, sounds fine in theory, it seems to work less well in practice. For example, later versions of Dotnet are supposed to be backward-compatible with earlier versions, yet you often end up with systems where there are multiple versions installed side-by-side.

Then there’s the question of Microsoft’s commitment to Dotnet. It got distracted by Silverlight at one point, then this WinRT thing. Currently it is pushing “Universal Windows Platform”, but already there are questions about how “Universal” this is going to be, with limitations being imposed on the XBox version (no games!) and the terminal condition of Windows Phone.

> Dealing with incompatible versions through version-numbered shareable
> image filenames or the occasional version-numbered object library is
> common on most platforms, including on OpenVMS.

How does VMS deal with it? I trust it doesn’t use file version numbers...

> Where this naming gets in trouble is now further along, around
> deploying applications and later with cleaning up any added
> dependencies, and with more subtle issues such as code signing and with

> ensuring that packages can't hijack each other ...

You don’t have that problem with open-source software, only with proprietary packages. The whole business with virtualization and containerization and all the rest is mainly because you cannot trust proprietary software to play nice.

Most Linux distros (even Android) have the concept of signing packages, rather than individual executables.

[Stephen Hoffman]

On 2016-06-30 22:47:26 +0000, lawren...@gmail.com said:

> On Friday, July 1, 2016 at 3:07:20 AM UTC+12, Stephen Hoffman wrote:
>
>> DLL Hell is a solved problem. Has been for many years. Apps are
>> adopting the available solutions, too. Slowly.
>>
>> Have a look at .NET, and with the way that applications now deploy
>> with> the expected frameworks, and with how applications are
>> increasingly> bundled together and don't need to splatter their giblets
>> around a> system. Or how the applications are prohibited from that.
>
> Windows Dotnet has been around since about 2001, hasn’t it? So “slowly”
> is a bit of an understatement.
>
> The idea of collecting all your dependencies together into
> “assemblies”, so that different apps can have different versions of the
> same dependency, sounds fine in theory, it seems to work less well in
> practice. For example, later versions of Dotnet are supposed to be
> backward-compatible with earlier versions, yet you often end up with
> systems where there are multiple versions installed side-by-side.
>
> Then there’s the question of Microsoft’s commitment to Dotnet. It got
> distracted by Silverlight at one point, then this WinRT thing.
> Currently it is pushing “Universal Windows Platform”, but already there
> are questions about how “Universal” this is going to be, with
> limitations being imposed on the XBox version (no games!) and the
> terminal condition of Windows Phone.

OpenVMS application development tools and practices are not exactly
bleeding edge. VAX C has been deprecated for longer than Windows 95
has been available and various folks still haven't reviewed and
reworked their application code for compatibility ANSI/ISO C, after all.

As for Microsoft? Learning from the ideas and implementations of
others does not mean reimplementing bugs, limitations, misfeatures or
errata. Best to learn and improve and avoid.

>> Dealing with incompatible versions through version-numbered shareable>
>> image filenames or the occasional version-numbered object library is>
>> common on most platforms, including on OpenVMS.
>
> How does VMS deal with it? I trust it doesn’t use file version numbers...

Badly. Manually. Rdb has a command procedure used to set the Rdb
version, and that's about the most reasonable approach presently in
use. Entirely unintegrated, of course.

>> Where this naming gets in trouble is now further along, around
>> deploying applications and later with cleaning up any added
>> dependencies, and with more subtle issues such as code signing and with
>> ensuring that packages can't hijack each other ...
>
> You don’t have that problem with open-source software, only with
> proprietary packages.

False.

> The whole business with virtualization and containerization and all the
> rest is mainly because you cannot trust proprietary software to play
> nice.

Nor can open source be particularly trusted. Either the specific wad
of code, or the package in general. This is why sandboxes are part of
most any container strategy, and part of app stacking. To isolate
damage and bugs that can be latent whether intentionally or otherwise,
and to (try to) isolate dependencies for that matter.

> Most Linux distros (even Android) have the concept of signing packages,
> rather than individual executables.

OpenVMS does not have that mechanism for anything past PCSI and
VMSINSTAL kits, and the closest analog is CDSA. That's long been
deprecated, too. Whether those existing checks are reliable is not
something I'm willing to affirm; not without reviewing details of the
implementation.

[Kerry Main]

> -----Original Message-----
> From: Info-vax [mailto:info-vax...@rbnsn.com] On Behalf Of
> lawrencedo99--- via Info-vax
> Sent: 30-Jun-16 6:47 PM
> To: info...@rbnsn.com
> Cc: lawren...@gmail.com
> Subject: Re: [Info-vax] Shared Objects And DLL Hell
>
> On Friday, July 1, 2016 at 3:07:20 AM UTC+12, Stephen Hoffman wrote:
>
> > DLL Hell is a solved problem. Has been for many years. Apps are
> > adopting the available solutions, too. Slowly.
> >
> > Have a look at .NET, and with the way that applications now deploy
> with
> > the expected frameworks, and with how applications are increasingly
> > bundled together and don't need to splatter their giblets around a
> > system. Or how the applications are prohibited from that.
>
> Windows Dotnet has been around since about 2001, hasn’t it? So
> “slowly” is a bit of an understatement.
>

Let's not forget the OS culture issue here with both Windows & perhaps
to a somewhat lesser degree, Linux as well.

How many new Windows / Linux projects start with the assumption that
their target platform is an existing server running other groups apps?

Following .Net standards would require different App groups to develop
their App standards strategy by actually working together.

In this culture, even with higher mgmt. , monitoring and security costs
associated with VM sprawl, It is just culturally (read politically) easier to
simply give each group their own VM or even physical server.

Heck, in many cases, it is easier to give each developer their own VM.

[snip...]

Regards,

Kerry Main
Kerry dot main at starkgaming dot com

[lawren...@gmail.com]

On Friday, July 1, 2016 at 12:35:04 PM UTC+12, Kerry Main wrote:

> How many new Windows / Linux projects start with the assumption that
> their target platform is an existing server running other groups apps?

Open Source ones? All of them--at least on Linux. They know how to stay out of each other’s way.

[lawren...@gmail.com]

On Friday, July 1, 2016 at 11:19:16 AM UTC+12, Stephen Hoffman wrote:

> On 2016-06-30 22:47:26 +0000, Lawrence D’Oliveiro said:
>> You don’t have that problem with open-source software, only with
>> proprietary packages.
>
> False.

ldo@theon:~> dpkg-query -l '*' | grep -c ^i
4657

That means I have 4657 separate packages all installed at once on my primary Debian Unstable system. Which remains reliable enough for all my day-to-day work. Typical uptime is about a month, because that’s how often I decide to update things.

How many packages can your Windows or VMS servers have installed at once without crashing?

[David Froble]

lawren...@gmail.com wrote:
> On Friday, July 1, 2016 at 11:19:16 AM UTC+12, Stephen Hoffman wrote:
>> On 2016-06-30 22:47:26 +0000, Lawrence D’Oliveiro said:
>>> You don’t have that problem with open-source software, only with
>>> proprietary packages.
>> False.
>
> ldo@theon:~> dpkg-query -l '*' | grep -c ^i
> 4657
>
> That means I have 4657 separate packages all installed at once on my primary Debian Unstable system. Which remains reliable enough for all my day-to-day work. Typical uptime is about a month, because that’s how often I decide to update things.

Because you haven't had a problem is not proof that it cannot happen. Your
claim is not proven. I'm sure you are happy with it. Still not proof.

> How many packages can your Windows or VMS servers have installed at once without crashing?

I can place as many things as I want on my VMS systems. My systems don't crash.
Sometimes power failure. No computer runs without power.

[lawren...@gmail.com]

On Friday, July 1, 2016 at 3:52:59 PM UTC+12, David Froble wrote:

> Lawrence D’Oliveiro wrote:
>> On Friday, July 1, 2016 at 11:19:16 AM UTC+12, Stephen Hoffman wrote:
>>> On 2016-06-30 22:47:26 +0000, Lawrence D’Oliveiro said:
>>>> You don’t have that problem with open-source software, only with
>>>> proprietary packages.
>>> False.
>>
>> ldo@theon:~> dpkg-query -l '*' | grep -c ^i
>> 4657
>>
>> That means I have 4657 separate packages all installed at once on my
>> primary Debian Unstable system. Which remains reliable enough for all my
>> day-to-day work. Typical uptime is about a month, because that’s how often
>> I decide to update things.
>
> Because you haven't had a problem is not proof that it cannot happen. Your
> claim is not proven. I'm sure you are happy with it. Still not proof.

Mumble-mumble. Is that really the best you can do? I could start counting how many systems I have running, and for how many years, if you want.

>> How many packages can your Windows or VMS servers have installed at once
>> without crashing?
>
> I can place as many things as I want on my VMS systems. My systems don't
> crash.

How many?

[Richard Levitte]

Den fredag 1 juli 2016 kl. 05:09:08 UTC+2 skrev lawren...@gmail.com:
> On Friday, July 1, 2016 at 11:19:16 AM UTC+12, Stephen Hoffman wrote:
> > On 2016-06-30 22:47:26 +0000, Lawrence D’Oliveiro said:
> >> You don’t have that problem with open-source software, only with
> >> proprietary packages.
> >
> > False.
>
> ldo@theon:~> dpkg-query -l '*' | grep -c ^i
> 4657
>
> That means I have 4657 separate packages all installed at once on my primary Debian Unstable system. Which remains reliable enough for all my day-to-day work. Typical uptime is about a month, because that’s how often I decide to update things.

This doesn't prove anything about open source in general. It does, however, show that the Debian maintainers are doing a fairly good job. There's a difference.

> How many packages can your Windows or VMS servers have installed at once without crashing?

VMS can have as many as you need, as long as you have space for it.

Cheers,
Richard

[Richard Levitte]

Den torsdag 30 juni 2016 kl. 17:07:20 UTC+2 skrev Stephen Hoffman:
> Dealing with incompatible versions through version-numbered shareable
> image filenames or the occasional version-numbered object library is
> common on most platforms, including on OpenVMS. DECwindows did this
> with version-numbered naming ~twenty years ago, and — for an entire
> package — Oracle Rdb has one of the best multi-version implementation
> available on OpenVMS.

I'm confused... DECWindows? A 'dir sys$share:decw*.exe' doesn't show much version naming at all. Actually, looking through sys$share: doesn't show very much version numbering at all, really. I don't have Oracle RDB on the systems I play with, so can't see how they've done things.

Cheers,
Richard

[lawren...@gmail.com]

On Friday, July 1, 2016 at 7:24:51 PM UTC+12, Richard Levitte wrote:
> Den fredag 1 juli 2016 kl. 05:09:08 UTC+2 skrev lawren...@gmail.com:
>> On Friday, July 1, 2016 at 11:19:16 AM UTC+12, Stephen Hoffman wrote:
>>> On 2016-06-30 22:47:26 +0000, Lawrence D’Oliveiro said:
>>>> You don’t have that problem with open-source software, only with
>>>> proprietary packages.
>>>
>>> False.
>>
>> ldo@theon:~> dpkg-query -l '*' | grep -c ^i
>> 4657
>>
>> That means I have 4657 separate packages all installed at once on my
>> primary Debian Unstable system. Which remains reliable enough for all my
>> day-to-day work. Typical uptime is about a month, because that’s how often
>> I decide to update things.
>
> This doesn't prove anything about open source in general. It does,
> however, show that the Debian maintainers are doing a fairly good job.
> There's a difference.

It’s a testament to the fact that no system built on proprietary software can scale to anywhere near that level.

>> How many packages can your Windows or VMS servers have installed at once
>> without crashing?
>
> VMS can have as many as you need, as long as you have space for it.

How many?

[John E. Malmberg]

On 6/30/2016 10:07 AM, Stephen Hoffman wrote:
> On 2016-06-30 06:06:08 +0000, lawren...@gmail.com said:
>
>> I saw some discussion elsewhere on why Microsoft Windows is prone to
>> “DLL Hell”, and whether Linux had a similar problem.
>
> DLL Hell is a solved problem. Has been for many years. Apps are
> adopting the available solutions, too. Slowly.

Only for some definitions of "Solved".

Basically the problem is solved as long as your Windows system disk is a
minimum of 250 GB.

The current solution is to track in a special mostly hidden directory
the private and usually redundant copies of the DLLs that are in
potential conflict, and in another mostly hidden directory keep all past
versions of the DLLs that are replaced by updates.

Set up a Windows 7 VM with only 30 to 40 GB of system disk and keep it
up to date with Windows update. After between 1 to 3 years, it will be
out of disk space.

A contrived example? Not according to the wild wild web, as there were
a lot of embedded and laptops shipped with small SSDs.

Cleaning up the disk space to remove the useless duplicate copies.
Official tools have shown up, and then moved to more hidden locations,
and generally do not work to clean up the mess.

The only known supported fix is to completely re-install the OS and the
applications.

And this feature got back ported (apparently with out the cleanup tool)
to Windows XP through updates.

And to help keep things non-compatible:

DLLs support both call by name and call by ordinal. In order for call
by ordinal to be backward compatible, the order of the symbols in the
DLL must not change.

Apparently call by ordinal is not used much because none of the open
source tools I have seen for building DLLs take any steps to insure that
new symbols are always added to the end of the list.

Regards,
-John
wb8...@qsl.net_work

[Richard Levitte]

Den fredag 1 juli 2016 kl. 14:32:17 UTC+2 skrev lawren...@gmail.com:
> On Friday, July 1, 2016 at 7:24:51 PM UTC+12, Richard Levitte wrote:
> > Den fredag 1 juli 2016 kl. 05:09:08 UTC+2 skrev lawren...@gmail.com:
> >> On Friday, July 1, 2016 at 11:19:16 AM UTC+12, Stephen Hoffman wrote:
> >>> On 2016-06-30 22:47:26 +0000, Lawrence D’Oliveiro said:
> >>>> You don’t have that problem with open-source software, only with
> >>>> proprietary packages.
> >>>
> >>> False.
> >>
> >> ldo@theon:~> dpkg-query -l '*' | grep -c ^i
> >> 4657
> >>
> >> That means I have 4657 separate packages all installed at once on my
> >> primary Debian Unstable system. Which remains reliable enough for all my
> >> day-to-day work. Typical uptime is about a month, because that’s how often
> >> I decide to update things.
> >
> > This doesn't prove anything about open source in general. It does,
> > however, show that the Debian maintainers are doing a fairly good job.
> > There's a difference.
>
> It’s a testament to the fact that no system built on proprietary software can scale to anywhere near that level.

I'm not sure that proprietary has anything to do with it. Either way, no one has built that kind of community around VMS (yet?), so your anecdote isn't much more than just that.

(and believe you me, I think the Debian project is a fantastic effort, precisely because of the community that has been built around it)

> >> How many packages can your Windows or VMS servers have installed at once
> >> without crashing?
> >
> > VMS can have as many as you need, as long as you have space for it.
>
> How many?

What, you're seriously asking for a number? Are we going to stoop to that level, when you know full well that there aren't a gazillion packages available today? You cannot compare with Debian on the number of available packages, but that has very little to do with the capability to scale and everything to do with an amazing community and popularity (on the Debian side).

How many VMS systems have you seen crash because of the sheere number of installed packages?

Cheers,
Richard

[Stephen Hoffman]

On 2016-07-01 03:09:06 +0000, lawren...@gmail.com said:

> On Friday, July 1, 2016 at 11:19:16 AM UTC+12, Stephen Hoffman wrote:
>> On 2016-06-30 22:47:26 +0000, Lawrence D’Oliveiro said:
>>> You don’t have that problem with open-source software, only with>>
>>> proprietary packages.
>>
>> False.
>
>

> How many packages can your Windows or VMS servers have installed at
> once without crashing?

You can't trust open source software to always play nice — whether in
terms of what it does, or whether it's been breached or is vulnerable
to breach — either. Which means treating all software with
skepticism.

[Stephen Hoffman]

On 2016-07-01 08:24:49 +0000, Richard Levitte said:

> Den torsdag 30 juni 2016 kl. 17:07:20 UTC+2 skrev Stephen Hoffman:
>> Dealing with incompatible versions through version-numbered shareable
>> image filenames or the occasional version-numbered object library is
>> common on most platforms, including on OpenVMS. DECwindows did this
>> with version-numbered naming ~twenty years ago, and — for an entire
>> package — Oracle Rdb has one of the best multi-version implementation
>> available on OpenVMS.
>
> I'm confused... DECWindows? A 'dir sys$share:decw*.exe' doesn't show
> much version naming at all. Actually, looking through sys$share:
> doesn't show very much version numbering at all, really.

You'll tend to find a few of these specifically on OpenVMS systems that
have been upgraded from older releases of DECwindows.

$ dir sys$share:decw*1*.exe,sys$share:decw*5*.exe, etc

You'll find some OpenGL bits, and bits such as DECW$DXMLIBSHR.EXE and
DECW$DXMLIBSHR12.EXE, for instance.

Unfortunately, some of the renames I've encountered have been done
after the incompatibility arises, which means folks can need to rework
some procedures — SSL1 is a whole lot of reworking of procedures — or
sometimes define logical names to find the required old or new
libraries. The Rdb approach to this works comparatively well, given
the confines of current OpenVMS tools and practices.

[Stephen Hoffman]

On 2016-07-01 12:43:28 +0000, John E. Malmberg said:

> On 6/30/2016 10:07 AM, Stephen Hoffman wrote:
>> On 2016-06-30 06:06:08 +0000, lawren...@gmail.com said:
>>
>>> I saw some discussion elsewhere on why Microsoft Windows is prone to
>>> “DLL Hell”, and whether Linux had a similar problem.
>>
>> DLL Hell is a solved problem. Has been for many years. Apps are
>> adopting the available solutions, too. Slowly.
>
> Only for some definitions of "Solved".
>
> Basically the problem is solved as long as your Windows system disk is
> a minimum of 250 GB.
>
> The current solution is to track in a special mostly hidden directory
> the private and usually redundant copies of the DLLs that are in
> potential conflict, and in another mostly hidden directory keep all
> past versions of the DLLs that are replaced by updates.

> ...

My reply was referencing "DLL Hell", a mess which can and does exist on
some platforms, and particularly with older applications. But if you
believe that "DLL Hell" is still a problem for recent approaches and
applications that are new or have been overhauled — and in the case of
OpenVMS, yes, things are still stuck in "DLL Hell" whenever upward
compatibility isn't available or isn't feasible — maybe look around at
what other approaches and what other solutions are available?

Can what Microsoft does now with their frameworks be done better?
Certainly. Can disk or file de-dup or clever run-time tricks or
compression or other techniques help somewhat with the bloat? Sure.
Has software bloat been discussed over the past several decades? Most
definitely, and more than a few folks here remember the 2GL/3GL/4GL
debates, and OMG how big did application code get from VAX to Alpha to
Itanium, and many other discussions and contexts. is it even
appropriate to be optimizing application software that's targeting
desktop and laptop environments for 250 GB hard disk drives or for 2016
computing in general? If that's your current installed base and you're
squeezed, sure, some investment is warranted, but — if your changes are
not going to be ready far enough ahead of the next-generation
deployment to really matter — then again maybe not.

OpenVMS development and even OpenVMS application developers needs to
stop looking at the worst of other platforms — except as a lesson in
mistakes to avoid — and start looking at the best of the other
platforms, and — where it's feasible — how to make OpenVMS competitive
with that, and how to make your applications better.

If your user interfaces are character cell using terminal emulators, or
if your operations are based on serial terminals for your management,
or if your applications require folks to hand-edit configuration files,
or if your periodic management requirements are not already automated,
maybe your front-end or your application or your environment needs some
work? If you are headed toward your own "DLL Hell", maybe learn how
others have avoided similar problems?

Look at other platforms. Learn what works, and why. Apply that to
your own environment, and your own approaches and applications. If
you're in VSI, apply the best of that learning and those other
platforms to OpenVMS itself. If you haven't figured out how to
version and manage code upgrades, then definitely take the time to
learn from the missteps of "DLL Hell" and even of the trade-offs
inherent in server uptime, upward compatibility and otherwise.

[Richard Levitte]

Den fredag 1 juli 2016 kl. 17:37:03 UTC+2 skrev Stephen Hoffman:
> The Rdb approach to this works comparatively well, given
> the confines of current OpenVMS tools and practices.

Again, the mystical Oracle Rdb...

would someone mind telling the rest of us how they have approached the problem, or point at some document that shows it?

(I'm particularly interested in good practices, since I'm build OpenSSL shared libraries)

Cheers,
Richard

[Richard Levitte]

Den fredag 1 juli 2016 kl. 19:23:15 UTC+2 skrev Richard Levitte:
> Den fredag 1 juli 2016 kl. 17:37:03 UTC+2 skrev Stephen Hoffman:
> > The Rdb approach to this works comparatively well, given
> > the confines of current OpenVMS tools and practices.
>
> Again, the mystical Oracle Rdb...
>
> would someone mind telling the rest of us how they have approached the problem, or point at some document that shows it?

Never mind that, I just found the installation manual. Darn, it's thorough!

http://download.oracle.com/otndocs/products/rdb/pdf/rdb073_install_guide.pdf

[Jan-Erik Soderholm]

I have not installed anything but the multiversion kit for decades back.
And today (as the manual says) there are only multiversion kits anyway.

Adding (or adjusting) the "$ @SYS$LIBRARY:RDB$SETVER 7.3 /SYSTEM" line
in systartup is "automatic" today... :-)

Note that the multiversioning is on the x.x level, but the kits comes in
x.x.x.x versions. But databases are compatible within one x.x level.

This feature makes it possible to install (say) 7.4 (not existing today)
on a Rdb 7.3 system and then just change my own local process to 7.4
and use a copy of the database (converted to 7.4) to test things out.
The rest of the box remains on 7.3, until the main database is converted
and the line in systartup is changed from 7.3 to 7.4. Both those things
are fast (a couple of minutes) when you already have everything (the new
version) installed.

Jan-Erik.

[Kerry Main]

> -----Original Message-----
> From: Info-vax [mailto:info-vax...@rbnsn.com] On Behalf Of
> Richard Levitte via Info-vax
> Sent: 01-Jul-16 1:31 PM
> To: info...@rbnsn.com
> Cc: Richard Levitte <ric...@levitte.org>
> Subject: Re: [Info-vax] Shared Objects And DLL Hell
>

In a previous life, it was great supporting RDB as we would get Cust issues
and we could test on the same system whether a new version was going to
fix an issue or perform better on a new version. It also allowed rollbacks to
the older version if required. Changing versions was as simple as running a
DCL command file which reset process logicals.

Richard - thx for pointer.

For benefit of others, here is sect 2.5 extract that deals with multi-version
support:

"The following images are installed in SYS$COMMON:[SYSLIB] by VMSINSTAL:
- RDB$SHARE73.EXE
- RDBSHR.EXE

Many layered products and third−party products call RDBSHR.EXE at image
activation time. With the multiversion variant of Oracle Rdb, more than one
version of Oracle Rdb is available to an application. The version required
depends on the parameter set by RDB$SETVER.COM.

Applications still call RDBSHR.EXE but RDBSHR.EXE checks only what version
the application wants to use by examining the logical name
RDMS$VERSION_VARIANT. If RDMS$VERSION_VARIANT is not defined,
RDBSHR.EXE calls RDB$SHARE.EXE, which contains the current released
version code. For example, if RDMS$VERSION_VARIANT translates to 72,
RDBSHR.EXE calls RDB$SHARE72.EXE, which contains the release 7.2 code."

[Stephen Hoffman]

On 2016-07-01 17:23:13 +0000, Richard Levitte said:

> would someone mind telling the rest of us how they have approached the
> problem, or point at some document that shows it?
>
> (I'm particularly interested in good practices, since I'm build OpenSSL
> shared libraries)

Start with page 32 here, for a high-level introduction and the end-user
view of how this works:

http://download.oracle.com/otndocs/products/rdb/pdf/rdb073_install_guide.pdf

Which version is selected is based on the user either specifying
explicit Rdb specific commands (by including the version number on
verbs) or by using the default verbs which get the process or system
default files and tools and shareable images, and this selection is
based on logical names and symbols, and particularly on the RDB$SETVER
procedure discussed in the above manual.

[Stephen Hoffman]

On 2016-07-01 18:07:47 +0000, Kerry Main said:

> In a previous life, it was great supporting RDB as we would get Cust

> issuesand we could test on the same system whether a new version was
> going tofix an issue or perform better on a new version. It also
> allowed rollbacks tothe older version if required. Changing versions
> was as simple as running aDCL command file which reset process logicals.

Next steps past this are sandboxes and jails to keep those apps from
stepping on each other either accidentally or secondary to a
vulnerability or malicious code, or the use of VM sprawl^Wguests.
If OpenVMS wants to play here, providing better tools, guidance and
documentation around making multiple versions easy and secure would be
among the obvious next steps.
As well as work around allowing OpenVMS components — for instance the
not-yet-integrated web server and related, various language compilers
(and some already have multiversion support), etc — to have some
consistency here, too.

[David Froble]

At some point I've got to ask what he's calling an "application" or a "package"?
4675 packages / applications? Can one person even run that many? Something
doesn't smell too good about this claim.

[Richard Levitte]

Packages can basically contain anything. A set of shared libraries, the corresponding development files, some small app (remember, we're talking Unix... you know, the system where you can pipe together a series of small simple apps to perform greater tasks), documentation (yup, documentation usually comes as its own package). The basic operating system itself comes as a buttload of packages in itself (I think they are classified "essential"), of which just the kernel and the corresponding development files are a few.
etc etc etc etc etc.

Just as an example, here's the set of packages Debian [unstable] (*) delivers for OpenSSL:

openssl - Secure Sockets Layer toolkit - cryptographic utility
libssl1.0.2 - Secure Sockets Layer toolkit - shared libraries
libssl-dev - Secure Sockets Layer toolkit - development files
libssl-doc - Secure Sockets Layer toolkit - development documentation
libssl1.0.2-dbg - Secure Sockets Layer toolkit - debug information

Cheers,
Richard

(*) For a distribution variant called "unstable", it's incredibly stable ;-)

[johnwa...@yahoo.co.uk]

"Re-install the OS and the applications" has been the preferred
Windows-centric IT department approach for decades (literally),
so these folks are used to that "solution". In many cases it's
all they've got - no attempt to gather evidence (even where
facilities exist eg NT minidump), no attempt to diagnose, no
attempt to do anything beyond "fixed in the next release" or
"hardware glitch" and hope no one notices the next time the
same issue causes the same problem on a (potentially
different) box.

I'm all in favour of restoring service as soon as possible, but
sometimes, there's a need for more than that.

[Stephen Hoffman]

On 2016-07-02 17:23:43 +0000, johnwa...@yahoo.co.uk said:

> "Re-install the OS and the applications" has been the preferred
> Windows-centric IT department approach for decades (literally),

Which works surprisingly well on some platforms and with either minimal
and usually no disruptions, and where that preserves user settings and
related data. I find myself wishing for this capability on OpenVMS —
along with server migration support — and looking at what would be
involved in implementing a subset. This is most useful for cases
where there have been disk corruptions or related problems, or when the
installation itself is sufficiently scrozzled that starting over is
preferable, and OpenVMS is not immune to those cases. More than a few
OpenVMS servers are filled with the accumulated cruft of a decade or
more, too.