Package loading problem

[Martin Lafferty]

I have a problem with a set of designtime packages which I am trying to
port to D2006. I have used this code for many years in D5, D6 & D7. I
suspect my problem might be related to demand-loading of packages.

The symptoms are complicated, but one easily described manifestation is
as follows:

One package is called TimecodeD10.bpl and contains two components.
Another is called TimelineD10.bpl and contains one component.

If I start the IDE and create a new application, then try to drop a
component from TimelineD10.bpl on the form I get the following error:

"---------------------------
Error
---------------------------
Can't load package
C:\Projects\Pro-bel\NA_00692_PrelCommon\D10Build\TimelineD10.bpl.
The specified module could not be found.
---------------------------
OK Details >>
---------------------------

+ $36[51F1A0DF]{rtl100.bpl } SysUtils.SysUtils.LoadPackage (Line
16128, "sysutils.pas" + 12) + $36
+ $8[20C3BA68]{coreide100.bpl} PakLoad.PakLoad.TPackage.DoLoadPackage
(Line 150, "PakLoad.pas" + 0) + $8
+ $2[20C0A325]{coreide100.bpl} PakMgr.PakMgr.TDesignPackage.Load (Line
583, "PakMgr.pas" + 2) + $2
+ $3[21B9BA30]{delphicoreide100.bpl}
PasCppPakMgr.PasCppPakMgr.TIDEDesignPackage.Load (Line 821,
"PasCppPakMgr.pas" + 4) + $3
+ $2[20C0AFB7]{coreide100.bpl}
PakMgr.PakMgr.TDesignPackages.LoadDelayLoadPackage (Line 926,
"PakMgr.pas" + 2) + $2
+ $4[21B98591]{delphicoreide100.bpl}
CompPalMgr.CompPalMgr.TComponentPalettePageItemDelegate.CreateComponent
(Line 2478, "CompPalMgr.pas" + 1) + $4
+ $17[00358979]{designide100.bpl}
ComponentDesigner.ComponentDesigner.TComponentRoot.DoCreateComponent
(Line 2283, "ComponentDesigner.pas" + 10) + $17

(... lower parts of stack omitted ...)

"

If I restart the IDE, create a new application, then drop a component
from TimecodeD10.bpl on the form, that works fine. I can then drop a
component from TimelineD10.bpl without a problem.

It seems that if A.bpl is demand-loaded because a component from A is
used, then this works, but if A.bpl is demand-loaded because a
component from B.bpl, which requires A, is used then the loading of A
fails.

Does anyone have any suggestions as to how I might work around this
problem?

--

regards

Martin Lafferty

[Liz]

Martin Lafferty wrote:

>
> I have a problem with a set of designtime packages which I am trying
> to port to D2006. I have used this code for many years in D5, D6 &
> D7. I suspect my problem might be related to demand-loading of
> packages.
>
>
> The symptoms are complicated, but one easily described manifestation
> is as follows:
>
> One package is called TimecodeD10.bpl and contains two components.
> Another is called TimelineD10.bpl and contains one component.
>
> If I start the IDE and create a new application, then try to drop a
> component from TimelineD10.bpl on the form I get the following error:

Is it in your path?

--
Liz the Brit
Delphi things I have released: http://www.xcalibur.co.uk/DelphiThings

[Martin Lafferty]

Liz wrote:

> Martin Lafferty wrote:
>
> >
> >
> > If I start the IDE and create a new application, then try to drop a
> > component from TimelineD10.bpl on the form I get the following
> > error:
>
> Is it in your path?

Yes. If I use a component located in the "missing" package it loads
fine. It is only if it is loaded as a result of being required by
another package that I get the error. In fact, all my library binaries
(bpl, dcp, dcu) are in the same directory which is on my library path.

--

regards

ML

[Martin Lafferty]

> >
> > I have a problem with a set of designtime packages which I am trying
> > to port to D2006. I have used this code for many years in D5, D6 &
> > D7. I suspect my problem might be related to demand-loading of
> > packages.
> >
> >

I have done some more testing on this and am now pretty sure that this
is a delphi bug, or at the very least eccentric behaviour. I have
submitted a reproducable case to QC - # 23225

--

regards

Martin

[Jeanne (Borland)]

Martin Lafferty wrote:

i have promoted to RAID 238184. thanks

--jeanne

[Martin Lafferty]

Jeanne (Borland) wrote:

>
>
> i have promoted to RAID 238184. thanks

Thanks, Jeanne.

I have uploaded an IDE add-in to CodeCentral (#23899) that provides a
workaround to this problem.

--

regards

Martin Lafferty

[Allen Bauer]

> It seems that if A.bpl is demand-loaded because a component from A is
> used, then this works, but if A.bpl is demand-loaded because a
> component from B.bpl, which requires A, is used then the loading of A
> fails.
>
> Does anyone have any suggestions as to how I might work around this
> problem?

Make sure your packages (the .BPL files) are in a folder that is on the
*system* search path. Since B.bpl *implicitly* requires A.bpl, Delphi
relies on the standard Windows DLL loading logic to load the dependent
packages. Apparently, the D2006 install doesn't pre-configure a folder on
your system into which all packages are built that is also added to the
system search path (see the PATH environment variable). This is something
that previous Delphi versions did. I'll be investigating getting this
fixed.

You can fix this yourself by going to Tools|Options|Delphi Options|Library -
Win32 and setting the Package Output Directory to be a directory that you've
placed on the system search path. Now when you compile your packages, all
the .bpl files will go to this folder and since it is on the system search
path, it will find the package dependencies. In order for this to work, you
need to make sure that the package projects' Package Output Directory is
*not* set to some other folder. Finally, if there is already something set
in the Tools|Options|Delphi Options... edit box, simply make sure that
folder is on the system search path.

NOTE: I'm talking about the *system* or *Windows* search path referenced
from the PATH environment variable. Seems that there are a *lot* of folks
who confuse the Library path with the system search path. The *compiler*
uses the Library path to locate .pas, .dcu, .dcp, files. The IDE relies
solely on the Windows loader and the system search path for loading
implicitly referenced packages.

--
Allen Bauer
Delphi/C#Builder/C++Builder Chief Scientist
Borland Software Corporation.
http://blogs.borland.com/abauer

[Rudy Velthuis [TeamB]]

At 22:43:04, 12.01.2006, Allen Bauer wrote:

> Apparently, the D2006 install doesn't pre-configure a folder
> on your system into which all packages are built that is also added to
> the system search path (see the PATH environment variable).

Actually, it does. It configures

<homedir>\My Files\Borland Studio Projects\Bpl

(or the local equivalent of that) for it, and puts it in the path.
Problem with this is that it is used for D2005 as well, and that although
packages (.bpl files) might have suffixes like 90 or 100, .dcp files
don't.

--
Rudy Velthuis [TeamB] http://rvelthuis.de/

"Inanimate objects can be classified scientifically into three major
categories; those that don't work, those that break down and those
that get lost." -- Russell Baker.

[Allen Bauer]

>Actually, it does. It configures
>
> <homedir>\My Files\Borland Studio Projects\Bpl

That's good. However this can still be defeated if the package projects
override that setting.

> (or the local equivalent of that) for it, and puts it in the path.
> Problem with this is that it is used for D2005 as well, and that although
> packages (.bpl files) might have suffixes like 90 or 100, .dcp files
> don't.

.dcp files should *not* be going there. They should remain with the .dpk
file (or go to the .dcp output folder which is a different option). Then
that folder should go in the *library* search path.

[Martin Lafferty]

Allen Bauer wrote:

> > problem?
>
> Make sure your packages (the .BPL files) are in a folder that is on

> the system search path. Since B.bpl implicitly requires A.bpl,

> Delphi relies on the standard Windows DLL loading logic to load the
> dependent packages. Apparently, the D2006 install doesn't
> pre-configure a folder on your system into which all packages are
> built that is also added to the system search path (see the PATH

Dear Allen,

Thanks for your reply. I understand why there is a requirement to have
designtime bpl's on the System path, but think the requirement is wrong.

I need my common libraries to work with a range of versions of delphi -
currently 5, 6, 7, and 10. I build my designtime packages to a (delphi)
version specific directory which is NOT on the system path. I do not
want to add all of these directories to the system path because:

a) I don't like having a really long system path. If a dll is required
by a single application then that application should know where to find
it.

b) The IDE, in my opinion, should not behave differently depending on
whether a packge is explicitly or implicitly loaded. I understand why
this is the case, but I still don't think it is right.

In any event, this is a breaking change. I have never had this problem
in any previous (non demand-loading) version of delphi.

I have worked around this problem by writing an 'Expert' dll that
modifies the IDE's enviromnent block by appending the Library path to
the PATH variable. (CodeCentral #23899).

--

regards

Martin Lafferty

[Allen Bauer]

> Thanks for your reply. I understand why there is a requirement to have
> designtime bpl's on the System path, but think the requirement is wrong.

It's been that way since packages were introduced in Delphi 3. As long as
there are no interdependencies among design-time packages, then you can get
away with not doing that (or placing all the design-time packages into the
IDE's .exe folder). We rely on the Windows loader simply to ensure
consistency with the whole system, and also to reduce the amount of code
needed for each application. Had we gone and written our own loader (and,
it has been discussed many times), it would have been extra code with which
all applications would have been burdened. You might want to look into the
"App Paths" registry key.

> a) I don't like having a really long system path. If a dll is required
> by a single application then that application should know where to find
> it.

The best way to do that is place the required packages in the same folder as
the .EXE.

> b) The IDE, in my opinion, should not behave differently depending on
> whether a packge is explicitly or implicitly loaded. I understand why
> this is the case, but I still don't think it is right.

Believe it or not, you've been surviving only by relying on a specific
loading order. If the a package is loaded before the one that depends upon
it, then the Windows loader will link to the existing one in memory
regardless of its location. Had the IDE decided to load in a different
order (and you cannot/should not depend upon nor control that), you'd have
run into the exact same problem.

> In any event, this is a breaking change. I have never had this problem
> in any previous (non demand-loading) version of delphi.

You can disable demand loading for your packages. In one of the Register
procedures simply put, ForceDemandLoadState(dlDisable);

> I have worked around this problem by writing an 'Expert' dll that
> modifies the IDE's enviromnent block by appending the Library path to
> the PATH variable. (CodeCentral #23899).

That's a clever solution, I'll admit ;-)

[Martin Lafferty]

Allen Bauer wrote:
>
> > a) I don't like having a really long system path. If a dll is
> > required by a single application then that application should know
> > where to find it.
>
> The best way to do that is place the required packages in the same
> folder as the .EXE.
>

Agreed, but I don't think this is appropriate for my libraries.
($BDS)\Bin is entirely your property; I have got no business putting my
stuff in there.

When I distribute libraries to my colleagues, I want to be able to give
them a single directory structure containing all compiled binaries and
source (for reference only). I do it like this:

PrelCommon\
PrelCommon_D5.bpg
PrelCommon_D7.bpg

D5Build\
*.dcu
*.bpl
*.dcp
D7Build\
*.dcu
*.bpl
*.dcp
LibA\
*.pas
LibA_D5.dpk (dcu & bpl & dcp output to ..\D5Build)
LibA_D7.dpk (dcu & bpl & dcp output to ..\D7Build)
LibB\
*.pas
LibB_D5.dpk
LibB_D7.dpk
LibC\
*.pas
LibC_D5.dpk
LibB_D7.dpk

Everything is contained within this structure. This is important
because the libraries can be reliably tracked and updated - there is no
possibility of binaries laying about in other places that might be
causing unexpected variations in our application builds. We add
PREL_COMMON\D?Build to our library path & install the bpls from there.
If we want to browse source we add PrelCommon\Lib? to the 'Browsing
Path'. The libraries never get built except as part of our release
process.

>
> Believe it or not, you've been surviving only by relying on a
> specific loading order.
>

Yikes! I suddenly feel reckless.

>>I have worked around this problem by writing an 'Expert' dll that
>>modifies the IDE's enviromnent block by appending the Library path to
>>the PATH variable. (CodeCentral #23899).

>That's a clever solution, I'll admit ;-)

I was going to use SetDllDirectory (introduced in XP SP1) but it turns
out to be pretty much useless. My API help says it takes a semi-colon
delimited path of directories - but in fact it fails if you pass it
more than one directory. The Online MSDN help has been recently updated
to reflect this - I guess it is easier to change the documentation than
update Kernel32.dll.

--

regards

ML

[Rudy Velthuis [TeamB]]

Allen Bauer wrote:

> > Actually, it does. It configures
> >
> > <homedir>\My Files\Borland Studio Projects\Bpl
>
> That's good. However this can still be defeated if the package
> projects override that setting.

Sure. I always compile my packages to $(BDS)\lib and my .dcps go to
$(BDS)\DCP (I always create that directory).

> > (or the local equivalent of that) for it, and puts it in the path.
> > Problem with this is that it is used for D2005 as well, and that
> > although packages (.bpl files) might have suffixes like 90 or 100,
> > .dcp files don't.
>

> .dcp files should not be going there. They should remain with the

> .dpk file (or go to the .dcp output folder which is a different
> option).

Problem is that that would be in <homedir>\My Files\Borland Studio
Projects, to which D2005 and D8 files also go, by default. So D2005 and
D2006 .dcps would be in the same directory, unless users create their
own directories (I do, but I learned that many don't).

Until D7, there was a $(Delphi)\projects dir, which was set as thew
default project dir, so each version would store files in a different
default directory. Much better, IMO.

--
Rudy Velthuis [TeamB] http://velthuis.homepage.t-online.de

"Future historians will be able to study at the Jimmy Carter Library,
the Gerald Ford Library, the Ronald Reagan Library, and the Bill
Clinton Adult Bookstore." -- George Carlin

[Rudy Velthuis [TeamB]]

Martin Lafferty wrote:

> Thanks for your reply. I understand why there is a requirement to have
> designtime bpl's on the System path, but think the requirement is
> wrong.

AFAIK, it is a Windows requirement, not one Borland invented.

--
Rudy Velthuis [TeamB] http://velthuis.homepage.t-online.de

"It's clearly a budget. It's got a lot of numbers in it."
-- George W. Bush

[Rudy Velthuis [TeamB]]

Martin Lafferty wrote:

> Agreed, but I don't think this is appropriate for my libraries.
> ($BDS)\Bin is entirely your property; I have got no business putting
> my stuff in there.

Then put your stuff

- in another directory, and add it to the path, or
- in another directory already on the path, or
- in the directory of the .exe.

--
Rudy Velthuis [TeamB] http://velthuis.homepage.t-online.de

"One of the symptoms of an approaching nervous breakdown is the belief
that one's work is terribly important."
- Bertrand Russell (1872-1970)

[Martin Lafferty]

Rudy Velthuis [TeamB] wrote:

> Martin Lafferty wrote:
>
> > Thanks for your reply. I understand why there is a requirement to
> > have designtime bpl's on the System path, but think the requirement
> > is wrong.
>
> AFAIK, it is a Windows requirement, not one Borland invented.

Of course, unless the IDE takes the fairly trival steps necessary to
circumvent it. I have done this now so I am happy.

--

regards

ML

[Martin Lafferty]

Rudy Velthuis [TeamB] wrote:

> Martin Lafferty wrote:
>
> > Agreed, but I don't think this is appropriate for my libraries.
> > ($BDS)\Bin is entirely your property; I have got no business putting
> > my stuff in there.
>
> Then put your stuff
>
> - in another directory, and add it to the path, or
> - in another directory already on the path, or
> - in the directory of the .exe.

I thought I was trying to explain why I don't like any of these
options; clearly I didn't do this particularly well. Please feel free
to consider me excessively particular. Perhaps I am.

I don't know whether you work in a team or on your own, but I have
found that dealing with libraries/components etc. becomes much more
complicated when working with others and trying to maintain ISO9000
quality control procedures across multiple projects.

--

regards

ML

[Liz]

Martin Lafferty wrote:

> I thought I was trying to explain why I don't like any of these
> options; clearly I didn't do this particularly well. Please feel free
> to consider me excessively particular. Perhaps I am.

Forgive me for butting in but you have only 2 choices, 1 to put it in
the current path, 2 make your own and add to the path.. You're saying
you dont want to do either?? While thats your choice, I dont see how
you expect it to work.

> I don't know whether you work in a team or on your own, but I have
> found that dealing with libraries/components etc. becomes much more
> complicated when working with others and trying to maintain ISO9000
> quality control procedures across multiple projects.

ISO9000 just means documenting what you need to do, it doesnt actually
stop you from doing anything.

[Rudy Velthuis [TeamB]]

At 09:47:19, 14.01.2006, Martin Lafferty wrote:

> I don't know whether you work in a team or on your own, but I have
> found that dealing with libraries/components etc. becomes much more
> complicated when working with others and trying to maintain ISO9000
> quality control procedures across multiple projects.

I have never worked with ISO9000 QC procedures, but it should not be too
hard to set up ONE directory for .bpls, and add it to the system path. It
is IMO the only proper way to deal with this. You may not like it, but I
don't quite understand why not.

One directory will do, BTW. Just use lib suffixes, or distinct package
names (like dmylib60.dpk, dmylib70.dpk, dmylib90.dpk and dmylib100.dpk)
to distinguish the packages.

--
Rudy Velthuis [TeamB] http://rvelthuis.de/

"Do you program in Assembly ?" she asked. "NOP," he said.

[Rudy Velthuis [TeamB]]

Why circumvent it? You only need ONE directory for all versions, add that
to the path and use appropriate package names to distinguish the
versions, either using a lib suffix or different project names.
--
Rudy Velthuis [TeamB] http://rvelthuis.de/

"Never raise your hands to your kids. It leaves your
groin unprotected." -- Red Button

[Florent Ouchet]

Hi,

Allen Bauer a écrit :
> ..dcp files should *not* be going there. They should remain with the .dpk

> file (or go to the .dcp output folder which is a different option). Then
> that folder should go in the *library* search path.

I reported this issue some days ago on QC:
Report No: 23153 Status: Reported
Conflict in dcp files because of identical DCP paths.
http://qc.borland.com/wc/qcmain.aspx?d=23153

I will withdraw this issue after adding your answer in a quoted comment.
Thanks for your answer.

--
Florent Ouchet