Re: Specifying an output directory for the build

[Jey Michael]

Is it possible to tell the make-system to place the output in a
different directory?

(A while ago, when I tried to play with the OUT_DIR and build the
source, I had problems)

In the larger picture, I am trying to have something like...
make OUT_DIR=$FEATURE_OUTDIR
// work work work...

// Notice a different bug, so Switch to the bugfix branch
git checkout bugfix
make OUT_DIR=$BUGFIXES_OUTDIR
git commit etc. etc.

// Go back to work on the main feature...
git checkout default
make OUT_DIR=$FEATURE_OUTDIR

-Jey

[Dave Bort]

OUT_DIR *should* do that, but I don't know how many people have tried
changing it. What sorts of problems did you run into before?

--dbort

[Jey]

I think it has got to do with the confusion with that "partner_setup"
script.
Are we supposed to source that build/envsetup.sh and run
partner_setup ?
(Either before/after setting the OUT_DIR?)
I always run that script once. I see the ANDROID_PRODUCT_OUT
directory pointed to
out/target/product/generic, after sourcing that script.

My current build with OUT_DIR has not finished yet, but this is what I
do.

$ . build/envsetup
$ partner_setup
$ echo ANDROID_PRODUCT_OUT
$ mydroid/out/target/product/generic
$ make -j2 OUT_DIR=~/outMain

So far, it seems to go okay. Would you recommend the above, Dave?

thanks,
Jey

On Nov 25, 4:15 pm, Dave Bort <db...@android.com> wrote:
> OUT_DIR *should* do that, but I don't know how many people have tried
> changing it. What sorts of problems did you run into before?
>
> --dbort
>

[Dave Bort]

I think ANDROID_PRODUCT_OUT will point to the right place if you define OUT_DIR before running partner_setup (you can also use choosecombo instead of partner_setup).  E.g.,

$ . build/envsetup.sh
$ export OUT_DIR=other-out
$ partner_setup

The build system doesn't care about the value of ANDROID_PRODUCT_OUT, although some of the tools do (like emulator, adb).  But you shouldn't have any problems actually building the system even if ANDROID_PRODUCT_OUT doesn't agree with OUT_DIR.

You could also define OUT_DIR in a buildspec.mk file instead of in the environment, so that it would be sticky to the tree rather than the shell:

$ cp build/buildspec.mk.default buildspec.mk  # copy to the root of the tree
$ echo "OUT_DIR := other-out" >> buildspec.mk  # or use an editor :)
$ partner_setup
# ANDROID_PRODUCT_OUT should agree with OUT_DIR at this point

--dbort

[Jey]

You are correct, Dave. The simple build went through fine, with my
partner_setup; make -j2 OUT_DIR=~/out10

ANDROID_PRODUCT_OUT (pointing to a different area) did not cause
problems. Verified.

However, when I do a "sdk" build as the next step, I get errors. ie,
make -j2 OUT_DIR=~/out10 sdk

I dont see sdk directory created, and see these errors in the log:

SDK: warning: including GNU target /Volumes/Goo/out10/target/product/
generic/system/xbin/opcontrol
SDK: warning: including GNU target /Volumes/Goo/out10/target/product/
generic/system/xbin/oprofiled
can't create directory /Volumes/Goo/out10/host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86 (No such file or directory)
make: *** [/Volumes/Goo/out10/host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86.zip] Error 44

-Jey
PS: /Volumes/Goo/out10/target/product/generic/sdk/ exists and has one
sdk-build.prop file in it

On Nov 25, 4:54 pm, Dave Bort <db...@android.com> wrote:
> I think ANDROID_PRODUCT_OUT will point to the right place if you define
> OUT_DIR before running partner_setup (you can also use choosecombo instead
> of partner_setup).  E.g.,
>
> $ . build/envsetup.sh
> $ export OUT_DIR=other-out
> $ partner_setup
>
> The build system doesn't care about the value of ANDROID_PRODUCT_OUT,
> although some of the tools do (like emulator, adb).  But you shouldn't have
> any problems actually building the system even if ANDROID_PRODUCT_OUT
> doesn't agree with OUT_DIR.
>
> You could also define OUT_DIR in a buildspec.mk file instead of in the
> environment, so that it would be sticky to the tree rather than the shell:
>
> $ cp build/buildspec.mk.default buildspec.mk  # copy to the root of the tree
> $ echo "OUT_DIR := other-out" >> buildspec.mk  # or use an editor :)
> $ partner_setup
> # ANDROID_PRODUCT_OUT should agree with OUT_DIR at this point
>
> --dbort
>

[Dave Bort]

Hmm, maybe give this edit to build/core/Makefile a shot:

diff --git a/core/Makefile b/core/Makefile
index 224c3cf..bd4143a 100644
--- a/core/Makefile
+++ b/core/Makefile
@@ -1002,6 +1002,7 @@ $(INTERNAL_SDK_TARGET): $(deps)
done; \
if [ $$FAIL ]; then exit 1; fi
$(hide) ( \
+ mkdir -p $(PRIVATE_DIR) \
$(HOST_OUT_EXECUTABLES)/atree \
$(addprefix -f ,$(PRIVATE_INPUT_FILES)) \
-m $(PRIVATE_DEP_FILE) \

If that doesn't work, try adding "showcommands" to your make line to
see what it's trying to do, like "make -j2 OUT_DIR=~/out10 sdk
showcommands".

--dbort

[Jey]

This change created the sdk dir, but still there are errors.

Errors:
~~~~~~
SDK: warning: including GNU target /Volumes/Goo/out10//target/product/
generic/system/xbin/oprofiled
mkdir: /Volumes/Goo/out10//host/darwin-x86/bin/atree: File exists
mkdir: development/build/sdk.exclude.atree: File exists
mkdir: development/build/sdk.atree: File exists
mkdir: development/build/sdk-darwin-x86.atree: File exists
make: *** [/Volumes/Goo/out10//host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86.zip] Error 44

showcommands:
~~~~~~~~~~~~~
Package SDK: /Volumes/Goo/out10//host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86.zip
rm -rf /Volumes/Goo/out10//host/darwin-x86/sdk/android-sdk_eng.Jey_mac-
x86 /Volumes/Goo/out10//host/darwin-x86/sdk/android-sdk_eng.Jey_mac-
x86.zip
for f in /Volumes/Goo/out10//target/product/generic/system/bin/dbus-
daemon /Volumes/Goo/out10//target/product/generic/system/bin/iptables /
Volumes/Goo/out10//target/product/generic/system/lib/libdbus.so /
Volumes/Goo/out10//target/produ
ct/generic/system/xbin/dbus-monitor /Volumes/Goo/out10//target/product/
generic/system/xbin/dbus-send /Volumes/Goo/out10//target/product/
generic/system/xbin/opcontrol /Volumes/Goo/out10//target/product/
generic/system/xbin/oprofiled; do
\
if [ -f $f ]; then \
echo SDK: warning: \
including GNU target $f >&2; \
FAIL=; \
fi; \
done; \
if [ $FAIL ]; then exit 1; fi
( \
mkdir -p /Volumes/Goo/out10//host/darwin-x86/sdk/
android-sdk_eng.Jey_mac-x86 \
/Volumes/Goo/out10//host/darwin-x86/bin/atree \
-f development/build/sdk.exclude.atree -f development/
build/sdk.atree -f development/build/sdk-darwin-x86.atree \
-m /Volumes/Goo/out10//host/darwin-x86/sdk/
sdk_deps.mk \
-I . \
-I /Volumes/Goo/out10//target/product/generic
\
-I /Volumes/Goo/out10//host/darwin-x86 \
-I /Volumes/Goo/out10//target/common \
-o /Volumes/Goo/out10//host/darwin-x86/sdk/
android-sdk_eng.Jey_mac-x86 && \
mkdir -p /Volumes/Goo/out10//host/darwin-x86/sdk/
android-sdk_eng.Jey_mac-x86/tools/lib/images/ && \
cp -f /Volumes/Goo/out10//target/product/generic/obj/
NOTICE.txt \
/Volumes/Goo/out10//host/darwin-
x86/sdk/android-sdk_eng.Jey_mac-x86/tools/lib/images/NOTICE.txt && \
cp -f /Volumes/Goo/out10//host/darwin-x86/obj/
NOTICE.txt /Volumes/Goo/out10//host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86/tools/NOTICE.txt && \
HOST_OUT_EXECUTABLES=/Volumes/Goo/out10//host/darwin-
x86/bin HOST_OS=darwin \
development/tools/scripts/sdk_clean.sh /Volumes/Goo/
out10//host/darwin-x86/sdk/android-sdk_eng.Jey_mac-x86 && \
chmod -R ug+rwX /Volumes/Goo/out10//host/darwin-x86/
sdk/android-sdk_eng.Jey_mac-x86 && \
cd /Volumes/Goo/out10//host/darwin-x86/sdk/ && zip -rq
android-sdk_eng.Jey_mac-x86.zip android-sdk_eng.Jey_mac-x86 \
) || ( rm -rf /Volumes/Goo/out10//host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86 /Volumes/Goo/out10//host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86.zip && exit 44 )




-Jey

[Dave Bort]

Oops, sorry, forgot a semicolon. :) I hope I didn't create too many
spurious directories on you system with this.

diff --git a/core/Makefile b/core/Makefile
index 224c3cf..800cf32 100644

--- a/core/Makefile
+++ b/core/Makefile
@@ -1002,6 +1002,7 @@ $(INTERNAL_SDK_TARGET): $(deps)
done; \
if [ $$FAIL ]; then exit 1; fi
$(hide) ( \
+ mkdir -p $(PRIVATE_DIR); \
$(HOST_OUT_EXECUTABLES)/atree \
$(addprefix -f ,$(PRIVATE_INPUT_FILES)) \
-m $(PRIVATE_DEP_FILE) \

--dbort

[Jey]

> Oops, sorry, forgot a semicolon. :)

Is it the semicolon, or the '&&' ?
Anyway, I see the directory getting created, but the error comes from
within the 'atree' binary.

I cut paste this command from showcommands, and executed it from the
shell.
I still get the same error:

/Volumes/Goo/out20/host/darwin-x86/bin/atree -f development/build/

sdk.exclude.atree -f development/build/sdk.atree -f development/build/

sdk-darwin-x86.atree -m /Volumes/Goo/out20/host/darwin-x86/sdk/
sdk_deps.mk -I . -I /Volumes/Goo/out20/target/product/generic -I /
Volumes/Goo/out20/host/darwin-x86 -I /Volumes/Goo/out20/target/common
-o /Volumes/Goo/out20/host/darwin-x86/sdk/android-sdk_eng.Jey_mac-x86
can't create directory /Volumes/Goo/out20/host/darwin-x86/sdk/android-

sdk_eng.Jey_mac-x86 (No such file or directory)

-Jey

[Jey]

I tried syncing to the latest version now, and then added the extra
line to that Makefile.
But it still fails with the same message as below.

-Jey

[Dave Bort]

I think this may be a problem with atree. You could try adding some
debugging to mkdir_recursively() in build/tools/atree/fs.cpp, to see
why it's failing in this case.

--dbort

[Jey]

You are correct. The fix need to be added to the atree/fs.cpp
The bug is, my OUT_DIR is absolute path, and so the pos initialized to
0 causes a problem.
The fix could be to initialize pos = 1 for those cases, as below:

diff --git a/tools/atree/fs.cpp b/tools/atree/fs.cpp
index a233832..b0d60e3 100644
--- a/tools/atree/fs.cpp
+++ b/tools/atree/fs.cpp
@@ -107,6 +107,10 @@ mkdir_recursively(const string& path)
{
int err;
size_t pos = 0;
+ // For absolute pathnames, that starts with leading '/'
+ // use appropriate initial value.
+ if (0 == path.find('/')) pos++;
+
while (true) {
pos = path.find('/', pos);
-------------------------------
This however does not completely solve the problem. It goes forward
and runs into another issue.

I now get this error:
development/build/sdk.atree:33: couldn't locate source file: out/
target/common/obj/PACKAGING/android_jar_intermediates/android.jar
make: *** [/Volumes/Goo/pandroid.out/host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86.zip] Error 44



The build should not look for "out/target/common/obj/PACKAGING/
android_jar_intermediates/android.jar"
It ought to look for "$OUT_DIR/target/common/obj/PACKAGING/
android_jar_intermediates/android.jar"
I have not been able to find the proper place to fix this. Do you
have a suggestion, Dave?

-Jey

On Dec 4, 12:12 pm, Dave Bort <db...@google.com> wrote:
> I think this may be a problem with atree.  You could try adding some
> debugging to mkdir_recursively() in build/tools/atree/fs.cpp, to see
> why it's failing in this case.
>
> --dbort
>

[Jey]

On Dec 10, 1:05 am, Jey <jey.mich...@gmail.com> wrote:
> You are correct.  The fix need to be added to the atree/fs.cpp
> The bug is, my OUT_DIR is absolute path, and so the pos initialized to
> 0 causes a problem.
> The fix could be to initialize pos = 1 for those cases,  as below:
>

I have added the change to gerrit [http://review.source.android.com/
5336], if you want to review it.

-Jey

[Jey]

On Nov 25, 4:54 pm, Dave Bort <db...@android.com> wrote:
> I think ANDROID_PRODUCT_OUT will point to the right place if you define
> OUT_DIR before running partner_setup (you can also use choosecombo instead
> of partner_setup).  E.g.,
>
> $ . build/envsetup.sh
> $ export OUT_DIR=other-out
> $ partner_setup
>

ANDROID_PRODUCT_OUT is not pointing to the right place, if I follow
the above.
And Its clear that the root cause of all these problems is due to my
absolute paths.
The OUT_DIR use-case is probably not tested for absolute paths
specification.

The rationale for my trying to resolve this is..

Had the directory structure been something like...
mydroid/src/packages
mydroid/src/framework
mydroid/src/external

I would have settled with using relative paths, to do something
like...
mydroid/out1/...
mydroid/out2/...

But, currently 'output' seems to have been hard-coded to sit within
the source area.

Would be very useful, to get this fixed.

thanks,
-Jey

[Dave Bort]

On Wed, Dec 10, 2008 at 1:45 PM, Jey <jey.m...@gmail.com> wrote:
> The OUT_DIR use-case is probably not tested for absolute paths
> specification.
>

That could be; very few people have tried changing OUT_DIR.

>
> Would be very useful, to get this fixed.
>

If you find the underlying problem, please send me a patch and I'll
take a look at it. Thanks for the debugging you've done so far!

--dbort

[Jey]

> > Would be very useful, to get this fixed.
>
> If you find the underlying problem, please send me a patch and I'll
> take a look at it.  Thanks for the debugging you've done so far!

My tools/atree/fs.cpp fix is in the queue. I think Joe is planning to
commit it.
I can followup further.

Dave, Do you have suggestions for this error? I see where it is
coming from in the tools,
but the context is not very obvious to me.

development/build/sdk.atree:33: couldn't locate source file: out/
target/common/obj/PACKAGING/android_jar_intermediates/android.jar
make: *** [/Volumes/Goo/pandroid.out/host/darwin-x86/sdk/android-
sdk_eng.Jey_mac-x86.zip] Error 44

-Jey