Building XNU?

[Mountainstorm]

so I've got XCode 3.2.3 and XCode developer preview (4A251a) installed
on my box. Unfortunately, try as I might, I can;t get XNU to build
(or CF .. anything infact).

Darwinbuild builds and installs fine (I'm on Snow Leopard), and it
even init fine. But once I try building anything I get loads of
errors like

DEVELOPER_DIR = '/Developer' via '/usr/share/xcode-select/
xcode_dir_path'
Database Key is: cc____/Developer____/Users/cooper/Documents/Dev/
darwin/darwinbuild/10J869/BuildRoot
Cache version is '1'
Looking up with ''/Developer/usr/bin/xcodebuild' -sdk '/Users/cooper/
Documents/Dev/darwin/darwinbuild/10J869/BuildRoot' -find 'cc''
2011-05-25 06:06:44.356 xcodebuild[6988:607] Unable to load platform
at path /Developer/Platforms/Darwinbuild_9G55_20100818131354.platform
2011-05-25 06:06:44.361 xcodebuild[6988:607] Warning: SDK name
collision for name 'macosx0.0': SDK at path '/Developer/SDKs/
MacOSX10.5.sdk' being superceded by SDK at path '/Developer/SDKs/
MacOSX10.6.sdk'.
2011-05-25 06:06:44.388 xcodebuild[6995:607] Unable to load platform
at path /Developer/Platforms/Darwinbuild_9G55_20100818131354.platform
2011-05-25 06:06:44.391 xcodebuild[6995:607] Warning: SDK name
collision for name 'macosx0.0': SDK at path '/Developer/SDKs/
MacOSX10.5.sdk' being superceded by SDK at path '/Developer/SDKs/
MacOSX10.6.sdk'.
2011-05-25 06:06:44.431 xcodebuild[7002:607] Unable to load platform
at path /Developer/Platforms/Darwinbuild_9G55_20100818131354.platform
2011-05-25 06:06:44.434 xcodebuild[7002:607] Warning: SDK name
collision for name 'macosx0.0': SDK at path '/Developer/SDKs/
MacOSX10.5.sdk' being superceded by SDK at path '/Developer/SDKs/
MacOSX10.6.sdk'.
/Developer/usr/bin/xcodebuild fails with 11 - Resource deadlock
avoided
make[3]: execvp: /Users/cooper/Documents/Dev/darwin/darwinbuild/10J869/
BuildRoot/SourceCache/xnu/xnu-1504.9.37: Permission denied
/Developer/usr/bin/xcodebuild fails with 11 - Resource deadlock
avoided
make[3]: execvp: /Users/cooper/Documents/Dev/darwin/darwinbuild/10J869/
BuildRoot/SourceCache/xnu/xnu-1504.9.37: Permission denied
/Developer/usr/bin/xcodebuild fails with 11 - Resource deadlock
avoided
xcrun via /usr/bin/xcrun (xcrun)

I also don't understand why its trying to load the 9G55 platform as I
darwinbuild -init 10J869.

any ideas?

Also, how do you go about changing the gcc build settings? I want to
change the compiler options so that it only compiles using the 486
instruction set.

Rich

[Aladin]

Hi Rich,

Are you able to successfully build a simple project like "zlib" or
"file_cmds" per example?
Does it work with the -nochroot option?
Did you use the "trunk" (http://svn.macosforge.org/repository/
darwinbuild/trunk/) version of DarwinBuild? Cause this one is intended
for Snow Leopard (Mac OS X 10.6.x, Darwin 10).

Concerning using the 486 instruction set only, you have to set
RC_ARCHS = "i386"; instead of RC_ARCHS = "i386 x86_64"; in a custom
plist.

Unfortunately, I don't have a Mac right now to test or reproduce it,
but perhaps you didn't properly prepare the environment and initialize
DarwinBuild.

I hope it can help you a bit!

Best,
Aladin.



On 25 mai, 16:57, Mountainstorm <mountainstor...@googlemail.com>
wrote:

[Mountainstorm]

I'm using the trunk build of darwinbuild and can sort of build CF. It
actually builds and I don't get all these strange errors but it fails
as the Foundation/* headers are missing (is this normal).

I was also having trouble building the kernel via its makefile (same
sort of errors), but have now solved that by following these
instructions (but for the 10.* series kernel)
http://osx86.boeaja.info/2009/10/building-xnu-kernel-on-snow-leopard/

It appears that xcodebuild moans about all sorts of random things like
relpath not set when what it means is you don't have /usr/local/bin/
relpath. Once I build and installed dtrace, cxxfilt, kext_tools and
bootstrap_cmds the make file worked and managed to build a kernel.

The kernel panic'd when I popped it on the darwin nano img (in
VirtualBox) but that appears to be a much more manageable problem;
i.e. it doesn't like the EFI interface version info VirtualBox
provided it.

So in summary, some progress. I've just tried building cxxfilt using
darwinbuild and that seems to fail. I've got errors about sdk
versions, resource deadlock avoided:
2011-05-26 01:39:13.018 xcodebuild[31448:607] Warning: SDK name

collision for name 'macosx0.0': SDK at path '/Developer/SDKs/
MacOSX10.5.sdk' being superceded by SDK at path '/Developer/SDKs/
MacOSX10.6.sdk'.
/Developer/usr/bin/xcodebuild fails with 11 - Resource deadlock
avoided

but it actually fails in configure with this error:
checking whether the C compiler ( -arch i386 -g -Os -
pipe ) works... no
configure: error: installation or configuration problem: C compiler
cannot create executables.
make: *** [/Users/cooper/Documents/dev/darwin/darwinbuild/10J869/
BuildRoot/private/var/tmp/cxxfilt/cxxfilt-9.obj/configure-stamp] Error
1

erm ... I'm sort configure but that C compiler does work ... I know, I
used it to build you manually!

Any thoughts?

Rich

[Aladin]

Re,

Maybe you will find more details on the "config.log" file.

I remember that I used to "impersonate" darwinbuild configure/build process in order to troubleshoot problematic projects, I was simply "cd" to the directory where the source are mount and should be build, and then run manually configure, xcodebuild or make depending the project with more verbose flags.

Also concerning cxxfilt, take a look between http://www.puredarwin.org/developers/darwinbuild/troubleshooting/darwin-9 and http://www.puredarwin.org/developers/darwinbuild/troubleshooting/darwin-10.

On Darwin 9, because of many dependencies, it tooks 5 big passes in order to be able to build successfully cxxfile; meaning others projects need to be built and installed in the buildroot, prior to cxxfile.

On Darwin 10, it failed on 10B504 but succeeded at first pass on 10C540.

Also the latest binaryroot of cxxfile built and available from macosforge is http://src.macosforge.org/Roots/10C540/cxxfilt.root.tar.gz

Best,

Aladin.

[Aladin]

Concerning PureDarwin nano, your problem could not only come from the EFI of VirtualBox.

Keep in mind nano was targetting Darwin 9 and I'm not sure xnu based on Darwin 10 will work with the kexts, the libs and the binaries based on D9.

It is also possible (and even plausible) that few things changed and need to be adapted. And of course it depends the hardware support.

Pure curiosity, but on which kind of lines the panic occured?

Two pointers in case: http://www.puredarwin.org/downloads/puredarwin-nano and http://www.puredarwin.org/developers/virtualbox

Best,

Aladin.

[Mountainstorm]

How do I build system.kext? they introduced the kpi dependency stuff
in 10 so I need a new system.kext.

I dont see one built in the BUILD/obj/RELEASE ... dir, is it a
separate build target on the XNU makefile? Theres something in the
config/ directory but its not right as all the versions show as
####KERNEL_VERSION_LONG###

Thoughts?

Rich

[Aladin]

Indeed it is part of the XNU project, as we can see here: http://www.opensource.apple.com/source/xnu/xnu-1504.7.4/config/Makefile

Proof is inside http://src.macosforge.org/Roots/10J869/xnu.root.tar.gz

Maybe building your kernel outside/without DarwinBuild is the cause of your missing System.kext or it has been built but you simply didn't see it.

Keep in mind that I cannot verify anything without my mac (far away from me) and I dropped myself out of PureDarwin since a year.

FYI, system.kext is just a container which contains more kexts inside.

[Mountainstorm]

you may be correct about the darwinbuild thing. I've been playing
around with darwinbuild again and just tried:

sudo darwinbuild -nochroot xnu

and, it appears to have worked ... I don't know why and if I'm honest
I'm not completely sure what difference not using chroot makes but ...
yay? I now have a nice tidy mach_kernel, system.kext, plus all the
headers in a nice clean folder structure.

A couple of questions:

1. how (via darwinbuild) can I specify build options e.g. the
equivalent of ARCH_CONFIGS, do I do it via exporting the variables
into the current shell?
2. if I want to build my own project in the darwinbuild environment
how do I? I've compiled AppleI386GenericPlatform (imported from
Darwin 8) and when darwin boots its complaining about not being able
to patch the classes parent vtables. I suspect its due to compiling
against the wrong platform but cant see how I'd change the xcodeproj
SDK to be my darwin build env.
3. why did you leave the puredarwin project ... and is there anyone
else left?

Thanks for your help

Rich

[Aladin]

If it worked with the "-nochroot" option, I believe your build root directory is not complete.

There is a big difference when you're using or not the chroot. Your result is "built on Darwin" in the chroot, and "built on Mac OS X" outside the chroot. In terms of dependencies too.

1. Yes you can specify build options as RC_ARCHS = "i386 x86_64"; , in a custom .plist file which inherits from a default one (or not recommended, in the default ones you retrieved). Exporting the variables to the current shell env is not a good idea, and i'm not sure it is still working as in the past.

Take a look at http://svn.macosforge.org/repository/darwinbuild/trunk/plists/10A432.plist (for options) and http://svn.macosforge.org/repository/darwinbuild/trunk/plists/10B504.plist (for inheritance) and http://svn.macosforge.org/repository/darwinbuild/trunk/plists/10J869.plist (for xml style).

2. To build your own project via/with DarwinBuild, first you have to be more familiar with DarwinBuild. Then it should be straightforward, look libelf or libdwarf in http://puredarwin.googlecode.com/svn/trunk/plists/9G55pd1.plist. There are few subtil things like where the Makefile (per example) is placed in the level of directories. If it is not your source code, you could need to "repackage" it order to make it suitable for DarwinBuild build process.

Concerning AppleI386GenericPlatform (you said imported from Darwin8), you will probably need to adapt the sources.

Take a look at https://sites.google.com/a/puredarwin.org/puredarwin/developers/pe#TOC-Other-Platform-Experts

3. https://groups.google.com/d/msg/puredarwin/P6AbgK4_PzU/a5F2HZvlw4MJ

[Mountainstorm]

Cheers for that, I'll have a look into my darwinbuild; can you think
of any ways I can try to debug my build root? I'm guessing what you
mean is that I'm missing some tools from it and thus need to
darwinbuild xxx (something else) before I it will work ... are there
any tips for finding out what.

1. cheers, I'll look into it
2. cheers, I've solved my problem, it wasn't an SDK one but instead a
kernel dependency one. I've now got a compiled version up and running
although it still relies on com.apple.kernel dependencies (and kpi
ones) and I've currently got some of the code commented out as I
haven't put the corresponding code back in pexpert/i386 yet.
3. Thats sad, particularly as it appears your still the most active
member :(

I can see why you stopped though. My plan was to see if its possible
to get darwin running on a bifferboard (http://bifferos.bizhat.com/)
and when its released a raspberry pi (http://www.raspberrypi.org/); no
X11, not gui ... just a minimal kernel and console (nano style); with
the hope of learning more about the internals of xnu along the way.

I got VirtualBox to load my minimal kernel, and things looked good
although I cant get it to boot the root device; it keeps saying
"waiting for root device". As my actual target platform is much older
I've switched to QEMU.

Problems I've noticed so far are:
1. if you run QEMU as default you just get a black screen (I suspect
the compilers used an unknown instruction or chameleon has loaded a
64bit variant) - temporarily solved by use of "-cpu core2duo".
2. You get the local APIC version 0x11, darwin needs 0x14 ... but my
bifferboard wont have an APIC as its a 486; humm
3. I suspect I'll have to implement (or forward port) non-PAE paging

Fun

Rich

[Aladin]

I think you're on your own concerning going deeper in DarwinBuild. I remember in the past it was needed to load some binaryroots in the build root via "darwinbuild -load ...", however these steps were not needed anymore when targeting D10 cause missing dependencies should be handled by Darwinbuild.

Concerning QEMU, indeed it has to be tweaked a bit in order to run PD. I made PD run with network support under QEMU on mac, linux and windows, take a look at: http://www.puredarwin.org/developers/qemu

About VirtualBox, take a look at http://www.puredarwin.org/developers/virtualbox to see what to do to make it run.

About APIC 0x14, ACPI seems to be a mandatory requirement from XNU but maybe there are ways to bypass that. There is also other requirements like SSEx support from your cpu, not sure a 486 family has that, but it is/shouldbe also possible to circumvent it via soft emulation: http://code.google.com/p/xnu-dev/downloads/detail?name=SSE3%20Emulator.pdf

I would be glad to step again inside PD, but according to the existing blockers in D9 and the missing of any binary drivers for D10.. it's not realistic at this time.