[mruby]

[pbosetti]

@jon: I paste hereafter your last message on GH, not to spam the rest of mruby devels.

If I understand correctly what you have in mind, you are planning to use directly cmake to actually build the iOS framework. I think that this could be unnecessarily complicated: we only need to customize the cmake generator in order to create three Xcode targets:

1. an external target that uses the official Makefile and provides the native mrbc and mrblib.ctmp
2. a native target that builds an OS X framework, having 1. as dependency
3. a native target that builds an iOS framework, having 1. as dependency

So that the workflow would be: run cmake -G Xcode .. > open Xcode project > select the desired target (iOS or OS X) > click build > copy the product (mruby.framework) in your project and link it to the code > happy coding.

This way, it should all be easier and more robust. But there is a high probability I'm misunderstanding the whole matter... What do you think?

About Xcode crosscompiling, you are absolutely correct: once you have working target to build a native framework (which is nothing less than a folder containing the binaries of a dynamic lib, the headers, and possibly resources), it is a pretty trivial task to cross compile it for ARM/iOS. So, IMHO the first thing to do is to edit the cake generator template to streamline the Xcode project and add a target for building the framework

I think we're close with the current cross compiling impl.

When cross compiling (i.e - define the CMAKE_TOOLCHAIN_FILE cmd line var) this section of code starts a cross build. Once the cross build needs the mrblib.ctmp dep (created by a native mrbc) this code fires off a full native build (since it's an unmet dependency) and places the native artifacts in build/native.

Effectively, the cross build starts a new native build and uses this code until the native build completes, then switches back to finish cross mode. Check out the 2nd shell example from my cross build post to see what I mean.

The configure phase sets up for a cross build. Make starts in cross mode and runs the firstScanning dependencies of target mruby_object in cross mode. When the build reachesScanning dependencies of target mruby-native, it fires off a complete native build (so mrblib.ctmp is created). Finally, the build switches back to cross mode in the secondScanning dependencies of target mruby_object and completes.

CMake is quite clever in allowing you to this single-pass cross and native build with such clean code and only a single command line invocation. Many CMake examples on the web show cross examples that require users to perform multiple steps. We don't want or need that complexity.

Try cross compiling for Window from your OS X box using this toolchain after downloading a darwin mingw-w64 cross toolchain mingw-w64 prefix indicates a cross tool for building for 64bit windows, and the mingw-32 prefix indicatescross tools for 32bit Windows

This may seem convoluted, but I believe it's fundamentally what needs to happen to get iOS built from Xcode. The existing cross machinery should work with minimal or no modification, and we just need to create a working iOS toolchain file similar to the google links.

The first priority is to tweak the non-cross part of the CMake recipe so that it creates Xcode project files that cleanly build native OS X code. I hope you and @carsonmcdonald can crack the code on that. I'll keep googling, but I'm handicapped without a Mac :(

[Jon]

On Thursday, May 24, 2012 3:02:43 PM UTC-4, pbosetti wrote:

@jon: I paste hereafter your last message on GH, not to spam the rest of mruby devels.

If I understand correctly what you have in mind, you are planning to use directly cmake to actually build the iOS framework. I think that this could be unnecessarily complicated: we only need to customize the cmake generator in order to create three Xcode targets:

1. an external target that uses the official Makefile and provides the native mrbc and mrblib.ctmp
2. a native target that builds an OS X framework, having 1. as dependency
3. a native target that builds an iOS framework, having 1. as dependency

So that the workflow would be: run cmake -G Xcode .. > open Xcode project > select the desired target (iOS or OS X) > click build > copy the product (mruby.framework) in your project and link it to the code > happy coding.

This way, it should all be easier and more robust. But there is a high probability I'm misunderstanding the whole matter... What do you think?

Ah, it looks like I don't understand what an iOS framework is and how it relates to building from the Xcode IDE.

The current Makefile-based cross build support currently builds both native versions and cross versions from a single command line invocation. Since the cross artifacts have a dependency on native mrbc and mrblib.ctmp, the Makefile does the right thing and fires off a native build as part of the cross build when you run something like `cmake -D CMAKE_TOOLCHAIN_FILE=~/path/to/toolchain .. && make` on Linux (and I think on OS X if you've got a cross toolchain installed).

I was hoping that CMake, when run via `cmake -G Xcode -D CMAKE_TOOLCHAIN_FILE=~/crossdev/Toolchain_Xcode_ios ..`, would generate an Xcode project file that would work the way you suggest. You simply (a) import the project (b) select the iOS target, and (c) click build and get an iOS mruby.

To get an OS X mruby, you'd run `cmake -G Xcode ..`, import the project, select OS X, and click build.

But if I understand you correctly, this may not be how Xcode works.

How about we do this first? Let's get the `cmake -G Xcode` scenario working for OS X targets and then move onto cross builds for iOS.  From Carson's comments, it seems the current CMake recipe doesn't generate an Xcode file that works for native OS X builds...missing `libmruby_static` or something. Let's fix this and get things so both you and he can build native OS X mruby using the `cmake -G Xcode ..` generated project file.

Please refresh my memory, does the 'cmake -G Xcode ..` project file allow you to build native OS X from Xcode on your system?

Jon

[pbosetti]

Now, cmake -G Xcode .. creates an Xcode project with 14 targets (= build configurations). You can see a screenshot of the generated project here: http://cl.ly/34040j1t2i2p1Y3f1F0O

The targets with the bull's eye icon are "external" targets, which means there are based on a different building system (GNU make). Those with the small greek temple icon represent "native" targets that build a library or a framework. Those with the dark terminal icon are "native" target that build command line programs.

The ALL_BUILD is the main interface, since it has the dependencies shown here: http://cl.ly/0j2L3h3b1s2J0s45132P

All of these are generated by the cmake generator. Too bad, currently they don't work: somehow, the libmruby.a library is not placed where dependent tasks expect it. I suppose that the way to go is to find the problem in the target settings, then figure out how to edit the cmake settings in order to have the Xcode project generated correctly and accordingly to those changes.

THEN, the next step will be to add a new target for building the framework for iOS. Generally, this simply means duplicating the OS X target and change the two settings you see in the first screenshot named "Architecture" (becomes ARM 7) and "Base SDK" (becomes iOS 5). Then the cmake settings have to be accordingly edited, again.

So yes, I do agree with your suggestion "Let's get the `cmake -G Xcode` scenario working for OS X targets and then move onto cross builds for iOS".

I plan to have a look at the issue starting from friday afternoon, Pacific time.

I'm sure we'll have a lot of fun in the process, since you don't know Xcode and I don't know cmake ;) I hope my explanation was clear enough...

Cheers,

-P.

[Jon]

> Now, cmake -G Xcode .. creates an Xcode project with 14 targets (= build
> configurations). You can see a screenshot of the generated project here:
> http://cl.ly/34040j1t2i2p1Y3f1F0O
> The targets with the bull's eye icon are "external" targets, which means
> there are based on a different building system (GNU make). Those with the
> small greek temple icon represent "native" targets that build a library or
> a framework. Those with the dark terminal icon are "native" target that
> build command line programs.
> The ALL_BUILD is the main interface, since it has the dependencies shown
> here: http://cl.ly/0j2L3h3b1s2J0s45132P
>
> All of these are generated by the cmake generator. Too bad, currently they
> don't work: somehow, the libmruby.a library is not placed where dependent
> tasks expect it. I suppose that the way to go is to find the problem in the
> target settings, then figure out how to edit the cmake settings in order to
> have the Xcode project generated correctly and accordingly to those changes.
>
> THEN, the next step will be to add a new target for building the framework
> for iOS. Generally, this simply means duplicating the OS X target and
> change the two settings you see in the first screenshot named
> "Architecture" (becomes ARM 7) and "Base SDK" (becomes iOS 5). Then the
> cmake settings have to be accordingly edited, again.

Fantastic!...the pics help a ton, thanks.

It looks like CMake generates an ugly looking Xcode project by default ;) I'm going to dig into the CMake doco and to see if some of the internal targets and stuff can be hidden.

Maybe there's a few Xcode specific CMake variables that do this. I see a bunch of darwin stuff in CMake's `share/cmake-2.8/Modules/Platform` dir and there's a `share/cmake-2.8/Modules/CMakeFindXCode.cmake` that looks promising. Time to spelunk.

> So yes, I do agree with your suggestion "Let's get the `cmake -G Xcode`
> scenario working for OS X targets and then move onto cross builds for iOS".
> I plan to have a look at the issue starting from friday afternoon, Pacific
> time.
>
> I'm sure we'll have a lot of fun in the process, since you don't know Xcode
> and I don't know cmake ;) I hope my explanation was clear enough...

:)

Since I don't have Mac/Xcode and you may not have Windows/Visual Studio, I've created a new repo to hold the different types of CMake generated project files. I push the Visual Studio 10 project files to the `vs10` branch and have created empty `xcode` (for native OS X builds) and `xcode-cross` (for iOS builds) branches.

When you get a moment, would you run `cmake -G Xcode ..` from `build/`, copy all the generated dirs/files (except for the `build/CMakeFiles/CMakeTmp/` tree, and push them to the `xcode` branch? I'd like to review and see how they correspond to the pics you sent.

Thanks,

Jon

---
Fail fast. Fail often. Fail publicly. Learn. Adapt. Repeat.
http://thecodeshop.github.com | http://jonforums.github.com/
twitter: @jonforums

[Jon]

Since I don't have Mac/Xcode and you may not have Windows/Visual Studio, I've created a new repo to hold the different types of CMake generated project files.  I push the Visual Studio 10 project files to the `vs10` branch and have created empty `xcode` (for native OS X builds) and `xcode-cross` (for iOS builds) branches.

When you get a moment, would you run `cmake -G Xcode ..` from `build/`, copy all the generated dirs/files (except for the `build/CMakeFiles/CMakeTmp/` tree, and push them to the `xcode` branch?  I'd like to review and see how they correspond to the pics you sent.

unless you're better at reading minds than I am, here's the cmake project files repo: https://github.com/thecodeshop/cmake-project-files

[Jon]

Paolo,

So in Xcode-speak, a "framework" is effectively a sub-project under the main project containing a bunch of code specific to a particular component?

You have to make sure the main target has a build step that builds the framework before continuing, right?

So it's a way of modularizing your code...building a bunch of internal libraries that then get linked to your main target's executable?

I've been googling XCode and it appears there was a location and file structure change between 4.2.x and 4.3.x...from /Developer to /Applications. CMake 2.8.8 had a fix so that it could support 4.3 without users patching CMake and building from source.

IIRC, Carson said he installed Xcode to a non-default location. Seems we need to ensure the CMake recipe is smart enough to support Xcode 4.2, 4.3 and custom install locations. I don't yet know what needs to change in our CMake code, but what do you think should be done?

[pbosetti]

I'll try to give you the short version. The whole structure of OS X (and iOS) development is based on the concept of "bundles". A bundle is a folder with a formalized structure of contents. The Finder (the OS X file browser) sees bundles as files, i.e. does not allow the simple see to open them as a folder (unless with a ctrl-click). The bundle can contain code, configuration files, resources (graphics, sounds, etc), and other bundles too.

An Application in OS X is a bundle: so to install a properly designed application you only have to copy it, since it carries in it every resources and library.

A Framework is a bundle containing a library. it MAY also contain headers (if so it is called a public framework), as well as resources and even other frameworks.

So, when you want to create modular code in Xcode, you usually put libraries in frameworks (and this means that you add to the project a target—which corresponds to a rule in GNU make—that builds a framework), and then build your application with a separate target that has the framework target as dependancy. And it works as with GNU make's rules: if rule A depends on rule B and you type make A, make also executes the rule B (unless it's already done).

Regarding the file structure change, you are right, it changed. Apple deprecated the installation outside /Applications. And honestly, I think that we should accept it: if we try to cope with deprecated setups, we will simply lengthen the "fade out time" of these non-standard configurations. I know that this can be rather a matter of principles: some like the freedom, some like strict rules... But the undeniable good point of OS X is that it "simply works", and it does so by forcing a consistent (and reasonable) set of rules...

There is another file structure change, though, that is more subtle and disruptive: now Xcode—by default, but this can be customized in preferences—puts the results of each build phase NOT in the source directory, but in a subfolder of a standard location ~/Library/Xcode/DerivedData. I haven't checked thoroughly, but I suspect that this is the reason why the current Cmake generated project does not work: its targets are searching for the mruby framework in an absolute path that points to the source directory, instead to a relative path to the build directory (DerivedData). I will check it shortly.

[Jon]

> I'll try to give you the short version. The whole structure of OS X (and
> iOS) development is based on the concept of "bundles".

> [...SNIP...]

Got it. Thanks, you're a good teacher. I just built my first Hello World application on MyBrainBook Pro, and it worked ;)

From your first screenshot, why is everything red under the Products folder?



> Regarding the file structure change, you are right, it changed. Apple *
> deprecated* the installation outside /Applications. And honestly, I think

> that we should accept it: if we try to cope with deprecated setups, we will
> simply lengthen the "fade out time" of these non-standard configurations. I
> know that this can be rather a matter of principles: some like the freedom,
> some like strict rules... But the undeniable good point of OS X is that it
> "simply works", and it does so by forcing a consistent (and reasonable) set
> of rules...
>
> There is another file structure change, though, that is more subtle and
> disruptive: now Xcode—by default, but this can be customized in
> preferences—puts the results of each build phase NOT in the source
> directory, but in a subfolder of a standard location
> ~/Library/Xcode/DerivedData. I haven't checked thoroughly, but I suspect
> that this is the reason why the current Cmake generated project does not
> work: its targets are searching for the mruby framework in an absolute path
> that points to the source directory, instead to a relative path to the
> build directory (DerivedData). I will check it shortly.

After looking at `share/cmake-2.8/Modules/Platform/Darwin.cmake` I think it's just a matter of setting CMake properties on the key targets (libmruby_static library/framework target) to make sure they're built in the locations expected by other targets. Take a look as the syntax is pretty simple. Look for the "SET(<VARIABLE_NAME> ...)" statements.

We may even need to override some of the variables from Darwin.cmake such as:

OSX_DEVELOPER_ROOT
CMAKE_OSX_ARCHITECTURES
CMAKE_SYSTEM_FRAMEWORK_PATH

BTW, have you tried the Mac version of TeamViewer QuickSupport? http://www.teamviewer.com/en/download/index.aspx

We may want to try a session next week to see if we can make any progress.

Finally, please run `xcode-select -print-path` and `sw_vers -productVersion` and reply with output.

[pbosetti]

Got it. Thanks, you're a good teacher. I just built my first Hello World application on MyBrainBook Pro, and it worked ;)

Because it's a Mac. "It just works" ;;;)

From your first screenshot, why is everything red under the Products folder?

Because the products of the build phase are not yet available, i.e. the binary files don't exist yet. After a successful build, they become black.

 

I have TeamViewer, that can be done, absolutely. But first I wanted to tell you the results of what I've found yesterday. 

It turns out that the mruby make uses the -MMD switch, which automatically generates the list of include files for each *c and puts it in the *.d counterparts. Now, Apple compilers (Apple LLVM 3.1 and LLVM GCC 4.2) do not support the -MMD switch. And this is a mess, because creating an Xcode native target is impossible, since the source files lack a lot of #include directives. I don't know if there is a workaround, I'm still investigating. 

You know, I think that this matter of the "native target" is important, if we want to have mruby easily embeddable in iOS projects. I would be curious to know how Carson built its mruby framework, but I suspect he made it by hand (crafting the framework bundle one file at a time), because both of his project on github include a premade framework, but not an Xcode target to build it. So probably he had my very same problems, and he took a shortcut.

Finally, my Xcode installation is standard (path is /Applications/Xcode.app/Contents/Developer) and my OS X version is 10.7.4.

[Jon]

 

I have TeamViewer, that can be done, absolutely. But first I wanted to tell you the results of what I've found yesterday. 

Great!

 

It turns out that the mruby make uses the -MMD switch, which automatically generates the list of include files for each *c and puts it in the *.d counterparts. Now, Apple compilers (Apple LLVM 3.1 and LLVM GCC 4.2) do not support the -MMD switch. And this is a mess, because creating an Xcode native target is impossible, since the source files lack a lot of #include directives. I don't know if there is a workaround, I'm still investigating.

The CMake Xcode project files have -MMD? I know that's in the standard Makefiles, but I thought I made CMake kill that in it's generated files by doing this

https://github.com/thecodeshop/mruby/blob/master/cmake/modules/IntrospectSystem.cmake#L4-13

Would you try changing line 4 to this and see if it makes any difference re: -MMD?

  if(CMAKE_COMPILER_IS_GNUCC OR XCODE)
 
When you get a moment, please clone https://github.com/thecodeshop/cmake-project-files and push the files generated in the `build` subdir when you run `cmake -G Xcode ..` to the `xcode` branch.  Hopefully it's as easy as

git clone g...@github.com:thecodeshop/cmake-project-files.git
cd cmake-project-files
git checkout -b xcode origin/xcode
# copy all the CMake generated Xcode dirs and files (except for build/CMakeFiles/CMakeTmp dir tree)
git push origin xcode

Also, please copy your CMakeCache.txt to a gist or http://hastebin.com/

Also, have you tried configuring using the cmake gui and to see what all the variables it allows you to tweak? Maybe everything works fine and we're just not setting the right cmd line vars when running `cmake -G Xcode ..`.  Here's the gui I get on Win7 when running `cmake-gui` in the build dir: http://www.mediafire.com/imageview.php?quickkey=9tdul9urhuw5xs2

You know, I think that this matter of the "native target" is important, if we want to have mruby easily embeddable in iOS projects. I would be curious to know how Carson built its mruby framework, but I suspect he made it by hand (crafting the framework bundle one file at a time), because both of his project on github include a premade framework, but not an Xcode target to build it. So probably he had my very same problems, and he took a shortcut.

Makes sense. I hope he swings by here and jumps in with his thoughts on how to best address in CMake.

[Jon]

Just a found a cool project called Luminence HDR that uses CMake and supports OS X http://qtpfsgui.sourceforge.net/?page_id=2

From lines 215-238 of this file, they're setting source file properties on the icons and translation files using a CMake property MACOSX_PACKAGE_LOCATION. Wonder if something like that isn't needed, but instead on some of the target files, specifically libmruby.a...

http://qtpfsgui.git.sourceforge.net/git/gitweb.cgi?p=qtpfsgui/qtpfsgui;a=blob;f=CMakeLists.txt;h=f4fe993de59fbcb175d53a539ac82cfd2d740f15;hb=HEAD#l215