I'm going to try an experiment.

Over time, AOSP has added files related to various hardware targets.
We started with just a few scripts on a web page (1.0), then we had a
git project (Cupcake), then we released some of the exact source files
that were used on retail flagship devices (Froyo), then distributed
proprietary binaries (Gingerbread), then we were able to run on
PandaBoard (Ice Cream Sandwich).

For a new challenge, I'd like to try to go one step further, and to
target some hardware beyond the usual categories. I've added a git
project for the Sony LT26, i.e. Xperia S. This seems like a good
target: it's a powerful current GSM device, with an unlockable
bootloader, from a manufacturer that has always been very friendly to
AOSP.

That git project is currently empty. I'm open to suggestions about the
best way to populate it. I think I'll start by putting together a
skeleton set of makefiles, followed by a kernel. Contributions are
strongly encouraged, and there should be more freedom than usual to
submit experimental changes since that won't impact the devices that
Google is most directly involved in.

I don't know how far that'll go, and there are so many unknowns that
the only way to know is to try it.

As usual, please be very careful about handling any proprietary files,
for Xperia S or any other device. Don't copy them, use them, or
distribute them without an appropriate license. Obviously, don't
upload them to AOSP if you don't own them. When in doubt, please ask
ahead of time, it's easier to answer an email than to fix things in an
emergency.

JBQ

--
Jean-Baptiste M. "JBQ" Queru
Technical Lead, Android Open Source Project, Google.

Questions sent directly to me that have no reason for being private
will likely get ignored or forwarded to a public forum with no further
warning.

TL;DR: For a long while this'll probably only be useful for low-level
engineering, but won't be usable as a daily driver.

In theory, AOSP is designed such that it should be possible to plug in
the files related to additional hardware targets. In practice, that
has never happened.

The short-term goal is to investigate the difference between theory
and practice, on a favorable real-world example, and to see whether
the result is worth the effort. The long-term goal is to try to
eliminate whatever hurdles we find, both the expected and the
unexpected ones.

I expect that we'll need at least a few iterations before that can
become usable as a day-to-day phone, and it's unclear whether that'll
ever happen. There's a reasonable chance that it could be useful for
people working at a lower level, literally several layers below the UI
and applications that everyone is familiar with. If you've seen the
classic Android Architecture diagram, we're going to be in the kernel
and just above it.

I don't want to use the word "supported" because there's no way to
agree on a single unambiguous definition.

JBQ

On Tue, Aug 14, 2012 at 12:43 PM, Artem Russakovskii
<arch...@gmail.com> wrote:
> Fantastic. So what's the high level goal here, in layman's terms? Does the
> future hold direct support in AOSP for building more than just Nexus
> devices? Are you thinking of targeting only a certain subset of devices
> (say, ones that satisfy a list of parameters - GSM, unlockable bootloader,
> etc)? What about the lack of proprietary drivers (though even the Nexus
> phones have this problem, and they're supported by AOSP)?
> --
> You received this message because you are subscribed to the "Android
> Building" mailing list.
> To post to this group, send email to android-...@googlegroups.com
> To unsubscribe from this group, send email to
> android-buildi...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/android-building?hl=en

What would be the main differences between the CM device target and the AOSP device target ?

Hi Andreas,

I've submitted my skeleton, with the caveat that it's very bare.

Beware that there's no actual kernel (I've put an empty file in its
place). My next step is to get the real kernel sources in AOSP and to
build my own kernel.

I think the other urgent step is to get a proper BoardConfig.mk in
there. You probably have something suitable.

I'm not incredibly worried about recovery (yet). I guess I'm biased,
since I only work on devices with unlocked bootloaders, so I'm used to
relying solely on fastboot.

JBQ

On Wed, Aug 15, 2012 at 4:47 AM, Andreas Makris
<andreas...@gmail.com> wrote:
> Hey JBQ,
>
> nice challenge you put up here :D
> I am one of the guys maintaining Sony Devices on CM tree. So this device is
> very common to me. OFC it is a powerful device, but it has some tricky stuff
> in it.
> For example the recovery is not this useable than a recovery on a Nexus
> device, so we trigger the recovery boot from boot.img and not from
> bootloader itself.
> Some more special stuff is going on here, but all is possible to handle ;)
>
> So start the skeleton, I am happy to help getting this up and running on
> AOSP.
>
> Best Regards
> Andreas aka. Bin4ry

Well, assuming that we can get to the same level of hardware support
(which is challenging on the AOSP side), we'd obviously have a
slightly different feature set.

I expect that the biggest difference might be more on the philosophy:
while CM is more likely to focus on stability, being based on a stable
tagged release, my work in AOSP is more likely to aim for the bleeding
edge, by using the master branch.

JBQ

Thanks for the information.

I'd love to hear pros and cons of using Sony's unmodified kernel vs CM's.

JBQ

On Wed, Aug 15, 2012 at 3:24 PM, Giulio Cervera
<giulio....@gmail.com> wrote:
> For kernel the start point can be
>
> https://github.com/CyanogenMod/sony-kernel-msm8660/tree/android-msm-3.0-jb
>
> which is based on Sony 6.1.A.0.452 (3.0.8) with kernel/common android-3.0 +
> caf/jb_chocolate merged in
>
> Il giorno mercoledì 15 agosto 2012 17:18:23 UTC+2, Jean-Baptiste Queru ha
> scritto:

>>
>> Beware that there's no actual kernel (I've put an empty file in its
>> place). My next step is to get the real kernel sources in AOSP and to
>> build my own kernel.
>>

>> JBQ
>>

For what i guessed Sony source is based on Arpil/March CAF code for
ICS, and CAF code is based on October/November 2011 AOSP

Sony kernel can be considered more stable, but is not designed for JB

The big difference of CM JB branch is what CAF userspace code can be
used. Few example can be the video dec/enc or the vsync'ed composition
which require updated kernel.

2012/8/16 Jean-Baptiste Queru <j...@android.com>:

CM is "cherry picked" from various sources including but not limited to improvements donated by community. Since I do not have Xperia S, I can't say what exactly, but as a CM nightly user, I can say it would be the right "bleeding edges" AOSP desperately needs.

For one thing, CM kernel would likely be more open to change then Sony's unmodified

The answer will depend on the success of the project :)

In all seriousness, we're made enough progress already to get us to
the point where we're facing the harder problems:
-making changes in the common Android tree that aren't actually
necessary for the devices that Google works on.
-handling the proprietary files.
-restoring the device to its factory state.

I see that as good news. That means that we've already taken care of a
number of smaller problems. Those also don't surprise me, from my
experience managing other devices in AOSP, and so far we haven't hit
any unforeseen hurdle.

JBQ

On Sat, Aug 18, 2012 at 10:16 PM, Tom Gascoigne <t...@gascoigne.me> wrote:
> An interesting idea. Depending on the success of this project, is there any
> chance we might see more non-nexus devices being supported in AOSP in
> future? There are a number of people (myself included) who are maintaining
> AOSP builds for various devices, and I for one would love to contribute back
> to the project.
>
> Tom

As far as end-users are concerned, the potential end result that could
be visible would be upgrades arriving very slightly earlier for Sony
devices, for two reasons:

-it could make the Android tree slightly more friendly to Sony devices
by merging changes that are specifically needed for those devices.

-by providing a concrete support for discussions, it could help Sony
engineers better organize their work to match the way Google engineers
work.

Now, that's entirely speculative, there's definitely now way to
quantify how big a difference this could make or when it would start
to be visible.

JBQ

On Sun, Aug 19, 2012 at 12:45 AM, ken sama <kens...@gmail.com> wrote:
> Salut Jean-Baptiste,
>
> Pourrais-tu concrètement me dire ce que ton projet peut amener pour
> l'utilisateur final que je suis?
> Pouvons-nous nous attendre à voir une version AOSP pour LTi26 débarquer
> rapidement?
>
> Merci d'avance.

To be extremely clear, I'm definitely not in the business of forcing
OEMs to release source code. This part of the discussion ends here.

JBQ

On Sun, Aug 19, 2012 at 1:00 AM, RS <rajes...@gmail.com> wrote:
> Short version:
> JBQ, kudos. Coax manufacturers to release more driver sources.
>
> Long version:
> JBQ, incredible work with AOSP and Nexus already. Kudos.
>
> Now about this experiment, don't we all know from past that the real trouble
> is manufacturers' support for drivers.
> Whenever we move across a major kernel change, it becomes a nightmare to
> hack the old (binary) drivers in without the sources.
>
> Options:
> 1) Utopia: encourage manufacturers to submit sources to such drivers on AOSP
> and let them use "Google f/w update guaranteed" tag through CTS as long as a
> fresh firmware compiled from AOSP passes CTS on their device.
>
> 2) Real-world: Coax them to release as much to AOSP but have an internal
> Google branch where they could share sources with Google just enough of it
> to help rebuild drivers on major kernel change.
>
> Why hang on Sony's word today that they'll assign a couple of engineers to
> provide you updated drivers .... pulling the plug is just a non-tech dumb
> manager's whim away in a corporate. Google might be different but how do you
> see Sony, HTC, Moto-google-rola, LG, ... with dwindling android revenue to
> be reliable?
>
> Convince Samsung, the only non-nexus success on android to open-source to
> the extent possible and others might follow suit.
>
> If they submit bug-fixes well and good otherwise at least the old stuff will
> run on new android flavors.
>
> Thanks,
> RajS

Just so that I get the right kernel, which defconfig should I be using?

I'm going to guess fuji_nozomi_defconfig for now, but I'd like confirmation.

Thanks,
JBQ

On Wed, Aug 15, 2012 at 4:31 PM, Giulio Cervera

Maybe partially answering my own question. The build seems happier
with cyanogen_nozomi_defconfig

JBQ

correct, it's mostly stock sony defconfig whit jb changes from tuna_defconfig

just to make u aware, sony bootloader have some issue and sometime boot fail
we have guessed about certain boot.img size broke boot, change compression
or add a filler file to make the image smaller or bigger fix the issue

also boot partition is very big (20MB) and i would recommend to use
uncompressed
kernel image

G.

2012/8/21 Jean-Baptiste Queru <j...@android.com>:

In jellybean there's a new value called BOARD_RAMDISK_OFFSET in the device-git that the build-git uses
for mkbootimg, could that be the problem?

Regards
//Ernst Sjöstrand -- Sony Mobile

I guess this well timed announcement could come in handy then:
http://developer.sonymobile.com/wp/2012/08/20/sony-opens-up-the-dynamic-android-sensor-hal-dash-developers-can-contribute-open-source/

I'm not at all familiar with how it works unfortunately so I can't help much!

Regards
//Ernst Sjöstrand -- Sony Mobile

https://www.google.com/search?q=unlock+xperia+s+bootloader

JBQ

On Thu, Sep 6, 2012 at 12:59 AM, Akhil Pachaury <akhi...@gmail.com> wrote:
> Thanks for information, but i need help regarding binary installation, I
> download the binary zip file for sony site, how i have to unlock boot
> downloader. Kinldy tell me steps that considering to install those file so
> that i can install AOSP in my sony xperia S.

You need to use the master branch.

JBQ

On Thu, Nov 8, 2012 at 1:06 AM, Jim <zong...@gmail.com> wrote:
>
> Hello,
>
> I am new in this workgroup, and I am available to make some tests.
>
> For the moment,
> - I have retrieved the android-4.0.4_r2.1.
> - I have copied kernel 6.1.A.0.452 files from Sony (kernel, vendor, external
> directories).
> - I have copied device/sony/lt26 files from
> https://android.googlesource.com/device/sony/lt26.git
> - I have copied SW_binaries_for_Xperia_S_v1 files into vendor directory
>
> Now I would like to build the project, but I do not have so much experience
> in android build...
> I ran build/envsetup.sh, and I can see that device/sony/lt26/vendorsetup.sh
> is included.
> But when I run "lunch" and I select full_lt26-userdebug in the list, I have
> the following error message:
>
> build/core/product_config.mk:193: ***
> _nic.PRODUCTS.[[device/sony/lt26/full_lt26.mk]]:
> "frameworks/native/build/phone-xhdpi-1024-dalvik-heap.mk" does not exist.
> Stop.
>
> ** Don't have a product spec for: 'full_lt26'
> ** Do you have the right repo manifest?
>
> Does anybody could tell me how I can solve the issue ? Does my overall
> method correct ?
>
> Thanks a lot !

Your commands are correct.

You can also turn your 4.0.4 client into a master one, with "repo init
-b master" in that client (followed by repo sync, of course).

master is bleeding edge, so it's ahead of Jelly Bean.

JBQ

On Thu, Nov 8, 2012 at 7:58 AM, Jim <zong...@gmail.com> wrote:
> Thanks a lot jean-Baptiste.
> Just to be sure, I guess you talk about the android master branch, i.e. I
> have to execute:
> repo init -u https://android.googlesource.com/platform/manifest
> instead of
> repo init -u https://android.googlesource.com/platform/manifest -b
> android-4.0.4_r2.1
>
> But how to know if the master branch is android ICS or JB or whatever ?
>
> Jim

>
>
> Le jeudi 8 novembre 2012 16:38:47 UTC+1, Jean-Baptiste Queru a écrit :
>>
>> You need to use the master branch.
>>
>> JBQ
>>
>>

On Thursday, November 08, 2012 at 12:26 EST,
     Jim <zong...@gmail.com> wrote:

> ok perfect.
> So I have retrirved the master branch, and th elunch is now ok.
>
> but the make fails :o(
>
>  make -j8 VERBOSE=1
>
> ============================================
> PLATFORM_VERSION_CODENAME=AOSP
> PLATFORM_VERSION=4.1.2.3.4.5.6.7.8.9
> TARGET_PRODUCT=full_lt26
> TARGET_BUILD_VARIANT=userdebug
> TARGET_BUILD_TYPE=release
> TARGET_BUILD_APPS=
> TARGET_ARCH=arm
> TARGET_ARCH_VARIANT=armv7-a-neon
> HOST_ARCH=x86
> HOST_OS=linux
> HOST_OS_EXTRA=Linux-2.6.35-30-generic-x86_64-with-Ubuntu-10.04-lucid
> HOST_BUILD_TYPE=release
> BUILD_ID=OPENMASTER
> OUT_DIR=out
> ============================================
> Checking build tools versions...
> build/core/host_shared_library.mk:22: *** external/e2fsprogs/e2fsck:
> Can not set module stem for a library.  Stop.

This indicates that you haven't actually synced and checked out
the current master of external/e2fsprogs, because the reference
to LOCAL_MODULE_STEM that make complains about was removed in
commit dc3069a about a year ago. The most recent commit in that
git should be 5886dc5, and that's the commit you should have
checked out. Try this (and post the output):

cd external/e2fsprogs
git rev-parse HEAD
repo sync .
git branch -a | grep remotes/m/
git rev-parse HEAD

--
Magnus Bäck
ba...@google.com

On Monday, November 12, 2012 at 02:28 EST,
     Jim <zong...@gmail.com> wrote:

> Thanks a lot for your support, and sorry for the late answer, I was
> not there during the last three days.
> I think I have already the latest commit. Here is the git log:
>
> external/e2fsprogs]$ git log
> commit 5886dc5cdcccd3d09a208d41d8c23748c25a2a22
> Merge: 0a02698 fada366
> Author: Theodore Ts'o
> Date:   Tue Jun 12 12:53:18 2012 -0700
>
>     am fada3660: e2fsck: correctly propagate error from journal to
> superblock
>    
>     * commit 'fada366033e80c119867eb303e8b48a8e027a9be':
>       e2fsck: correctly propagate error from journal to superblock
>
>
> I have executed on last Thursday a repo sync on master branch, so I
> guess I have the latest commits almost every where.

If you look in the Android.mk files below external/e2fsprogs, is
LOCAL_MODULE_STEM set for any shared library module (actually,
anything but executable or host executable)? Show the output of the
following commands (run from the e2fsprogs root):

git status
grep LOCAL_MODULE_STEM */*.mk

--
Magnus Bäck
ba...@google.com

On Monday, November 12, 2012 at 11:42 EST,

     Jim <zong...@gmail.com> wrote:

> > If you look in the Android.mk files below external/e2fsprogs, is
> > LOCAL_MODULE_STEM set for any shared library module (actually,
> > anything but executable or host executable)? Show the output of
> > the following commands (run from the e2fsprogs root):
> >
> > git status
> > grep LOCAL_MODULE_STEM */*.mk
>

> Hi Magnus, here are the logs:
>
> /external/e2fsprogs]$ git status
> # Not currently on any branch.
> nothing to commit (working directory clean)
>
> /external/e2fsprogs]$ grep LOCAL_MODULE_STEM */*.mk
> e2fsck/Android.mk:155:LOCAL_MODULE_STEM := e2fsck
> misc/Android.mk:77:LOCAL_MODULE_STEM := mke2fs
> misc/Android.mk:150:LOCAL_MODULE_STEM := tune2fs
> misc/Android.mk:221:LOCAL_MODULE_STEM := badblocks
> resize/Android.mk:64:LOCAL_MODULE_STEM := resize2fs

It looks like your source workspace is okay. I don't have time to help
you any further. You should get to the bottom of which module it's
actually complaining about (e.g. by commenting out parts of the
makefiles, piece by piece until it stops yelling) and take it from
there.

--
Magnus Bäck
ba...@google.com

On Tuesday, November 13, 2012 at 02:34 EST,
     Jim <zong...@gmail.com> wrote:

> I cann fully understand that yo do not have more time to support me.
> In fact on top of the master branch, I have copied the sources files
> from the Sony delivery 6.1.A.0.452, in order to have kernel source
> files:
> http://developer.sonymobile.com/downloads/xperia-open-source-archives/open-source-archive-for-build-6-1-a-0-452-6-1-a-0-453-and-6-1-a-0-454/

6.1.A.0.452 is ICS-based and won't work out of the box with the
JB-based master branch. Unless you really know what you're doing
I don't recommend trying it.

> This delivery contains several directories in /external,
> including external/e2fsprogs
> I will make new tests, and try to include directories one by one.

What do you mean, did you get the build problems when including the
source code from Sony's 6.1.A.0.45[234] drop? That would explain the
build error but wouldn't explain why Git reported external/e2fsprogs
to be clean and have the correct commit checked out.

--
Magnus Bäck
ba...@google.com

On Wednesday, November 14, 2012 at 03:13 EST,
     Jim <zong...@gmail.com> wrote:

> I am a little bit confused: from Jean-Baptiste answer, I understood
> that master branch contains ICS android sources, not JB.

No, the master branch contains Jellybean and some other stuff. You can
assume that the master branch is always ahead of anything on release
branches.

> Here is what I want to do:
> Be able to build a full android image, with the associated kernel, and
> flash it in my Xperia S.
>
> For the kernel sources, I only found ICS (6.1.A.0.452). So I guess it
> is easier to make this exercise with ICS.
> Then I first tried to use branch android-4.0.4_r2.1 to be sure to have
> ICS, but I understood that the lt26i is not supported in this branch.
> Jean-Baptiste then told me to simply retrieve the master branch, that
> is in fact ICS. Is it right ?

The master branch is closer to Jellybean than to ICS. You need the
master branch to get the device configuration, but the code that Sony's
released is for ICS. Yes, this makes things harder. You either have to
retrofit the LT26 setup from the master branch to an ICS branch (sounds
pretty easy) or use the ICS-based Sony software on top of Jellybean.

> The compilation of this matser branch is ok in lt26i device mode
> (if I download vendor/sony/lt26 git).
> However, I also made the test to copy external and vendor files
> from 6.1.A.0.452 delivery, and then the compilation issue occurs.

You can't just copy the files. You'd have to figure out what Sony has
done in those directories and apply the same patches to a Jellybean
tree. They've changed about 80 files in the Webkit tree but I don't
think they've made any significant changes anywhere and probably none
that are necessary for the phone to boot. The reason those directories
are included in the archive is because they're under GPL or LGPL.

> (And the error does not disapear if I only change the
> external/e2fsprogs to the original one).

Then you didn't restore the original tree.

--
Magnus Bäck
ba...@google.com

On Friday, November 16, 2012 at 04:36 EST,
     Jim <zong...@gmail.com> wrote:

> Something is still not very clear for me: the dependence between the
> kernel and the android source code.
>
> I have the kernel source code 6.1.A.0.452. I understood that it was ICS.

Correct.

> So I guess that if I switch to android master branch that is JB,
> I will need the JB kernel.
>
> Am I right ?

An ICS kernel might work, it depends on the actual kernel changes
between ICS and JB. It might be possible to apply the changes between
the stock ICS kernel and Sony's ICS kernel onto the stock JB kernel,
but that probably wouldn't be a trivial task.

> Where could I find this kernel ?

Sony hasn't released any JB-based kernel sources. I think they've
announced that JB updates for their phones will be released during
Q1 2013, so I wouldn't expect kernel sources to be publicly available
prior to that.

--
Magnus Bäck
ba...@google.com

> I have executed a repo sync on the branch android-4.0.4_r2.1.
>
> Then I made a git clone of :
> https://android.googlesource.com/device/sony/lt26.git
> (directory device/sony did not exist, so no merge needed)

You can also do this by adding the lt26.git in a local manifest file, as described in the README on the github project [1].  Then when you repo sync it will automatically fetch it for you.

[1] https://github.com/sonyxperiadev/device-sony-lt26

--
David Pursehouse
Sony Mobile Communications Japan, Inc

Jim,