Upcoming new rpi specific package feed for openwrt

[Osku]

Hi.

I previously posted a patch on the trunk tree for getting a working 3.12-y kernel for OpenWrt in Raspberry Pi.

Now I am announcing that I have been working real hard here building a new package feed for OpenWRT which is specific for Raspberry Pi.

Currently I have succesfully built following tools for native development on raspberry pi:

autoconf

automake

bison

gcc (4.6-linaro, host must use same version)

m4

Following libraries:

cairo (with egl support)

libX11

libXau

libXcursor

libXdmcp

libXfixes

libXrender

libmpc

libmtdev

libunwind

libwayland

libxcb

libxkbcommon

pixman

mpfr

following protocols:

fixesproto

inputproto

kbproto

renderproto

xcbproto

xextproto

xproto

following utils:

fontconfig

xtrans

following data packages:

xkeyboard-config

following raspberry Pi support:

rpi-userland ( tomeuv's wayland branch )

+ all examples under submenu:

rpi-helloaudio

rpi-hellodispmanx

rpi-helloencode.. etc..

And I am also bringing in the game:

weston

That needs most of the packages listed (except the development packages).

Weston also has options:

enable egl?

build alternative compositors: wayland-compositor? fbdev-compositor?

support xwayland?

support libunwind?

support dbus?

---

All this works, except EGL enabled weston, it compiles and runs just fine, but screen is empty and if I force start, for example; weston-terminal, screen goes back to console, although weston still runs- didn't find anything ugly in debug log either (yet)..

So EGL is disabled as default until I or someone gets it to work.

---

So I told that I am announcing and yes, this isn't available yet as I want to include few more packages:

 - Enlightenment Foundation Libraries + samples (with egl)

 - tslib ( haven't checked if it's already in repository, I just provided it a update awhile ago, so it propably needs to be updated in openwrt packages as well..)

 - opengalax2 (this isn't available anywhere yet, it's my own fork of opengalax and it uses tslib to provide touch screen support for.. weston and others.. so common "cheap" 7" hdmi display set with opengalax touchscreen will be supported)

And maybe some more, haven't decided completely yet..

Also then there is a huge work ahead: cleaning up my package feed, removing unnecessary comments, last minute changes.. Adding of possible init.d files.. And so on..

---

I know you guys are fond to using eglibc- unfortunately, I built this to work with the upstream openwrt, so it's designed for gcc-4.6-linaro and default uclibc.

So what do you guys think about it?

[Osku]

I also added option to build weston with weston launch which then requires libpam as dependancy.

---

Building with everything I provide-- Makes openwrt quite big. (but still, all that I currently have and all that I am about to include, should fit to less than 300mb) but idea is to bring openwrt to same level with other distributions available for Raspberry Pi,

how big your image grows depends on what you do on your pi, if you just are wanting a basic openwrt image without development tools and can do with a stripped down version of weston and decide to leave out rpi demos as well.. Image won't be a lot

larger from normal openwrt image..

[Osku]

Included libtool utilities inside automake package.

Just tested native build environment by building weston natively. That works. I also wanted to add cmake package - but it seems to be impossible- cmake build does not seem to support cmake creation for different architechture (cross compilation).

EGL still not working. Don't really know why. I was able to get EGL working with weston in ArchLinux.. Well, cairo-egl is there.. Hopefully I also get EFL egl when I start with EGL..

---

Added some extra packages to rpi-userland package:

rpi-tvservice

rpi-raspicam

rpi-vcgencmd

I also plan to convert vcfiled's init script for openwrt so I can add that too..

---

No free week from work at now, so all this slows down a bit..

[Osku]

Finally I have managed to get EFL to build, some minor things still need to be done until I'm done with EFL. It's a rather minimal build of EFL, meaning that I left _most_ of the stuff out as it would otherwise be dependant on so many things, and as the library needs to be built for both; host and target, I also leaved audio support out from it. Still, before stripping binaries, package is huge, about 500kb - containing only libraries and support files. Also, cross compilation of EFL examples isn't going to happen anytime soon, so there are no examples for EFL. But.. I am adding also Elementary package which allows easy/simple build of applications, elementary needs EFL and I have had good experiences with it in the past, so I hope this will help people with what ever they are building with RPi - if they need graphical UI..

Also, I think, I have gotten EGL issues sorted out now. Some more testing on the way, the problem was too old firmware that OpenWRT ships with. I upgraded it and now I have plans to create my own rpi-update utility that can be used to update firmware. Yes, I know, rpi-update already exists, but openwrt has it's own kernel, and I don't think it's a good idea to mess with kernel and modules with this tool - and I want it to work with openwrt packages. But more about that later, when I first get current issues worked out.

btw. I also thought, that I could try to add a minimal web browser as a package, but don't yet know how that will go..

[Osku]

libEFL before stripping: 1.5megabytes

Elementary before stripping: 7.9megabytes

now fine tuning these before moving on to tslib..

Then I need to check my opengalax2 driver before I publish it on github and then make a package of it..

Then I guess I am ready to start last minute checkings before publishing my package tree..

---

But I'm doing something else during christmas, so stay tuned..

[Osku]

libEFL comes with 2 packages: library package + native development support package

elementary comes with 3 packages: libraries + native development support package + pre-compiled examples.

libEFL and elementary are compiled to support fb, wayland-shm and wayland-egl

---

Forgot that I had promised to try add that web browser package as well..

[Navid Nadali]

Great work Osku.

Best wishes,

--
You received this message because you are subscribed to the Google Groups "Raspberry Pi Openwrt Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to raspberrypi-openw...@googlegroups.com.
To post to this group, send email to raspberrypi...@googlegroups.com.
Visit this group at http://groups.google.com/group/raspberrypi-openwrt-dev.
For more options, visit https://groups.google.com/groups/opt_out.

[Osku]

Thanks.

Couldn't resist. I finalized my opengalax2 driver, renamed it to opengalax2 so I can support both, original serio using version and my usb supporting version.
Now I really should try to take some time off.. Marry christmas to all..

Oh, what is it? It acts as a bridge between tslib and weston, so devices supported by tslib can be automaticly supported by weston if not supported natively by weston:

https://github.com/oskarirauta/opengalax2

[Osku]

Okay- I was promising a web browser.. Well.. We got to forget that for the moment, I was thinking about this: Eve

Just because I was previously adding some enlightenment libraries and those would had supported this browser;

Also, chromium for example needs to use some sort of "ninja" build system, too compilicated.. Might make cross compiling

impossible or.. atleast openwrt buildroot lacks support for it and support cannot be added in package tree..

But.. This requires building of some additional stuff like webkit and webkit-efl..  And.. Well, maybe I'll go for this idea

later, or maybe I have been providing a lot of tools so I can leave some for others as well- it's just that at this moment,

I don't think I have enough time for starting to add a huge project like webkit.. And webkit-efl didn't seem very simple

either..

---

Anyway.. Good news in this case, as my todo list so far regarding to missing packages has only one item;

which is custom rpi-update script for openwrt.

---

Then there's a lot of shining to do, adding md5sums of packages.. Weston needs an upgrade.. (fixes nested-server and nested-client examples)..

and some init scripts missing.. Then there's patches that I have made to buildroot, some might be included in mainstream already, but some propably aren't,

and some.. I propably haven't sent at all, therefore, I need to install my package tree to "fresh" git clone of openwrt tree, so I can make directions on

which patches to apply, if not yet applied in mainstream..

And xorg tree cannot be used if you use my tree. And if I remember right.. One package from package feed also needs to be removed, as my tree provides

the same package with different options..

Too bad, I need to go to work tomorrow.. But things are going to a better direction.

[Osku]

Okay.

rpi-update script is finally done now. It is just a shell script, but I made it so, it ain't dependant on git or actually nothing that doesn't come out of the box; except: kmod-fs-vfat and kmod-nls-cp437 which are modules necessary for mounting boot partition.

It is a simple script, that does not allow upgrading to a specific revision like original script, it only makes it possible to update to latest revision of chosen branch.

Without arguments it gives a help notification like this:

rpi-update v0.2(OpenWRT version)

usage: rpi-update [options]

       -c, --c: check for updated version of firmware

       -u, --u: upgrade firmware (if not running latest)

       -n, --n: use branch "next" instead of "master"

You can run it with -c just to check for firmware updates, add -n if you are using branch next. Branch used is not recorded anywhere, just like the original rpi-update doesn't record it. First time, it doesn't know your current firmware revision, because openwrt build root currently does not create .firmware_revision file containing revision of firmware. I might send a patch to openwrt, but it is very unlikely to be added.. But this is not a problem.

If there is a upgrade and you decide to go for it.. Run with -u and add -n if you want to use branch next.

---

It doesn't ask anything like are you sure and so on, so be careful with it. It automaticly mounts /dev/mmcblk0p1 to temporary location, or if already mounted, uses that location. I added few sync's to the soup to avoid getting into problems.

---

So more ore less- I have created all the packages I wanted and now I need to start "cleaning up" the tree before uploading this to github.. Most boring part.. Although, I have some testing to do first..

[Osku]

Sorry that status updates took some time - been really busy here lately..

First of all, some changes:

 - now both Linaro-gcc compilers, 4.6 and 4.8 are currently supported. Choose one for your toolchain, and if you build a package for native gcc, it will be using the same version.

  I would recommend personally 4.8 version, because it solves some problems, for example, it doesn't warn you all the time causing for example, compiling cairo with mutex support to fail (actually it's configure that fails) -

  although, 4.8 is quite a lot bigger than 4.6.

 - Some more updates to rpi-update script are necessary; need to take in account that system's multi user features are enabled and restrict usage for only root, also, instead of static tmp directory, I should use mktemp..

And then why the delay? Well, I've got quite a lot of stuff here, meaning lots of files.. And I noticed something. Failures with filesystem. Even though, I have increased inode count of rootfs, with lot of files I start to notice following:

 - filesystem corruption: files that were supposed to be on image have gone missing.

 - filesystem corruption: inode's modes getting corrupted. Files are loosing their attributes (for e.g. chmod a+x file no longer executable)

 - filesystem corruption: reported free space is enough for writing on disk - but when trying, disk is reported to be full.

 - filesystem corruption: disk goes read only. Permanently. fsck does not help.

 - filesystem corruption: lot's of errors popping up when doing fsck.

Oops. They were all filesystem corruptions ;)

Reason for this? It is openwrt specific problem and I have narrowed down cause of it after studying other distributions available for rpi. OpenWRT uses genext2fs and tune2fs to generate ext4 image. This is the most propable reason

for all the problems mentioned. This way, you for example cannot set fs features, as tune2fs supports only few features.

Is there a fix? Oh yes. Well.. Not yet.

ext2fsprogs now ships with a script, populate-extfs.sh that should be a tool replacing genext2fs - but it's not quite there.. Most other distributions (for e.g. yocto) already use this script. Or slightly modified version.

There although was a small problem. OpenWRT's rootfs should contain /dev/console. genext2fs used to generate this to image from filespec and this could be added with populate-extfs.sh as well, if directory holding

contents of rootfs contains dev/console created properly with mknod. And this causes a problem, as mknod, both as executable program and as c function, require root priviledges. OpenWRT should be compilable

without root priviledges. I first started to change populate-extfs.sh to have support for filespec as well, but when I noticed that there were some changes in it between different distributions I came to conclusion that

it isn't quite yet perfect. Problems might cause system specific settings, arch specific settings, environment specific settings and even.. shell specific settings (and there are numerous; bash, sh, zsh..)

Therefore, I have written a new populatefs utility, which - well mostly does the same as populate-extfs.sh script, but is coded in C - it needs ext2fsprogs libraries and kernel of host needs support for debugfs.

So what is better than in script:

 - support for setting all ownerships to root:root

 - support for larger scale of filetypes

 - support for combining multiple rootfs directories

 - support for filespec, meaning adding for e.g. /dev/console properly (but this feature is not yet done, just finished other features)

So does this then fix other problems as well than just filesystem corruption? Glad you asked; yes it does!

Common rpi distributions come with images that don't have a lot of free space on rootfs. This is because people might be having different sizes of SD cards and 8gb cards aren't being used always. Someone might be using even 256mb from old days.

Then there's a trick; with use of fdisk/parted/etc and resize2fs - one can grow rootfs, let's say, to fill in the whole rest of card. This wasn't possible before, as when using genext2fs and tune2fs, some ext4 features couldn't be set. One of these features

is resize_inode which allows operation mentioned. After using a new way to generate "proper" ext4 image, user should overcome this problem.

Some rpi distributions ship with rpi-config utility that does the job for them. I am not planning on porting rpi-config to these packages, or building a simpler version of it. Atleast now. Busy with populatefs currently.

[Osku]

Utility is nearly finished. Final touches left before publishing it..

Too bad I am sick which slows me down.. (bad flu)

[Osku]

New populatefs utility has been published. It is available here:

https://github.com/oskarirauta/populatefs

[Osku]

And populatefs patch is now sent to OpenWrt dev trac..

https://dev.openwrt.org/ticket/14770

[Osku]

Anyone to suggest a free hosting site suitable for binary package repository?

I should be able to publish this soon, but I could first publish packages as pre-compiled..

I am not at home right now but 1gb should be enough, propably I could fit it in 500mb but not sure if it is enough..

Wondering if github/google code/google drive could be used..

[Michel Daggelinckx]

Op 15/01/2014 10:34, Osku schreef:

sourceforge ?

it has the advantage that you get a nice mirror network.

[Osku]

sourceforge ? 

it has the advantage that you get a nice mirror network.

It needs to be a standard directory on the web without bells and whistles, opkg does not support  services like sf.. :/ 

[Osku]

Just like this repository here:

http://downloads.openwrt.org/snapshots/trunk/ar71xx/packages/

[Osku]

Updated kernel patch. Now latest version 3.12.7y is being utilized.

https://sites.google.com/site/variousopenwrt/rpi-patch

[Osku]

Updated my kernel patch again. Some minor fixes. Does not depend on any of my old patches..

Added a new tool: rpi-extendfs

It was supposed to be named rpi-config - port of raspi-config for OpenWrt, meaning a lightweight version of raspi-config. But as I implemented only the feature to extend root filesystem

I decided to name it like this. If I create later rest of the features I planned (gui for changing config.txt) - I'll rename it to rpi-config.

So, this is using dialog to provide a menu-driven gui for extending root filesystem. If you use my populatefs patch, this will do the job without problems.

I also made it a bit different from raspi-config, with this, you don't need to use FULL amount of sd card, you can increase with amount you want to or set partitions last used sector.

This tool can only grow the root partition, no shrunking.

Bugs:

1) My algorithm that I use to do the math for calculating amount of megabytes for amount of sectors is bit off. If I resize my root filesystem to 52MB I end up with something like 48MB..

    But it's just a cosmetic bug..

 

 

[Osku]

Testing phase begins..

I have allowed testing by uploading binaries to a free hosting service.

Image is available at http://rpi-openwrt.host22.com/brcm2708/

and packages at http://rpi-openwrt.host22.com/brcm2708/packages

This package tree is also listed in image's opkg.conf in /etc - 

It's not a production ready image, or package feed. I did not compile _every_ package, it contains some packages and all of packages I created for rpi.

Image also contains something that is not in default image by openwrt buildroot:

resize2fs, rpi-extendfs, rpi-update, keyboard support (kmod-hid-generic), nano, kmod-vfat, kmod-nls-cp437, kmod-nls-iso8859-1

I added these to make getting started with testing a bit easier.

Try weston and some rpi-demos.. Maybe elementary demos as well while you at it..? Or test what ever I've made available..

Just opkg update and install from the tree. Luci is also there. Atleast most of it..

It's on free hosting site, so bandwidth isn't a lot.. And while writing this, I was just uploading package repository, so it might take about 10 minutes from

this post to have all available, but feel free to start downloading image from there, when you have written it on your sdcard and got system to bootup - 

all the packages are propably already available..

[Osku]

I need more testers for this before putting out the source code, so if someone could try it out - this would be great..

And remember, also if it works, give then good feedback, so I know it's been tested.. Or if something's terribly going

wrong, tell me about that too..

[Osku]

Kernel patch has once again been updated.

https://sites.google.com/site/variousopenwrt/rpi-patch

[Osku]

I have started to build repository for sources of my build. OpenWrt buildroot is not included, as the supposed way to build your own image is to follow these steps:

 - get official openwrt buildroot source from dev.openwrt.org

 - install packages feed to your sources (required for so many things) + luci and others you desire, excluding X repository (it will conflict with rpi-packages repository)

 - get patches from repository and patch your freshly cloned source tree with 'em.

 - patch with rpi-kernel patch.

 - add rpi-packages repository to package feeds and install rpi-packages repository

 - follow the guide ( not yet anywhere ) to setup some basic settings, including few _recommended_ compiler settings.

 - make your own settings and build your image - if there are exceptions for some packages (there is for elementary and efl library) - follow the guide again to get them compiling properly.

 - build

There is not much yet.. But some categories exist - if you have ideas on re-forming this, I'm open for ideas. I want to make this easy to maintain (that's why multiple categories) + easy for the "end-user" to understand.

It's a organization - so I can give rights to other people for committing as well, this will help to keep the project alive, as I have other projects as well and it's very likely.. That I won't be maintaining this regular enough and/or forever..

Current skeleton is at: https://github.com/rpi-openwrt

Tell me what you think. And interested parties for committing might request rights from me after this is complete.

[Osku]

Updates to feed sources have been uploaded. Some issues still, but I am working on it. Sorry, it's been awhile, I have been doing other things lately..

[Osku]

Oh, and thanks to Florian from OpenWrt, he rebased patches from rpi kernel to openwrt, so my version of kernel is no longer needed. Afaik, I personally haven't tested it yet, but soon..

[Osku]

Updates on the rpi-packages have occurred:

 - wayland and weston no longer support EGL

 - userland now is derived from stock branch, wayland-egl version hasn't been updated in a long time and is considered as deprecated

 - dialog is now part of rpi-packages, as it's lost from packages feed.

 - lesser amount of patches is now on needed as all patches against packages tree are now in mainstream

 - some packages have been updated to more recent versions (cairo, weston, wayland, userland, and so on..)

 - new package repository / image downloading site

Package development still is at: https://github.com/rpi-openwrt/rpi-packages

Patches from here: https://github.com/rpi-openwrt/patches

And there's a new site, hosting image and packages compiled at 23.11.2014, find it from here:

http://rpi-openwrt.github.io

Sorry about robustness of page, but it serves it's purpose and was done pretty quickly..

p.s. this post was sent before files to the new site were uploaded, so a hour more from publishing and it's all set up.