emdebian.org offline?

[Aaron Edwards]

Hi all,

Trying to play with ONL, working on setting up a build environment on a fresh 13.10 server install. I had a few issues:

• www.emdebian.org looks to be offline. Is there another host with this repo/packages that can be dropped in the install-ws-deps make area?
  
• on Ubuntu 13.10, powerpc items are seemingly removed/(retired?) from the archive & security ubuntu.com repos. Creating a new apt source with [arch=powerpc] and replacing the standard repo hosts with ports.ubuntu.com/ubuntu-ports seems to have worked (so far).
• I had resolve.conf issues when going to the workspace. 13.10 seems to have /etc/resolv.conf symlinked to /run/resolvconf/resolv.conf, and /var/run symlinked to /run. Upon entering the workspace, /run seems to be pointing to <ws>/var/run, which is an empty dir. Putting <ws>/var/run/resolvconf/resolv.conf there fixed my DNS issues. <ws>/run seems to be populated with what should be in <ws>/var/run ?
• Got a 'Missing privilege separation directory: /var/run/sshd' on entering the workspace, even though the dir is there. Assume it's a similar issue with symlinks as above, but didn't spend any time on this. This error message seems generated by 'sshd -t' 

Thanks,

Aaron

[Rob Sherwood]

Hi Aaron -- good to hear from you again.

It looks like emdebian is back online so please give it a try again.  Fwiw, we also saw it go down as well from our internal build scripts so agreed that we need to make this a bit more robust.  With the new ORC and ONLP code that I just pushed we now have an official ONL apt repository so in theory we could mirror the emdebian packages there as well.  Let me open an github issue to track this (https://github.com/opennetworklinux/ONL/issues/51).

About Ubuntu 13.10 - we haven't yet done the work to figure out what needs to change for building from newer Ubuntu versions.  The build instructions are for 12.04 currently.  Any help there you can would be appreciated.  Fwiw, when built correctly under 12.04, things look like this:

(iso1:ws.amd64)robs@ubuntu:/var$ ls -l 

total 36

drwxr-xr-x  2 root root  4096 Jun 11 14:07 backups

drwxr-xr-x  7 root root  4096 Oct  2 05:25 cache

drwxr-xr-x 23 root root  4096 Oct  2 05:25 lib

drwxrwsr-x  2 root staff 4096 Jun 11 14:07 local

lrwxrwxrwx  1 root root     9 Oct  2 05:10 lock -> /run/lock

drwxr-xr-x  6 root root  4096 Oct  2 05:26 log

drwxrwsr-x  2 root mail  4096 Oct  2 05:10 mail

drwxr-xr-x  2 root root  4096 Oct  2 05:10 opt

lrwxrwxrwx  1 root root     4 Oct  2 05:10 run -> /run

drwxr-xr-x  6 root root  4096 Oct  2 05:26 spool

drwxrwxrwt  2 root root  4096 Oct  5 03:18 tmp

(iso1:ws.amd64)robs@ubuntu:/var$ cd /run

(iso1:ws.amd64)robs@ubuntu:/run$ ls -l

total 8

-rw-r--r-- 1 root root 14 Nov  1 09:55 debian_chroot

drwxrwxrwt 2 root root 40 Nov  1 09:55 lock

drwxr-xr-x 2 root root 60 Nov  1 09:55 resolvconf

drwxr-xr-x 2 root root 40 Nov  1 09:55 sshd

-rw-r--r-- 1 root root  3 Nov  1 09:55 sshd.pid

-rw-rw-r-- 1 root utmp  0 Nov  1 09:55 utmp

About the missing separation directory error, I saw that error once as well in the past but don't recall how I worked around it.  I certainly don't see it regularly so I don't know off the top of my head what a fix would be.  Perhaps it's a function of building for 13.10.  If it helps, I attached output from my `onl-mkws` session.

Please take a look at these things and let me know if it helps.  Also, there are pre-compiled binaries if you just want to play with the output.

- Rob

.

--
You received this message because you are subscribed to the Google Groups "opennetworklinux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opennetworklin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aaron Edwards]

Hi Rob - nice to see you again as well.

Thanks for the info, it helped a bunch. I rebuilt my build system with 12.04.5 instead of 13.10, as I'd much rather be on a LTS version. I originally started with 13.10 due to the ONL Quickstart.md - it only lists 12.10, 13.04 and 13.10. 

Going through a build from a fresh 12.04 install went better, but I still ran into a few issues:

• emdebian.org is back up, but seems to be missing the powerpc repos/Packages. See below. Any idea where to grab those?
• Ran into a QEMU issue where QEMU on 12.04 cores on powerpc ws create. Saw someone else on the list had the same issue. Seems to be very similar to this and this. Since some of the launchpad comments say it's fixed in later qemu versions, I manually downloaded and installed the raring exp qemu-user-static package. This seems to have completely fixed the QEMU issue with no "apparent" dependency issues or side effects (yet). 

Thanks!

Aaron

sudo apt-get update

Ign http://apt.opennetlinux.org unstable Release.gpg

Ign http://apt.opennetlinux.org unstable Release                                                     

Hit http://mirrors.kernel.org wheezy Release.gpg                                                     

Ign http://apt.opennetlinux.org unstable/main powerpc Packages/DiffIndex

Hit http://mirrors.kernel.org wheezy Release                           

Get:1 http://emdebian.org wheezy Release.gpg [490 B]                   

Get:2 http://apt.opennetlinux.org unstable/main powerpc Packages           

Hit http://emdebian.org wheezy Release                          

Ign http://apt.opennetlinux.org unstable/main Translation-en_US

Ign http://apt.opennetlinux.org unstable/main Translation-en                                 

Hit http://mirrors.kernel.org wheezy/main Sources                      

Ign http://emdebian.org wheezy Release           

Hit http://mirrors.kernel.org wheezy/main powerpc Packages

Hit http://mirrors.kernel.org wheezy/main Translation-en          

Err http://emdebian.org wheezy/main powerpc Packages              

  404  Not Found

Ign http://emdebian.org wheezy/main Translation-en_US

Ign http://emdebian.org wheezy/main Translation-en

Fetched 3,294 B in 16s (201 B/s)   

[Rob Sherwood]

Hmm... regarding Emdebian, we've known that they've stopped maintaining the fork from Debian a few months ago, but it seems like their main mirror was compromised (see below for details).  I'm going to try to track down some backup mirror of the files, mirror them to apt.opennetlinux.org, and at least update our own build scripts until we can find a better alternative.

Then I'll look into Qemu - that's clearly an old hack that needs to get fixed.

- Rob

.

[from http://www.emdebian.org/News/2014/20141117.html]

Emdebian Grip Server Downtime

November 17th, 2014

Server Downtime due to compromise.

The emdebian server was cracked in November so was taken offline to limit the damage.

A new virtual server has kindly been provided by Nathael Pajani, but this took a few days to move data across to so the site was offline for a week.

The emdebian site is now largely in maintenance mode, providing just the website and cross-toolchains, for jessie and older releases. Unstable onwards has cross-toolchains in the Debian archive.

Grip for squeeze and wheezy is available from mirrors, but no longer from this site.

[Rob Sherwood]

Sorry for the delay, but I've just hacked in a temporarily work around for this.

We're only using emdebian for the cross-compile utilities.  The Right Thing would be to move to the new cross compile utilities that are being upstreamed into Debian mainline (and now are proported to be in the unstable branch, but I haven't had a chance to look), but that would involve changing our build process.  So, for the short term, I've cached the relevant .deb files at http://apt.opennetlinux.org and updated the top-level Makefile to point to them.

This at least unblocks building a new workspace.

Keep the feedback coming,

- Rob

.

[Aaron Edwards]

Thanks Rob, got a lot further.

Looks like the install-cross-deps section in the makefile is the only thing failing now. Seems that apt.opennetlinux.org mirror seems to be missing some of the -cross packages from emdebian? 

sudo apt-get install -y libc6-dev-powerpc-cross
Reading package lists... Done
Building dependency tree       
Reading state information... Done
E: Unable to locate package libc6-dev-powerpc-cross
make: *** [install-cross-deps] Error 100
(iso1:ws.powerpc)testbuild@onl-build:~/work/ONL#

As a side note - looking at the embedded-debian list, seems these are known to be offline from emdebian.org with an eta of 'reasonably soon'.

The Emdebian.org server got a) hacked and b) moved, so the old
toolchains (squeeze, wheezy and older) are currently non online, but
will be restored reasonably soon.

https://lists.debian.org/debian-embedded/2014/11/msg00053.html

Aaron

[Rob Sherwood]

Hmm.  This works for me.

I've verified that that .deb is both in the apt.opennetlinux.org repo and that my repository can see it:

robs@ubuntu:~$ apt-cache search libc6-dev-powerpc-cross

libc6-dev-powerpc-cross - Embedded GNU C Library: Development Libraries and Header Files (for cross-compiling)

Are you sure you've run `apt-get update` (the Makefile should do this automatically for you in any case)?

At worst, you should be able to get the file directly from the web server: 

curl -O http://apt.opennetlinux.org/debian/dists/unstable/main/binary-amd64/libc6-dev-powerpc-cross_2.17-93_all.deb

- Rob

.

[Aaron Edwards]

Hmm, I wonder if it's an architecture issue. install-cross-deps is being called by install-ws-deps, which is in a powerpc workspace.

(iso3:ws.powerpc)testbuild@onl-build:~/work/ws.powerpc# dpkg --print-architecture
powerpc
(iso3:ws.powerpc)testbuild@onl-build:~/work/ws.powerpc#

The cross packages are amd64, right? I tried to manually add the amd64 as a secondary arch to the workspace, and now I get a different 'no candidate' error.

Will play with it a bit more and let you know.

Aaron

[Rob Sherwood]

Aah -- it looks like you may have made the wrong workspace for yourself.

When you're calling `onl-mkws`, you're trying to create a workspace for your host build architecture, which is presumably amd64, so you should be calling:

# onl-mkws -a amd64 ws.amd64

# cd ws.amd64

# onl-chws

The fact that you can also build a powerpc workspace is a by product of how we actually create the ultimate switch root file system and is not meant for building the dev environment.  If this ends up being the problem, take a look at the build guide (http://opennetlinux.org/docs/build) and let me know if there's anything I can do to clarify this.

Hopefully this is the issue,

- Rob

.

[Aaron Edwards]

That was it. Cleaned out my system, and made a workspace with amd64, and got a lot farther.

You might consider removing the "-a arch" from the onl-mkws directions. The onl-mkws script seems to have logic to detect the host arch and use it if the arch is not specified. When I was looking to build for a different arch, I was mostly replacing amd64 with powerpc. If the arch should in most cases match the build system, seems best to leave it out of the directions.

After getting past this - I only had one more issue. When building, the script had several failures building ONL/components/powerpc/onlp-powerpc-as5710-54x-r0b. I looked, and saw that they were requiring repo onl-accton. I saw in the git info that onl-accton was closed/restricted, and that onlp-powerpc-accton directories were exempted from build without repo access. 

It seems that onlp-powerpc-as* components require onlp-accton according to the Makefiles in that directroy. Is that correct? I was able to build successfully by removing onlp-powerpc-as* from the components directory (haven't tested it yet though). Should these be exempted from build, or should I simply remove the onl-accton module from the Makefiles? 

Thanks,

[Rob Sherwood]

Thanks for continuing to try to compile this.

You're describing a build error that has been fixed a while back (I think).  Can you do a `git clean && git pull` to make sure your repository is up to date?  The latest branch of the code will only build the onlp-powerpc-as5710-54x-r0b package if you have the environmental variable ONL_CLOSED_SOURCE set (which you should not set :-).  The idea is that there are some packages which are still closed source that I need to build (at least for now; in particular that package is in the process of being open sourced).

Try the above and let me know if you make any progress.

- Rob

.

[Aaron Edwards]

Removed the old ONL repository and did a fresh clone to make sure I had the latest code. Issue still exists. Last few lines below, Can send full log  of build process if needed.

dpkg-deb: building package `onlp' in `../onlp_0.2.0bsn1~ubuntu1_powerpc.deb'.
 dpkg-genchanges -b >../onlp_0.2.0bsn1~ubuntu1_powerpc.changes
dpkg-genchanges: binary-only upload - not including any source code
 dpkg-source --after-build debuild-powerpc
dpkg-buildpackage: binary only upload (no source included)
Now running lintian...
W: onlp: package-name-doesnt-match-sonames libonlp1
W: onlp: hardening-no-relro lib/powerpc-linux-gnu/libonlp.so
W: onlp: hardening-no-fortify-functions lib/powerpc-linux-gnu/libonlp.so
E: onlp: missing-dependency-on-libc needed by lib/powerpc-linux-gnu/libonlp.so
W: onlp: latest-debian-changelog-entry-without-new-date
E: onlp: description-starts-with-package-name
E: onlp: description-synopsis-is-duplicated
E: onlp: missing-pre-dependency-on-multiarch-support
E: onlp: ldconfig-symlink-missing-for-shlib lib/powerpc-linux-gnu/libonlp.so.1 lib/powerpc-linux-gnu/libonlp.so libonlp.so.1
E: onlp: postinst-must-call-ldconfig lib/powerpc-linux-gnu/libonlp.so
Finished running lintian.
GNUPGHOME=/home/aedwards/work/ONL/make/default_sign/gnupg dpkg-sig --sign builder *powerpc*.deb  \
&& echo Packages Signed # sign packages
Processing onlp_0.2.0bsn1~ubuntu1_powerpc.deb...
gpg: WARNING: unsafe permissions on homedir `/home/aedwards/work/ONL/make/default_sign/gnupg'
Signed deb onlp_0.2.0bsn1~ubuntu1_powerpc.deb
Packages Signed
/home/aedwards/work/ONL/tools/onlpkg.py --add-pkg *powerpc*.deb
INFO:onlpkg:adding new package onlp_0.2.0bsn1~ubuntu1_powerpc.deb...
rm *powerpc*.deb
rm -rf debuild-powerpc/debian/tmp debuild-powerpc/debian/onlp/ debuild-powerpc/debian/onlp-dbg debuild-powerpc/debian/onlp-dbg/ debuild-powerpc/debian/onlp-dbg-dbg || echo rm failed
make[3]: Leaving directory `/home/aedwards/work/ONL/components/any/onlp/onlp'
make[2]: Leaving directory `/home/aedwards/work/ONL/components/powerpc/onlp/onlp'
INFO:onlpkg:extracting /home/aedwards/work/ONL/debian/repo/powerpc/onlp_0.2.0bsn1~ubuntu1_powerpc.deb...
WARNING:onlpkg:no matching packages for onlp-powerpc-as5710-54x-r0a (powerpc)
INFO:onlpkg:found at /home/aedwards/work/ONL/components/powerpc/onlp/onlp-powerpc-as5710-54x-r0a/deb/debuild/debian
INFO:onlpkg:can be built locally at /home/aedwards/work/ONL/components/powerpc/onlp/onlp-powerpc-as5710-54x-r0a
make[2]: Entering directory `/home/aedwards/work/ONL/components/powerpc/onlp/onlp-powerpc-as5710-54x-r0a'
Submodule 'submodules/onlp-accton' (g...@github.com:opennetworklinux/onlp-accton) registered for path 'submodules/onlp-accton'
Cloning into 'submodules/onlp-accton'...
Warning: Permanently added the RSA host key for IP address '192.30.252.129' to the list of known hosts.
Permission denied (publickey).
fatal: The remote end hung up unexpectedly
Clone of 'g...@github.com:opennetworklinux/onlp-accton' into submodule path 'submodules/onlp-accton' failed
Traceback (most recent call last):
  File "/home/aedwards/work/ONL/tools/submodules.py", line 100, in <module>
    submodule_update(module, os.getenv("SUBMODULE_DEPTH"))
  File "/home/aedwards/work/ONL/tools/submodules.py", line 59, in submodule_update
    if subprocess.check_call(args) != 0:
  File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['git', 'submodule', 'update', '--init', 'submodules/onlp-accton']' returned non-zero exit status 1
make[2]: *** [component-deps] Error 1
make[2]: Leaving directory `/home/aedwards/work/ONL/components/powerpc/onlp/onlp-powerpc-as5710-54x-r0a'
Traceback (most recent call last):
  File "/home/aedwards/work/ONL/tools/onlpkg.py", line 276, in <module>
    check_call(('make', '-C', buildpath, 'deb',))
  File "/home/aedwards/work/ONL/tools/onlpkg.py", line 146, in check_call
    subprocess.check_call(cmd, *args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 511, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '('make', '-C', '/home/aedwards/work/ONL/components/powerpc/onlp/onlp-powerpc-as5710-54x-r0a', 'deb')' returned non-zero exit status 2
make[1]: *** [component-deps] Error 1
make[1]: Leaving directory `/home/aedwards/work/ONL/builds/components'
make: *** [all-components] Error 2
(iso3:ws.amd64)aedwards@onl-build:~/work/ONL$ 

Not sure the cause, but I'm guessing it's one of the following:

1. onlp-powerpc-as5710-54x-r0a requires onl-accton, and is not correctly excluded (In tools/onlpkg.py I see closed source  modules, and this one is not listed.)
   
2. onlp-powerpc-as5710-54x-r0a does not require onl-accton, and incorrectly lists onl-accton in ONL_REQUIRED_SUBMODULES in ONL/components/powerpc/onlp/onlp-powerpc-as5710-54x-r0a/Makefile
   
3. Something else I'm completely missing :)

in any case, the build succeeds if I 'rm -rf ONL/components/powerpc/onlp/onlp-powerpc-as*', not sure if it's functional though yet..

Aaron

[Aaron Edwards]

As a follow-up, I tested the code built after 'rm -rf ONL/components/powerpc/onlp/onlp-powerpc-as*', and it booted/ran fine on the platforms I have access to.

Thanks for your help Rob.

Aaron

...

[kkma...@gmail.com]

hello guys

i got a Accton as5710-54X (just 54X,i don't know the difference between 54x and 54x-r0b)

and i got compiling error as same as Edward

i found that in ONL/tools/onlpkg.py

it says that i can't built as5710 package without magic priviledges

how can i get magic privileges?

thanks

...

[kkma...@gmail.com]

i set the magic env and i still got the same error

...

[Rob Sherwood]

Apologies for the delays -- I've been traveling (and sleeping off the jetlag :-).  

It looks like there's a problem with the build system  -- looking into it now.  Just so everyone is on the same page, I've recently added a hack to the build system which lets everyone build all of the open source packages but only tries to build the Big Switch only closed source packages if you have $ONL_CLOSED_SOURCE set.  

In the mean time, the binaries for the packages you're trying to build are already built and available off of apt.opennetlinux.org and that repo has been added to the default apt sources list, so you actually don't have to build that package from source.

Fwiw, the packages that are binary only right now (e.g., everything from accton - for the moment) are in the process of being open sourced, so this binary-only thing is a short term work around.  Getting access to the closed source code before we get a chance to open it is not really possible - -just bear with us until we do.

I'm trying to reproduce the build errors now and will post a patch ASAP.

Thanks,

- Rob

.

--

[Rob Sherwood]

OK - I've verified and (I believe) fixed the problem:

https://github.com/opennetworklinux/ONL/pull/60

It's basically as Aaron said -- there were packages that should be in the exceptions list which were not.  In case anyone is interested, we do have an autobuilder setup to catch these types of problems, but the autobuilder runs with privileges to checkout the binary-only packages and so it didn't have a problem.  I would setup another autobuilder without those privileges, but I'm hoping to get those packages opened up ASAP, thus removing the need for the hack.

Give it a try and let me know if it works for you.

- Rob

.

[Rob Sherwood]

FYI, the naming convention is actually from ONIE, and is basically of the form 

$arch-$vendor-$product-r$version

so any platform specific packages keep that convention (e.g., onlp-$arch-$vendor-$product-r$version for the onlp drivers for that platform).

Most products (e.g.., Accton 5710-54X) actually have multiple internal hardware versions (e.g., r0, r1, r3) that are not supposed to be customer facing differences (read: the box looks and acts the same) but might require different drivers, so ONL has to be aware of the specific version.

Hope that helps and keep the questions coming,

- Rob

.

--

[Rob Sherwood]

Great to hear -- did you get a chance to test out ORC out of curiosity?

- Rob

.

--