failed to build ONL

[Mark]

Hello,

I was struggling to build ONL on a freshly installed Debian-8 as per instructions http://opennetlinux.org/docs/build however it keeps on failing with the following errors:

make[2]: Entering directory '/home/mark/work/OpenNetworkLinux.git/packages/base/powerpc/fit/buildroot/builds'

/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py --initrd onl-buildroot-initrd:powerpc,onl-buildroot-initrd-powerpc.cpio.gz --add-platform all --itb onl-buildroot-fit.itb

onl-platform-config-powerpc-accton-as6700-32x-r0:powerpc

Traceback (most recent call last):

  File "/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py", line 297, in <module>

    fit.add_platform(platform)

  File "/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py", line 178, in add_platform

    self.add_platform_package("onl-platform-config-%s:powerpc" % platform)

  File "/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py", line 172, in add_platform_package

    self.add_yaml(platform, y)

  File "/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py", line 165, in add_yaml

    d = yaml.load(open(fname))

IOError: [Errno 2] No such file or directory: '{:timestamp=>"2016-02-11T19:43:03.159177+0000", :message=>"Created package", :path=>"/tmp/tmpYffZG2/onl-platform-config-powerpc-accton-as6700-32x-r0_1.0.0_powerpc.deb"}\n/tmp/tmpYffZG2/home/mark/work/OpenNetworkLinux.git/REPO/jessie/extracts/onl-platform-config-powerpc-accton-as6700-32x-r0_powerpc/lib/platform-config/powerpc-accton-as6700-32x-r0/onl/powerpc-accton-as6700-32x-r0.yml'

Makefile:4: recipe for target 'onl-buildroot-fit.itb' failed

make[2]: *** [onl-buildroot-fit.itb] Error 1

make[2]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/packages/base/powerpc/fit/buildroot/builds'

ERROR:onlpm:'Build failed.'

/tmp/tmpY6b3On/tmp/tmpOCkF4x/tmp/tmpqqxJh5/tmp/tmpcXhjF7/tmp/tmpzZcVmM/home/mark/work/OpenNetworkLinux.git/make/pkg.mk:21: recipe for target 'pkgall' failed

make[1]: *** [pkgall] Error 1

make[1]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/packages/base'

Makefile:25: recipe for target 'ppc' failed

make: *** [ppc] Error 2

I installed docker as suggested in https://docs.docker.com/engine/installation/debian/.I do have python-yaml package installed.

What else am I missing?

Thanks.

[Steven Noble]

Hi Mark,

We have noticed on certain systems that this error happens. If you continue to run make, it will eventually get past it.  I know Jeff has done some work to correct it, are you using the latest source code?

The current SHA-1 is 90c8fddb755903cea079cd971b87b4a673ecdaa6

Mark

February 11, 2016 at 11:49 AM

--
You received this message because you are subscribed to the Google Groups "opennetworklinux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opennetworklin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mark]

Hi Steven,

Yes I'm using the very recent source code (git clone), and SHA-1 matches mine. Actually yesterday I repeated "make docker" after error, but it didn't help though.

[Steven Noble]

Hi Mark,

We are still debugging this and if anyone has input, we would be glad to take it.

Can you do the direct build and see if you can get around it?

docker/tools/onlbuilder
source setup.env
apt-cacher-ng
sudo mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc
sudo /etc/init.d/binfmt-support start
make onl-ppc

Mark

February 11, 2016 at 12:49 PM

Hi Steven,

Yes I'm using the very recent source code (git clone), and SHA-1 matches mine. Actually yesterday I repeated "make docker" after error, but it didn't help though.

On Thursday, February 11, 2016 at 2:49:41 PM UTC-5, Mark wrote:

--
You received this message because you are subscribed to the Google Groups "opennetworklinux" group.
To unsubscribe from this group and stop receiving emails from it, send an email to opennetworklin...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Mark

February 11, 2016 at 11:49 AM

[Jeffrey Townsend]

I'm reproducing the problem now to see if I can resolve it. Hold tight. 

[Mark]

Steven, I tried your suggestion, but it fails at the same spot:

mark@sdnlab:~/work/OpenNetworkLinux.git$ ./docker/tools/onlbuilder -8

Unable to find image 'opennetworklinux/builder8:1.0' locally

1.0: Pulling from opennetworklinux/builder8

f8efbffe7b95: Already exists

a3ed95caeb02: Already exists

8819950c38de: Already exists

fc8f8648f60d: Already exists

67e093370f2f: Already exists

94515acdfa41: Already exists

7be36e1a5bb7: Already exists

0f9319780d72: Already exists

657316ce36fe: Already exists

923a9a79e824: Already exists

52ab93a1d52c: Already exists

Digest: sha256:1ef5b23d3c91ee3fcca9a72725ab590ed5da0ed5ea139aef64f53fb804b6b06e

Status: Downloaded newer image for opennetworklinux/builder8:1.0

mark@sdnlab:~/work/OpenNetworkLinux.git$

mark@sdnlab:~/work/OpenNetworkLinux.git$ source ./setup.env

mark@sdnlab:~/work/OpenNetworkLinux.git$ apt-cacher-ng

mark@sdnlab:~/work/OpenNetworkLinux.git$ sudo mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc

mark@sdnlab:~/work/OpenNetworkLinux.git$ mount

none on / type aufs (rw,relatime,si=743c0c7482817d7,dio,dirperm1)

proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)

tmpfs on /dev type tmpfs (rw,nosuid,mode=755)

devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=666)

sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)

tmpfs on /sys/fs/cgroup type tmpfs (rw,nosuid,nodev,noexec,relatime,mode=755)

cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd)

cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)

cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)

cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)

cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)

cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)

cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)

cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)

/dev/sda1 on /home/mark type ext4 (rw,relatime,errors=remount-ro,data=ordered)

/dev/sda1 on /lib/modules type ext4 (rw,relatime,errors=remount-ro,data=ordered)

/dev/sda1 on /etc/resolv.conf type ext4 (rw,relatime,errors=remount-ro,data=ordered)

/dev/sda1 on /etc/hostname type ext4 (rw,relatime,errors=remount-ro,data=ordered)

/dev/sda1 on /etc/hosts type ext4 (rw,relatime,errors=remount-ro,data=ordered)

shm on /dev/shm type tmpfs (rw,nosuid,nodev,noexec,relatime,size=65536k)

mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime)

devpts on /dev/console type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)

binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)

mark@sdnlab:~/work/OpenNetworkLinux.git$ sudo /etc/init.d/binfmt-support start

[ ok ] Enabling additional executable binary formats: binfmt-support.

mark@sdnlab:~/work/OpenNetworkLinux.git$ make onl-ppc

make -C packages/base ARCHES=powerpc,all

make[1]: Entering directory '/home/mark/work/OpenNetworkLinux.git/packages/base'

Created package {:path=>"/tmp/tmpTlMV6u/onl-vendor-config-onl_1.0.0_all.deb"}

INFO:onlpm:adding package '/home/mark/work/OpenNetworkLinux.git/packages/base/all/vendor-config-onl/onl-vendor-config-onl_1.0.0_all.deb'...

INFO:onlpm:dstdir=/home/mark/work/OpenNetworkLinux.git/REPO/jessie/packages/all

Created package {:path=>"/tmp/tmpLIm0pF/onl-loader-initrd-files_1.0.0_all.deb"}

Created package {:path=>"/tmp/tmppyDOUO/onl-loader-initscripts_1.0.0_all.deb"}

INFO:onlpm:adding package '/home/mark/work/OpenNetworkLinux.git/packages/base/all/initrds/loader-initrd-files/onl-loader-initrd-files_1.0.0_all.deb'...

INFO:onlpm:dstdir=/home/mark/work/OpenNetworkLinux.git/REPO/jessie/packages/all

INFO:onlpm:adding package '/home/mark/work/OpenNetworkLinux.git/packages/base/all/initrds/loader-initrd-files/onl-loader-initscripts_1.0.0_all.deb'...

INFO:onlpm:dstdir=/home/mark/work/OpenNetworkLinux.git/REPO/jessie/packages/all

make[2]: Entering directory '/home/mark/work/OpenNetworkLinux.git/packages/base/powerpc/faultd/builds'

    Linking[release]: faultd-module::faultd.bin

make[2]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/packages/base/powerpc/faultd/builds'

Created package {:path=>"/tmp/tmpvTGsUW/onl-faultd_1.0.0_powerpc.deb"}

INFO:onlpm:adding package '/home/mark/work/OpenNetworkLinux.git/packages/base/powerpc/faultd/onl-faultd_1.0.0_powerpc.deb'...

INFO:onlpm:dstdir=/home/mark/work/OpenNetworkLinux.git/REPO/jessie/packages/powerpc

INFO:onlpm:Requiring prerequisite package onl-buildroot-initrd:powerpc...

make[2]: Entering directory '/home/mark/work/OpenNetworkLinux.git/packages/base/powerpc/fit/buildroot/builds'

/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py --initrd onl-buildroot-initrd:powerpc,onl-buildroot-initrd-powerpc.cpio.gz --add-platform all --itb onl-buildroot-fit.itb

onl-platform-config-powerpc-accton-as6700-32x-r0:powerpc

onl-platform-config-powerpc-accton-as6700-32x-r1:powerpc

onl-platform-config-powerpc-accton-as5710-54x-r0b:powerpc

Traceback (most recent call last):

  File "/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py", line 297, in <module>

    fit.add_platform(platform)

  File "/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py", line 178, in add_platform

    self.add_platform_package("onl-platform-config-%s:powerpc" % platform)

  File "/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py", line 172, in add_platform_package

    self.add_yaml(platform, y)

  File "/home/mark/work/OpenNetworkLinux.git/tools/flat-image-tree.py", line 165, in add_yaml

    d = yaml.load(open(fname))

IOError: [Errno 2] No such file or directory: '{:timestamp=>"2016-02-11T21:49:39.043580+0000", :message=>"Created package", :path=>"/tmp/tmpo1zKyn/onl-platform-config-powerpc-accton-as5710-54x-r0b_1.0.0_powerpc.deb"}\n/tmp/tmpo1zKyn/home/mark/work/OpenNetworkLinux.git/REPO/jessie/extracts/onl-platform-config-powerpc-accton-as5710-54x-r0b_powerpc/lib/platform-config/powerpc-accton-as5710-54x-r0b/onl/powerpc-accton-as5710-54x-r0b.yml'

Makefile:4: recipe for target 'onl-buildroot-fit.itb' failed

make[2]: *** [onl-buildroot-fit.itb] Error 1

make[2]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/packages/base/powerpc/fit/buildroot/builds'

ERROR:onlpm:'Build failed.'

/tmp/tmpTlMV6u/tmp/tmpLIm0pF/tmp/tmppyDOUO/tmp/tmpvTGsUW/home/mark/work/OpenNetworkLinux.git/make/pkg.mk:21: recipe for target 'pkgall' failed

make[1]: *** [pkgall] Error 1

make[1]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/packages/base'

Makefile:25: recipe for target 'onl-ppc' failed

make: *** [onl-ppc] Error 2

mark@sdnlab:~/work/OpenNetworkLinux.git$

[Mark]

Jeffrey, thanks for you efforts!

[Jeffrey Townsend]

Mark, 

Please pull your tree and try again. Let me know if you're still having issues. 

Jeff

[Mark]

Hi Jeffrey,

Thank you, I can build both targets now.

When "make docker" has completed its work, it stays in the docker I guess, and doesn't provide access to command line. So only "Ctrl+C" returns me back to host shell:

...

/tmp/tmpv1SWrwmake[1]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/builds/powerpc/installer/legacy'

make -C REPO build-clean

make[1]: Entering directory '/home/mark/work/OpenNetworkLinux.git/REPO'

make[1]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/REPO'

mark@onlbuilder8:~$

^CTraceback (most recent call last):

  File "docker/tools/onlbuilder", line 194, in <module>

    sys.exit(subprocess.call(g_docker_arguments, shell=True))

  File "/usr/lib/python2.7/subprocess.py", line 522, in call

    return Popen(*popenargs, **kwargs).wait()

  File "/usr/lib/python2.7/subprocess.py", line 1384, in wait

    pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)

  File "/usr/lib/python2.7/subprocess.py", line 476, in _eintr_retry_call

    return func(*args)

KeyboardInterrupt

Makefile:56: recipe for target 'docker' failed

make: *** [docker] Error 1

Is it normal? What is a "clean" way to properly complete the build process?

[Mark]

Hi Jeffrey,

Forgot to mention that when building "by hand" I get errors:

% cd OpenNetworkLinux
% docker/tools/onlbuilder -8
% source ./setup.env
% make amd64 ppc

...

E: Unable to locate package apt

E: Unable to locate package apt-utils

E: Unable to locate package base-files

E: Package 'binutils' has no installation candidate

E: Unable to locate package bsdmainutils

E: Unable to locate package cpio

E: Unable to locate package debian-archive-keyring

E: Unable to locate package dmidecode

E: Unable to locate package dosfstools

E: Unable to locate package ethtool

E: Unable to locate package fancontrol

E: Unable to locate package file

E: Unable to locate package grub2

E: Unable to locate package i2c-tools

E: Unable to locate package ifupdown

E: Unable to locate package iproute

E: Package 'iptables' has no installation candidate

E: Unable to locate package iputils-ping

E: Unable to locate package isc-dhcp-client

E: Unable to locate package kexec-tools

E: Unable to locate package less

E: Unable to locate package lm-sensors

E: Unable to locate package localepurge

E: Unable to locate package locales

E: Unable to locate package lsof

E: Unable to locate package make

E: Unable to locate package mingetty

E: Unable to locate package module-init-tools

E: Unable to locate package mtd-utils

E: Unable to locate package nano

E: Unable to locate package net-tools

E: Unable to locate package netbase

E: Unable to locate package netplug

E: Unable to locate package nfs-common

E: Unable to locate package ntp

E: Unable to locate package ntpdate

E: Unable to locate package openssh-server

E: Unable to locate package parted

E: Unable to locate package pciutils

E: Unable to locate package procps

E: Unable to locate package psmisc

E: Package 'python' has no installation candidate

E: Unable to locate package python-cherrypy3

E: Unable to locate package python-pyinotify

E: Unable to locate package python-pypcap

E: Unable to locate package python-tz

E: Unable to locate package realpath

E: Unable to locate package resolvconf

E: Unable to locate package rssh

E: Unable to locate package rsyslog

E: Unable to locate package scapy

E: Unable to locate package smartmontools

E: Unable to locate package snmpd

E: Unable to locate package sudo

E: Unable to locate package sysvinit-core

E: Unable to locate package telnetd

E: Unable to locate package traceroute

E: Unable to locate package u-boot-tools

E: Unable to locate package usbutils

E: Unable to locate package util-linux

E: Unable to locate package vim-tiny

E: Unable to locate package watchdog

E: Unable to locate package wget

E: Unable to locate package zile

apt download failed. Exit value: 100

ERROR:onlrfs:Multistrap APT failure.

/home/mark/work/OpenNetworkLinux.git/make/rfs.mk:32: recipe for target 'RFS' failed

make[2]: *** [RFS] Error 1

make[2]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/builds/amd64/rootfs/builds'

ERROR:onlpm:'Build failed.'

/home/mark/work/OpenNetworkLinux.git/make/pkg.mk:21: recipe for target 'pkgall' failed

make[1]: *** [pkgall] Error 1

make[1]: Leaving directory '/home/mark/work/OpenNetworkLinux.git/builds/amd64/rootfs'

Makefile:19: recipe for target 'amd64' failed

make: *** [amd64] Error 2

Looks like, for some reason it is unable to install required packages in the docker environment?

[Jeffrey Townsend]

Mark, 

This problem was addressed right after fixing the build itself. Thanks for following up with it. 

The fix has been integrated: https://github.com/opencomputeproject/OpenNetworkLinux/commit/fbaccf6cb23fe15cb00ee330f34d58f62a45d83d

Please let me know if you continue to see an issue. 

Jeff

[Jeffrey Townsend]

Mark, 

The onlbuilder container is both an automated build environment and a development environment. 

When you run onlbuilder -8 you are running it in developer mode (for jessie). Developer mode does not use a new network namespace but instead uses the host's namespace. It give the build container full access to the local network resources. 

In developer mode it is assumed that you will have apt-cacher-ng running on the host build machine, and that it can be accessed directly from the container via 127.0.0.1:3142 This allows the apt-cache to be persistent and shared across multiple developers and multiple builds. 

If you would like to use this mode to build then you will need to install apt-cacher-ng on the host machine prior to entering the container. This will also speed up future root filesystem creations. 

If you want to run interactively with the same environment as make docker then you should run make docker-debug. 

Jeff

[Mark]

Thanks again, it works.