Jenkins fails to start in gofabric8-v0.4.61

[Mike]

Errors (most recent first):

• Error syncing pod, skipping: failed to "StartContainer" for "jenkins" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=jenkins pod=jenkins-3604705456-xf7ae_default(0e0dfc16-76e1-11e6-9d02-ea4cdd9b5383)"
  
• Back-off restarting failed docker container
  
• Error syncing pod, skipping: failed to "StartContainer" for "jenkins" with RunContainerError: "PostStart handler: Error executing in Docker Container: 1"
  

System / Environmental's:

• Mac 10.11.6
  
• 16GB
  
• minikube with virtualbox
  
• 9216MB for minikube
  

Commands:

• minikube start --memory=9216 --cpus=2 --insecure-registry=[] --vm-driver="virtualbox"
  
• ./gofabric8/gofabric8-darwin-amd64-v0.4.61  deploy --insecure-skip-tls-verify=true -y
  

[James Strachan]

Sorry to hear that. Did all the other pods startup OK? It was just the jenkins pod that can't start?

I wonder can you attach the output the following commands:

kubectl get pods

kubectl describe deployment jenkins

kubectl logs jenkins-XXXx (whatever the jenkins pod id is)

--
You received this message because you are subscribed to the Google Groups "fabric8" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fabric8+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

James
-------
Red Hat

Twitter: @jstrachan
Email: james.s...@gmail.com

Blog: https://medium.com/@jstrachan/

fabric8: http://fabric8.io/

open source microservices platform

[mrayar...@flyingraydesign.com]

Everything else started up fine. 

[Mike Ray]

$ kubectl get pods

NAME                                       READY     STATUS             RESTARTS   AGE

elasticsearch-2594944935-o56j8             2/2       Running            2          16h

exposecontroller-3657226338-kqtvw          1/1       Running            1          17h

fabric8-docker-registry-1778841248-fchav   1/1       Running            1          17h

fabric8-forge-1812418832-89xil             1/1       Running            1          17h

fabric8-pzynp                              2/2       Running            2          17h

fluentd-s2ugu                              1/1       Running            1          16h

gogs-625799268-y2bjs                       1/1       Running            1          17h

jenkins-3604705456-jnc40                   0/1       CrashLoopBackOff   8          12h

kibana-54543160-uh1rj                      2/2       Running            2          16h

message-broker-2732014047-6tyzj            1/1       Running            1          16h

message-gateway-2475834259-aoy60           1/1       Running            1          16h

nexus-2018182024-3latu                     1/1       Running            2          17h

zookeeper-1732469276-gwu90                 1/1       Running            1          16h

$ kubectl describe deployment jenkins

Name: jenkins

Namespace: default

CreationTimestamp: Fri, 09 Sep 2016 20:13:19 -0700

Labels: group=io.fabric8.devops.apps

project=jenkins

provider=fabric8

version=2.2.248

Selector: group=io.fabric8.devops.apps,project=jenkins,provider=fabric8

Replicas: 1 updated | 1 total | 0 available | 1 unavailable

StrategyType: RollingUpdate

MinReadySeconds: 0

RollingUpdateStrategy: 1 max unavailable, 1 max surge

OldReplicaSets: <none>

NewReplicaSet: jenkins-3604705456 (1/1 replicas created)

No events.

$ kubectl logs jenkins-3604705456-jnc40

Running from: /usr/share/jenkins/jenkins.war

webroot: EnvVars.masterEnvVars.get("JENKINS_HOME")

Jenkins home directory: /var/jenkins_home found at: EnvVars.masterEnvVars.get("JENKINS_HOME")

--> disabling master executors

--> setting agent port for jnlp

--> setting agent port for jnlp... done

On Sep 9, 2016, at 10:28 PM, James Strachan <james.s...@gmail.com> wrote:

[James Strachan]

Thanks. Still none the wiser on why that pod is fialing. I wonder does this offer any more information:

  kubectl describe pod jenkins-3604705456-jnc40

  kubectl get event

[James Strachan]

Yay we figured it out! It seems this commit broke the jenkins image: 

https://github.com/fabric8io/jenkins-docker/commit/06eedb22f8d263ff825cb7b32d63dbfc05fb9622

James Rawlings fixed this and releeased it. So if you grab the latest gofabric8 and re-run `gofabric8 deploy` you should be good to go. Sorry about that!

[Mike Ray]

OUTSTANDING!!!

You guys ROCK!!!

Thank you so much!

[James Strachan]

Yeah. For now you can use 'gofabric8 service foo" to open the URL (the same command is in minikube / minishift). There's a pending issue to do a DNS-like thing; though that would require also running an ingress controller on kubernetes or a router on minishift:
https://github.com/kubernetes/minikube/issues/347

Also from the fabric8 console, the drop down menu on the right lets you quickly switch to other apps:
http://fabric8.io/guide/images/console-tools.png

more details here:
http://fabric8.io/guide/console.html

your previous vagrant issue looks like the fabric8-forge pod is trying to access the external URL for gogs; I wonder if the external IP its getting  - 172.30.75...) (from 'kubectl get node') is not an IP you can see from your browser? We've seen cases of vagrant installs not correctly exposing the actual host IP address into the kubernetes node.

 

On 12 September 2016 at 19:57, Mike Ray <mrayarchitect@flyingraydesign.com> wrote:

 

I just tried again with Minikube (instead of Vagrant) and everything appears to work.  :-)

The one benefit to Vagrant method is that the Landrush DNS services allows one to simply bookmark sites in the browser rather than being required to launch the browser from the command line.

Begin forwarded message:

Errors (most recent first):
System / Environmental's:
Commands:

-- 
You received this message because you are subscribed to the Google Groups "fabric8" group.
To unsubscribe from this group and stop receiving emails from it, send an email to fabric8+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Error syncing pod, skipping: failed to "StartContainer" for "jenkins" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=jenkins pod=jenkins-3604705456-xf7ae_default(0e0dfc16-76e1-11e6-9d02-ea4cdd9b5383)"
Back-off restarting failed docker container
Error syncing pod, skipping: failed to "StartContainer" for "jenkins" with RunContainerError: "PostStart handler: Error executing in Docker Container: 1"

Mac 10.11.6
16GB
minikube with virtualbox
9216MB for minikube

minikube start --memory=9216 --cpus=2 --insecure-registry=[] --vm-driver="virtualbox"
./gofabric8/gofabric8-darwin-amd64-v0.4.61  deploy --insecure-skip-tls-verify=true -y

--

[Mike]

I'm getting permission problems ( /data ) for gofabric8 version 0.4.73. I used the following script on a 16GB i7 Macbook Pro with no success:

# clear out any previous vm

minkube delete

minikube start --memory=9216 --cpus=2

minikube status

kubectl get services

minikube dashboard

# Install fabric8 - takes a LONG time to download, monitor progress with minikube dashboard (kubernetes)

./gofabric8/gofabric8-darwin-amd64-v0.4.73  deploy --insecure-skip-tls-verify=true -y

minikube service fabric8

minikube service fabric8 --url

On Tuesday, September 13, 2016 at 11:45:44 AM UTC-7, James Strachan wrote:

Yeah. For now you can use 'gofabric8 service foo" to open the URL (the same command is in minikube / minishift). There's a pending issue to do a DNS-like thing; though that would require also running an ingress controller on kubernetes or a router on minishift:
https://github.com/kubernetes/minikube/issues/347

Also from the fabric8 console, the drop down menu on the right lets you quickly switch to other apps:
http://fabric8.io/guide/images/console-tools.png 

more details here:

To unsubscribe from this group and stop receiving emails from it, send an email to fabric8+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Error syncing pod, skipping: failed to "StartContainer" for "jenkins" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=jenkins pod=jenkins-3604705456-xf7ae_default(0e0dfc16-76e1-11e6-9d02-ea4cdd9b5383)"
Back-off restarting failed docker container
Error syncing pod, skipping: failed to "StartContainer" for "jenkins" with RunContainerError: "PostStart handler: Error executing in Docker Container: 1"
Mac 10.11.6
16GB
minikube with virtualbox
9216MB for minikube
minikube start --memory=9216 --cpus=2 --insecure-registry=[] --vm-driver="virtualbox"
./gofabric8/gofabric8-darwin-amd64-v0.4.61  deploy --insecure-skip-tls-verify=true -y

[James Rawlings]

Hi Mike, do you have more details of the permission error you're seeing or do you have any errors during gofabric8 deploy that would help, do you have any failed pods?  

kubectl get pods

if you do then using the pod name there maybe some information in the pod events:

kubectl describe pod foo

On minikube we create persistent volumes out of the box which should have the right permissions set.  

If all else fails you could try our new gofabric8 commands that aim to make life easier:

minikube delete

rm -rf ~/.minikube

gofabric8 start --console

The --console flag will only download the console and fabric8 system related docker images, so you shouldn't have to wait as long to pull the full CD pipeline images.

James.

--
James Rawlings
Engineering

Red Hat, Inc.

Blog: https://medium.com/@jdrawlings

Twitter: @jdrawlings

http://fabric8.io

https://medium.com/fabric8-io

[Mike Ray]

The fabric8 console starts up fine.

When I run the CD Pipeline, I get the following (similar errors for Gogs, and Nexus):

Unable to mount volumes for pod "jenkins-983013807-vb98s_default(e172b9c5-85cf-11e6-9c0f-52f30356860f)": timeout expired waiting for volumes to attach/mount for pod "jenkins-983013807-vb98s"/"default". list of unattached/unmounted volumes=[jenkins-jobs jenkins-workspace]

[Mike Ray]

I reinstalled and noticed when I added CD Pipeline:

Here is what it was before running / install:

On Sep 28, 2016, at 3:08 PM, James Rawlings <rawli...@gmail.com> wrote:

[James Strachan]

There is a current limitation running the cd-pipeline from the console - if there are no PVs in your cluster & dynamic PV generation is not configured then Jenkins, nexus & gogs fail with unmourned volume errors. 

The workaround is to use 'gofabric8 volumes' 

Or just install cd-pipelines on minikube/minishift directly with 'gofabric8 start' which should create the PVs and chmod them properly

To unsubscribe from this group and stop receiving emails from it, send an email to fabric8+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "fabric8" group.

To unsubscribe from this group and stop receiving emails from it, send an email to fabric8+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.