[Jenkins X] How to run npm install as part of a build
2,295 views
Skip to first unread message
Matt Raible
Jul 3, 2018, 1:03:42 AM7/3/18
to jenkins...@googlegroups.com
Hello everyone,
I’m trying to port a Spring Boot + Ionic project from using Travis CI to using Jenkins X. I was able to setup a cluster on Google Cloud and I imported my project using
jx import —url https://github.com/mraible/okta-spring-boot-angular-auth-code-flow-example.git
I quickly found out that Jenkins thought I had a Node.js project, because it used container(‘nodejs’) everywhere.
I changed it to use Maven instead and modified the steps to match what I formerly had in Travis CI.
Now I’m getting an error when I try to run ’npm install ionic’ because npm is not found. I tried installing the NodeJS plugin and configure it to install ionic globally, but that doesn’t seem to help.
Here’s my Travis script that works:
On a related note, I have to install jx every time I log in to Google Cloud. Is that normal?
curl -L https://github.com/jenkins-x/jx/releases/download/v1.3.52/jx-linux-amd64.tar.gz | tar xzv
sudo mv jx /usr/local/bin
Thanks,
Matt
Matt Raible
Jul 3, 2018, 7:07:39 PM7/3/18
to jenkins...@googlegroups.com
I tried to use Jenkin’s “multiple containers” feature today and got a little further.
Then I realized I could do everything from Maven and configured the frontend-maven-plugin to do “npm install" for me. Now I’m getting the following error.
[ERROR] Unable to save binary /home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/crypto-pwa/node_modules/node-sass/vendor/linux-ia32-57 : { Error: EACCES: permission denied, mkdir '/home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/crypto-pwa/node_modules/node-sass/vendor'
[ERROR] at Object.fs.mkdirSync (fs.js:885:18)
[ERROR] at sync (/home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/crypto-pwa/node_modules/mkdirp/index.js:71:13)
[ERROR] at Function.sync (/home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/crypto-pwa/node_modules/mkdirp/index.js:77:24)
[ERROR] at checkAndDownloadBinary (/home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/crypto-pwa/node_modules/node-sass/scripts/install.js:111:11)
[ERROR] at Object.<anonymous> (/home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/crypto-pwa/node_modules/node-sass/scripts/install.js:154:1)
[ERROR] at Module._compile (module.js:652:30)
[ERROR] at Object.Module._extensions..js (module.js:663:10)
[ERROR] at Module.load (module.js:565:32)
[ERROR] at tryModuleLoad (module.js:505:12)
[ERROR] at Function.Module._load (module.js:497:3)
[ERROR] errno: -13,
[ERROR] code: 'EACCES',
[ERROR] syscall: 'mkdir',
[ERROR] path: '/home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/crypto-pwa/node_modules/node-sass/vendor' }
Is there something I have to configure in Jenkins X to allow node-sass to be installed?
Here’s my current Jenkinsfile: https://github.com/mraible/okta-spring-boot-angular-auth-code-flow-example/blob/PR-node-container/Jenkinsfile
Thanks,
Matt
Matt Raible
Jul 5, 2018, 10:57:54 AM7/5/18
to jenkins...@googlegroups.com
It seems that this issue is caused by Jenkins X running the build as the root user.
I was able to fix this by using node-sass’s troubleshooting instructions:
And adjusting my pom.xml to run with unsafe permissions:
<execution>
<id>npm install</id>
<goals>
<goal>npm</goal>
</goals>
<phase>generate-resources</phase>
<configuration>
<arguments>install --unsafe-perm</arguments>
</configuration>
</execution>
Now I’m getting the following error, which seems to be related to the fact that my repo name is so long.
[preview] Running shell script
+ jx preview --app mraible --dir ../..
Creating a preview
Commit author email is empty for: 83821e4392538adb80e65242b880c2d86b4f1b27
Looking for commits in: /home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/
Commit author email is empty for: 9bb8a3636341c3ea612686ad7d3ff0f9f187a73d
Looking for commits in: /home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/
Commit author email is empty for: 5911be9f2ef820bd9dbda0419f5e69f3a4d25e7b
Looking for commits in: /home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/
Commit author email is empty for: 72b76849c4cd2a324f513d0cdb8588a484c847e3
Looking for commits in: /home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/
Commit author email is empty for: 62a286af01ec19b129f5d13ac57ed6718386eff8
Looking for commits in: /home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/
Commit author email is empty for: ba663183438a56a7fafe90c6043b8cc9f1265cda
Looking for commits in: /home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/
Commit author email is empty for: d94ffea2a6beaf164d42cbbe5a97621766f3fdd6
Looking for commits in: /home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/
CreateOrUpdateUser: mraible <matt....@okta.com>
Adding missing user: matt.raible.okta.com
Created environment mraible-okta-spring-boot-angular-auth-code-flow-example-pr-1
expose:
config:
domain: 35.236.161.165.nip.io
exposer: ""
http: ""
tlsacme: ""
preview:
image:
repository: 10.51.245.83:5000/mraible/mraible
tag: 0.0.0-SNAPSHOT-PR-1-7
[debug] Created tunnel using local port: '44282'
[debug] SERVER: "127.0.0.1:44282"
Error: UPGRADE FAILED: invalid release name, must match regex ^(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])+$ and the length must not longer than 53
Error: Command failed helm upgrade jx-mraible-okta-spring-boot-angular-auth-code-flow-example-pr-1 . --force --install --debug --wait --namespace jx-mraible-okta-spring-boot-angular-auth-code-flow-example-pr-1 --values=/home/jenkins/workspace/auth-code-flow-example_PR-1-MFELMYVCFZVXOAELC7G2YU6BAUFSINH5K2KN532X72ETDA3JTKQQ/charts/preview/extraValues.yaml
I’ll try to rename it and see if that works.
Thanks for following along y’all! :)
Matt
Matt Raible
Jul 6, 2018, 12:36:13 PM7/6/18
to jenkins...@googlegroups.com
Renaming my repo to something shorter did solve the problem. Yesterday, I was able to get my Docker image refactored to build things thanks to James Strachan’s https://github.com/jstrachan/hipster3 repo. Then, it was failing to deploy:
export VERSION=$PREVIEW_VERSION && skaffold run -f skaffold.yaml
Step 5/5 : ADD holdings-api/target/*.jar /app.jar
---> 4df4f93bc7b1
Successfully built 4df4f93bc7b1
Successfully tagged 10.51.245.83:5000/mraible/mraible:0.0.0-SNAPSHOT-PR-1-16
The push refers to a repository [10.51.245.83:5000/mraible/mraible]
e1bd9fec5742: Preparing
7caae1a37299: Preparing
c9b26f41504c: Preparing
cd7100a72410: Preparing
7caae1a37299: Layer already exists
c9b26f41504c: Layer already exists
cd7100a72410: Layer already exists
e1bd9fec5742: Pushed
0.0.0-SNAPSHOT-PR-1-16: digest: sha256:6fc95f11f94ea1f4e9f050a2a95ec02f8e58176494f885265039cb059b44cebe size: 1159
Build complete in 5.918334422s
Starting deploy...
time="2018-07-05T22:41:59Z" level=warning msg="image [changeme] is not used by the deployment"
error: no objects passed to apply
time="2018-07-05T22:42:00Z" level=fatal msg="deploy step: deploying manifests: exit status 1"
script returned exit code 1I decided to try a different approach today and started by creating a Spring Boot app using the following command:
$ jx create spring -d web -d actuator
Then, I copied all of my code into the Spring Boot project and changed the paths in Dockerfile and Jenkinsfile to accommodate my nested apps structure.
I created a PR and got things to pass and deploy to a preview environment.
It took me a while to find the logs for the pod (is there an easier way using jx?), but when I did, it shows my Spring Boot app starting, then stopping. I guess things are working, but since it doesn’t stay up in a preview environment, I’m not sure.
I 2018-07-06 16:08:08.544 INFO 1 --- [ main] o.s.b.a.w.s.WelcomePageHandlerMapping : Adding welcome page: class path resource [static/index.html] I 2018-07-06 16:08:09.025 INFO 1 --- [ main] o.s.j.e.a.AnnotationMBeanExporter : Registering beans for JMX exposure on startup I 2018-07-06 16:08:09.147 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path '' I 2018-07-06 16:08:09.223 INFO 1 --- [ main] c.o.d.h.HoldingsApiApplication : Started HoldingsApiApplication in 21.08 seconds (JVM running for 22.929) I 2018-07-06 16:08:09.226 INFO 1 --- [ main] c.o.d.h.HoldingsApiApplication : I ---------------------------------------------------------- I Application 'Crypto Wealth Tracker' is running! Access URLs: I Local: http://localhost:8080 I External: http://10.52.2.21:8080 I Profile(s): [prod] I ---------------------------------------------------------- I 2018-07-06 16:08:13.545 INFO 1 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring FrameworkServlet 'dispatcherServlet' I 2018-07-06 16:08:13.546 INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization started I 2018-07-06 16:08:13.728 INFO 1 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : FrameworkServlet 'dispatcherServlet': initialization completed in 182 ms I 2018-07-06 16:09:05.838 INFO 1 --- [ Thread-4] ConfigServletWebServerApplicationContext : Closing org.springframework.boot.web.servlet.context.AnnotationConfigServletWebServerApplicationContext@340f438e: startup date [Fri Jul 06 16:07:52 UTC 2018]; root of context hierarchy I 2018-07-06 16:09:05.925 INFO 1 --- [ Thread-4] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown
Cheers,
Matt
Michael Neale
Jul 8, 2018, 8:32:07 PM7/8/18
to Jenkins Users
Hi Matt.
Can't answer all of it (sounds like you got further) - but yeah, there are a few pre-made images, and in fact javascript projects when imported use this build pack: https://github.com/jenkins-x/draft-packs/tree/master/packs/javascript (although it sounds like you are happy with maven image).
Can't answer all of it (sounds like you got further) - but yeah, there are a few pre-made images, and in fact javascript projects when imported use this build pack: https://github.com/jenkins-x/draft-packs/tree/master/packs/javascript (although it sounds like you are happy with maven image).
No - you should not need to install jx repeatedly, that is most odd.
Matt Raible
Jul 9, 2018, 12:06:20 AM7/9/18
to jenkins...@googlegroups.com
I got most things figured out by using Maven to do them. The only thing I wasn’t able to get working was running Protractor tests. I installed the Xvfb and chromedriver plugins too. In a Maven profile, I have a profile to run Protractor, which I can run with “mvn install -Pprod,e2e”. It works locally, but when running in Jenkins X, I get the following error:
[INFO] --- frontend-maven-plugin:1.6:npm (ionic e2e) @ holdings-api ---
[INFO] Running 'npm run e2e' in /home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa
[INFO]
[INFO] > cryp...@0.0.3 e2e /home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa
[INFO] > webdriver-manager update --standalone false --gecko false && protractor ./test/protractor.conf.js
[INFO]
[INFO] [06:13:44] I/file_manager - creating folder /home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/protractor/node_modules/webdriver-manager/selenium
[INFO] [06:13:45] I/update - chromedriver: unzipping chromedriver_2.33.zip
[INFO] [06:13:46] I/update - chromedriver: setting permissions to 0755 for /home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/protractor/node_modules/webdriver-manager/selenium/chromedriver_2.33
[ERROR] (node:858) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
[INFO] [06:13:48] I/launcher - Running 1 instances of WebDriver
[INFO] [06:13:48] I/direct - Using ChromeDriver directly...
[INFO] [06:13:48] E/launcher - Server terminated early with status 127
[INFO] [06:13:48] E/launcher - Error: Server terminated early with status 127
[INFO] at earlyTermination.catch.e (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/selenium-webdriver/remote/index.js:252:52)
[INFO] at <anonymous>
[INFO] at process._tickCallback (internal/process/next_tick.js:188:7)
[INFO] From: Task: WebDriver.createSession()
[INFO] at Function.createSession (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/selenium-webdriver/lib/webdriver.js:769:24)
[INFO] at Function.createSession (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/selenium-webdriver/chrome.js:761:15)
[INFO] at Direct.getNewDriver (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/protractor/built/driverProviders/direct.js:77:33)
[INFO] at Runner.createBrowser (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/protractor/built/runner.js:195:43
[INFO] at q.then.then (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/protractor/built/runner.js:339:29)
[INFO] at _fulfilled (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/q/q.js:834:54)
[INFO] at self.promiseDispatch.done (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/q/q.js:863:30)
[INFO] at Promise.promise.promiseDispatch (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/q/q.js:796:13)
[ERROR] npm ERR! code ELIFECYCLE
[ERROR] npm ERR! errno 199
[ERROR] npm ERR! cryp...@0.0.3 e2e: `webdriver-manager update --standalone false --gecko false && protractor ./test/protractor.conf.js`
[ERROR] npm ERR! Exit status 199
[ERROR] npm ERR!
[ERROR] npm ERR! Failed at the cryp...@0.0.3 e2e script.
[ERROR] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
[INFO] at /home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/q/q.js:556:49
[INFO] at runSingle (/home/jenkins/workspace/mraible_jx-demo_PR-1-MSUIFYHSV63RPK5FBOMHYLPRX3VKXXIENUIT2JO4UEEZFWAHCS4Q/crypto-pwa/node_modules/q/q.js:137:13)
[INFO] [06:13:48] E/launcher - Process exited with error code 199
Any ideas why?
Thanks,
Matt
P.S. For the jx command disappearing from Google Cloud, it appears they wipe it after an hour. https://cloud.google.com/shell/docs/limitations#custom_installed_software_packages_and_persistence
<profile>
<id>e2e</id>
<properties>
<http.port>8000</http.port>
</properties>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<id>pre-integration-test</id>
<goals>
<goal>start</goal>
</goals>
<configuration>
<arguments>
<argument>--server.port=${http.port}</argument>
</arguments>
</configuration>
</execution>
<execution>
<id>post-integration-test</id>
<goals>
<goal>stop</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>com.github.eirslett</groupId>
<artifactId>frontend-maven-plugin</artifactId>
<version>${frontend-maven-plugin.version}</version>
<configuration>
<workingDirectory>../crypto-pwa</workingDirectory>
</configuration>
<executions>
<execution>
<id>ionic e2e</id>
<goals>
<goal>npm</goal>
</goals>
<phase>integration-test</phase>
<configuration>
<environmentVariables>
<PORT>${http.port}</PORT>
<CI>true</CI>
</environmentVariables>
<arguments>run e2e</arguments>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/54b332b6-75bb-4122-bcf2-fc42e93b1c53%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Carlos Sanchez
Jul 9, 2018, 4:33:31 AM7/9/18
to Jenkins Users
Error 127 seems to be command not found
https://stackoverflow.com/questions/1763156/127-return-code-from
Maybe you can get more info from the container logs: kubectl logs $(kubectl get pods -o name | grep something)
Maybe you can get more info from the container logs: kubectl logs $(kubectl get pods -o name | grep something)
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/159100F4-E4EA-416F-8B1B-1259818C7DA4%40raibledesigns.com.
Michael Neale
Jul 9, 2018, 5:21:58 AM7/9/18
to Jenkins Users
Hrm - I couldn't find it in my samples but I remember needing to do something like:
Xvfb :99 &
export DISPLAY=:99
when headless to get it all to work, not sure if relevant here.
Oh gotcha on the cloud shell (not used that myself) but that does sound like an unsurprising limitation (didn't realise that... but it kind of makes sense on that link).
BTW, if you are a slack user http://slack.k8s.io/ and https://kubernetes.slack.com/messages/C9MBGQJRH if you are interested in real time chat
Matt Raible
Jul 9, 2018, 12:10:41 PM7/9/18
to jenkins...@googlegroups.com
I added these steps and it doesn’t seem to help:
sh "Xvfb :99 &"
sh "DISPLAY=:99 mvn install -Pprod,e2e"
With Travis, I have to install Chrome and set the path to it. Does Jenkins X need a similar setup?
Relevant .travis.yml snippets:
addons:
apt:
sources:
- google-chrome
packages:
- google-chrome-stable
apt:
sources:
- google-chrome
packages:
- google-chrome-stable
before_install:
- export CHROME_BIN=/usr/bin/google-chrome
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
- export CHROME_BIN=/usr/bin/google-chrome
- export DISPLAY=:99.0
- sh -e /etc/init.d/xvfb start
I found this guide, will try it, but hoping for something easier:
Cheers,
Matt
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0fc504fa-06fe-4418-b180-d8296e348eee%40googlegroups.com.
Carlos Sanchez
Jul 9, 2018, 12:15:37 PM7/9/18
to Jenkins Users
You could also use different containers to run the selenium tests
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/5BA82421-6411-4BFD-B015-3049005A2826%40raibledesigns.com.
Matt Raible
Jul 9, 2018, 12:26:16 PM7/9/18
to jenkins...@googlegroups.com
Great idea! Are you saying I should be able to change from:
agent {
label "jenkins-maven"
}To:
agent {
label "maven-chrome"
}Or is there more setup that needs to happen? I tried it and all I get is the following message:
There are no nodes with the label ‘maven-chrome’
On a related note, is it possible to get the preview URL that Jenkins X deploys to? I see the following at the end:
Preview application is now available at: http://jx-demo.jx-mraible-jx-demo-pr-2.35.189.36.9.nip.io
And I want to run a process that updates my Okta app with this redirect URI. I have a program written, so I just need to run the following at the end of the build.
mvn exec:java@add-redirect -DappId=$APP_ID -DredirectUri=${previewURL}/login
Thanks,
Matt
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALHFn6NkjF40psTKr3B6%2BGyFM01JVzy_44zQDiRBKNnZAgVXsg%40mail.gmail.com.
Makkaoui Ahmed
Jul 9, 2018, 12:27:44 PM7/9/18
to jenkins...@googlegroups.com
HELLO im using jenkins in my machine CENTOS and i need to clone the project from gitlab but i have a error :
stderr: fatal: unable to access 'https://gitlab.s.com/jihed/byzaneo.git/': Failed connect to gitlab.s.com:443; Connection refused
Failed to connect to repository : Command "/usr/bin/git ls-remote -h https://gitlab.s.com/j/byzo.git HEAD" returned status code 128:
THX
stderr: fatal: unable to access 'https://gitlab.s.com/jihed/byzaneo.git/': Failed connect to gitlab.s.com:443; Connection refused
Failed to connect to repository : Command "/usr/bin/git ls-remote -h https://gitlab.s.com/j/byzo.git HEAD" returned status code 128:
THX
Mekkaoui Ahmed
E-mail : ahmed.m...@esprit.tn
Réseaux sociaux : https://www.linkedin.com/in/mekkaouiahmed/
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0fc504fa-06fe-4418-b180-d8296e348eee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/5BA82421-6411-4BFD-B015-3049005A2826%40raibledesigns.com.To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
Matt Raible
Jul 9, 2018, 1:35:23 PM7/9/18
to jenkins...@googlegroups.com
I’ll answer my own question. No, this does not work. When I tried surrounding my pipeline with podTemplate/containerTemplate, it didn’t work.
https://github.com/mraible/jx-demo/blob/cabf04fd5b93e721dc87d0490afce951bddc9143/Jenkinsfile
I think I’m going to have to give up on running Protractor tests on Jenkins X. I’ve lost too many hours trying to make it work. I’ll share my findings in a blog post this week.
As far as the previewURL, I’m still interested in seeing if that’s possible.
Thanks!
Matt
Carlos Sanchez
Jul 9, 2018, 1:35:35 PM7/9/18
to Jenkins Users
On Mon, Jul 9, 2018 at 6:26 PM Matt Raible <ma...@raibledesigns.com> wrote:
Great idea! Are you saying I should be able to change from:agent {
label "jenkins-maven"
}To:agent {
label "maven-chrome"
}Or is there more setup that needs to happen? I tried it and all I get is the following message:There are no nodes with the label ‘maven-chrome’
you would need to configure your agent pod template to have those containers in the pod and use `container(maven-chrome)` It runs all as a one pod agent with multiple containers. There is no preconfigured pod template to do that/
On a related note, is it possible to get the preview URL that Jenkins X deploys to? I see the following at the end:Preview application is now available at: http://jx-demo.jx-mraible-jx-demo-pr-2.35.189.36.9.nip.io
yes, although it is not trivial. See my demo using selenium against the preview deployment
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/EEC90DD9-F006-438F-A36B-6E4BA4BFD108%40raibledesigns.com.
jraw...@cloudbees.com
Jul 9, 2018, 1:57:26 PM7/9/18
to Jenkins Users
In Jenkins X we have a number of builder images, the nodejs one has chome and xvfb https://github.com/jenkins-x/builder-nodejs/blob/master/Dockerfile#L4 which we use in the node quickstart https://github.com/jenkins-x/draft-packs/blob/master/packs/javascript/Jenkinsfile#L21-L23.
So you could try adding a step like that which uses the OOTB Jenkins X nodejs agent. Fingers crossed that will work.
FWIW we also hang out in the Jenkins X rooms from the Kubernetes Slack channel, incase there's any realtime help we can provide https://jenkins-x.io/community/
Hope that helps, James.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsub...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/0fc504fa-06fe-4418-b180-d8296e348eee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/5BA82421-6411-4BFD-B015-3049005A2826%40raibledesigns.com.
For more options, visit https://groups.google.com/d/optout.
--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALHFn6NkjF40psTKr3B6%2BGyFM01JVzy_44zQDiRBKNnZAgVXsg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Matt Raible
Jul 9, 2018, 6:03:08 PM7/9/18
to jenkins...@googlegroups.com
I tried creating a maven-chrome container with your example and have the same issue.
stage('CI Build and push snapshot') {
agent {
kubernetes {
label 'protractor'
yaml """
apiVersion: v1
kind: Pod
metadata:
labels:
some-label: some-label-value
spec:
serviceAccountName: jenkins
containers:
- name: maven-chrome
image: jenkinsxio/builder-maven:0.0.305
command:
- cat
tty: true
"""
}
}
when {
branch 'PR-*'
}
environment {
PREVIEW_VERSION = "0.0.0-SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER"
PREVIEW_NAMESPACE = "$APP_NAME-$BRANCH_NAME".toLowerCase()
HELM_RELEASE = "$PREVIEW_NAMESPACE".toLowerCase()
}
steps {
container('maven-chrome') {
dir ('./holdings-api') {
sh "mvn versions:set -DnewVersion=$PREVIEW_VERSION"
sh "Xvfb :99 &"
sh "DISPLAY=:99 mvn install -Pprod,e2e"
}
sh 'export VERSION=$PREVIEW_VERSION && skaffold build -f skaffold.yaml'
sh "jx step post build --image $DOCKER_REGISTRY/$ORG/$APP_NAME:$PREVIEW_VERSION"
}
dir ('./charts/preview') {
container('maven') {
sh "make preview"
sh "jx preview --app $APP_NAME --dir ../.."
}
}
// Add redirect URI in Okta
dir ('./holdings-api') {
container('maven') {
sh '''
yum install -y jq
previewUrl=$(jx get preview -o json|jq -r ".items[].spec | select (.previewGitInfo.name==\\"$CHANGE_ID\\") | .previewGitInfo.applicationURL")
mvn exec:java@add-redirect -DappId=$OKTA_APP_ID -DredirectUri=${previewUrl}/login
'''
}
}
}
}
[INFO] --- frontend-maven-plugin:1.6:npm (ionic e2e) @ holdings-api ---
[INFO] Running 'npm run e2e-test' in /home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa
[INFO]
[INFO] > cryp...@0.0.3 e2e-test /home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa
[INFO] > protractor ./test/protractor.conf.js
[INFO]
[ERROR] (node:825) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
[INFO] [20:12:57] I/launcher - Running 1 instances of WebDriver
[INFO] [20:12:57] I/direct - Using ChromeDriver directly...
[INFO] [20:12:57] E/launcher - Server terminated early with status 127
[INFO] [20:12:57] E/launcher - Error: Server terminated early with status 127
[INFO] at earlyTermination.catch.e (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/remote/index.js:252:52)
[INFO] at <anonymous>
[INFO] at process._tickCallback (internal/process/next_tick.js:188:7)
[INFO] From: Task: WebDriver.createSession()
[INFO] at Function.createSession (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/webdriver.js:769:24)
[INFO] at Function.createSession (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/chrome.js:761:15)
[INFO] at Direct.getNewDriver (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/protractor/built/driverProviders/direct.js:77:33)
[INFO] at Runner.createBrowser (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/protractor/built/runner.js:195:43)
[INFO] at q.then.then (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/protractor/built/runner.js:339:29)
[INFO] at _fulfilled (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/q/q.js:834:54)
[INFO] at self.promiseDispatch.done (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/q/q.js:863:30)
[INFO] at Promise.promise.promiseDispatch (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/q/q.js:796:13)
[ERROR] npm ERR! code ELIFECYCLE
[INFO] at /home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/q/q.js:556:49
[ERROR] npm ERR! errno 199
[INFO] at runSingle (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/q/q.js:137:13)
[INFO] [20:12:57] E/launcher - Process exited with error code 199
[ERROR] npm ERR! cryp...@0.0.3 e2e-test: `protractor ./test/protractor.conf.js`
I also tried using a nodejs container to run against the preview environment:
stage('Run e2e tests') {
agent {
label "jenkins-nodejs"
}
steps {
container('nodejs') {
sh "echo 'Running e2e tests on ${PREVIEW_URL}...'"
dir ('./crypto-pwa') {
sh "npm install"
sh "Xvfb :99 &"
sh "DISPLAY=:99 npm run e2e -- --baseUrl=$PREVIEW_URL"
}
}
}
}Unfortunately, this fails too, but at the least the error message is a little better.
(node:195) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.
[21:50:08] I/launcher - Running 1 instances of WebDriver
[21:50:08] I/direct - Using ChromeDriver directly...
[21:51:08] E/launcher - unknown error: DevToolsActivePort file doesn't exist
(Driver info: chromedriver=2.40.565383 (76257d1ab79276b2d53ee976b2c3e3b9f335cde7),platform=Linux 4.4.111+ x86_64)
[21:51:08] E/launcher - WebDriverError: unknown error: DevToolsActivePort file doesn't exist
(Driver info: chromedriver=2.40.565383 (76257d1ab79276b2d53ee976b2c3e3b9f335cde7),platform=Linux 4.4.111+ x86_64)
at Object.checkLegacyResponse (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/error.js:546:15)
at parseHttpResponse (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/http.js:509:13)
at doSend.then.response (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/http.js:441:30)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:182:7)
From: Task: WebDriver.createSession()
Google led me to this https://github.com/GoogleChrome/puppeteer/issues/1834, which says it’s a Chrome on Linux issue. Angular has a related issue too https://github.com/angular/protractor/issues/4850. I’m going to try this solution:
webdriver-manager update --versions.chrome 2.37
And then I’ll try specifying the following chromeOptions:
['--headless', '--disable-gpu', '--no-sandbox', '--disable-extensions', '--disable-dev-shm-usage’]
BBS,
Matt
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CALHFn6PrCDVqCA1hKzU-BeoirTNwfDQ_vvet2xKh0rttgiQ%3DGQ%40mail.gmail.com.
Matt Raible
Jul 9, 2018, 6:56:29 PM7/9/18
to jenkins...@googlegroups.com
On Jul 9, 2018, at 4:02 PM, Matt Raible <ma...@raibledesigns.com> wrote:Unfortunately, this fails too, but at the least the error message is a little better.(node:195) [DEP0022] DeprecationWarning: os.tmpDir() is deprecated. Use os.tmpdir() instead.[21:50:08] I/launcher - Running 1 instances of WebDriver[21:50:08] I/direct - Using ChromeDriver directly...[21:51:08] E/launcher - unknown error: DevToolsActivePort file doesn't exist(Driver info: chromedriver=2.40.565383 (76257d1ab79276b2d53ee976b2c3e3b9f335cde7),platform=Linux 4.4.111+ x86_64)[21:51:08] E/launcher - WebDriverError: unknown error: DevToolsActivePort file doesn't exist(Driver info: chromedriver=2.40.565383 (76257d1ab79276b2d53ee976b2c3e3b9f335cde7),platform=Linux 4.4.111+ x86_64)at Object.checkLegacyResponse (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/error.js:546:15)at parseHttpResponse (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/http.js:509:13)at doSend.then.response (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/http.js:441:30)at <anonymous>at process._tickCallback (internal/process/next_tick.js:182:7)From: Task: WebDriver.createSession()Google led me to this https://github.com/GoogleChrome/puppeteer/issues/1834, which says it’s a Chrome on Linux issue. Angular has a related issue too https://github.com/angular/protractor/issues/4850. I’m going to try this solution:webdriver-manager update --versions.chrome 2.37And then I’ll try specifying the following chromeOptions:['--headless', '--disable-gpu', '--no-sandbox', '--disable-extensions', '--disable-dev-shm-usage’]
Downgrading to 2.37 did not help. Now I get an unknown error:
[22:28:15] I/direct - Using ChromeDriver directly...
[22:29:15] E/launcher - unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.4.111+ x86_64)
[22:29:15] E/launcher - WebDriverError: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Linux 4.4.111+ x86_64)
at Object.checkLegacyResponse (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/error.js:546:15)
at parseHttpResponse (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/http.js:509:13)
at doSend.then.response (/home/jenkins/workspace/mraible_jx-demo_PR-2-C7V4GOQ2ULMKN4TRTR4MOAZTPV7ZLFKNQTNPVQZ5WOVWBQBUXFUA/crypto-pwa/node_modules/selenium-webdriver/lib/http.js:441:30)
at <anonymous>
at process._tickCallback (internal/process/next_tick.js:182:7)
From: Task: WebDriver.createSession()
capabilities: {
'browserName': 'chrome',
'chromeOptions': {
'args': ['--headless', '--disable-gpu', '--no-sandbox', '--disable-extensions', '--disable-dev-shm-usage']
}
},
And it worked!
I’ll trace back from here to try and figure the optimal configuration.
Thanks for the help y’all! 🍻
Matt
Matt Raible
Jul 10, 2018, 8:12:49 AM7/10/18
to jenkins...@googlegroups.com
12 hours later and I almost have all my protractor tests passing! I’ve had to increase the timeouts from 5s to 30s for many, but this is likely because I created my cluster in australia-southeast1. I’ve pasted the stages I’m using for CI build and e2e tests below.
I also found my environment variables from Jenkins X aren’t passed to the Docker images, so I echo them to an application.properties file, then copy that to the image. In Dockerfile, I have:
COPY holdings-api/target/application.properties /opt/.
A few questions:
1. Is there an easier way to get the previewURL so I only have to retrieve it once? Currently, I have the following code block twice:
sh '''
yum install -y jq
previewURL=$(jx get preview -o json|jq -r ".items[].spec | select (.previewGitInfo.name==\\"$CHANGE_ID\\") | .previewGitInfo.applicationURL")
mvn exec:java@add-redirect -DappId=$OKTA_APP_ID -DredirectUri=$previewURL
‘''
2. Is it possible to add HTTPS support for preview/staging with Let’s Encrypt? Since I’m building a PWA (progressive web app), one of the requirements is HTTPS.
3. Is it possible to transfer environment variables/secrets to the Docker image from Jenkins X?
stage('CI Build and push snapshot') {
when {
branch 'PR-*'
}
environment {
PREVIEW_VERSION = "0.0.0-SNAPSHOT-$BRANCH_NAME-$BUILD_NUMBER"
PREVIEW_NAMESPACE = "$APP_NAME-$BRANCH_NAME".toLowerCase()
HELM_RELEASE = "$PREVIEW_NAMESPACE".toLowerCase()
}
steps {
container('maven') {
dir ('./holdings-api') {
sh "mvn versions:set -DnewVersion=$PREVIEW_VERSION"
sh "mvn install -Pprod"
sh "echo 'okta.client.token='$OKTA_CLIENT_TOKEN > target/application.properties"
}
sh 'export VERSION=$PREVIEW_VERSION && skaffold build -f skaffold.yaml'
sh "jx step post build --image $DOCKER_REGISTRY/$ORG/$APP_NAME:$PREVIEW_VERSION"
}
dir ('./charts/preview') {
container('maven') {
sh "make preview"
sh "jx preview --app $APP_NAME --dir ../.."
}
}
// Add redirect URI in Okta
dir ('./holdings-api') {
container('maven') {
sh '''
yum install -y jq
previewURL=$(jx get preview -o json|jq -r ".items[].spec | select (.previewGitInfo.name==\\"$CHANGE_ID\\") | .previewGitInfo.applicationURL")
mvn exec:java@add-redirect -DappId=$OKTA_APP_ID -DredirectUri=$previewURL
'''
}
}
}
}
stage('Run e2e tests') {
agent {
label "jenkins-nodejs"
}
steps {
container('nodejs') {
sh '''
yum install -y jq
previewURL=$(jx get preview -o json|jq -r ".items[].spec | select (.previewGitInfo.name==\\"$CHANGE_ID\\") | .previewGitInfo.applicationURL")
cd crypto-pwa && npm install --unsafe-perm && npm run e2e-update
Xvfb :99 &
echo 'Running e2e tests on \$previewURL in 60s...'
sleep 60s
DISPLAY=:99 npm run e2e-test -- --baseUrl=$previewURL
'''
}
}
}
Thanks,
Matt
Carlos Sanchez
Jul 10, 2018, 9:34:16 AM7/10/18
to Jenkins Users
This is how I pass the variables to docker build PREVIEW_VERSION GIT_COMMIT
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/63651303-600A-4321-8EEE-CE77C8907D1D%40raibledesigns.com.
Michael Neale
Jul 10, 2018, 9:18:02 PM7/10/18
to Jenkins Users
well that is good more is working - on #1:
$(cat .previewUrl) - I believe the preview writes the url in the content of that file (there may be another way now that is cleaner).
For environments - they should be made available if they are declared in environment section.
Reply all
Reply to author
Forward
0 new messages