Jenkins and monitoring (python jupyter) artifacts

434 views
Skip to first unread message

Martin Holeček

unread,
Jun 18, 2018, 3:59:23 AM6/18/18
to Jenkins Developers
Hello!
Please, what approach would be the best for the following task? We use Jenkins for running data science experiments (experiment = jenkins build).

I would like to have (for a job I would configure) each build to display either:

a) its already produced jupyter notebook (the job had done as a part of the build). Maybe this is called 'build artifact'? Am I right?

b) live (like it is monitoring stdout on 'console output') jupyter notebook view, that the build is running. (For people not familiar with ipython notebooks - the build would, as a part of pipeline just start the jupyter notebook server using bash command and run a specific file. The jupyter service then creates -at the node it is running on- a localhost webserver on a a specific port ... and I want this web-app output of the port displayed in jenkins->job->build ... meybe on a plugin's page, i do not want to overwrite consoleoutput, of course)

Is there already a plugin for that? Or should I make my own one? Or should it be way easier using some trick? What is the best way to do it?

Thanks!

Martin

Also I can give you credit on stackoverflow ;)
https://stackoverflow.com/questions/50627444/jenkins-and-monitoring-artefacts-like-jupyter-notebooks

Lloyd Chang

unread,
Jan 21, 2019, 9:01:45 PM1/21/19
to Jenkins Developers
Hello Martin Holeček,

Would you be interested and have time availability to champion a new Jenkins plugin (with code to be written in Python) for Data Science and Jupyter Notebook workflows?


Thank you,
Lloyd

Oleg Nenashev

unread,
Jan 22, 2019, 7:10:52 PM1/22/19
to Jenkins Developers
As discussed in the GSoC Gitter channel, Martin has created a GSoC project idea draft: https://docs.google.com/document/d/19ignQBMUr3qxfmkf8Sa9KG7wJlxs3js_kg4mJhX_dXo/edit?usp=sharing . I have copy-edited this draft, and I think it could be published as a draft project idea on the Jenkins GSoC page. Any reviews and feedback will be appreciated.

Best regards,
Oleg

Ioannis Moutsatsos

unread,
Feb 28, 2019, 12:02:19 PM2/28/19
to Jenkins Developers
I have updated the GSoC document and added my self as a Champion/Mentor
This project is highly interesting to me and the BioUno project
Refs: http://biouno.org/ , http://biouno.org/2016/03/07/Creating-a-Jenkins-Data-Science-Platform.html

Oleg Nenashev

unread,
Feb 28, 2019, 1:59:35 PM2/28/19
to Jenkins Developers
Hi Ioannis,

Thanks a lot for the interest! Any contribution will be appreciated, and we will welcome more mentors for this project.
I recommend scheduling a sync-up with Martin and Denis so that you can discuss collaboration (e.g. it might be one or multiple project ideas).

If BioUno project members are interested to propose more project ideas or to advertise Jenkins GSoC among student contributors, it will be more than appreciated as well.

Best regards,
Oleg

Ioannis Moutsatsos

unread,
Feb 28, 2019, 2:53:09 PM2/28/19
to jenkin...@googlegroups.com
Thank you Oleg

Glad to provide my support and expertise. I'm currently communicating with the BioUno contributors to see if we can include additional mentors.
Could you update us though about the the status of the proposal? Has it been submitted to Google?

Thanks and best regards
Ioannis 

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/VrBoOWVsZ-o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/2c56a881-5040-4579-9785-3aa6d2ab0965%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


--
Best regards
--Ioannis--

Ioannis K. Moutsatsos
imout...@gmail.com

Oleg Nenashev

unread,
Feb 28, 2019, 3:15:32 PM2/28/19
to JenkinsCI Developers, dart...@gmail.com, djou...@cloudbees.com
Hi Ioannis,

Also added other mentors to Cc.

Could you update us though about the the status of the proposal? Has it been submitted to Google?

GSoC process works a bit differently. Organizations do not submit projects to GSoC, students are expected to submit project applications instead. Organizations only propose project ideas. They help students to focus their exploration on particular areas and then to submit project proposals based on these ideas. Students are expected to submit their project proposals till April 9, and then we will select project proposals and students.

So please consider the project idea as active. It is in the Draft state, because there is an ongoing discussion in which all interested mentors and students are welcome to participate. And we already several students asking about this project. But I hope we will be able to move it to the published state soon.

Hopefully it helps,
Oleg
 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAOwNbFtdYANuwPU1-Cq8L10vhV5-nh7Uj6gEuNzn93GQngh%3DFw%40mail.gmail.com.

Ioannis Moutsatsos

unread,
Feb 28, 2019, 3:19:46 PM2/28/19
to jenkin...@googlegroups.com
Indeed very helpful info, and good to know that there is interest!
Thank you Oleg!

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAPfivLBMQfM-twBM_u-DgQjf0bSqEdPoFW%3DAKmwVkX4_q2bj8w%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

marky.r...@gmail.com

unread,
Feb 28, 2019, 5:08:59 PM2/28/19
to Jenkins Developers
I have also signed on as a potential mentor


On Monday, June 18, 2018 at 12:59:23 AM UTC-7, Martin Holeček wrote:

Martin Holeček

unread,
Feb 28, 2019, 5:14:16 PM2/28/19
to Jenkins Developers
Guys, I love, that the idea received so much positive attitude, thak You all!

We could sync ourselves somehow of course (so far there are tons of conversations, arent there? First is the gitter one, then the discussions in the document and then this thread :) ).

Ioannis Moutsatsos

unread,
Feb 28, 2019, 5:23:14 PM2/28/19
to jenkin...@googlegroups.com
Yes, I also love all the attention the proposal is receiving! I would suggest we find a commonly agreeable time for a Google hangout to discuss and virtually meet with all the mentors. 

Timezones for each mentor should be entered in the proposal document. I'll be happy to organize a hangout then . I commonly organize these for the BioUno project 

Best regards 
Ioannis 

On Thu, Feb 28, 2019, 5:14 PM Martin Holeček <dart...@gmail.com> wrote:
Guys, I love, that the idea received so much positive attitude, thak You all!

We could sync ourselves somehow of course (so far there are tons of conversations, arent there? First is the gitter one, then the discussions in the document and then this thread :) ).

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/VrBoOWVsZ-o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
On Thu, Feb 28, 2019, 5:14 PM Martin Holeček <dart...@gmail.com> wrote:
Guys, I love, that the idea received so much positive attitude, thak You all!

We could sync ourselves somehow of course (so far there are tons of conversations, arent there? First is the gitter one, then the discussions in the document and then this thread :) ).

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/VrBoOWVsZ-o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/dba2db0f-27f4-46fc-8747-23ee311ceb3d%40googlegroups.com.

Marky Jackson

unread,
Feb 28, 2019, 5:26:44 PM2/28/19
to jenkin...@googlegroups.com
Agreed. 
I entered my time zone

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAOwNbFtq2D7mCCUYe5_aXgGSZrQNwMiYQwfs72fiX6vJHz9nYA%40mail.gmail.com.

Oleg Nenashev

unread,
Feb 28, 2019, 5:32:52 PM2/28/19
to JenkinsCI Developers
Taking the interest in this project, we could probably create a new dedicated Gitter channel for this project.
E.g. jenkinsci/gsoc-machine-learning-project or so. WDYT?


To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/171F7F6E-9A01-4889-9294-CB804F77EA13%40gmail.com.

Marky Jackson

unread,
Feb 28, 2019, 5:33:33 PM2/28/19
to jenkin...@googlegroups.com
I think that is a good idea

Martin Holeček

unread,
Feb 28, 2019, 5:34:30 PM2/28/19
to Jenkins Developers
Also thumb up!

Thx!

Ioannis Moutsatsos

unread,
Feb 28, 2019, 5:39:12 PM2/28/19
to Jenkins Developers
Definitely a good idea! Thanks Oleg!

Oleg Nenashev

unread,
Feb 28, 2019, 5:40:50 PM2/28/19
to JenkinsCI Developers

On Thu, Feb 28, 2019 at 11:39 PM Ioannis Moutsatsos <imout...@gmail.com> wrote:
Definitely a good idea! Thanks Oleg!

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/VrBoOWVsZ-o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/1c1046f0-a3c1-48ac-a231-e3bae7ac34d0%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages