OK,I know what you means. But I test these pipeline that data is not synchronized many times( about more than 700 times,I use periodical build to test it automatically). Shared data that is not synchronized is processed correctly every time. Do you really encounter the situation that shared data is corrupt in parallel task?
In some scenarios, I want to parallel build one more artifacts that used in different platforms from the same source code. And then I parallel test all artifacts in their corresponding platforms(One artifact might be used in one more platforms).
build platform: A B C
test platform: A(A1 A2 A3), B(B1 B2), C(C1 C2)
-> A1
-> A -> A2
| -> A3
/
source code --> B -> B1
\ -> B2
|
-> C -> C1
-> C2
-> C3
Pipeline code like this:
pipeline {
agent any
stages {
stage('Parallel Build') {
steps {
script {
def builds = [:]
def tests = [:]
stash name: 'src', include: 'src/**'
def build_action = { platform, tests ->
//build in some slave node.
node ("${platform}") {
//unstash need thread mutex lock?
unstash name: 'src'
sh "make ${platform}"
//archive need thread mutex lock?
archive name: "./${platform}"
}
//nested parallel, outside of node block
parallel tests
}
def test_action = { platform ->
//test in some slave node.
node ("${platform}") {
sh "make ${platform}-test"
}
}
tests["A1"] = test_action("A1")
tests["A2"] = test_action("A2")
tests["A3"] = test_action("A3")
builds["A"] = build_action("A",tests)
tests = [:]
tests["B1"] = test_action("B1")
tests["B2"] = test_action("B2")
builds["B"] = build_action("B",tests)
tests = [:]
tests["C1"] = test_action("C1")
tests["C2"] = test_action("C2")
builds["C"] = build_action("B",tests)
parallel builds
}
}
}
}
}
I have been running this pipeline for some time. It works well. But I worry about the thread safety of unstash(and archive,or other commands). I want to find some evidence that Jenkins garantee these parallel thread safety and nested parallel is ok. Otherwise I must split these platforms into different build jobs(which I think it's more difficult to manage projects.) Because Jenkins must garantee parallel build jobs more safe? I just don't know the underlying works that Jenkins do about parallel.I'm just curious about these thread safety.If Jenkins take care of these synchronization,that is the best. If such, I don't need to care about these thread things. If not,I want to find correct ways to do parallel tasks.
------------------ 原始邮件 ------------------
发送时间: 2021年10月31日(星期天) 晚上7:19
主题: Re: thread safety of scripted pipeline parallel and usage of nested parallel