Could someone please help me understand using API?

Gabriel Callaghan

unread,

Feb 22, 2021, 7:03:00 PM2/22/21

to go-cd

Hi,

I am running into some different results than what I wanted. My goal is to use jq to filter results of the "get stage history" (https://api.gocd.org/current/#get-stage-history) so that I can get the most recent run and to check it for if it has been passed or not.

At the moment, I am using that API with v2+ on the third line as it provides results. If I change it to v3+, it gets "<html><body><h2>404 Not found</h2></body></html>". This is why I have been using it as v2+. However, when using v2+ I am getting back weird results. It does not seem to be in order from most recent results to the oldest, it is being filtered somehow? It means that as a result, my script is trying to run a pipeline that is an older version, when it should be checking and running the most recent version of a pipeline.

Why is it doing that, and what can I do to make sure that the stage history json provides results from the most recent to the latest?

If it helps, my current script is this:

#! /bin/bash

fetchPipelineHistory=$( curl "{COMPANY NAME}/go/api/stages/Games-AcesHigh-PHY-Test/Approve/history" \

-H "Authorization: Bearer {MY TOKEN)" \

-H 'Accept: application/vnd.go.cd.v2+json')

getResultOfCurrentStage=$( echo $fetchPipelineHistory | jq -r '.stages[].result')

counter=$( echo $fetchPipelineHistory | jq -r '.stages[0].counter')

if [ "$getResultOfCurrentStage" != "Passed" ]

then

echo "{COMPANY NAME}/api/stages/Games-AcesHigh-PHY-Test/$counter/Approve/run"

curl "{COMPANY NAME}/go/api/stages/Games-AcesHigh-PHY-Test/$counter/Approve/run" \

-H "Authorization: Bearer {MY TOKEN}" \

-H 'X-GoCD-Confirm: true' \

-H 'Accept: application/vnd.go.cd.v2+json' \

-X POST

fi

Thank you for reading,
Gabes.

Gabriel Callaghan

unread,

Feb 22, 2021, 9:06:56 PM2/22/21

to go-cd

I just saw that our GOCD is 20.1.0 so it wouldn't be able to use v3 of this api, as it requires v20.9.0. So that answers my question for why v3 doesn't work. However in GOCD api list, it only shows v3+, so i cant check for what it looks like when using v2. Would anyone know how to get the most recent history rather than filtered? I don't want to be only shown passed results, I want to see failed ones as well.

Marques Lee

unread,

Feb 23, 2021, 12:05:12 AM2/23/21

to go...@googlegroups.com

What version of GoCD are you using? If your version is old, v3 may not be available. You can always use the `latest` version by using `-H 'Accept: application/vnd.go.cd+json`. Adding `-v` to `curl` will also print response header which will indicate the version `latest` resolves to, which might be `v2` in your case.

I'm running 21.1.0. v3 was first available since 20.9.0 according to api.gocd.org.

# works for me
curl -v \
-H 'Accept: application/vnd.go.cd+json' \
-H "Authorization: Bearer $token" \
"http://${service_host_port}/go/api/stages/$pipeline/$stage/history"

As far as the history goes, it should be in the correct order. Perhaps it's because you aren't filtering out current running stages (i.e., stages with "Unknown" as the result)?

Try this:

# Get the whole JSON result into a variable of the latest stage that finished
# NOTE: stages that are scheduled/running will have result == "Unknown", so we
# filter those out with `jq`
last_stage_result="$(curl -v \
-H 'Accept: application/vnd.go.cd+json' \
-H "Authorization: Bearer $token"
"https://$host_and_port/go/api/stages/$pipeline/$stage/history" | \
jq '[.stages[] | select (.result != "Unknown")][0]')"

# extract the result state from json blob
result_state="$(echo "$last_stage_result" | jq -r .result)"

if [ "$result_state" != "Passed" ]; then
# extract the counter of that same result
counter="$(echo $last_stage_result | jq -r .counter)"

# do the rest of your code here
fi

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/bcb78dac-5cdb-4216-ad87-a9e56291ef4an%40googlegroups.com.

Marques Lee

unread,

Feb 23, 2021, 12:06:54 AM2/23/21

to go...@googlegroups.com

Yes, so if you're on say, 20.7.0, just add that to your url: https://api.gocd.org/20.7.0/

It looks pretty similar.

-Marques

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/5cb8a558-ed85-4021-b4ed-564d1d11f460n%40googlegroups.com.

Message has been deleted

Marques Lee

unread,

Feb 23, 2021, 4:15:14 AM2/23/21

to go...@googlegroups.com

oh also, in the code snippet I sent a couple of messages ago, you may want to null check the json blob in case there aren't any matches, for good measure.

that is, do:

if [ "null" = "$last_stage_result" ]; then

echo "No pipelines have completed yet" >&2

exit 1

fi

Gabriel Callaghan

unread,

Feb 23, 2021, 4:33:54 AM2/23/21

to go-cd

hi just checking, did you delete the message I wrote recently? showing up as message has been deleted, not sure if I deleted by accident.

Marques Lee

unread,

Feb 23, 2021, 5:12:16 AM2/23/21

to go...@googlegroups.com

Hi, no I haven’t seen any message actually. Not sure what happened?

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/c8ca81b4-c68f-43b3-ba49-f505b66ed0d5n%40googlegroups.com.

Gabriel Callaghan

unread,

Feb 23, 2021, 5:14:45 AM2/23/21

to go-cd

ok i think i have the wrong approach to this problem..

so when im doing stage history, it's not in order. because if an earlier instance is run, that goes to the top of the stage history, even tho I only want to approve the most recent instance. also stage history doesn't show the stage history of an instance that hasn't had a stage run yet, so it won't show up to run.

so I need to rethink the soloution to this problem.

Gabriel Callaghan

unread,

Feb 23, 2021, 5:16:00 AM2/23/21

to go-cd

yeah I posted some stuff, which seems to not have posted correctly which was odd. basically i had a few json objects that didnt match what I expected.. but figured that out, typed the answer in my last message.

Gabriel Callaghan

unread,

Feb 23, 2021, 5:18:53 AM2/23/21

to go-cd

sorry, i mean if an older instance is run, it goes to the top of stage history. which isnt helpful when I need to approve the latest instance.

Gabriel Callaghan

unread,

Feb 23, 2021, 5:29:09 AM2/23/21

to go-cd

im a moron lol.

so ive been trying to get info from stage history, when I should be getting from pipeline history. if i use pipeline history, I can get the latest pipeline instance, so just need to approve that. i believe this is the solution.. just gotta think this thru and should be able to finish this hopefully before 1am lol

Marques Lee

unread,

Feb 27, 2021, 1:08:55 AM2/27/21

to go...@googlegroups.com

Hey there, sorry for the delay. I missed this email notification. What version of GoCD are you using? I'd like to take a look at the ordering of the results for pipeline history.

-Marques

On Tue, Feb 23, 2021 at 7:01 AM Gabriel Callaghan <gab...@rubyplaynetwork.com> wrote:

So the reason why I've been confused over this bit is because when I run your command to get stage history, I get a result which look like this:
{
"_links" : {
"previous" : {
"href" : "https://{COMPANY NAME}/go/api/stages/Games-AcesHigh-PHY-Test/Approve/history?before=10780"
}
},
"stages" : [ {
"name" : "Approve",
"counter" : "1",
"approval_type" : "manual",
"approved_by" : "test1",
"result" : "Passed",
"rerun_of_counter" : null,
"pipeline_name" : "Games-AcesHigh-PHY-Test",
"pipeline_counter" : 20,
"jobs" : [ {
"name" : "Approve",
"state" : "Completed",
"result" : "Passed",
"scheduled_date" : 1611717690867
} ]
}, {
"name" : "Approve",
"counter" : "1",
"approval_type" : "manual",
"approved_by" : "test1",
"result" : "Passed",
"rerun_of_counter" : null,
"pipeline_name" : "Games-AcesHigh-PHY-Test",
"pipeline_counter" : 18,
"jobs" : [ {
"name" : "Approve",
"state" : "Completed",
"result" : "Passed",
"scheduled_date" : 1605826768229
} ]
}, {
"name" : "Approve",
"counter" : "1",
"approval_type" : "manual",
"approved_by" : "test2",
"result" : "Passed",
"rerun_of_counter" : null,
"pipeline_name" : "Games-AcesHigh-PHY-Test",
"pipeline_counter" : 5,
"jobs" : [ {
"name" : "Approve",
"state" : "Completed",
"result" : "Passed",
"scheduled_date" : 1601954345153
} ]
} ]

The above JSON are not the most recent results.

However if I were to use this bit of code (with page size set to 100 as it seems to be the only way to get the more recent results):

curl "https://{COMPANY NAME}/go/api/stages/Games-AcesHigh-PHY-Test/Approve/history?page_size=100" \
-H "Authorization: Bearer $TOKEN" \
-H 'Accept: application/vnd.go.cd.v2+json'

I then get (this is just a brief snippet):

{
"_links" : {
"previous" : {
"href" : "https://{COMPANY NAME}/go/api/stages/Games-AcesHigh-PHY-Test/Approve/history?before=11468"
}
},
"stages" : [ {
"name" : "Approve",
"counter" : "1",
"approval_type" : "manual",
"approved_by" : "gabe",
"result" : "Passed",
"rerun_of_counter" : null,
"pipeline_name" : "Games-AcesHigh-PHY-Test",
"pipeline_counter" : 7,
"jobs" : [ {
"name" : "Approve",
"state" : "Completed",
"result" : "Passed",
"scheduled_date" : 1614046479825
} ]
}, {
"name" : "Approve",
"counter" : "13",
"approval_type" : "manual",
"approved_by" : "gabe",
"result" : "Failed",
"rerun_of_counter" : null,
"pipeline_name" : "Games-AcesHigh-PHY-Test",
"pipeline_counter" : 1,
"jobs" : [ {
"name" : "Approve",
"state" : "Completed",
"result" : "Failed",
"scheduled_date" : 1614037130394
} ]

Neither results matches the stage history shown in GOCD (after clicking on job details of the stage I am working on, at the right side, the numbers do not match). The pipeline counter is 40+ and the run numbers are either 2, 3 or 4, which doesn't match the counter in the stage history which shows 1 at the most recent instance, the next being 13, 12 and so on. I've attatched an image of the stage history. The part that I've circled, is where I believe the bottom JSON in the example above matches (since the counter is 13 in both JSON and the image).

Am I doing something wrong to get a different stage history from what's shown in GOCD?

Again, thank you for your patience in assisting me :)

Cheers,
Gabes.

To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/3b8debc9-2efb-4b57-8d74-a78a6378212an%40googlegroups.com.

Aravind SV

unread,

Feb 27, 2021, 2:00:38 AM2/27/21

to Marques Lee, go...@googlegroups.com

Marques: I think this post came through later, after being stuck in a spam check queue. I don’t think this part of the question is valid any more.

Marques Lee

unread,

Feb 27, 2021, 2:05:13 AM2/27/21

to Aravind SV, go...@googlegroups.com

Ah ok, thanks for letting me know :)

Reply all

Reply to author

Forward