Is there a way to capture the name of a task that has been executed in SnapLogic? - rest

We have a lot of triggered Tasks that run on the same pipelines, but with different parameters.
My question regarding this - is there a possible way, like a function or an expression to capture the name of the triggered task so that we could use the information while writing the reports and e-mails of which Task started the error pipeline. I can't find anything even close to it.
Thank you in advance.

This answer addresses the requirement of uniquely identify the invoker task in the invoked pipeline
For triggered tasks, there isn't anything provided out of the box by SnapLogic. Although, in case of ULTRA tasks you can get $['task_name'] from the input to the pipeline.
Out of the box, SnapLogic provides the following headers that can be captured and used in the pipeline being initiated by the triggered task. These are as follows.
PATH_INFO - The path elements after the Task part of the URL.
REMOTE_USER - The name of the user that invoked this request.
REMOTE_ADDR - The IP address of the host that invoked this request.
REQUEST_METHOD - The method used to invoke this request.
None of these contains the task-name.
In your case, as a workaround, to uniquely identify the invoker task in the invoked pipeline you could do one of the following three things.
Pass the task-name as a parameter
Add the task-name in the URL like https://elastic.snaplogic.com/.../task-name
Add a custom header from the REST call
All the three above methods can help you capture the task-name in the invoked pipeline.
In your case, I would suggest you go for a custom header because the parameters you pass in the pipeline could be task-specific and it is redundant to add the task-name again in the URL.
Following is how you can add a custom header in your triggered task.
From SnapLogic Docs -
Custom Headers
To pass a custom HTTP header, specify a header and its value through the parameter fields in Pipeline Settings. The
request matches any header with Pipeline arguments and passes those to
the Task, while the Authorization header is never passed to the
Pipeline.
Guidelines
The header must be capitalized in its entirety. Headers are case-sensitive.
Hyphens must be changed to underscores.
The HTTP custom headers override both the Task and Pipeline parameters, but the query string parameter has the highest precedence.
For example, if you want to pass a tenant ID (X-TENANT-ID) in a
header, add the parameter X_TENANT_ID and provide a default or leave
it blank. When you configure the expression, refer to the Pipeline
argument following standard convention: _X_TENANT_ID. In the HTTP
request, you add the header X-TENANT-ID: 123abc, which results in the
value 123abc being substituted for the Pipeline argument X_TENANT_ID.
Creating a task-name parameter in the pipeline settings
Using the task-name parameter in the pipeline
Calling the triggered task
Note: Hyphens must be changed to underscores.
References:
SnapLogic Docs - Triggered Tasks

I'm adding this as a separate answer because it addresses the specific requirement of logging an executed triggered task separate from the pipeline. This solution has to be a separate process (or pipeline) instead of being part of the triggered pipeline itself.
The Pipeline Monitoring API doesn't have any explicit log entry for the task name of a triggered task. invoker is what you have to use.
However, the main API used by SnapLogic to populate the Dashboard is more verbose. Following is a screenshot of how the response looks on Google Chrome Developer Tools.
You can use the invoker_name and pipe_invoker fields for identifying a triggered task.
Following is the API that is being used.
POST https://elastic.snaplogic.com/api/2/<org snode
id>/rest/pm/runtime
Body:
{
"state": "Completed,Stopped,Failed,Queued,Started,Prepared,Stopping,Failing,Suspending,Suspended,Resuming",
"offset": 0,
"limit": 25,
"include_subpipelines": false,
"sort": {
"create_time": -1
},
"start_ts": null,
"end_ts": null,
"last_hours": 1
}
You can have a pipeline that periodically fires this API then parses the response and populates a log table (or creates a log file).

Related

Trigger Azure Pipeline during PR with parameters

I have a pipeline in Azure DevOps which requires the parameter libName. The pipeline starts like this:
name: ${{ parameters.libName }}
trigger: none
pr: none
When I start the pipeline manually, a dialog box appears where I can provide the libName. So far so good.
However, I want to trigger this pipeline when I create a Pull Request. In that case I receive an error
A value for the 'libName' parameter must be provided.
Is there a way to provide this parameter when I create the PR?
It is currently not supported to pass parameter along with the Pull Request.
If you want to define a specific value for parameter, you must manually trigger this pipeline and put in the value.
For PR trigger, you are not allowed to define the parameter value automatically, try adding some default value for it.
To fix this error:
A value for the 'libName' parameter must be provided.
You could provide a default value for your parameter. For example:
parameters:
- name: libName
default: some library
Refer to this official doc for details: https://learn.microsoft.com/en-us/azure/devops/pipelines/process/runtime-parameters?view=azure-devops&tabs=script

Chain ADF pipe line with feeding output

I would like to chain 2 Azure Data Factory pipelines A -> B so that the pipeline B consumes the output of the Pipe line A.
So, basically A is a authorization pipeline that has few activities, the last of which produces the authentication token. Now that token I want use in the Pipeline B (and C, and D.. etc...). So, basically I want to reuse pipe line A.
I used Execute Pipleline activity but, its output is NOT what I expect .. How can use the actual output of A in B ?
Create Pipeline B
Click on Parameters
Add a parameter value ->inputfromA and give Default value as blank
Create a variable fromA and in the value,click on add dynamic content->#pipeline().parameters.inputfromA
Create Pipeline A
Store the value of authorization token in some variable
Add a execute pipeline and on InvokedPipeline->add pipelineB
Then it will ask to input the value of inputfromA ->click add dynamic content and then you can add the token value there.
So, I figured out how to solve this, for my case at least.
Since, I am dealing with (in the particular case) with the authentication token, I made use of the Azure Key Vault. Where, in the last activity of a pipe line A I POST the acquired token to the Azure key vault as a named secret.
In the pipeline B - what I do is I call 'execute Pipeline' activity - executing pipe line A. After that on completion, calling the web activity to read the token acquired in 'A'. And as a next activity in "B" I am using that token in API.

How to trigger azure pipeline via API in a way it does not report it was manually triggered

We have an Azure pipeline building a static site. When there is a change in a content repository the site needs to be rebuilt. For that, we're using webhooks and Azure DevOps API. The request to queue the build is very simple and is illustrated for example here.
What I don't like about this is that int the build listing it says "Manually triggered for person XY", where the person XY is the one who generated the credentials used in the API request. It seems quite confusing because any API request seems strange to be labeled as "manually requested". What would be the best way how to achieve more semantically correct message?
I've found there is a reason property which can be sent in the request. But none of the values seems to represent what I want and some of them do not work (probably need additional properties and there is no documentation for that).
Based on my test, when you use the Rest API to queue a build and set the build reason, the reason could be shown in the Build(except:batchedCI and buildCompletion).
Here is the Rest API example:
Post https://dev.azure.com/Organization/Project/_apis/build/builds?api-version=4.1
Request Body:
{
"definition": {
"id": 372
},
"reason":"pullRequest"
}
The value : checkInShelveset individualCI pullRequest schedule could show their own names.
The value: manual and none could show manually trigger.
The other value(e.g. All, userCreated) will show Other Build Reason.
For the value: batchedCI and buildCompletion.
BatchedCI: Continuous integration (CI) triggered by a Git push or a TFVC check-in, and the Batch changes was selected.
This means that batch changes are required to achieve this trigger. So it doesn't support to queue build in Rest API .
buildCompletion: you could refer to this ticket This reason doesn't support in Rest API-queue Build.
Note: If you enter a custom value or misspelling, it will always display manual trigger.
In the end, I went with all value and also overriding the person via requestedFor property. This leads to the message "Other build reason", which seems usable to me.
{
"definition": {
"id": 17
},
"reason":"all",
"requestedFor": {
"id": "4f9ff423-0e0d-4bfb-9f6b-e76d2e9cd3ae"
}
}
However, I'm not sure if there aren't any unwanted consequences of this "All reasons" value.

Is it possible to read the PR tag on a pipeline task?

My question is basically the title. I couldn't find such information reading this page and searching the web.
My scenario is the following: whenever I create a PR to master I add a tag with some information, like the lib version seen here:
Then, during the build process, I generate the release notes and would like to access that PR tag inside a task like I do with $(Build.BuildId) here:
How can I accomplish that? Something like $(PullRequest.Tag)
Thanks!
How can I accomplish that? Something like $(PullRequest.Tag) Thanks!
There's no predefined variable for Pull Request tag. (I use printenv command in CMD task to confirm this!)
Here're my workaround:
Use Powershell task to call this rest api, the response would contain the tag of specific PR
$url = "$($env:SYSTEM_TEAMFOUNDATIONSERVERURI)$env:SYSTEM_TEAMPROJECTID/_apis/git/repositories/$($env:BUILD_REPOSITORY_NAME)/pullRequests/$($env:SYSTEM_PULLREQUEST_PULLREQUESTID)/labels?api-version=5.1-preview.1"
$response = Invoke-RestMethod -Uri $url -Method Get -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "##vso[task.setvariable variable=PullRequestTag;isOutput=true]$($response.value.name)"
Then pass the variable $response.value.name(come from rest api response, the name represents the tag name) to output variable PullRequestTag(custom variable) so that next tasks can access the returned tag name.
Notes:
1.Make sure the job which contains the Powershell task allow scripts to access the OAuth token cause my script uses OAuth token instead of PAT to call rest api. (Click Agent Job Name=>Additional options)
2.Since it's a output variable, we should use format $(referenceName.variablename).
After that, we can use $(PS.PullRequestTag) in next tasks to access the tag name.
3.Since your scenario is in a pipeline run triggered by PR, so actually the Powershell task should only run when current pipeline is triggered by PR instead of manual run/CI.
Use and(succeeded(), eq(variables['Build.Reason'], 'PullRequest')) in PS task's control options. See conditions and Build.Reason variable.
Update:
If I added several tags when creating the PR:
The $(PS.PullRequestTag)'s value would be:
I doubt there's a predefined variable for a pull request tag, but you can achieve the goal with the REST API. For example, you can have a build task (e.g. a PowerShell script) which:
gets all tags (aka labels) of a pull request
parses the list of tags to find out the necessary one
set the build variable to the name of the necessary tag

Unable to flag / trigger "Merge when pipeline succeeds" via Gitlab Api (v3/v4)

So as a part of some tests to automatically accept / merge successful pipelines in our git repository i was running some tests to flag the "merge when pipeline succeeds" feature when the pipeline is still running:
So this button is available when the pipeline is still running and will convert to a green 'Accept merge' button when the pipeline succeeds:
(note that this picture was taken afterwards not to confuse the use-case)
additionally i have set these general settings:
So when checking the Gitlab API Documentation it says i should use the following endpoint:
PUT /projects/:id/merge_requests/:merge_request_iid/merge
when using the parameter ?merge_when_pipleline_succeeds=true it should flag the button.
However when i call this endpoint when the pipeline is still running (i built in a wait for 10 mins while testing this) i get the following result:
i am getting a Method Not Allowed. My assumption is that the endpoint i am using is correct because otherwise i would've gotten a bad request / not found return code.
when checking the gitlab merge request i am seeing that indeed the flag is not set to true:
However, when i manually click the blue button the mergerequest looks like this:
Also if i let the pipeline finish and then proceed to call the merge api (w/ or w/o the merge when pipeline succeeds flag) it will accept the merge. It just does not work when the pipeline is running (which is odd because even the button itself only shows when the pipeline is running)
so i am wondering what I am doing wrong here.
I am using a Powershell module to call the GitLab API. The Accept part of the module is not official and was made by myself because i found this feature missing.
I am using the same credentials for the API w/ a personal access token to authenticate to the API. Other functionality of the API work with this token like creating merge requests, retrieving status of a current MR and accepting a MR when the pipeline is finished.
I have tried the following variants :
Use the V3 api with merge_when_build_succeeds=true --> nets the same
result
Uncheck the "Only allow merge request to be merged if the
pipeline succeeds" --> nets the same result
Use ID of the merge request instead of IID
use /merge_when_pipeline_succeeds instead of ?merge_when_pipeline_succeeds=true
use True instead of true --> nets the same result
I get a similar issue with the python-gitlab library on v4. It works sometimes when I use:
mr.merge(merge_when_pipeline_succeeds=True)
Where mr is a ProjectMergeRequest object. However, if the MR has a merge conflict in it I get that 405 Method Not Allowed error back.
My best guess is to see if I can apply logic before calling mr.merge() to check for problems. Will update this if that works.
UPDATE: Looks like there is no feature to check for conflicts as of today. https://gitlab.com/gitlab-org/gitlab-ce/issues/41762
UPDATE 2: You can check merge_status when looking at the MR information, so either that attribute or an exception then mr.merge() fails would let you identify when it won't work.