I'm trying to use the email-ext plugin in a Jenkins pipeline job, but most of the tokens I take for granted when using email-ext in a freestyle job aren't being expanded?
e.g.
node {
emailext body: '${DEFAULT_CONTENT}', mimeType: 'text/html', replyTo: '$DEFAULT_REPLYTO', subject: '${DEFAULT_SUBJECT}', to: '$DEFAULT_RECIPIENTS'
}
Generates the email:
Subject:
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!
Body:
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS: Check console output at $BUILD_URL to view the results.
Which are, in fairness, what is set in the general configuration. However, none of those tokens are expanded, and if I use them directly from workflow, they're not expanded either, e.g.
emailext body: '$PROJECT_NAME', mimeType: 'text/html', replyTo: '$DEFAULT_REPLYTO', subject: '${DEFAULT_SUBJECT}', to: '$DEFAULT_RECIPIENTS'
This would be 'okay' if I was able to reference my existing Groovy-based email templates using $SCRIPT, but those aren't expanded either.
Apologies if I'm missing something incredibly obvious - I don't have much experience with Pipeline yet - the lack of email-ext support has stopped me from moving across thus far.
I tested today by clean installing Jenkins 1.651, email-ext 2.41.3 and Pipeline 1.14.
The tokens are highly tied to AbstractBuild, which workflow doesn't use. So, the tokens won't be replaced correctly right now. I need to modify the token-macro plugin (which is the basis for the token expansion) so that it will work with a Run object instead of just AbstractBuild, but this is a large task that I haven't had the change to do yet. You could generate the same content using the workflow DSL.
Related
I would like to only build my static web app and not deploy it. I saw there is a env setting "skip_deploy_on_missing_secrets' but after setting that in the pipeline it just gets ignored and the pipeline fails with error saying the deployment token is not set. How exactly should I use this env setting? Does it actually work?
There's not much info on the internet about this parameter. However, at least Dapr docs suggest that it should work, and I doubt they'd put it in their docs if it didn't (here).
However, I had problems getting it working as well.
One thing to notice there is that Dapr docs actually show a GitHub Action, and they work a little bit differently than Azure CICD YAML Pipelines, which I was using.
Finally I stumbled upon this comment on a similar issue on GitHub which hints that this magic undocumented parameter should be passed as an environment variable. I was passing it as an input. Maybe GitHubActions forward these params to envs automatically?
So I tried setting it as ENV and it worked!
- task: AzureStaticWebApp#0
inputs:
app_location: ...blahblahblah
....
#skip_deploy_on_missing_secrets: true
# ABOVE: this one is documented in few places, but it's expected to be a ENV var!
#see https://github.com/Azure/static-web-apps/issues/679
env:
SKIP_DEPLOY_ON_MISSING_SECRETS: true
This is my first question posted on stackoverflow, hence in case I did something incorrectly pleaselet me know.
Description
I am currently working on translation of freestyle projects to declarative pipelines in Jenkins (jenkinsfiles kept in Git repo). The original freestyle job was triggering PowerShell script which needed access to Global name/password pairs defined in Mask Passwords plugin section in Configure System. Solution to this problem was an additional tick in project itself (unfortunately I am not allowed adding screenshots to posts yet, hence editor uploaded screen to imgur and pasted link - please see Screenshot 1):
Screenshot 1
Therefore I started looking for possible implementation of such solution to jenkinsfile, however wothout luck.
My problem
When the script is triggered from the pipeline, it errors out stating that it cannot find relevant passwords (powershell refers to those credentials as to environment variables). This works fine when ran from freestyle project.
Which I reckon is caused by pipeline not being able to reach out to previously mentioned credentials.
What I tried
Wrapping the step into below block:
wrap([$class: 'MaskPasswordsBuildWrapper']) {
bat(batch file launching ps script)
}
Then the above block containing relevant step wrapping into
script {
wrap(...)
}
But none of them worked.
I have taken a look at other plugins like Credentials Binding Plugin or Credentials Plugin but those allow to bind/pass one credential per step, and I need to pass all credentials specified in Jenkins (I am open to move saved credentials to any other location within Jenkins).
I have looked at adding environment variable:
credentials('Credentials-ID')
But the problem is the same as with mentioned plugins.
By any chance, have anyone came across similar situation and know what can be done in order to allow pipeline to access/pass to pipeline all the credentials specified in Jenkins instead of binding/passing them one a time?
All tips are very welcome!
You can do this and the env variable will then be available throughout your job. You could define multiple env variable too.
environment {
// Use credentials() to hide the environment variable's output
MY_PERSONAL_TOKEN = credentials('Credentials-ID')
}
stages {
stage('Test Stage') {
steps {
script {
// do what you need to
}
}
}
}
We have a number of tasks that we carry out every time we create a new GIT repository in our project, and I would like to know if there's a way to script (PowerShell or any other method) these out. for e.g. every these are the steps we follow everytime we create a new repo
Create a new GIT repo
Create a Build pipeline for Build validations during
pull request
Add branch policies to Master including a step to validate build using the above build
Create a Build pipeline for releases
Create a Release pipeline
Is there a way to script repetitive tasks in Azure DevOps?
Of course yes! As Daniel said in comment, just use REST API can achieve these all. But since the steps you want to achieve are little much, the script might be little complex.
Create a new GIT repo
If you also want to use API to finish this step, it needs 3 steps to finish that( Since this does not be documented in doc, I will described it very detailed ):
Step1: Create the validation of importing repository
POST https://dev.azure.com/{org name}/{project name}/_apis/git/import/ImportRepositoryValidations?api-version=5.2-preview.1
Request body:
{
"gitSource":
{
"url":"${ReposURL}",
"overwrite":false
},
"tfvcSource":null,
"username":"$(username}"/null,
"password":"${pw}"/"${PAT}"/null
}
Step2: Create the new repos name
POST https://dev.azure.com/{org name}/{project name}/_apis/git/Repositories?api-version=5.2-preview.1
Request body:
{
"name":"${ReposName}",
"project":
{
"name":"{project name}",
"id":"{this project id}"
}
}
Step3: Import repos
POST https://dev.azure.com/{org name}/{project name}/_apis/git/repositories/{the new repos name you create just now}/importRequests?api-version=5.2-preview.1
Request body:
{
"parameters":
{
"deleteServiceEndpointAfterImportIsDone":true,
"gitSource":
{
"url":"${ReposURL}",
"overwrite":false
},
"tfvcSource":null,
"serviceEndpointId":null
}
}
In these script, you can set variables in Variable tab, then use ${} to get them in the script.
Create a Build pipeline for Build validations during pull request
This step you'd better finish manually, because you can configure more about tasks and trigger with UI. If still want use API, refer to this doc: create build definition. There has detailed sample you can try with.
Add branch policies to Master including a step to validate build using the above build
This API still be documented in doc: create build policy. Just refer to that, and ensure use the correct policy type and the corresponding buildDefinitionId.
Create a Build pipeline for releases
This still recommend you finish manually, same with the step3 you mentioned.
Create a Release pipeline
See this doc: create release.
Note: For some parameter which will be used many times, you can set it as variable. For the parameter which need get from previous API response, you can define a variable to get its value then pass this variable into the next API to use.For e.g. :
$resultT= $result.Headers.ETag
Write-Host "##vso[task.setvariable variable=etag;]$resultT"
Now, you can directly use the $(etag) in the next API.
I've got a Concourse job that uses the appearance of a file in an Amazon S3 bucket as a trigger to a suite of tests. Using this resource --> https://github.com/concourse/s3-resource . Problem is, the job is not firing when the file appears. When I trigger the job manually, it does see the file and start the test suite.
Yaml config looks like this:
- name: s3-trigger-file
type: s3
source:
bucket: my-bucket-name
regexp: qabot_request_(.*).json
access_key_id: {{s3-access-key-id}}
secret_access_key: {{s3-secret-access-key}}
jobs:
- name: my-job
public: true
plan:
- get: s3-trigger-file
trigger: true
When I click on the trigger itself in the Concourse UI, I see what looks like a running monitor:
As I said, the job isn't firing when the file appears, but a manual trigger does verify the S3 input is found.
How can I debug why the automatic trigger isn't firing? Also, how much latency is expected for the s3 resource to detect a new file has appeared?
Concourse 3.4. Thanks ~~
The capturing group in your regexp must refer to a semver compliant version.
See the documentation:
The version extracted from this pattern is used to version the resource. Semantic versions, or just numbers, are supported. Accordingly, full regular expressions are supported, to specify the capture groups.
Your capturing group is currently making the captured "version" quote2. You should probably delete the pipeline and regenerate it with a modified regex (e.g. qabot_request_quote(\d+).json)
How is it possible to use Metadata in on_success/on_failure? For example, to send emails via https://github.com/pivotal-cf/email-resource?
I haven't found a way, as I can't change content of files where email resources reside (subject/body), as the metadata is not available to tasks.
And yep, that might be a duplicate for Concourse CI and Build number
But still my question IMHO is a valid use case for notifications.
The metadata you are referring to, I assume, is the environment variables provided to resources, not tasks.
This can be used with the slack resource to provide information about what build failed.
For example:
on_failure:
put: slack-alert
params:
text: |
The `science` pipeline has failed. Please resolve any issues and ensure the pipeline lock was released. Check it out at:
$ATC_EXTERNAL_URL/teams/$BUILD_TEAM_NAME/pipelines/$BUILD_PIPELINE_NAME/jobs/$BUILD_JOB_NAME/builds/$BUILD_NAME
The email resource, you're referencing has an open PR to support these environment variables. I'd discuss your need for that feature there.