I want to download a TeamCity artifact via powershell. It needs to be the last successful build of a specific branch.
I've noticed two common url paths to access the artifacts. One seems to be
/repository/download/BUILD_TYPE_EXT_ID/.lastSuccessful/ARTIFACT_PATH
The problem is that the file at the end relies on the release version. Within TeamCity there is syntax to specify all files \*.msi. Is there any way to specify an artifact starting with FileName-{version.number}.msi when trying to access this url?
EDIT:
The other url I noticed is for the REST API.
http://teamcity/guestAuth/app/rest/builds/branch:[BRANCH],buildType:[BUILD TYPE],status:SUCCESS,state:finished/artifacts/[BUILD PATH]
The problem is that I can't download artifacts from here. If I want to download the artifacts I have to use the current build id. The above url gives the following url: /guestAuth/app/rest/builds/id:[Build ID]/artifacts/content/[Artifact Path] to download the artifact.
I can use the first REST url to eventually get the second through the returned xml, but would prefer a more straightforward approach.
Unfortunately as TeamCity artifacts are not browsable the usual workarounds like wget recursive download or wildcards are not applicable.
Using wildcards in wget or curl query
How do I use Wget to download all Images into a single Folder
One workaround you could try is formatting the link in the job, saving the link to a text file and storing that as an artifact as well, with a static name. Then you just need to download that text file to get the link.
I found you can format the artifact URL in TeamCity job by doing:
%teamcity.serverUrl%/repository/download/%system.teamcity.buildType.id%/%teamcity.build.id%:id/<path_to_artifact>
In a command line step. You can write this to a file by doing:
echo %teamcity.serverUrl%/repository/download/%system.teamcity.buildType.id%/%teamcity.build.id%:id/myMsi-1.2.3.4.msi > msiLink.txt"
Now you have an artifact with a constant name, that points to the installer (or other artifact) with the changing name.
If you use the artifact msiLink.txt you don't need to use the REST interface (it's still two calls, both through the same interface).
You can easy download the latest version from batch/cmd by using:
wget <url_server>/repository/download/BUILD_TYPE_EXT_ID/.lastSuccessful/msiLink.txt ---user #### --passsword ####
set /P msi_url=<msiLink.txt
wget %msi_url% --user #### --passsword ####
Hope it helps.
Update:
Sorry I just realized the question asked for PowerShell:
$WebClient = New-Object System.Net.WebClient
$WebClient.Credentials = New-Object System.Net.Networkcredential("yourUser", "yourPassword")
$WebClient.DownloadFile( "<url_server>/repository/download/BUILD_TYPE_EXT_ID/.lastSuccessful/msiLink.txt", "msiLink.txt" )
$msi_link = [IO.File]::ReadAllText(".\msiLink.txt")
$WebClient.DownloadFile( $msi_link, "yourPath.msi" )
Related
I have been testing the Jenkins "Poll Mail Trigger" plugin to trigger a project via email. To develop the project I need to download the attached file that is sent.
In the project settings, I enable the only download option that exists: "Download to a timestamped Directory".
Also, the only information regarding the download is this: "The Attachments field, allows you to determine what to do with email attachments. If attachments are downloaded, the pmt_attachmentsDirectory variable with be set to the download directory."
Then, after doing several tests without configuring any directory for the download of the attached file -assuming that the download should have been automatic-, I proceeded to change the environment variables in the Jenkins configuration, I put the following:
Name: pmt_attachmentsDirectory
Value: *D: *
So, what I'm trying to do is save the file to my Drive D: \
The file is never downloaded to the specified disk, and just for verification, I am trying to move the file with the following Windows command:
move "% pmt_attachmentsDirectory%" "D: \ Users \ wrodriguez \ Downloads \ Test_Programs"
And finally it throws me the following output: "The system cannot find the specified file."
So, I am really lost here, can anybody help me with this issue? Is there another plugin on Jenkins that I can use to download an attatchment?
Thanks for your help.
I haven't worked on that plugin (or Jenkins) in 5 years, so forgive my rustiness.
Looking at the code, the plugin creates a timestamped folder, and saves any attachments there.
It then injects an environment variable named pmt_attachmentsDirectory into the Jenkins job instance's environment variables - the value is the absolute path of folder (it's not configurable).
Troubleshooting:
have you enabled saving Attachments in the Plugin options?
check the job instance's environment variables - does it have pmt_attachmentsDirectory listed?
print the value of pmt_attachmentsDirectory - e.g. echo "%pmt_attachmentsDirectory%" - what does it say?
does the directory exist?
does the directory contain any files?
does the "View Polling Log" have any errors?
do the Jenkins logs have any errors?
Other notes:
"% pmt_attachmentsDirectory%" - shouldn't have a space e.g. should be "%pmt_attachmentsDirectory%"
Is there an option to handle the next situation:
I have a pipeline and Copy Files task implemented in it, it is used to upload some static html file from git to blob. Everything works perfect. But sometimes I need this file to be changed in the blob storage (using hosted application tools). So, the question is: can I "detect" if my git file is older than target blob file and ignore this file for the copy task to leave it untouched. My initial idea was to use Azure file copy and use an "Optional Arguments" textbox. However, I couldn't find required option in the documentation. Does it allow such things? Or should this case be handled some other way?
I think you're looking for the isSourceNewer value for the --overwrite option.
--overwrite string Overwrite the conflicting files and blobs at the destination if this flag is set to true. (default true) Possible values include true, false, prompt, and ifSourceNewer.
More info: azcopy copy - Options
Agree with ickvdbosch. The isSourceNewer value for the --overwrite option could meet your requirements.
error: couldn't parse "ifSourceNewer" into a "OverwriteOption"
Based on my test, I could reproduce this issue in Azure file copy task.
It seems that the isSourceNewer value couldn't be set to Overwrite option in Azure File copy task.
Workaround: you could use PowerShell task to run the azcopy script to upload the files with --overwrite=ifSourceNewer
For example:
azcopy copy "filepath" "BlobURLwithSASToken" --overwrite=ifSourceNewer --recursive
For more detailed info, you could refer to this doc.
For the issue about the Azure File copy task, I suggest that you could submit a feedback ticket in the following link: Report task issues.
I'm on a project that uses TeamCity for builds.
I have a VM, and have written a PowerShell script that backs up a few files, opens a ZIP artifact that I manually download from TeamCity, and then copies it to my VM.
I'd like to enhance my script by having it retrieve the ZIP artifact (which always has the same name).
The problem is that the download path contains the build number which is always changing. Aside from requesting the download path for the ZIP artifact, I don't really care what it is.
An example artifact path might be:
http://{server}/repository/download/{project}/{build_number}:id/{project}.zip
There is a "Last Successful Build" page in TeamCity that I might be able to obtain the build number from.
What do you think the best way to approach this issue is?
I'm new to TeamCity, but it could also be that the answer is "TeamCity does this - you don't need a PowerShell script." So direction in that regard would be helpful.
At the moment, my PowerShell script does the trick and only takes about 30 seconds to run (which is much faster than my peers that do all of the file copying manually). I'd be happy with just automating the ZIP download so I can "fire and forget" my script and end up with an updated VM.
Seems like the smallest knowledge gap to fill and retrieving changing path info at run-time with PowerShell seems like a pretty decent skill to have.
I might just use C# within PS to collect this info, but I was hoping for a more PS way to do it.
Thanks in advance for your thoughts and advice!
Update: It turns out some other teams had been using Octopus Deploy (https://octopus.com/) for this sort of thing so I'm using that for now - though it actually seems more cumbersome than the PS solution overall since it involves logging into the Octopus server and going through a few steps to kick off a new build manually at this point.
I'm also waiting for the TC administrator to provide a Webhook or something to notify Octopus when a new build is available. Once I have that, the Octopus admin says we should be able to get the deployments to happen automagically.
On the bright side, I do have the build process integrated with Microsoft Teams via a webhook plugin that was available for Octopus. Also, the Developer of Octopus is looking at making a Microsoft Teams connector to simplify this. It's nice to get a notification that the new build is available right in my team chat.
You can try to get your artefact from this url:
http://<ServerUrl>/repository/downloadAll/<BuildId>/.lastSuccessful
Where BuildId is the unique identifier of the build configuration.
My implementation of this question is, in powershell:
#
# GetArtefact.ps1
#
Param(
[Parameter(Mandatory=$false)][string]$TeamcityServer="",
[Parameter(Mandatory=$false)][string]$BuildConfigurationId="",
[Parameter(Mandatory=$false)][string]$LocalPathToSave=""
)
Begin
{
$username = "guest";
$password = "guest";
function Execute-HTTPGetCommand() {
param(
[string] $target = $null
)
$request = [System.Net.WebRequest]::Create($target)
$request.PreAuthenticate = $true
$request.Method = "GET"
$request.Headers.Add("AUTHORIZATION", "Basic");
$request.Accept = "*"
$request.Credentials = New-Object System.Net.NetworkCredential($username, $password)
$response = $request.GetResponse()
$sr = [Io.StreamReader]($response.GetResponseStream())
$file = $sr.ReadToEnd()
return $file;
}
Execute-HTTPGetCommand http://$TeamcityServer/repository/downloadAll/$BuildConfigurationId/.lastSuccessful | Out-File $LocalPathToSave
}
And call this with the appropriate parameters.
EDIT: Note that the current credential I used here was the guest account. You should check if the guest account has the permissions to do this, or specify the appropriate account.
Try constructing the URL to download build artifact using TeamCity REST API.
You can get a permanent link using a wide range of criteria like last successful build or last tagged with a specific tag, etc.
e.g. to get last successful you can use something like:
http://{server}/app/rest/builds/buildType:(id:{build.conf.id}),status:SUCCESS/artifacts/content/{file.name}
TeamCity has the capability to publish its artifacts to a built in NuGet feed. You can then use NuGet to install the created package, not caring about where the artifacts are. Once you do that, you can install with nuget.exe by pointing your source to the NuGet feed URL. Read about how to configure the feed at https://confluence.jetbrains.com/display/TCD10/NuGet.
Read the file content of the path in TEAMCITY_BUILD_PROPERTIES_FILE environment variable.
Locate the teamcity.configuration.properties.file row in the file, iirc the value is backslash encoded.
Read THAT file, and locate the teamcity.serverUrl value, decode it.
Construct the url like this:
{serverurl}/httpAuth/repository/download/{buildtypeid}/.lastSuccessful/file.txt
Here's an example (C#):
https://github.com/WideOrbit/buildtools/blob/master/RunTests.csx#L272
In trying to download a file with PowerShell I have the following
$client = new-object System.Net.WebClient
$client.DownloadFile($AGRIDATAMISCURL,$TESTAGRIDATAMISCZIP)
Where $AGRIDATAMISCURL is a URL that looks like "https://drive.google.com/file/d/<...>" and $TESTAGRIDATAMISCZIP looks like "C:\test\A.zip"
This script doesn't return an error but the file it downloads is basically an HTML file with a prompt to sign in to Google. Is there another way to download a file that is "shared with me"?
Share the file first
Files in Google Drive must be made available for sharing before they can be downloaded. There's no security context when running from PowerShell, so the file download fails. (To check this, rename the file with a `.html` extension, and view in a text editor).
Note: the following solution assumes that the links are to non-security-critical files, or that the links will only be given to those with whom access can be trusted (links are https, so are encrypted with transmission). The alternative is to programatically authenticate with Google - something not addressed in this answer.
To Share the file, in Google Drive:
Right-click the file, and choose Get shareable link
2. Turn link sharing on
Click Sharing Settings
Ensure that Anyone with the link can view (Note that in corporate environments, the link must be shared with those outside the organization in order to bypass having to login)
Then Download Programatically
Then, code can be used to download the file as such (in this case with Windows PowerShell):
# Download the file
$zipFile = "https://drive.google.com/uc?export=download&id=1cwwPzYjIzzzzzzzzzzzzzzzzzzzzzzzz"
Invoke-WebRequest -Uri $zipFile -OutFile "$($env:TEMP)\myFile.doc"
Replace the 1cwwPzYjIzzzzzzzzzzzzzzzzzzzzzzzz with the ID code from the shareable link setting back in step #2, above.
I'm using Perl's File::Fetch to download a file from the lastfinished build in Teamcity. This is working fine except the file is versioned, but I'm not getting the version number.
sub GetTeamcityFiles {
my $latest_version = "C:/dowloads"
my $uri = "http://<teamcity>/guestAuth/repository/download/bt11/.lastFinished/MyApp.{build.number}.zip";
# fetch the uri to extract directory
my $ff = File::Fetch->new(uri => "$uri");
my $where = $ff->fetch( to => "$latest_version" );
This gives me a file:
C:\downloads\MyApp.{build.number}.zip.
However, the name of the file downloaded has a build number in the name. Unfortunately there is no version file within the zip, so this is the only way I have of telling what file i've downloaded. Is there any way to get this build number?
c:\downloads\MyApp.12345.zip
With build configs modification
If you have the ability to modify the build configs in TeamCity, you can easily embed the build number into the zip file.
Create a new build step - choose command line
For the script, do something like: echo %build.number% > version.txt
That will put version.txt at the root directory of your build folder in TeamCity, which you can include in your zip later when you create it.
You can later read that file in.
I'm not able to access my servers right now so I don't have the exact name of the parameter, but typing %build will pull up a list of TeamCity parameters to choose from, and I think it is %build.number% that you're after.
Without build configs modification
If you're not able to modify the configs, you're going to need something like egrep:
$ echo MyApp.12.3.4.zip | egrep -o '([0-9]+.){2}[0-9]+'
> 12.3.4
$ echo MyApp.1234.zip | egrep -o '[0-9]+'
> 1234
It looks like you're running on Windows; in those cases I use UnxUtils & UnxUpdates to get access to utilities like this. They're very lightweight and do not install to the registry, just add them to your system PATH.