We have certain Chocolatey packages stored in VSTS private package management feed. I'm trying to download a specific package from feed in VSTS in PowerShell. I'm using the below commands:
$user = "my-user-id"
$token = "my-pat-token"
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $user,$token)))
$url = "vsts-package-url"
$webClient = New-Object System.Net.WebClient
$webClient.Headers.Add('HttpRequestHeader.Authorization',"Basic $base64AuthInfo")
After this, I should be able to download the package when using $webClient.DownloadFile($url, $fileName).
However file contains dummy text expecting me to sign-in and has 401 related details.
Its basically a PowerShell code. Can anyone please advise how should I go about this? I do not want to install the package from feed, only download it.
I was wondering if there is a better way to do this as this process still needs me to prepare package url first which would be different for each package iteration?
It looks like you're creating the auth header incorrectly. Basically, with a PAT, no user name is needed.
I use this snippet of code all the time:
$pat = 'yourPatHere'
$bytes = [System.Text.Encoding]::ASCII.GetBytes(":$($pat)")
$encodedText =[Convert]::ToBase64String($bytes)
$header = #{Authorization = "Basic $encodedText"}
You can then use Invoke-WebRequest to download the feed information:
Invoke-WebRequest -Uri 'https://uri' -Header $header -Method Get
That looks roughly the same as your code, but Invoke-WebRequest is idiomatic PowerShell vs creating a WebClient. Shouldn't matter in the end.
Actually downloading a file from a feed is a different matter. The packages are exposed via standard mechanisms for that specific type of feed. See the documentation. Basically, if you need a NuGet package, you download it from the NuGet feed, not from package management.
Related
I need the list of available versions for a Universal Package stored in Azure Devops. My thought is to call the REST API Get Package Versions to get a list of the versions for packages on a feed.
GET https://feeds.dev.azure.com/{organization}/{project}/_apis/packaging/Feeds/{feedId}/Packages/{packageId}/versions?api-version=5.1-preview.1
The problem is that it requires a packageId, which is the GUID and I only know the name. The only way I've figured out so far to convert a package name to a GUID is using "Get Packages" but that returns every package on the feed (which for me includes thousands of NPM packages) and that makes the download very large for the handful of items I need. Is there some way to extract the packageId for a given package name? Or is there a better way to extract all the versions for a package?
Someone pointed out to me that the Get Packages API has options for IncludeAllVersions and packageNameQuery to achieve what I want rather than using GetAllVersions.
https://feeds.dev.azure.com/{organization}/{project}/_apis/packaging/Feeds/{feedId}/Packages?includeAllVersions=true&packageNameQuery={packageName}&protocolType=nuget
I assume you have checked some docs and found there's no direct API can let you get specified packaged ID, right? Also, as the doc said, the package name could not be used in this API:
In fact, you have very close to the answer. Just add some filter while you running Get Packages. But this need you execute some script in Powershell command line or Powershell ISE which is the most convenient approach can for you use. You can also run below script in Azure Devops pipeline, but compare with running in command line, it is a bit cumbersome.
Run below script in your Powershell command line or Powershell ISE:
$token = "{your PAT token}"
$url = 'https://feeds.dev.azure.com/{org name}/_apis/packaging/Feeds/{feed ID}/packages'
$token = [System.Convert]::ToBase64String([System.Text.Encoding]::ASCII.GetBytes(":$($token)"))
$response = Invoke-RestMethod -Uri $url -Headers #{Authorization = "Basic $token"} -Method Get
$results = $response.value | Where {$_.name -eq "{Package Name}"} #|
Write-Host "results = $($results | ConvertTo-Json -Depth 100)"
Note: In the above script, you just need to replace the value of PAT token, Organization name and your package name.
Then you will get the info of the specified package, and you can copy the package ID from the command line and apply it in another place :
Note: The above script can also be applied in the Powershell task of Azure Devops without change anything.
I have a script which grabs a file from GIT using the bitbucket REST API (1.0) however it has recently stopped working. I'm theorizing this may be due to the v1 REST API being depreciated but I'm not sure.
Anyway I am trying to retrieve the file using the new 2.0 REST API but I can't seem to get the syntax right as the request continually fails.
I'm starting out with curl since its easiest to test. This is what I'm trying:
curl -u myusername#mydomain.com "https://api.bitbucket.org/2.0/repositories/MyCompany/myrepo/downloads/Scripts/Environment Setup/test.txt"
Enter host password for user 'myusername#mydomain.com': redacted
{"type": "error", "error": {"message": "Resource not found", "detail": "There is no API hosted at this URL.\n\nFor information about our API's, please refer to the documentation at: https://developer.atlassian.com/bitbucket/api/2/reference/"}}
Here is the reference documentation I am using: https://developer.atlassian.com/bitbucket/api/2/reference/resource/repositories/%7Busername%7D/%7Brepo_slug%7D/downloads/%7Bfilename%7D
Maybe I am using the wrong function? I'm not sure.
For posterities sake, you don't want to use the following to download an individual file from bitbucket:
https://api.bitbucket.org/2.0/repositories/MyCompany/myrepo/downloads/path/to/your/file.txt
("Downloads" is to download entire repo files like a .zip file)
Instead you want to do:
curl --user myuser#mydomain.com:password "https://api.bitbucket.org/2.0/repositories/MyCompany/myrepo/src/master/path/to/file.txt"
If you're trying to use Invoke-RestRequest (in powershell) note there are some extra steps. With the old 1.0 API you could do:
$cred = Get-Credential
$uri = "https://api.bitbucket.org/1.0/repositories/MyCompany/$($filepath)"
# Get the files from bitbucket (GIT)
Invoke-RestMethod -Credential $cred -Uri $uri -Proxy $proxyUri -OutFile $destination
With the new 2.0 API that no longer works. Powershell's Invoke-RestMethod waits for a 401 response before sending the credentials, and the new 2.0 bitbucket api never provides one, so credentials never get sent causing a 403 forbidden.
To work around that you have to use the following ugly hack to force Invoke-RestMethod to send the credentials immediately in an Authorization header:
$cred = Get-Credential
$uri = "https://api.bitbucket.org/2.0/repositories/MyCompany/$($filepath)"
$username = ($cred.GetNetworkCredential()).username
$password = ($cred.GetNetworkCredential()).password
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
# Get the files from bitbucket (GIT)
Invoke-RestMethod -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -Uri $uri -Proxy $proxyUri -OutFile $destination
Hopefully that helps someone else out in the future!
Thanks #Jim Redmond for the help.
You can also use the PowerShell module BitbucketServerAutomation. There's not a ton of cmdlets, they do have Get-BBServerFile and Get-BBServerFileContent. I have found it is well written, very usable and being updated regularly. The Invoke-BBServerRestMethod cmdlet is available if you need a command it doesn't have.
In our azure devops organization, we have ~100 projects. Each projects holds between 1 and 20 git repositories. This makes it hard to find a specific repository unless you also remember which project it is located in.
The search box at the project list only searches by project names. Is there any way to search and find repository names?
There is a code search, but that returns results from files already in a repository, which gives both redundant and wrong results.
I think the simplest is to call this REST API.
You can call it even from the browser and use the Find feature.
Note that the project element is not required.
To automate, you can use a few lines of Powershell to search in the result JSON document, e.g.
$organization = ...
$PAT = ...a personal access token
$repoNameStart = ...initial name of repo
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes("dummy:${PAT}"))
$result = Invoke-RestMethod -Method Get -Uri "https://dev.azure.com/${organization}/_apis/git/repositories?api-version=4.1" -Headers #{Authorization = "Basic $base64AuthInfo" } -Credential $credential -ContentType "application/json"
$result.value | where {$_.name -like "${repoNameStart}*"}
(Untested code, use it as an example)
I am trying to perform CI/CD using Perfecto and hence I am trying to upload a file to perfecto when my Bamboo build is finished.
I was trying with the following cURL command when we have a Linux server.
curl -X POST --upload-file test.apk 'https://****.perfectomobile.com/services/repositories/media/PRIVATE:test.apk?operation=upload&user=<email>&password=<password>&overwrite=true'
Now our server is changed to Windows and hence I want a powershell script which I can use as an Inline Scripts in Bamboo.
Can you please tell me what is an equivalent script in Powershell for windows.
Many thanks in advance.
# Gather your information.
$email = "myEmail#website.com";
$password = "powershellR0cks!";
$subDomain = "****";
$url = "https://$subDomain.perfectomobile.com/services/repositories/media/PRIVATE:test.apk?operation=upload&user=$email&password=$password&overwrite=true";
$filePath = ".\test.apk";
# Make the request.
$response = Invoke-WebRequest -Uri $URL -Method Post -InFile $filePath -ContentType "application/octet-stream";
# Check for success.
if (-not ($response.StatusCode -eq 200)) {
throw "There was an error uploading the APK manifest.";
}
You may want to check the value of -ContentType, but I think that's correct. You don't necessarily need to include the scheme (HTTPS) if you don't want to, and semicolons in PowerShell are optional, but you can include them if you want.
The $response variable is an HtmlWebResponseObject that has the content of the response, the status code, and a bunch of other useful info. You can check out the available properties and methods on the object by running $response | Get-Member.
Finally, the Invoke-WebRequest cmdlet also has other parameters that may be useful to you, such as -Credential, -Headers, and more.
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/invoke-webrequest?view=powershell-5.1
As a side-note, if you run Get-Alias -Name "curl", you can see that anytime you use curl in PowerShell, you're really just calling Invoke-WebRequest. You can use the curl alias if you want, but it's generally not a good idea to use aliases in automation since they can be modified or deleted.
I am trying to download a google spreadsheet via an invoke-webrequestin powershell.
My link looks something like this and I can confirm that if I go to the link it prompts me to download the file I want...
$url = https://docs.google.com/spreadsheets/d/XXXXXXXXXXXXXXXX/export?format=csv
I have tried downloading the file via 2 ways:
Invoke-WebRequest $url -OutFile $saveLocation
(New-Object System.Net.WebClient).DownloadFile($url, $saveLocation)
Both of these just download the HTML for the page, and not the actual file.
The spreadsheet I am downloading is 100% public so it is not like I need to be logged into a google account to view it.
I made the test and I can quite safely say that your file is not public.
I have reproduced your issue before making my test sheet public, and I could not afterwards.
Tested with this :
$url = "https://docs.google.com/spreadsheets/d/1OOc5aiG3hh8mrGF4p7NJXKI8KqBSkS2ZjsBXtb4qpz8/export?format=csv"
Invoke-WebRequest $url -OutFile "C:\test.csv"