Download secure file with PowerShell - powershell

I'm trying to create a new release task for VSTS which needs to download a secure file from the library. However, when I run the following PowerShell script no secure files are displayed but there are two in there. Could this be not having enough rights? What should be changed.
Another question: when I'm able to list the secure files I want to download a specific one. I haven't found any examples on how to do that. Does anyone know of an example?
$url = "$($env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI)$env:SYSTEM_TEAMPROJECTID/_apis/distributedtask/securefiles"
Write-Host "URL: $url"
$secureFiles = Invoke-RestMethod -Uri $url -Headers #{
Authorization = "Bearer $env:SYSTEM_ACCESSTOKEN"
}
Write-Host "SecureFiles: $secureFiles"

I was able to download Secure Files using a REST API, the task's Access Token, and an Accept header for application/octet-stream. I enabled "Allow scripts to access the OAuth token". Here my task.json is using a secureFile named "SecureFile."
$secFileId = Get-VstsInput -Name SecureFile -Require
$secTicket = Get-VstsSecureFileTicket -Id $secFileId
$secName = Get-VstsSecureFileName -Id $secFileId
$tempDirectory = Get-VstsTaskVariable -Name "Agent.TempDirectory" -Require
$collectionUrl = Get-VstsTaskVariable -Name "System.TeamFoundationCollectionUri" -Require
$project = Get-VstsTaskVariable -Name "System.TeamProject" -Require
$filePath = Join-Path $tempDirectory $secName
$token= Get-VstsTaskVariable -Name "System.AccessToken" -Require
$user = Get-VstsTaskVariable -Name "Release.RequestedForId" -Require
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $User, $token)))
$headers = #{
Authorization=("Basic {0}" -f $base64AuthInfo)
Accept="application/octet-stream"
}
Invoke-RestMethod -Uri "$($collectionUrl)$project/_apis/distributedtask/securefiles/$($secFileId)?ticket=$($secTicket)&download=true&api-version=5.0-preview.1" -Headers $headers -OutFile $filePath
I am using "$(Build.QueuedById)" to get the user id in build tasks, but honestly I don't think it matters what string you use there.
If you don't have the Accept header, you'll get JSON metadata back for the file you're attempting to download.
Unfortunately I cobbled this together from other SO posts and the github issues pages; I can't find anywhere official that documents the URL I'm using there.

There has no such REST API to download secure file, but you can use Download secure file task for assistants.
And since the secure file only exist in temporary location during build, you should download the secure file by Download secure file task firstly, and copy the secure file to another directory secondly:
1. Download secure file
You can add a Download secure file task (for VSTS) and specify the filename to download.
Note: since the task is not available for TFS, you can install the similar task like Download Secure File extension for your TFS account.
2. Copy secure file to another directory
Such as copy the secure file to $(Build.ArtifactStagingDirectory), you can use PowerShell script:
Copy-Item -Path $(Agent.WorkFolder)\_temp\filename -Destination $(Build.ArtifactStagingDirectory)
Or use Copy Files task to copy the secure file to $(Build.ArtifactStagingDirectory).
BTW:
Since you are using Download Secure File task (developed by Matt Labrum) which can only select secure file from DropDownList (variables can not be used). But there has an issue Enable to use a variable to specify the secure file to download which suggests this feature, you can follow up.
And for the REST API to download secure file, it's not available for now. But there has an user voice Access "Secure files" from .NET client library, and you can vote and follow up.

Related

Azure Devops - Azure powershell task find release/deployment directory

I'm running Azure devops where i have a pipeline and a release (running on a self hosted agent), and the release is set to send up to an azure app service. The deployment works fine, the only issue is that i'd also like to be able to (based on some of my release variables) edit the web.config of the site AFTER it has already been deployed to the azure website.
I'm using the Azure Powershell task ( https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/deploy/azure-powershell?view=azure-devops ) ,and i can't find anywhere in the release variables ( https://learn.microsoft.com/en-us/azure/devops/pipelines/release/variables?view=azure-devops&tabs=batch ) that shows the directory of where the site exists. Looking through Kudu, it's pretty basic, like d:\home\site\wwwroot\ , but using that doesn't work at all.
Is this post config that i'm looking for not really possible, or should i be approaching it a different way?
I think you can use powershell task to call Kudu api to get the deployed web.config and edit it using Magic Chunks task or File Transform task in your release pipeline. Then using Kudu api to upload the changed web.config to azure website again.
1, Below script shows how to get web.config from azure website.
$srcResGroupName = "Test"
$srcWebAppName = "tstest12"
$outwebconfig="$(System.DefaultWorkingDirectory)\tempfolder\web.config"
# Get publishing profile for SOURCE application
$srcWebApp = Get-AzWebApp -Name $srcWebAppName -ResourceGroupName $srcResGroupName
[xml]$publishingProfile = Get-AzWebAppPublishingProfile -WebApp $srcWebApp
# Create Base64 authorization header
$username = $publishingProfile.publishData.publishProfile[0].userName
$password = $publishingProfile.publishData.publishProfile[0].userPWD
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
$apiBaseUrl = "https://$($srcWebApp.Name).scm.azurewebsites.net/api/vfs/site/wwwroot/web.config"
# Download the web.config file to $outwebconfig
Invoke-RestMethod -Uri "$apiBaseUrl" `
-Headers #{UserAgent="powershell/1.0"; `
Authorization=("Basic {0}" -f $base64AuthInfo)} `
-Method GET `
-OutFile $outwebconfig
Above script will download the web.config from the azure website and save it to $(System.DefaultWorkingDirectory)\tempfolder\web.config, where you can edit it with transform task later.
Above scripts get the username and password with scripts, you can also get them in the publish profile by going to the Overview blade on your App Service, clicking ...More at the top of the blade, and then clicking Get publish profile
2, Then you can add a config transform task to change your web.config according.
3, Last add a powershell task to upload the changed web.config to azure website
$srcResGroupName = "Test"
$srcWebAppName = "tstest12"
$webconfig="$(System.DefaultWorkingDirectory)\tempfolder\web.config"
# Get publishing profile for SOURCE application
$srcWebApp = Get-AzWebApp -Name $srcWebAppName -ResourceGroupName $srcResGroupName
[xml]$publishingProfile = Get-AzWebAppPublishingProfile -WebApp $srcWebApp
# Create Base64 authorization header
$username = $publishingProfile.publishData.publishProfile[0].userName
$password = $publishingProfile.publishData.publishProfile[0].userPWD
$base64AuthInfo = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $username,$password)))
$apiUrl = "https://$($srcWebApp.Name).scm.azurewebsites.net/api/vfs/site/wwwroot/web.config";
Invoke-RestMethod -Uri $apiUrl -Headers #{Authorization=("Basic {0}" -f $base64AuthInfo)} -UserAgent $userAgent -Method PUT -InFile $webconfig -ContentType "application/xml";
For more usage of Kudu api you can check here.

Downloading a file with PowerShell

I have a URL to a CSV file which, in a browser, I can download and open without issue.
I'm trying to download this file using PowerShell without success. I tried using Invoke-WebRequest, Start-BitsTransfer and using a webrequest object but no luck there.
Invoke-WebRequest comes with a parameter to store its result in a file: -OutFile
Invoke-WebRequest $myDownloadUrl -OutFile c:\file.ext
If you need authorization before you can send a request like this:
Invoke-WebRequest $myAuthUrl /* whatever is neccesary to login */ -SessionVariable MySession
Invoke-WebRequest $myDownloadUrl -WebSession $MySession
To determine the layout of the form where the login happens, you can use Invoke-WebRequests return object. It'll collect information about forms and fields on the HTML (might be Windows only). Mileage of logging in may vary with things like Two-Factor-Auth active or not. Probably you can create some secret link to your file which does not need Auth or possibly google allows you to create a private access token of some sort, which can be send aus Authorization-Header alongside your request.
TLDR answers*:
Method 1, by default synchronous**
Invoke-WebRequest $url -OutFile $path_to_file
(if you get error "...Could not create SSL/TLS secure channel." see Powershell Invoke-WebRequest Fails with SSL/TLS Secure Channel)
Method 2, by default synchronous**
(New-Object System.Net.WebClient).DownloadFile($url, $path_to_file)
Method 3, asynchronous and may be much slower than the other two but is very gentle on bandwidth usage (it uses the BITS service).
Import-Module BitsTransfer
Start-BitsTransfer -Source $url -Destination $path_to_file
Notes:
*: This answer is for those that google for "how to download a file with PowerShell".
**: Read the help pages if you want asynchronous downloading
For a while now I've been using a PS script to download PowerBI bi-monthly and using the BITS, it's been pretty solid and now so much stronger now since I removed the -Asynchronous at the end of the Start-BitsTransfer
$url = "https://download.microsoft.com/download/8/8/0/880BCA75-79DD-466A-927D-1ABF1F5454B0/PBIDesktopSetup.exe"
$output = "%RandomPath%\PowerBI Pro\PBIDesktopSetup.exe"
$start_time = Get-Date
Import-Module BitsTransfer
Start-BitsTransfer -Source $url -Destination $output
#Commented out below because it kept creating "Tmp files"
#Start-BitsTransfer -Source $url -Destination $output -Asynchronous

Powershell download file from redirecting url - TeamViewer & Intune

Thank you in advance for anyone taking a look into this.
I'm currently trying to deploy TeamViewer via Intune that only support MSI files for deployment. However, TeamViewer has a feature called account assignment which it comes in form of an executable. Since Intune doesn't allow you deploy exe files, please correct me if I'm wrong. I have resulted in using a PowerShell script that will download the necessary files and then install.
My goal is to have the files stored in the cloud like onedrive or Dropbox. The problem there is the public link doesn't point to the file directly as its a redirect.
For example https://www.dropbox.com/x/xyzd/TeamViewer_Assignment.exe?dl=0 --> https://www.dropbox.com/x/xyzd/TeamViewer_Assignment.exe
or
https://1drv.ms/u/s!Avjfi0upMYg9haNVTMpdoPGdstex --> https://1drv.ms/u/s/teamviewer.exe
if both links were to end with the file extension (.exe), then it would be no problem. But I would like to use Teamviewer links (get.teamviewer.com/myhost redirects https://download.teamviewer.com/u/id12345/TeamViewer.exe hoping this will help a lot more people. As opposed to having a cloud storage account.
https://download.teamviewer.com/u/id12345/TeamViewer.exe is not a permanent link either, and it has an expiration time.
Things I've tried:
$url = "https://get.teamviewer.com/mycustomhost"
$output = "$PSScriptRoot\myhost.exe"
$start_time = Get-Date
Invoke-WebRequest -Uri $url -OutFile $output
Write-Output "Time taken: $((Get-Date).Subtract($start_time).Seconds)
second(s)"
$url = "http://get.teamviewer.com/myhost"
$output = "$PSScriptRoot\myhost.exe"
$start_time = Get-Date
$wc = New-Object System.Net.WebClient
$wc.DownloadFile($url, $output)
#OR
(New-Object System.Net.WebClient).DownloadFile($url, $output)
Write-Output "Time taken: $((Get-Date).Subtract($start_time).Seconds)
second(s)"
$rep=Invoke-WebRequest
http://www.get.teamviewer.com/myhost -MaximumRedirection
0
$rep.Links | where {$_.innerText -eq "click here"} |select -expand href
None of those examples worked I tried other combination from bits and pieces over the net but no go.
You can use the following URI for all of your examples:
https://customdesign.teamviewer.com/download/version_12x/myhost/TeamViewerQS.exe
You can get this URI for your download in Chrome in the following way:
Download TeamViewer
Open the Download History
Right click the entry for the TeamViewer download and copy the download URI.
Edit:
You can parse the download site for the real link with the following command:
$downloadPage = Invoke-WebRequest -Uri https://get.teamviewer.com/myhost
$downloadLink = $request.ParsedHtml.getElementById('MasterBodyContent_btnRetry').href
Now you can use the '$downloadLink' variable to download the executable with any of your scripts. You may have to change this if the download page for TeamViewer changes.
Just search for the id of the 'Try again' button on the download page. Then you can edit my code to get the appropriate element and attribute.

Downloading a file from web using Powershell

I am trying to download a csv file using Powershell. If I am use a public link, I am able to download the file.
But when I am trying to download from my account (private link), the file downloads and I am getting some sort of html/css content. Screenshot is attached. For now, I was testing it with Dropbox. Although, I want to download the file from Huddle Workspace.
I am using two ways as follows:
$url = "url/path/to/file/Notification.csv"
$output = "pathToFolder/Notification.csv"
$wc = New-Object System.net.WebClient
$wc.DownloadFile($url,$output)
$cred = Get-Credential -UserName 'myEmailAddress#hotmail.com'
Invoke-WebRequest $url -OutFile $output -Credential $cred
Again, both methods works with Public link. What would be the best way to add credentials and download the file. Thanks

Create an Azure Website with PowerShell and FTP

I need to write a PowerShell script that automatically creates an Azure Website and deploy the content of a local file system. Seems that Azure PowerShell SDK doesn't provide a way to copy files to a website so we want to use FTP as a deploy method.
To get the correct FTP endpoints and credentials the only way that I have found is to call an Azure Management Rest API: Get a Site’s Publish Profile.
But this API as other Azure Management API requires a certificate. I have found the following tutorial
Building Real-World Cloud Apps with Windows Azure that explain how to get the certificate:
$s = Get-AzureSubscription -Current
$thumbprint = $s.Certificate.Thumbprint
Unfortunately seems that with the current SDK $s.Certificate is always null, this property doesn't exists. If I manually set the certificate thumbprint everything works as expected.
Do you have an idea on how to get the correct subscription certificate? Or do you have an easy alternative to deploy local files to an Azure website?
It seems that now you can access the certificate thumbprint using
$thumbprint = $s.DefaultAccount
instead of
#$thumbprint = $s.Certificate.Thumbprint
Seems that the DefaultAccount has exactly the same value as the certificate thumbprint.
Just for reference here is my complete script to obtain a publishing profile for a given website:
Function get-AzureWebSitePublishXml
{
Param(
[Parameter(Mandatory = $true)]
[String]$WebsiteName
)
# Get the current subscription
$s = Get-AzureSubscription -Current
if (!$s) {throw "Cannot get Windows Azure subscription."}
#$thumbprint = $s.Certificate.Thumbprint #this code doesn't work anymore
$thumbprint = $s.DefaultAccount
if (!$thumbprint) { throw "Cannot get subscription cert thumbprint."}
# Get the certificate of the current subscription from your local cert store
$cert = Get-ChildItem Cert:\CurrentUser\My\$thumbprint
if (!$cert) {throw "Cannot find subscription cert in Cert: drive."}
$website = Get-AzureWebsite -Name $WebsiteName
if (!$website) {throw "Cannot get Windows Azure website: $WebsiteName."}
# Compose the REST API URI from which you will get the publish settings info
$uri = "https://management.core.windows.net:8443/{0}/services/WebSpaces/{1}/sites/{2}/publishxml" -f `
$s.SubscriptionId, $website.WebSpace, $Website.Name
# Get the publish settings info from the REST API
$publishSettings = Invoke-RestMethod -Uri $uri -Certificate $cert -Headers #{"x-ms-version" = "2013-06-01"}
if (!$publishSettings) {throw "Cannot get Windows Azure website publishSettings."}
return $publishSettings
}
NOTE: this only works when you have connected to azure using Import-AzurePublishSettingsFile
Can anyone confirm that is safe to use DefaultAccount property?
UPDATE
If you use Kudu API to upload your site, like this, you don't need any certificate or publishing profile. You should read the user name and password using Get-AzureWebsite and the hostname is just yourwebsitename.scm.azurewebsites.net (note the scm segment). I suggest to use Kudu because is far more reliable and fast.