how to import files from SharePoint team site to local machines? - powershell

My client generating reports (excel files) monthly basis and they upload the files in the SharePoint document library. Again some reports are coming through the mail (outlook). 
The requirement is we need to copy the excel reports (which are in document library and the files that are coming through mail) to the Landing zone(FTP Server) and we need to automate this process every month.
Which is the best way to achieve this ? please help me on this.Thank you..

$uri = "https://intranet.company.com/departments/SampleSite/_api/web/getfilebyserverrelativeurl('/departments/SampleSite/Shared Documents/test.docx')/`$Value"
$FileBinary = $(Invoke-WebRequest -Uri $uri -Credential $cred -ContentType 'application/json;odata=verbose').Content
[System.IO.File]::WriteAllBytes('C:\tempfolder\test.docx',$FileBinary)
With this sample you can download a file and save it (it uses the SP REST API). This simple sample uses the credentials of the currently logged on user.
Edit : on request I've added a sample for SharePoint Online (you need the SP SDK for this because of the login flow). Code is not optimized though since this was one of my first samples. The problem is that you have to "hardcode" credentials. My solution for this within my company is encrypting the credentials with a certificate : I encrypt the credentials in a file using the public key of the service account that's running the script ; the service account uses the private key to decrypt it. Not ideal but safe enough for us.
$cred = Get-Credential # Get credentials
## Load the Assemblies
add-type -AssemblyName system.net.http
Add-type -Path "C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.SharePoint.Client.Runtime\v4.0_16.0.0.0__71e9bce111e9429c\Microsoft.SharePoint.Client.Runtime.dll"
# Put credentials in SPO credential object
$SPCred = [Microsoft.SharePoint.Client.SharePointOnlineCredentials]::new($cred.UserName,$cred.Password)
# Set the uri's
$uri = [uri]::new('https://YourTentant.sharepoint.com')
$fullUri = "https://YourTentant.sharepoint.com/sites/ApiTest/_api/web/getfilebyserverrelativeurl('/sites/ApiTest/Shared Documents/Document.docx')/`$Value"
# Create HTTPClientHandler object and set the properties
$handler = [System.Net.Http.HttpClientHandler]::new()
$handler.Credentials = $SPCred
$handler.CookieContainer.SetCookies($uri,$SPCred.GetAuthenticationCookie($uri))
# Create the client and set the options
$Client = [System.Net.Http.HttpClient]::new($handler)
$Client.DefaultRequestHeaders.Accept.Clear()
$Client.DefaultRequestHeaders.Accept.Add([System.Net.Http.Headers.MediaTypeWithQualityHeaderValue]::new('application/json'))
# Call the URL (async) and get the response
$Response = $Client.GetAsync($fullUri)
$data = $Response.Result
## now download the data as string (in the final version I'm going to call the ReadAsStream() method instead to get the binary file)
$a = $data.Content.ReadAsStringAsync()
$a.Result

Related

Powershell to Update Google Sheets

I have imported a module UMN-Google from https://github.com/umn-microsoft-automation/UMN-Google in PowerShell which is used to create, update Google Sheets.
I create a Project in https://console.cloud.google.com/
Enabled the Google Sheets and Drive API.
Create a Service Account and generated a private key(P12 format).
Following is the PowerShell Code to generate Access Token and same access token is used to create or update the sheet which is working perfectly.
Import-Module UMN-Google
# Set security protocol to TLS 1.2 to avoid TLS errors
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12
# Google API Authozation
$scope = "https://www.googleapis.com/auth/spreadsheets https://www.googleapis.com/auth/drive https://www.googleapis.com/auth/drive.file"
$certPath = "C:\Temp\sheets-script-336006-5db4e6b09111.p12"
$iss = 'sheets-script#sheets-script-336006.iam.gserviceaccount.com'
$certPswd = 'notasecret'
try {
$accessToken = Get-GOAuthTokenService -scope $scope -certPath $certPath -certPswd $certPswd -iss $iss
} catch {
$err = $_.Exception
$err | Select-Object -Property *
"Response: "
$err.Response
}
$accessToken
# Create new spreadsheet
$Title = 'Patching Spreadsheet'
$SpreadsheetID = (New-GSheetSpreadSheet -accessToken $accessToken -title $Title).spreadsheetId
$SpreadsheetID
# Create new sheet
$Sheet = 'Computers'
Add-GSheetSheet -accessToken $accessToken -sheetName $Sheet -spreadSheetID $SpreadsheetID
Now I have many google sheets to which I want the script to be integrated in-order to update the data in various sheets. We have disabled drive sharing outside domain due to which I am unable to give permission to service account for sheets editing.
Is there any way or possibility which can be done so that the service account has the permission to update the sheets.
This is related to something that I have run into recently, specifically with having a service account "impersonate" a user within our Google Workspace in order to perform calls to Google Drive API.
Our solution came from this guide here: Delegating domain-wide authority to the service account.
The idea is to have your service account assume the role of an account in your Google Workspace that does have the permissions you need, in this case to edit specific Sheets. The instructions are pretty straightforward in that guide; just a quick setup in the Admin console.
It's worth noting that we are not using Get-GOAuthTokenService to get an access token. We are building a JWT using Svyatoslav Pidgorny's JWT module here and using it to get an access token via the https://oauth2.googleapis.com/token endpoint.
I only took a quick peek at UMN-Google's implementation, but I see that their JWT-creation doesn't offer any sub parameter in the JWT payload. This is the parameter that let's you designate which Workspace user that you want the service account to impersonate.

Generate a JWT with an X.509 certificate signature in Powershell

I'm trying to get an API access token from Active Directory using certificate authentication as shown here:
Access token request with a certificate
The request needs the client_assertion property, which is a JWT created from the certificate with the format and specs mentioned here:
Assertion format
Is there a way of generating this token in Powershell that is non-interactive as this is part of a release pipeline?
Edit: To make it a little more clear, this C# code using the Microsoft.IdentityModel.Clients.ActiveDirectory library is what I'm trying to do in Powershell, specifically the second line:
AuthenticationContext authContext = new AuthenticationContext(authority);
IClientAssertionCertificate assertion = new ClientAssertionCertificate(clientId, certificate);
authenticationResult = await authContext.AcquireTokenAsync(resource, assertion);
Sample code (just tested this, this works for me when I register an app as a web app).
<#
Sample to connect to Graph using a certificate to authenticate
Prerequisite : ADAL (Microsoft.IdentityModel.Clients.ActiveDirectory.dll)
#>
# Load the ADAL Assembly
Add-Type -Path "E:\Assemblies\Microsoft.IdentityModel.Clients.ActiveDirectory.4.3.0\lib\net45\Microsoft.IdentityModel.Clients.ActiveDirectory.dll"
# Settings for the application
$AppID = '<ID OF THE WEB APP>'
$TenantDomain = '<TENANT>'
$LoginUri = 'https://login.microsoftonline.com/'
$Resource = 'https://graph.microsoft.com'
$Certificate = Get-Item 'Cert:\CurrentUser\My\<CERTIFICATE THUMBPRINT>' # This points to my own certificate
# Auth Authority Uri
$Authority = "$LoginUri/$TenantDomain"
# Create the authenticationContext
$Context = [Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext]::new($Authority)
# create the CAC
$CAC = [Microsoft.IdentityModel.Clients.ActiveDirectory.ClientAssertionCertificate]::new($AppID,$Certificate)
# Get the token
$TokenResponse = $Context.AcquireTokenAsync($Resource,$CAC)
Start-Sleep -Seconds 1 # Sleep for 1 second...
# Token should be present
$TokenResult = $TokenResponse.Result
As I know, you could not use PowerShell to create the JWT.
You could use jwt.io to edit each part (header, payload), and then jwt.io will automatically encode it into a JWT for the client_assertion.

Using Powershell to get Azure AD Token (jwt)

I am trying to get a jwt token from AAD using Powershell using Username/Password authentication.
I am writing a powershell script that will to call an API using a bearer token. What I have works if I copy & paste the token from an SPA that uses the API. I am looking for a way to retrieve the token from my powershell.
This looks really promising: https://github.com/Azure-Samples/active-directory-dotnet-native-headless/blob/master/TodoListClient/Program.cs
I feel like I'm smacking my head against a wall trying to create a 'UserPasswordCredential' object. Any clues to how I can do this would be super-helpful.
I have Add-Type-ed:
- Microsoft.IdentityModel.Clients.ActiveDirectory.dll
- Microsoft.IdentityModel.Clients.ActiveDirectory.platform.dll (adds nothing?)
- Microsoft.IdentityModel.Clients.ActiveDirectory.WindowsForms.dll
- Microsoft.IdentityModel.Clients.ActiveDirectory.WindowsForms.dll
The docs page for 'UserPasswordCredential' :
https://learn.microsoft.com/en-us/dotnet/api/microsoft.identitymodel.clients.activedirectory.userpasswordcredential
It should be in one of the first two dlls
This, under 'Constraints & Limitations', makes me think it may not actually be possible from powershell:
http://www.cloudidentity.com/blog/2014/07/08/using-adal-net-to-authenticate-users-via-usernamepassword/
Looking at the code below, the first acquire token succeeds, the second fails - possibly/probably because $cred is a UserCredential not a UserPasswordCredential.
Is is possible to do this with powershell?
Finally, on a totally different track, how do I find the values for redirectUri and resourceAppIdURI that my application needs? When I look in the AAD console, and browser to my Enterprise Application, I can find the AppId (which I can use as $clientId).
I'm not sure the redirectUri is strictly necessary for me as all I really want is the token, but I can have a good guess at what it should be.
When I try to call the first AquireToken method (without $cred) using my app details, it fails with this message:
Exception calling "AcquireToken" with "4" argument(s): "AADSTS50001: The application named https://myappwithapi/Login was not found in the tenant named me.onmicrosoft.com.
Is it possible for me to find the require value for resourceAppIdURI by looking in my azure portal?
'https://myappwithapi/Login' is from my azure portal > enterprise apps > [app' > properties > HomepageUrl
code:
#setup
$TenantName = "mme.onmicrosoft.com"
$clientId = "1950a258-227b-4e31-a9cf-717495945fc2" # Microsoft
$clientId = "03faf8db-..........................." #
$username = "me#me.onmicrosoft.com"
$password = Read-Host -AsSecureString -Prompt "Enter Password"
# add dlls
$adal = "${env:ProgramFiles(x86)}\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Services\Microsoft.IdentityModel.Clients.ActiveDirectory.dll"
$adalforms = "${env:ProgramFiles(x86)}\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Services\Microsoft.IdentityModel.Clients.ActiveDirectory.WindowsForms.dll"
$adalplatform = "${env:ProgramFiles(x86)}\Microsoft SDKs\Azure\PowerShell\ServiceManagement\Azure\Services\Microsoft.IdentityModel.Clients.ActiveDirectory.platform.dll"
[System.Reflection.Assembly]::LoadFrom($adal) | Out-Null
[System.Reflection.Assembly]::LoadFrom($adalforms) | Out-Null
[System.Reflection.Assembly]::LoadFrom($adalplatform) | Out-Null
#prep request
$redirectUri = "urn:ietf:wg:oauth:2.0:oob" # Microsoft
$resourceAppIdURI = "https://graph.windows.net"
$authority = "https://login.windows.net/$TenantName"
$authContext = New-Object "Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext" -ArgumentList $authority
# Get Token prompting for creds
$authResult = $authContext.AcquireToken($resourceAppIdURI, $clientId, $redirectUri, "Always")
$authResult
# Get the cred
$cred = New-Object -TypeName 'Microsoft.IdentityModel.Clients.ActiveDirectory.UserCredential' -ArgumentList $username, $password
#$cred = New-Object -TypeName 'Microsoft.IdentityModel.Clients.ActiveDirectory.UserPassCredential' -ArgumentList $username, $password
$authResult = $authContext.AcquireToken($resourceAppIdURI, $clientId, $cred)
$authResult
This post has more the one question in it.
Your base use case 'Using Powershell to get Azure AD Token (jwt)' is a common one and there are several samples and pre-built examples to leverage. For example:
https://github.com/pcgeek86/AzureADToken
A PowerShell module that allows you to get a JSON Web Token (JWT) from Azure Active Directory (AAD).
https://gallery.technet.microsoft.com/Get-Azure-AD-Bearer-Token-37f3be03
This script acquires a bearer token that can be used to authenticate to the Azure Resource Manager API with tools such as Postman. It uses the Active Directory Authentication Library that is installed with the Azure SDK.
See if those two resources resolves your use base line use case.
As for this...
"Is it possible for me to find the require value for resourceAppIdURI by looking in my azure portal?"
You can do this via a remote PowerShell logon to AzureAD. Install the AAD PowerShell module.
https://learn.microsoft.com/en-us/powershell/azure/overview?view=azurermps-5.1.1
https://msdn.microsoft.com/en-us/library/dn135248(v=nav.70).aspx
Download and install MSOL. Sign in with the MSOL
https://www.microsoft.com/en-US/download/details.aspx?id=39267
The Microsoft Online Services Sign-In Assistant provides end user sign-in capabilities
and use the built-in cmdlets to pull your information from your organization settings, and or hit the MSGraph API and query.
https://learn.microsoft.com/en-us/powershell/azure/active-directory/install-adv2?view=azureadps-2.0
You can use the Azure Active Directory PowerShell Module Version for Graph for Azure AD administrative tasks
As for this one:
"how do I find the values for redirectUri and resourceAppIdURI that my application needs?"
This is in your app registration section of your portal. The developer team provide the redir uri not Azure. It's part of the registration process all else is generated by Azure App Reg process.
The app registration process is here and of course you are someone else had to register this app in AzureAD, and thus can retrieve it at any time.:
https://blogs.msdn.microsoft.com/onenotedev/2015/04/30/register-your-application-in-azure-ad
Any registered apps and their details can be retrieved using...
Get-AzureADApplication
Get-AzureADApplication | Select -Property *
(Get-AzureADApplication).ReplyUrls
Get-AzureADApplication | Select -Property AppID, DisplayName,ReplyUrls
https://learn.microsoft.com/en-us/powershell/module/azuread/get-azureadapplication?view=azureadps-2.0

How can I authenticate to AAD and call the Graph API as a Daemon Application with PowerShell?

I am trying to do some very quick tests on Azure Active Directory, and I want to use a Daemon Application to access the Graph API without needing a user present to authenticate. I want to verify that my application registration can successfully authenticate to AAD, that my client secret is valid, and make calls to the AAD Graph API.
I have registered a "Web App/API" in my directory already, and I have set it up to have the appropriate permissions to call the AAD Graph API in the App Only Context. I have also generated an application key/certificate for my app so that I can authenticate as a confidential client.
I want to take a look at my AAD Token, and the output from the Graph API after my call. How can I use PowerShell to quickly accomplish this?
This question is very similar to this one where create a PowerShell script to authenticate as a Native Client Application. However, in this situation, there are some subtle and important differences because you want to authenticate as a confidential client. Specifically, we need to create a Client Credential so that we can authenticate without a user as a Daemon Application.
First you need to download and save the .NET dlls for ADAL. The download link can be found on Nuget.
Note: We specifically use ADAL v2 here.
You can extract the contents of the .nupkg with a File Extractor like
7z, WinZip, etc...
Extract the contents from \lib\net45\ and copy them into your working directory. I put the files in their own "ADAL" folder, to keep it separate.
Then you should be able to create a new PowerShell script with the following:
# Load ADAL
Add-Type -Path ".\ADAL\Microsoft.IdentityModel.Clients.ActiveDirectory.dll"
# Output Token and Response from AAD Graph API
$accessToken = ".\Token.txt"
$output = ".\Output.json"
# Application and Tenant Configuration
$clientId = "<AppIDGUID>"
$tenantId = "<TenantID>"
$resourceId = "https://graph.windows.net"
$login = "https://login.microsoftonline.com"
# Create Client Credential Using App Key
$secret = "<AppKey>"
# Create Client Credential Using Certificate
#$certFile = "<PFXFilePath>"
#$certFilePassword = "<CertPassword>"
#$secret = New-Object -TypeName System.Security.Cryptography.X509Certificates.X509Certificate -ArgumentList $certFile,$certFilePassword
# Get an Access Token with ADAL
$clientCredential = New-Object Microsoft.IdentityModel.Clients.ActiveDirectory.ClientCredential($clientId,$secret)
$authContext = New-Object Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext("{0}/{1}" -f $login,$tenantId)
$authenticationResult = $authContext.AcquireToken($resourceId, $clientcredential)
($token = $authenticationResult.AccessToken) | Out-File $accessToken
# Call the AAD Graph API
$headers = #{
"Authorization" = ("Bearer {0}" -f $token);
"Content-Type" = "application/json";
}
Invoke-RestMethod -Method Get -Uri ("{0}/{1}/users?api-version=1.6" -f $resourceId,$tenantId) -Headers $headers -OutFile $output
Note: You will need to update the App ID, Tenant ID, and your App Secret information in this script. If you use a certificate to authenticate, simply comment out the code that uses the App Key, and un-comment the code which uses the certificate. I have also pre-configured the AAD Graph API call to return the users in my tenant, but you can change this REST call to whatever you want.
After you successfully run the script, you should get 2 new files in your working directory: A text file that contains your encoded JSON access token, which can be base64 decoded on sites like this, and a JSON file with the response from the AAD Graph API.
Let me know if this helps!

Create an Azure Website with PowerShell and FTP

I need to write a PowerShell script that automatically creates an Azure Website and deploy the content of a local file system. Seems that Azure PowerShell SDK doesn't provide a way to copy files to a website so we want to use FTP as a deploy method.
To get the correct FTP endpoints and credentials the only way that I have found is to call an Azure Management Rest API: Get a Site’s Publish Profile.
But this API as other Azure Management API requires a certificate. I have found the following tutorial
Building Real-World Cloud Apps with Windows Azure that explain how to get the certificate:
$s = Get-AzureSubscription -Current
$thumbprint = $s.Certificate.Thumbprint
Unfortunately seems that with the current SDK $s.Certificate is always null, this property doesn't exists. If I manually set the certificate thumbprint everything works as expected.
Do you have an idea on how to get the correct subscription certificate? Or do you have an easy alternative to deploy local files to an Azure website?
It seems that now you can access the certificate thumbprint using
$thumbprint = $s.DefaultAccount
instead of
#$thumbprint = $s.Certificate.Thumbprint
Seems that the DefaultAccount has exactly the same value as the certificate thumbprint.
Just for reference here is my complete script to obtain a publishing profile for a given website:
Function get-AzureWebSitePublishXml
{
Param(
[Parameter(Mandatory = $true)]
[String]$WebsiteName
)
# Get the current subscription
$s = Get-AzureSubscription -Current
if (!$s) {throw "Cannot get Windows Azure subscription."}
#$thumbprint = $s.Certificate.Thumbprint #this code doesn't work anymore
$thumbprint = $s.DefaultAccount
if (!$thumbprint) { throw "Cannot get subscription cert thumbprint."}
# Get the certificate of the current subscription from your local cert store
$cert = Get-ChildItem Cert:\CurrentUser\My\$thumbprint
if (!$cert) {throw "Cannot find subscription cert in Cert: drive."}
$website = Get-AzureWebsite -Name $WebsiteName
if (!$website) {throw "Cannot get Windows Azure website: $WebsiteName."}
# Compose the REST API URI from which you will get the publish settings info
$uri = "https://management.core.windows.net:8443/{0}/services/WebSpaces/{1}/sites/{2}/publishxml" -f `
$s.SubscriptionId, $website.WebSpace, $Website.Name
# Get the publish settings info from the REST API
$publishSettings = Invoke-RestMethod -Uri $uri -Certificate $cert -Headers #{"x-ms-version" = "2013-06-01"}
if (!$publishSettings) {throw "Cannot get Windows Azure website publishSettings."}
return $publishSettings
}
NOTE: this only works when you have connected to azure using Import-AzurePublishSettingsFile
Can anyone confirm that is safe to use DefaultAccount property?
UPDATE
If you use Kudu API to upload your site, like this, you don't need any certificate or publishing profile. You should read the user name and password using Get-AzureWebsite and the hostname is just yourwebsitename.scm.azurewebsites.net (note the scm segment). I suggest to use Kudu because is far more reliable and fast.