I have a files in my SharePoint site that look something like this
ShardReportsFull/ShardData-Full-2022-03-28-03.00.12AM.csv
ShardReportsFull/ShardData-Full-2022-03-27-53.00.12AM.csv
ShardReportsFull/ShardData-Full-2022-03-25-34.00.12AM.csv
I'm just wondering how can I download the latest file. I have tried passing a date like this but the problem is the uploaded file name format. it has a time where is not consistent so I can't just pass a date like this so I need to find a way to download the latest file instead.
$Date = Get-Date
$ShardDate = $Date.ToString("yyyy-MM-dd")
$Global:ShardListURL = "/sites/msteams_88c7ed/ShardReportsFull/ShardData-Full-"+$ShardDate+".csv"
$Global:shardListCSV = "C:\scripts\Re-LitHold-OneDrive\Download-Files\ShardData-Full-"+$ShardDate+".csv"
$Global:SiteURL = "https://company.sharepoint.com/sites/"
$Global:ShardListURL = "/sites/msteams_88c7ed/ShardReportsFull/ShardData-Full-2022-03-27-03.00.11AM.csv"
$Global:shardListCSV = "C:\scripts\OneDrive\Download-Files\ShardData-Full-2022-03-27-03.00.11AM.csv"
Function Download-FileFromLibrary() {
param
(
[Parameter(Mandatory = $true)] [string] $SiteURL,
[Parameter(Mandatory = $true)] [string] $SourceFile,
[Parameter(Mandatory = $true)] [string] $TargetFile
)
Try {
#Setup Credentials to connect
$Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Global:adminUPN, $Global:adminPwd)
#Setup the context
$Ctx = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
$Ctx.Credentials = $Credentials
#sharepoint online powershell download file from library
$FileInfo = [Microsoft.SharePoint.Client.File]::OpenBinaryDirect($Ctx, $SourceFile)
$WriteStream = [System.IO.File]::Open($TargetFile, [System.IO.FileMode]::Create)
$FileInfo.Stream.CopyTo($WriteStream)
$WriteStream.Close()
Write-host -f Green "File '$SourceFile' Downloaded to '$TargetFile' Successfully!" $_.Exception.Message
}
Catch {
write-host -f Red "Error Downloading File!" $_.Exception.Message
}
}
Download-FileFromLibrary -SiteURL $Global:SiteURL -SourceFile $Global:ShardListURL -TargetFile $Global:shardListCSV
Related
Can anyone advise on how to validate credentials on a remote domain?
My environment has multiple domains that do not have trust relationships defined between them.
I have a Powershell script that needs to access a shared folder residing on a server in another domain which obviously requires authentication. Prior to accessing it, I need to validate credentials to avoid lock-outs (The script can be ran against multiple servers).
In the past I've used this wonderful script which used current domain for validation but I cannot get it to work against a remote domain.
I tried this is (slightly modified script from link above):
function Test-Cred {
[CmdletBinding()]
[OutputType([String])]
Param (
[Parameter(
Mandatory = $false,
ValueFromPipeLine = $true,
ValueFromPipelineByPropertyName = $true
)]
[Alias(
'PSCredential'
)]
[ValidateNotNull()]
[System.Management.Automation.PSCredential]
[System.Management.Automation.Credential()]
$Credentials
)
$Domain = $null
$Root = $null
$Username = $null
$Password = $null
If($Credentials -eq $null)
{
Try
{
$Credentials = Get-Credential "domain\$env:username" -ErrorAction Stop
}
Catch
{
$ErrorMsg = $_.Exception.Message
Write-Warning "Failed to validate credentials: $ErrorMsg "
Pause
Break
}
}
# Checking module
Try
{
# Split username and password
$Username = $credentials.username
$Password = $credentials.GetNetworkCredential().password
# Get Domain
###$Root = "LDAP://" + ([ADSI]'').distinguishedName
$Root = "LDAP://DC=remote_domain,DC=com" ### statically define the remote domain
$Domain = New-Object System.DirectoryServices.DirectoryEntry($Root,$UserName,$Password)
}
Catch
{
$_.Exception.Message
Continue
}
If(!$domain)
{
Write-Warning "Something went wrong"
}
Else
{
If ($domain.name -ne $null)
{
return "Authenticated"
}
Else
{
$Domain ### diagnosing the error
return "Not authenticated"
}
}
}
I get the following error:
format-default : The following exception occurred while retrieving member "distinguishedName": "The user name or
password is incorrect.
"
+ CategoryInfo : NotSpecified: (:) [format-default], ExtendedTypeSystemException
+ FullyQualifiedErrorId : CatchFromBaseGetMember,Microsoft.PowerShell.Commands.FormatDefaultCommand
The username/password is 100% correct.
Thank you
EDIT 1
I have found the following blog post that goes over how to work with Active Directory using .Net assemblies. The following has worked quite well
Add-Type -AssemblyName System.DirectoryServices.AccountManagement
#store credentials (of account with appropriate permissions)
$creds = Get-Credential
#set the domain name
$dn = 'contoso.com'
#Create the principal context object (so to say connect to a domain with the credentials provided)
$pc = [System.DirectoryServices.AccountManagement.PrincipalContext]::new([System.DirectoryServices.AccountManagement.ContextType]::`
Domain,$dn,$($creds.UserName),$($creds.GetNetworkCredential().Password))
I assume I could use this in an If statement to achieve what I need. Admittedly, I do not know the way of the .Net and it is a bit scary but I will have to learn it.
EDIT 2
Here is what I pieced together:
Function Test-Cred
{
[CmdletBinding()]
[OutputType([String])]
Param (
[Parameter(
Mandatory = $false,
ValueFromPipeLine = $true,
ValueFromPipelineByPropertyName = $true
)]
[Alias(
'PSCredential'
)]
[ValidateNotNull()]
[System.Management.Automation.PSCredential]
[System.Management.Automation.Credential()]
$Credentials
)
Add-Type -AssemblyName System.DirectoryServices.AccountManagement
# Checking module
$Validated = [System.DirectoryServices.AccountManagement.PrincipalContext]::new([System.DirectoryServices.AccountManagement.ContextType]::Domain,'remote_domain',$($Credentials.UserName),$($Credentials.GetNetworkCredential().Password))
If ($Validated.ConnectedServer)
{
Return "Authenticated"
}
Else
{
Return "Not authenticated"
}
}
Any feedback?
EDIT 3
Well, EDIT 2 does not work for Powershell 4, grrr
Method invocation failed because [System.DirectoryServices.AccountManagement.PrincipalContext] dies not contain method named 'new'
I had to make it work like this:
$ContextType = [System.DirectoryServices.AccountManagement.ContextType]::Domain
$ContextName = 'target_domain.com'
$Validated = New-Object -TypeName System.DirectoryServices.AccountManagement.PrincipalContext -ArgumentList $ContextType, $ContextName, $($Credentials.UserName),$($Credentials.GetNetworkCredential().Password)
Here is my final version of this test function that works with Powershell version older than 5.1.
Function Test-Cred
{
[CmdletBinding()]
[OutputType([String])]
Param (
[Parameter(
Mandatory = $false,
ValueFromPipeLine = $true,
ValueFromPipelineByPropertyName = $true
)]
[Alias(
'PSCredential'
)]
[ValidateNotNull()]
[System.Management.Automation.PSCredential]
[System.Management.Automation.Credential()]
$Credentials
)
# Checking module
Add-Type -AssemblyName System.DirectoryServices.AccountManagement
$ContextType = [System.DirectoryServices.AccountManagement.ContextType]::Domain
$ContextName = 'remote_domain.com'
$Validated = New-Object -TypeName System.DirectoryServices.AccountManagement.PrincipalContext -ArgumentList $ContextType, $ContextName, $($Credentials.UserName),$($Credentials.GetNetworkCredential().Password)
If ($Validated.ConnectedServer)
{
Return "Authenticated"
}
Else
{
Return "Not authenticated"
}
}
I've been reading a ton of these articles that say to use Get-SPWeb, but I've never been able to get those functions working due to authentication errors. I have build my own little functions to do what I need but I'm struggling to figure out what I'm doing wrong for my update function. Below are the functions I've built, and all of them work:
If (!$cred) {$cred = get-credential -UserName "$ENV:Username#$ENV:UserDomain.com" -Message "Enter your office 365 login"}
function Get-AuthenticationCookie($context)
{
$sharePointUri = New-Object System.Uri($context.site.Url)
$authCookie = $context.Credentials.GetAuthenticationCookie($sharePointUri)
if ($? -eq $false) #https://ss64.com/ps/syntax-automatic-variables.html
{
return $null
}
$fedAuthString = $authCookie.TrimStart("SPOIDCRL=".ToCharArray())
$cookieContainer = New-Object System.Net.CookieContainer
$cookieContainer.Add($sharePointUri, (New-Object System.Net.Cookie("SPOIDCRL", $fedAuthString)))
return $cookieContainer
}
function Get-SharepointContext
{
Param(
[Parameter(Mandatory = $true)]
$siteUrl,
[Parameter(Mandatory = $false)]
$cred)
If (!$cred) {$cred = get-credential -UserName "$ENV:Username#$env:USERDNSDOMAIN" -Message "Login"}
[string]$username = $cred.UserName
$securePassword = $cred.Password
[Void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
[Void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client.ClientContext")
$ctx = New-Object Microsoft.SharePoint.Client.ClientContext($siteUrl)
$ctx.RequestTimeOut = 1000 * 60 * 10;
$ctx.AuthenticationMode = [Microsoft.SharePoint.Client.ClientAuthenticationMode]::Default
$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $securePassword)
$ctx.Credentials = $credentials
Return $ctx
}
function Add-SharepointListEntry
{
#example
#Add-SharepointListEntry -cred $cred -clientName $DestinationPages
Param(
[Parameter(Mandatory = $true)]
$cred,
[Parameter(Mandatory = $true)]
$sitename,
$siteUrl = "https://$env:Userdomain.sharepoint.com/$sitename",
[Parameter(Mandatory = $true)]
$ListName,
$SharepointData
)
[Void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
# Bind to site collection
$Context = Get-SharepointContext -siteUrl $siteUrl -cred $cred
# Get List
$List = $Context.Web.Lists.GetByTitle($ListName)
$Context.Load($List)
$Context.ExecuteQuery()
# Create Single List Item
$ListItemCreationInformation = New-Object Microsoft.SharePoint.Client.ListItemCreationInformation
$NewListItem = $List.AddItem($ListItemCreationInformation)
#construct the entry to insert
$NewListItem["Title"] = $SharepointData.Title #Client Name
$NewListItem["Description"] = $SharepointData.Title
#These objects should pass right through
$NewListItem["Client"] = $SharepointData.Client
$NewListItem["Author"] = $SharepointData.Author
$NewListItem["Initials"] = $SharepointData.Author
$NewListItem["Created"] = $SharepointData.Created
$NewListItem.Update()
$Context.ExecuteQuery()
}
Function Get-SharepointListData
{
#example
#Get-SharepointListData -cred $cred
Param(
[Parameter(Mandatory = $true)]
$cred,
[Parameter(Mandatory = $true)]
$sitename,
$siteUrl = "https://$env:Userdomain.sharepoint.com/$sitename",
[Parameter(Mandatory = $true)]
$ListName
)
[Void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
# Bind to site collection
$Context = Get-SharepointContext -siteUrl $siteUrl -cred $cred
#Retrive the List
$List = $Context.web.Lists.GetByTitle($ListName)
#Get All List Items
#reference https://gallery.technet.microsoft.com/office/How-to-do-a-CAML-Query-6f5260cf
$Query = New-Object Microsoft.SharePoint.Client.CamlQuery
$ListItems = $List.GetItems($Query)
$context.Load($ListItems)
$context.ExecuteQuery()
# Turn item into a catch array
$ListItemCollection = #()
ForEach ($item in $ListItems)
{
$propertiesValues = New-Object PSObject
$currentItem = $item
$item.FieldValues.Keys | Where {$_ -ne "MetaInfo"} | ForEach {Add-Member -InputObject $propertiesValues -MemberType NoteProperty -Name $_ -Value $currentItem[$_]}
$ListItemCollection += $propertiesValues
Remove-Variable propertiesValues
}
Return $ListItemCollection
}
Now I'm building a new function and trying to use one list (which is querying a sharepoint folder) to update a sharepoint list. I query the directory with the get-sharepointlistdata, then loop through the results to add new entries if something is missing. This whole process works without issue. I'm trying to add a step in to update for any changes, but the function keeps failing on $list.Items.GetByID($index) throwing the error "You cannot call a method on a null-valued expression.":
Function Set-SharepointListData
{
Param(
[Parameter(Mandatory = $true)]
$cred,
[Parameter(Mandatory = $true)]
$sitename,
$siteUrl = "https://$env:userdomain.sharepoint.com/$sitename",
[Parameter(Mandatory = $true)]
$ListName,
[Parameter(Mandatory = $true)]
[int]$Index,
[Parameter(Mandatory = $true)]
$Time
)
[Void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SharePoint.Client")
# Bind to site collection
$Context = Get-SharepointContext -siteUrl $siteUrl -cred $cred
# Get List
$List = $Context.Web.Lists.GetByTitle($ListName)
$Context.Load($List)
$Context.ExecuteQuery()
# Select Single List Item
$ListItem = $List.Items.GetById($index)
$ListItem["Created"] = $time
$ListItem.Update();
$Context.ExecuteQuery();
}
I'm certain I'm overlooking something obvious here... anyone have any ideas?
The $Context.Web.Lists.GetByTitle($ListName) doesn't return the Items of the list. You have to load the Items... normally done via caml query. See here - Although the sample is in C# it should get you started.
Actually I rather suggest you to use PnPPowershell, there are plenty of cmdlets to work with Sharepoint.
While running a command to copy files from source to destination using powershell, I ran into the problem of how to only copy files which do not have any operation running on them?
If there are 3 files A/B/C, logging is still happening on A, while B and C are complete. I only want to copy B and C using powershell.
Any ideas will be helpful.
Here is something you can start with:
function Test-FileNotInUse
{
[cmdletbinding()]
param(
[parameter(Position = 0, Mandatory = $true, ValueFromPipeline = $true, ValueFromPipelineByPropertyName = $true)] [Alias('FullName')] [string] $filePath
)
process {
$inUse = $false
try {
$fileInfo = New-Object System.IO.FileInfo $filePath
$fileStream = $fileInfo.Open([System.IO.FileMode]::OpenOrCreate, [System.IO.FileAccess]::ReadWrite, [System.IO.FileShare]::None)
if ($fileStream) {
$fileStream.Close()
}
}
catch {
$inUse = $true
}
if (!$inUse) {
Write-Output $filePath
}
}
}
$source = "C:\source"
$destination = "C:\destination"
dir $source -recurse | Test-FileNotInUse | %{ copy $_ $destination }
I have an import tool that has support for running in unattended mode. It accepts arguments like this:
importer.exe -organization "DEV" -dataFile "E:\importData.txt" -rightsFile "E:\importRights.txt" -logFile "C:\LogFile.log"
Now above is how the tool itself accepts the arguments.
I'm writing a ps-script to launch the tool with the above parameters.
<#
.SYNOPSIS
Executes the PWR Bulk Data Importer tool in unattended mode.
.DESCRIPTION
This script executes the PWR Bulk Data Importer tool in unattended mode. The Import files: Data and rights must be supplied and log file must also be provided.
.PARAMETER ImportToolExe
The full path and exe of the tool.
.PARAMETER Organization
The identifier of the organization.
.PARAMETER DataFile
The full path and filename of data file.
.PARAMETER RightsFile
The full path and filename of rights file.
.PARAMETER LogFile
The full path and filename of log file (will be created and if already exists, it'll be over-writtien).
.PARAMETER IsForced
If true, tool will run in override mode omitting all deletion warnings.
.EXAMPLE
Executer -ImportToolExe "D:\tool\PWR Bulk Data Importer.exe" -Organization "VTDEV" -DataFile "E:\importData.txt" -RightsFile "E:\importRights.txt" -LogFile "C:\LogFile.log"
Executer -ImportToolExe "D:\tool\PWR Bulk Data Importer.exe" -Organization "VTDEV" -DataFile "E:\importData.txt" -RightsFile "E:\importRights.txt" -LogFile "C:\LogFile.log" -IsForced true
#>
param(
[Parameter(Mandatory = $true, Position = 0, ValueFromPipelineByPropertyName = $true)]
[string]$ImportToolExe,
[Parameter(Mandatory = $true, Position = 1, ValueFromPipelineByPropertyName = $true)]
[string]$Organization,
[Parameter(Mandatory = $true, Position = 2, ValueFromPipelineByPropertyName = $true)]
[string]$DataFile,
[Parameter(Mandatory = $true, Position = 3, ValueFromPipelineByPropertyName = $true)]
[string]$RightsFile,
[Parameter(Mandatory = $true, Position = 4, ValueFromPipelineByPropertyName = $true)]
[string]$LogFile,
[Parameter(Mandatory = $false)]
[bool]$IsForced
)
Write-Output ""
Write-Output "Script to execute Bulk Data Importer"
Write-Output ""
$params = "-organization " + $Organization + " -dataFile " + $DataFile + " -rightsFile " + $RightsFile + " -logFile " + $LogFile
Write-Output "Debuging"
Write-Output ($ImportToolExe + " " + $params)
Try
{
Write-Output " "
Write-Output "Executing..."
Invoke-Expression ($ImportToolExe + " " + $params)
Write-Output "Finished."
Write-Output "Checking exit code."
}
Catch [system.exception]
{
" "
"Exception while trying to execute"
Write-Output $_.Exception.GetType().FullName;
Write-Output $_.Exception.Message;
return
}
Finally
{
Write-Output " "
}
$IsImportSuccess = $false
IF ($lastexitcode -eq 0)
{
Write-Output "Import successful."
$IsImportSuccess = $true
}
ELSE
{
Write-Output "Import failed."
$IsImportSuccess = $false
}
IF ($IsImportSuccess -eq $true)
{
Try
{
$SMTPServer = "smtp.gmail.com"
$SMTPClient = New-Object Net.Mail.SMTPClient( $SmtpServer, 587 )
$SMTPClient.EnableSSL = $true
$SMTPClient.Credentials = New-Object System.Net.NetworkCredential( "GMAIL_USERNAME", "GMAIL_PASSWORD" );
$emailMessage = New-Object System.Net.Mail.MailMessage
$emailMessage.From = $EmailFrom
foreach ( $recipient in $Arry_EmailTo )
{
$emailMessage.To.Add( $recipient )
}
$emailMessage.Subject = $EmailSubj
$emailMessage.Body = $EmailBody
# Do we have any attachments?
# If yes, then add them, if not, do nothing
# if ( $Arry_EmailAttachments.Count -ne $NULL )
# {
# $emailMessage.Attachments.Add()
# }
$emailMessage.Attachments.Add($LogFile)
$SMTPClient.Send( $emailMessage )
}
Catch [system.exception]
{
" "
"Exception while emailing"
write-host $_.Exception.GetType().FullName;
write-host $_.Exception.Message;
return
}
}
No I'm getting output with errors:
Script to execute Bulk Data Importer
Debuging
D:\tool\PWR Bulk Data Importer.exe -organization VTDEV -dataFile E:\importData.txt -rightsFile E:\importRights.txt -logFile C:\LogFile.log
Executing...
Exception while trying to execute
System.Management.Automation.CommandNotFoundException
The term 'D:\tool\PWR' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
I'm facing a bit of learning curve with PS and had this script up by reading from SO so far.
I see the main issue where I print a debug line: All my quotes are gone already. And my manager told me that the Invoke-Expression is not a good idea. He recommends me to use Start-something
Now I'm stuck. Any pointer will be greatly appreciated and there will be upvotes as well.
Add -ErrorAction Stop inside try/catch, this should trigger desired action:
try{
...
Invoke-Expression ($ImportToolExe + " " + $params) -ErrorAction Stop
...
}
catch{
## catch code
}
You can also try Invoke-Command to start the exe so in your scenario replace Invoke-Expression with the following:
Invoke-command -ScriptBlock {&$ImportToolExe $args[0]} -ArgumentList $params
I"m able to put a file up to a remote FTP with a modified version of...
$File = "D:\Dev\somefilename.zip"
$ftp = "ftp://username:password#example.com/pub/incoming/somefilename.zip"
"ftp url: $ftp"
$webclient = New-Object System.Net.WebClient
$uri = New-Object System.Uri($ftp)
"Uploading $File..."
$webclient.UploadFile($uri, $File)
I'm running into the problem that I"m trying to upload a file to a directory that doesn't exist, the put fails. So I need to create the target directory first. GET-MEMBER doesn't seem to show any methods I can invoke to create a directory, only file operations.
I use function Create-FtpDirectory
function Create-FtpDirectory {
param(
[Parameter(Mandatory=$true)]
[string]
$sourceuri,
[Parameter(Mandatory=$true)]
[string]
$username,
[Parameter(Mandatory=$true)]
[string]
$password
)
if ($sourceUri -match '\\$|\\\w+$') { throw 'sourceuri should end with a file name' }
$ftprequest = [System.Net.FtpWebRequest]::Create($sourceuri);
$ftprequest.Method = [System.Net.WebRequestMethods+Ftp]::MakeDirectory
$ftprequest.UseBinary = $true
$ftprequest.Credentials = New-Object System.Net.NetworkCredential($username,$password)
$response = $ftprequest.GetResponse();
Write-Host Upload File Complete, status $response.StatusDescription
$response.Close();
}
Taken from Ftp.psm1 where you can find also other functions for FTP.
To others: sorry for not following well known verb-noun pattern. ;)