query on powershell script that downloads the Microsoft Ebook Giveaway books - powershell

Got a whatsapp message that Microsoft is giving away free ebooks from the below url.
URL : Microsoft Ebook Giveaway
To download all the books in one go, the following powershell script was used, which is available in the same url.
Now my problem is, if I run the powershell script as a whole, it is not throwing any error. All the books from the url gets downloaded to a single location in my computer.
But if I try to run the script line by line to understand what each statement does, it is giving the following error when the , $bookList = Invoke-WebRequest $downLoadList gets executed,
Now to resolve this error, there are many others posts in stack overflow, that passes the username and password to overcome this error. Those scripts / solutions are not working at my end.
More than the error, why is it, that the script runs without any errors/issues when I execute the full script, but throws an error when I execute line by line ?
Any inputs on the nature of execution or helpful tips in overcoming the error will be useful... Thank you.
Error :
Invoke-WebRequest : (my ip number )
Credentials are missing.
Make sure to specify a domain with your username
This website has been blocked by a cyber security policy
and SecureWeb does not currently support web exceptions
If you have an exception, copy the link below into a new tab
http://ligman.me/2tk1D2V
At line:1 char:13
+ $bookList = Invoke-WebRequest $downLoadList
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest)
[Invoke-WebRequest], WebException
+ FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.
InvokeWebRequestCommand
PowerShell Script :
###############################################################
# Eric Ligmans Amazing Free Microsoft eBook Giveaway
# https://blogs.msdn.microsoft.com/mssmallbiz/2017/07/11/largest-free-microsoft-ebook-giveaway-im-giving-away-millions-of-free-microsoft-ebooks-again-including-windows-10-office-365-office-2016-power-bi-azure-windows-8-1-office-2013-sharepo/
# Link to download list of eBooks
# http://ligman.me/2tk1D2V
# Thanks David Crosby for the template (https://social.technet.microsoft.com/profile/david%20crosby/)
#
# Modified by Robert Cain (http://arcanecode.me)
# Added code to check to see if a book was already downloaded,
# and if so was it the correct file size. If so, the book
# download is skipped. This allows users to simply rerun the
# script if their download process is interrupted.
###############################################################
# Set the folder where you want to save the books to
$dest = "I:\new_microsoft\" # Make sure the file path ends in a \
# Download the source list of books
$downLoadList = "http://ligman.me/2tk1D2V"
$bookList = Invoke-WebRequest $downLoadList
# Convert the list to an array
[string[]]$books = ""
$books = $bookList.Content.Split("`n")
# Remove the first line - it's not a book
$books = $books[1..($books.Length -1)]
$books # Here's the list
# Get the total number of books we need to download
$bookCount = $($books).Count
# Set a simple counter to let the user know what book
# number we're currently downloading
$currentBook = 0
# As an option, we can have it log progress to a file
$log = $true
if ($log -eq $true)
{
# Construct a log file name based on the date that
# we can save progress to
$dlStart = Get-Date
$dlStartDate = "$($dlStart.Year)-$($dlStart.Month)-$($dlStart.Day)"
$dlStartTime = "$($dlStart.Hour)-$($dlStart.Minute)-$($dlStart.Second)"
$logFile = "$($dest)BookDlLog-$dlStartDate-$dlStartTime.txt"
}
# Download the books
foreach ($book in $books)
{
# Increment current book number
$currentBook++
try
{
# Grab the header with the books full info
$hdr = Invoke-WebRequest $book -Method Head
# Get the title of the book from the header then
# make it a safe string (remove special characters)
$title = $hdr.BaseResponse.ResponseUri.Segments[-1]
$title = [uri]::UnescapeDataString($title)
# Construct the path to save the file to
$saveTo = $dest + $title
# If the file doesn't exist, download it
if ($(Test-Path $saveTo) -eq $false)
{
$msg = "Downloading book $currentBook of $bookCount - $title"
$msg
if ($log -eq $true) { "`n$($msg)" | Add-Content $logFile }
Invoke-WebRequest $book -OutFile $saveTo
}
else
{
# If it does exist, we need to make sure it wasn't
# a partial download. If the file size on the server
# and the file size on local disk don't match,
# redownload it
# Get the size of the file from the download site
$dlSize = $hdr.BaseResponse.ContentLength
# Get the size of the file on disk
$fileSize = $(Get-ChildItem $saveTo).Length
if ($dlSize -ne $fileSize)
{
# If not equal we need to download the book again
$msg = "Redownloading book $currentBook of $bookCount - $title"
$msg
if ($log -eq $true) { "`n$($msg)" | Add-Content $logFile }
Invoke-WebRequest $book -OutFile $saveTo
}
else
{
# Otherwise we have a good copy of the book, just
# let the user know we're skipping it.
$msg = "Book $currentBook of $bookCount ($title) already exists, skipping it"
$msg
if ($log -eq $true) { "`n$($msg)" | Add-Content $logFile }
}
}
} # end try
catch
{
$msg = "There was an error downloading $title. You may wish to try to download this book manually."
Write-Host $msg -ForegroundColor Red
if ($log -eq $true) { "`n$($msg)" | Add-Content $logFile }
} # end catch
} # end foreach
# Let user know we're done, and give a happy little beep
# in case they aren't looking at the screen.
#"Done downloading all books"
#[Console]::Beep(500,300)

Related

Issue with moving multiple items from one outlook folder to another - Powershell

I am trying to select multiple emails from on outlook inbox folder via mapi addressing and want to move a copy of these emails to another folder in the same inbox.
Unfortunately my script seems to do whatever it wants, sometimes copying 6 emails before stopping with following failure, sometimes stopping right with the first email.
Failure:
... "veeam")} | ForEach-Object {$_.Copy().Move($Namespace.Folders.Item("$ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [ForEach-Object], COMException
+ FullyQualifiedErrorId : System.Runtime.InteropServices.COMException,Microsoft.PowerShell.Commands.ForEachObjectCommand
I could not find any solution for this and I am sitting here confused since in another mailbox the code works just fine.
Of course I am setting the variables $Mailbox and $TempWorkPath beforehand.
Thanks in advance for your help.
Trying to run the code in a foreach-loop is less performant and ends with the same issue.
About 3 hours of google search did not help me at all.
Just moving the object causes the code to break, probably because of indexiation?
Add-Type -Assembly "Microsoft.Office.Interop.Outlook"
$OutlookSession = New-Object -ComObject Outlook.Application
$Namespace = $OutlookSession.GetNameSpace("MAPI")
$Namespace.Folders.Item("$Mailbox").Folders.Item("Posteingang").Items.Restrict('[UnRead] = True') | Where-Object {($_.Subject -match "ackup") -or ($_.SenderEmailAddress -match "veeam")} | ForEach-Object {$_.Copy().Move($Namespace.Folders.Item("$Mailbox").Folders.Item("Posteingang").Folders.Item("$TempWorkPath"))} | Out-Null
<# Do things with the selected/coppied emails #>
[System.Runtime.Interopservices.Marshal]::ReleaseComObject($OutlookSession) | Out-Null
$OutlookSession = $null | Out-Null
In Theory an based on my tests in another folder this should work perfectly fine, create a copy of the email, move it to my folder and afterwards I can do things with it.
Well, I think I found my way around the issue. Running the command in a while loop instead of an foreach loop seems to work better.
$Inbox = $Namespace.Folders.Item("$Mailbox").Folders.Item("Posteingang").Items.Restrict('[UnRead] = True') | Where-Object {($_.Subject -match "ackup") -or ($_.SenderEmailAddress -match "veeam")}
$MailCounter = $Inbox.Count
$HelperForCounting = 0
while ($MailCounter -gt $HelperForCounting)
{
$Inbox[$MailCounter].Copy().Move($Namespace.Folders.Item("$Mailbox").Folders.Item("Posteingang").Folders.Item("$TempWorkPath"))
$MailCounter = $MailCounter - 1
}
Greetings
I also had this issue with processing emails on Outlook. My overall scheme is to process emails folder by folder. I traced the issue to the Emails.getNext() function. My completely uneducated guess is it has something to do with parallel processing of Emails and how it grabs them in ForEach() and getNext(). The problem went away by using the getLast().
Note in the following code it will just move all read emails to archive folder and then some unread emails to corporate dump folder and most unread emails to the unread folder. This is itself just a mutation on the .p0r email script. There is a > $null at the end of the function block is where I originally had it on the ForEach loop and it worked as one would expect, but it does not work on the While loop blocking function. Instead that had to be moved to the location in the move unread section. Still a lot of room for improvement, getting some strange com errors but it will process through an inbox so long as GetLast() email is moved out of the folder.
As for my rationale on the root cause, I noticed that the failure to read a whole inbox is dependent on the size of the inbox. So each run my go through 2/3 of the remaining emails in the inbox.
# OUTLOOK RULES #
#################
# OUTLOOK RULES #
#################
#Import Object Library?
Add-Type -assembly "Microsoft.Office.Interop.Outlook"
# VARIABLES
$index=0;
$pstPath = "C:\YOURPATHHERE"
# DISPLAY INFO
function display( [string]$subject, [string]$color , [string]$out) {
# REQUIRED LENGTH OF STRING
$len = 20
# STRINGS THAT ARE LONGER WILL BE CUT DOWN,
# STRINGS THAT ARE TO SHORT WILL BE MADE LONGER
if ( $subject.length -lt 20 ){
$toadd=20-$subject.length;
for ( $i=0; $i -lt $toadd; $i++ ){
$subject=$subject+" ";
}
$len = $subject.length
}
else { $len = 20 }
$index=$index+1
Write-host -ForegroundColor $color -nonewline " |" ((($subject).ToString()).Substring(0,$len)).ToUpper()
}
# CREATING OUTLOOK OBJECT
$outlook = New-Object -comobject outlook.application
$namespace = $outlook.GetNameSpace("MAPI")
# GETTING PST FILE THAT WAS SPECIFIED BY THE PSTPATH VARIABLE
$pst = $namespace.Stores | ?{$_.FilePath -eq $pstPath}
# ROOT FOLDER
$pstRoot = $pst.GetRootFolder()
# SUBFOLDERS
$pstFolders = $pstRoot.Folders
$fArchive = $pstFolders.Item("Archive")
# PERSONAL SUBFOLDER
$personal = $pstFolders.Item("Personal")
# INBOX FOLDER
$DefaultFolder = $namespace.GetDefaultFolder(6)
# INBOX SUBFOLDERS
$InboxFolders = $DefaultFolder.Folders
# DELETED ITEMS
$DeletedItems = $namespace.GetDefaultFolder(3)
# EMAIL ITEMS
$Emails = $DefaultFolder.Items
$workingFile = [IO.Path]::GetTempFileName()
# PROCESSING EMAILS
$currentWriteFolder = $pstFolders.Item("Archive")
While ($Emails.count -gt 0) {
$Email = $Emails.GetLast()
#Move all reads into Archive
if (!$Email.Unread) {
$email.move($fArchive) > $null
continue
}
#Filter unread items by sender
$WriteString = $Email.SenderEmailAddress.ToString()
[IO.File]::WriteAllLines($workingFile, $WriteString)
if (Select-String -Path $workingFile -Pattern "company") {
$email.move($currentWriteFolder.Folders.Item("globalcorp"))
continue
}
$email.move($pstFolders.Item("Unread"))
} # > $null
[IO.File]::Delete($workingFile)
Write-host ""

email to check size / if it's complete

Running the following script:
$FileToCheck = Get-Item -Path $folder/test.zip -ErrorAction SilentlyContinue
$EmailSplat = #{
To = 'business#email.com'
CC = 'admin#email.com'
#SmtpServer = 'smtp.server.net'
From = 'my#email.com'
Priority = 'High'
}
$folder = "C:\test\"
# first condition: 'If the file does not exist, or was not created today, an e-mail should be sent that states "File not created" or similar.'
if ((-not $FileToCheck) -or ($FileToCheck.CreationTime -le (Get-Date).AddDays(-1))) {
$EmailSplat.Subject = 'File not Found or not created today'
$EmailSplat.building = 'This is the email building'
Send-MailMessage #EmailSplat
# second condition 'If the file exists and was created today, but has no content, no e-mail should be sent.'
} elseif (($FileToCheck) -and ($FileToCheck.Length -le 2)) {
#third condition and the default condition if it does not match the other conditions
} else {
$EmailSplat.Subject = 'Active Directory Accounts To Check'
$EmailSplat.building = Get-Content -Path/test.zip //maybe add the file??
Send-MailMessage #EmailSplat
}
Goal: Check if file .zip is complete, once is complete it sends an email to let business that file is good to go. I'm running the script, getting no errors but no alert emails either.
Build on: Add maybe a time that the email could be sent. For example, script will run every morning, at 6:00 the email gets sent to users to notify that the file is complete.
The $Folder variable needs to be on the line before the "$FileToCheck = Get-Item..." line since it uses that variable.
There is no such parameter "building" in the Send-MailMessage cmdlet. I think you are after Body, since you are trying to Get-Content...?
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/send-mailmessage?view=powershell-6
Another note is that Get-Content won't be able to read the contents of a zip file. You would need to unzip the file then read the file or add the file as an attachment.
This is an example of using Get-Content on an zip file that has only a text file in it:
PK
ï¸ÌN_S³ test.txtsdafasdfPK
ï¸ÌN_S³ test.txtPK 6 .
Add $ErrorActionPreference = "Stop" at the top of the script, so errors are shown.
Use the Attachments parameter to add files, building is not a valid parameter of Send-MailMessage
Get-Content is not needed, just add the path to the attachment:
$EmailSplat.Attachments = "Path/test.zip"
So something like this:
$ErrorActionPreference = "Stop"
Set-StrictMode -Version Latest
$folder = "C:\test"
$fileToCheck = Get-Item -Path (Join-Path $folder test.zip) -ErrorAction SilentlyContinue
$emailOptions = #{
"To" = "business#email.com"
"CC" = "admin#email.com"
"SmtpServer" = "smtp.server.net"
"From" = "my#email.com"
"Priority" = "High"
}
# first condition: If the file does not exist, or was not created today, an e-mail should be sent that states "File not created" or similar.
if ((-not $fileToCheck) -or ($fileToCheck.CreationTime -le (Get-Date).AddDays(-1)))
{
$emailOptions.Subject = "File not Found or not created today"
Send-MailMessage #emailOptions
}
elseif ($fileToCheck -and ($fileToCheck.Length -le 2))
{
# second condition: If the file exists and was created today, but has no content, no e-mail should be sent.
}
else
{
# third condition and the default condition if it does not match the other conditions
$emailOptions.Subject = "Active Directory Accounts To Check"
$emailOptions.Attachments = $fileToCheck.FullName
Send-MailMessage #emailOptions
}

powershell and FTP, script not waiting for transfer to complete

I have a script that, in a nutshell, does the following:
copies required files to a temporary folder
compresses the files in the temporary folder to a .zip file
FTPs the .zip file to our FTP server
tidies up and deletes the temporary folder and .zip file
I have pinched the FTP code from a previous post:
Upload files with FTP using PowerShell
and modified it where necessary (keeping the basics in tact - I think).
The issue I have is that while the .zip file is being FTP'd the script doesn't wait until it is complete. It gets part way through, anywhere from 20Mb to 60Mb before it continues executing, tidies up and deletes the file it is transferring.
The temporary folder is always the same but the .zip filename varies depending on the date so I can't really reverse the order of operations.
Can anyone suggest how I might get the script to wait until the FTP process has completed, success or fail, before it moves on?
Cheers,
Andrew.
Edit: For those that asked....
function FTPtoServer ()
{
<#
What this function has to/should do:
- accept the right number of parameters,
minimum/mandatory: username, password, file
optional: proxy server address/port, proxy username and password
- check that the source file exists, then extract the filename.
- if a proxy is specified, set the appropriate parameters
- transmit the file
- if any errors occur, throw and return
#>
param(
[string]$sourcefile=$(throw 'A sourcefile is required, -sourcefile'), <#fully qualified zip file name#>
[string]$FTPUser =$(throw 'An FTP username is required, -ftpuser'),
[string]$FTPPass =$(throw 'An FTP password is required, -ftppass'),
#
[string]$proxyServer, #proxySocket?? it is an address and port
[string]$proxyUser,
[string]$proxyPass
)
#local variables
$FTPserver = "ftp://ftp.servername.com.au"
#check if the sourcefile exists, if not return/throw an error
# The sourcefile should contain the full path to the file.
if (-not (test-path $sourcefile)){
throw "the source file could not be located: $sourcefile"
}
# extract the filename from the sourcefile.
$filename = split-path -path $sourcefile -leaf
# create the FtpWebRequest and configure it
$ftp = [System.Net.FtpWebRequest]::Create("$FTPserver/$filename")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential($FTPUser,$FTPPass)
$ftp.UseBinary = $true
$ftp.UsePassive = $false
#proxy info
# ******** DANGER Will Robinson - this proxy config has not been
# tested and may not work.
if ($proxyServer){
$proxy = New-Object System.Net.WebProxy $proxyServer
if ($proxyUser -and $proxyPass){
$proxy.Credentials = new-object System.Net.NetworkCredential($proxyUser,$proxyPass)
}
$ftp.Proxy = $proxy
$ftp.UsePassive = $true #apparently, must usePassive if using proxy
}
#now we have checked and prepared everything, lets try and send the file.
# read in the file to upload as a byte array
try{
#work out how much we are sending
$content = [System.IO.File]::ReadAllBytes("$sourceFile")
$ftp.ContentLength = $content.Length
try {
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
# be sure to clean up after ourselves
$rs.Close()
$rs.Dispose()
}
catch {
$errorMessage = "FTP failed. " + $_.exception.message
throw $errormessage
}
}
catch {
$errorMessage = "Unable to transmit file " + $sourceFile + "`r`n" + $_.exception.message
throw $errormessage
}
}
The above is in a separate file, but is called by the following:
try {
FTPtoServer -sourcefile $sourcefile -ftpuser $FTPUser -ftppass $FTPPass
}
catch {
$errorMessage = "FTPtoServer function failed with error: $_"
finishFail -failmessage $errorMessage
}
Cheers.
Found it.
I executed the FTP code above in isolation using a large file (~140Mb) and it threw the error; "The underlying connection was closed: An unexpected error occured on a receive."
I rebooted the FTP server, checked the user account etc etc.
I also tested the M$ FTP client with the same file and it transferred completely and correctly.
Anyway, I found this article: https://www.codeproject.com/Questions/597175/FileplusUploadplustoplusFTPplusserver which also has the error I received.
As it turns out, the Timeout value of FTPWebRequest is NOT -1 as in the doco but 100 seconds.
I checked my FTP logs and sure enough, time between logon and logoff was about 100 seconds.
I added the line: $ftp.Timeout = -1 to my code and first attempt transferred the entire file completely without error.
Previous transfers had worked as they fell below the 100 second timeout.
Many thanks for the posts and help.
I use an alternative oldschool method myself, it should work for you, and doesn't need any extra components on the server.
$ftp_user = "username"
$ftp_password = "password"
$ftp_address = "ftp.someserver.com"
$ftp_commands = #"
open $ftp_address
$ftp_user
$ftp_password
lcd c:\jobs\output
put estate_data_current.xml
bye
"#
set-content -encoding "ASCII" -path ftp_commands.txt -value $ftp_commands
ftp -s:ftp_commands.txt

In EWS/Powershell How Do I Remove Email Addresses From $EmailMessage.ToRecicpients?

Here's what is happening. I have this handy powershell script I've written up to connect to EWS using Impersonation, read plain-text files containing email messages, and send them one at a time in a loop. At the end there is a little Inbox and Sent Items folder clean up. (Kudos to my friends at stackoverflow, as many of the issues I already encountered in writing this have been solved by looking up the answers right here.)
Everything works great except ToRecipients.Add, which works a little _too_well_.
$EmailMessage.ToRecipients.Add($DName)
When the loop iterates to read the next file, the $EmailMessage.ToRecipients array retains it's information after executing
$EmailMessage.SendAndSaveCopy($SentItems.Id).
I had somehow expected a SendAndSaveCopy to discard all information in $EmailMessage, but it does not (Send, Save, and then Discard, right?).
So when sending one file, there is no issue whatsoever. But when sending multiple files, each subsequent file just adds its email addresses on to the end of the $EmailMessage.ToRecipients array. So the second file is sent to the first file's recipients AND the second file's recipients. The third file goes to first, second, and third file recipients. The more files, the bigger the mess.
I've tried various things at the end of the loop, before it iterates, to clear out the ToRecipients, but have not had any luck. Powershell rejects any direct assignment of any kind. I expected there to be a ToRecipients.Clear or .Empty or .Remove or something like that, but I haven't found anything that works. These approaches fail:
Remove-Variable $EmailMessage
Remove-Variable $EmailMessage.ToRecipients
Remove-Variable $EmailMessage.ToRecipients[x] #rejects all counter values
Clear-Variable $EmailMessage
Clear-Variable $EmailMessage.ToRecipients
Clear-Variable $EmailMessage.ToRecipients[x] #rejects all counter values
$EmailMessage.ToRecipients = NULL
$EmailMessage.ToRecipients = ""
I can loop through and read every value, get a count of how many ToRecipients there are, print them out, and all sorts of other nifty things, but it seems to be read-only and I haven't found any way to clear them out before iterating.
I have thought of a couple "band-aid" approaches, which I haven't tried yet:
1. Put it in a function with local variables, and $EmailMessage will poof on the function exit (with any luck).
2. Just process one file, and at the end of the script right before exiting, check to see if any files remain to process, and call the script again using Invoke-Expression.
Really, this seems like overkill, and a work around to the original problem rather than a direct fix. I should just be able to clear out the email addresses before the loop iterates, right?
Here is the full code:
<#
Script Overview
1. Opens EWS services for shared mailbox using an appid
2. Reads files from c:\outbox\ containing email to send, one per file
3. Sends the mail message in each file found
4. Renames each file found
5. Moves sent items into c:\sentemail\
6. Deletes any emails in the Inbox or Sent Items folders that are older than 14 days ($purgebeforedate))
#>
#Variables
## Define UPN of the Account that has impersonation rights
$AccountWithImpersonationRights = "myappid"
$appidpasswd = "mysupersecretpassword"
##Define the SMTP Address of the mailbox to impersonate
$MailboxToImpersonate = "autoemail#mysite.mydomain"
##Define CAS URL (Client Access Server) - can be found in system registry if necessary
$CASURL = "https://my.site.and.domain/ews/exchange.asmx"
$ewsApiDownload = "http://www.microsoft.com/en-us/download/details.aspx?id=35371"
## Define Exchange web services DLL path (requires 2.0 for our environment)
$dllpath = "C:\Program Files\Microsoft\Exchange\Web Services\2.0\Microsoft.Exchange.WebServices.dll"
##Define folders for incoming and archived email files
$GetFolder = "c:\outbox\" #folder where input email message files are in plaintext
$SaveFolder = "c:\sentemail\" #folder to move sent email files to after processed
$PurgeBeforeDate = (Get-Date (Get-Date).AddDays(-14) -format G) #format m/d/yyyy hh:mm:ss ap- ::Date is 14 days ago [AddDays(-14)]
##Define location of error log on the local windows server where this script runs
$errorLog = "c:\sentemail\MyAutoSentEmail.log"
# Post a note in the error log that the script has started
$date = Get-Date -format G #Default date and time output used in log file.
Add-Content $errorLog $date": Email Service script started."
Add-Content $errorLog $date": Old sent email delete cutoff - $PurgeBeforeDate"
## Load Exchange web services DLL
Import-Module $dllpath
#Exit script if importing EWS API fails.
if ($? -eq $false)
{
$date = Get-Date -format s #Default date and time output used in log file. (was G before)
Add-Content $errorLog $date": Faied to load EWS, ensure it is installed:"$ewsApiDownload
Add-Content $errorLog $date": EWS API expected location:" $dllpath
Add-Content $errorLog $date": Failed to import the EWS API. Script terminated."
exit
}
## Set Exchange Version (our site requires Exchange2010_SP2)
$ExchangeVersion = [Microsoft.Exchange.WebServices.Data.ExchangeVersion]::Exchange2010_SP2
## Create Exchange Service Object
$service = New-Object Microsoft.Exchange.WebServices.Data.ExchangeService($ExchangeVersion)
#Get valid Credentials using UPN for the ID that is used to impersonate mailbox
$service.Credentials = New-Object System.Net.NetworkCredential($AccountWithImpersonationRights, $appidpasswd);
## Set the URL of the CAS (Client Access Server)
$service.Url = New-Object Uri($CASURL)
##Login to Mailbox with Impersonation
#Write-Host 'Using ' $AccountWithImpersonationRights ' to Impersonate ' $MailboxToImpersonate
$service.ImpersonatedUserId = New-Object Microsoft.Exchange.WebServices.Data.ImpersonatedUserId([Microsoft.Exchange.WebServices.Data.ConnectingIdType]::SmtpAddress,$MailboxToImpersonate );
#Connect to the Inbox and display basic statistics
$SentFolder = new-object Microsoft.Exchange.WebServices.Data.FolderId([Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::SentItems,$ImpersonatedMailboxName)
$SentItems = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service,$SentFolder)
if ($? -eq $false) #Exit script if binding to folder fails.
{
Add-Content $errorLog $date": Failed to bind to the specified mailbox folder. Script terminated."
exit
}
#For info only, can uncomment to get a list of how many items are in the Inbox and how many are unread
#Write-Host 'Total Item count for Inbox:' $Inbox.TotalCount
#Write-Host 'Total Items Unread:' $Inbox.UnreadCount
#get the filenames in the outgoing-email directory into $files so they may be read one at a time
$files = Get-ChildItem $GetFolder
if ($files.Count -gt 0)
{ #if there are files to process
#Create new email message to send out
$EmailMessage = New-Object Microsoft.Exchange.WebServices.Data.EmailMessage -ArgumentList $service
$EmailMessage.Body = New-Object Microsoft.Exchange.WebServices.Data.MessageBody
$EmailMessage.Body.BodyType = [Microsoft.Exchange.WebServices.Data.BodyType]::HTML
$EmailMessage.From = $MailboxToImpersonate
for ($i=0; $i -lt $files.Count; $i++) { #start for $i loop for each filename found
$emailfile = Get-Content $files[$i].FullName #read file into $emailfile
$To = "empty" #init $To to empty
$Subject = "empty" #init $Subject to empty
$Body = "empty" #init $Body to empty
$Bodytext = "" #init $Bodytext - place to accumulate body lines before adding to email message, gets html return on end of each line
$Bodyhtml = "" #init $Bodyhtml - body lines are placed here in parallel, no html carriage return added
foreach ($Data in $emailfile) { #put each line of the file into $Data and process it
if ( ($Data.StartsWith("to:")) -and ($Subject -eq "empty") )
{ #add recipients to the email message, all to: lines must be before the subject: line
$To = "found" #mark $to line(s) as having been found
Write-Host "TO: line found - " $Data
$Data = $Data.Substring(3) #get all chars after subject:
if ($Data.Contains(";"))
{ #if the email address passed in contains multiples separated by semicolons, split and add them
$DataNames = $Data -split ';' #split $Data into an array of semicolon separated substrings
foreach ($DName in $DataNames) { #validate each semicolon separated email address and add it
$EmailMessage.ToRecipients.Add($DName)
Write-Host $date": TO: address added: "$DName
} #end foreach $DName in $DataNames
} else { #else if no semicolon, just add what you have
$EmailMessage.ToRecipients.Add($Data)
Write-Host $date": TO: address added: "$DName
} #end if email address passed doesn't contain a semicolon
} #end add recipients to $To
if ( ($Data.StartsWith("subject:")) -and ($Body -eq "empty") )
{ #add subject line to the email message, must come before any body: line(s)
$Data = $Data.Substring(8) #get all chars after subject:
$EmailMessage.Subject = $Data
$Subject = "found" #mark $subject line as having been found
Write-Host "SUBJECT: line found - " $Data
} #end add recipients to $To
if ($Data.StartsWith("body:"))
{ #first line of body found
$Data = $Data.Substring(5) #get all chars after subject:
$Bodytext = $Data
$Bodyhtml = $Data
$Body = "found" #mark $body line as having been found
Write-Host "BODY: line found - " $Data
} #end add recipients to $To
elseif ($Body -eq "found")
{ #accumulate the remaining lines into the $bodytext string with '<br />' as carriage returns between lines
$Bodytext = -join($Bodytext, '<br />', $Data); #text lines get an html carriage return
$Bodyhtml = -join($Bodyhtml, $Data); #html lines don't get an html carriage return added, shouldnt need it
Write-Host "body data found - " $Data
} #end accumulate the remaining lines into the $bodytext string with '<br />' as carriage returns between lines
} #end foreach - done processing this text file
$CurrentDateAndTime = $(get-date -f yyyy-MM-dd-HH-mm-ss) #date format to add to the front of the filename when moving/renaming
$OldFileName = $files[$i].FullName
$NewFileName = [io.path]::GetFileName($OldFileName) #just the filename without the path for rename later
Write-Host "OldFileName - " $OldFileName
if ( ($To -eq "found") -and ($Subject -eq "found") -and ($Body -eq "found") )
{ #if all parts found, add body text to the new email message and send it off
if ( ( $Bodyhtml -Match "<" ) -and ( $Bodyhtml -Match ">" ) ) {
$EmailMessage.Body.Text = $Bodyhtml } else {
$EmailMessage.Body.Text = $Bodytext
} #end if html set body.text to html, otherwise set to bodytext with '<br />' crlf's added
$EmailMessage.SendAndSaveCopy($SentItems.Id)
Add-Content $files[$i].FullName "EMAIL-SENT-AT: $CurrentDateAndTime"
Add-Content $errorLog $date": Email File Sent Successfully:"$OldFileName
$NewFileName = -join($CurrentDateAndTime, "-$i-", $NewFileName)
} Else { #add error messages to output file, and select a name that includes NOTSENT so it is obvious
Add-Content $errorLog $date": NOTSENT-ERROR in file: "$OldFileName
if ( $To -eq "empty") {
Add-Content $files[$i].FullName "NOTSENT-ERROR: NO TO: LINE FOUND"
Add-Content $errorLog $date": NOTSENT-ERROR: NO TO: LINE FOUND"
}
if ( $Subject -eq "empty") {
Add-Content $files[$i].FullName "NOTSENT-ERROR: NO SUBJECT: LINE FOUND"
Add-Content $errorLog $date": NOTSENT-ERROR: NO SUBJECT: LINE FOUND"
}
if ( $Body -eq "empty") {
Add-Content $files[$i].FullName "NOTSENT-ERROR: NO BODY: LINE FOUND"
Add-Content $errorLog $date": NOTSENT-ERROR: NO BODY: LINE FOUND"
}
$EmailMessage.Delete #delete new composition if it is not set up properly
$NewFileName = -join($CurrentDateAndTime, "-$i-NOTSENT-", $NewFileName) #note in filename if email was not sent
} #end send-or-not-to-send if
Write-Host "NewFileName - " $NewFileName
#rename and move the file with the date and time it was sent (or tried to be sent)
$NewFileName = -join($SaveFolder, $NewFileName)
Move-Item $OldFileName $NewFileName
$EmailMessage.ToRecipients.Clear() ##BenH - clears ToRecipients before next file is processed so they don't accumulate
} #end for $i loop for each filename found
} #end if there are files to process
if ($? -eq $false)
{
Add-Content error.log $date": error sending message. Script terminated."
exit
}
#Delete items from the Sent Items folder that are too old
#Get the ID of the folder to move to, by searching up from the mailbox root folder
$RootFolder = [Microsoft.Exchange.WebServices.Data.Folder]::Bind($service, [Microsoft.Exchange.WebServices.Data.WellKnownFolderName]::MsgFolderRoot)
$fvFolderView = New-Object Microsoft.Exchange.WebServices.Data.FolderView(1)
$fvFolderView.Traversal = [Microsoft.Exchange.WebServices.Data.FolderTraversal]::Deep
$SfSearchFilter = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsEqualTo([Microsoft.Exchange.WebServices.Data.FolderSchema]::DisplayName,"Sent Items")
$findFolderResults = $RootFolder.FindFolders($SfSearchFilter,$fvFolderView) #search from mailbox root folder for a folder named Sent Items
if ($? -eq $false)
{
#Write-Host 'Unable to locate destination folder:' $findFolderResults
Add-Content error.log $date": Unable to locate the Sent Items folder for cleanup. Script terminated."
exit
}
$sentitemsfolder = $findFolderResults.Folders[0] #save reference to the Sent Items folder in $sentitemsfolder
$SfSearchFilterI = new-object Microsoft.Exchange.WebServices.Data.SearchFilter+IsEqualTo([Microsoft.Exchange.WebServices.Data.FolderSchema]::DisplayName,"Inbox")
$findFolderResults = $RootFolder.FindFolders($SfSearchFilterI,$fvFolderView) #search from mailbox root folder for a folder named Inbox
if ($? -eq $false)
{
#Write-Host 'Unable to locate destination folder:' $findFolderResults
Add-Content error.log $date": Unable to locate the Inbox folder for cleanup. Script terminated."
exit
}
$inboxfolder = $findFolderResults.Folders[0] #save reference to the Inbox folder in $inboxfolder
#search through the Sent Items and Inbox folders for items older than $purgebeforedate and soft delete them
$puItemView = New-Object Microsoft.Exchange.WebServices.Data.ItemView(500, 0, [Microsoft.Exchange.WebServices.Data.OffsetBasePoint]::Beginning)
$puItemView.PropertySet = New-Object Microsoft.Exchange.WebServices.Data.PropertySet([Microsoft.Exchange.WebServices.Data.BasePropertySet]::IdOnly, [Microsoft.Exchange.WebServices.Data.ItemSchema]::Subject, [Microsoft.Exchange.WebServices.Data.ItemSchema]::DateTimeReceived)
$puItemView.Traversal = [Microsoft.Exchange.WebServices.Data.ItemTraversal]::Shallow
#Sent Items scan for old items to delete
$puItems = $null #init FindItems results to null before executing find
do
{ #start do
$puItems = $service.FindItems($sentitemsfolder.Id,$puItemView) #find all items in Sent Items
if ($puItems.Items.Count -gt 0)
{ #if Sent Items folder not empty, inspect all items found
foreach($Item in $puItems.Items)
{
if ($item.datetimereceived -le $PurgeBeforeDate) #compare email date to purge cutoff date
{
#Write-Host "Delete Item with date time received" $item.DateTimeReceived
[void]$item.Delete([Microsoft.Exchange.WebServices.Data.DeleteMode]::SoftDelete)
#Note: exchange email internal date is in format of : 5/31/2016 8:02:19 AM (get-date -format G)
} #end compare email date to purge cutoff date
} #end foreach $Item in $puItems.items
} #end if Sent Items folder not empty
$puItemView.Offset += $puItems.Items.Count
} #end do
while($puItems.MoreAvailable -eq $true)
#Inbox scan for old items to delete
$puItems = $null #init FindItems results to null before executing find
do
{ #start do
$puItems = $service.FindItems($inboxfolder.Id,$puItemView) #find all items in Sent Items
if ($puItems.Items.Count -gt 0)
{ #if Sent Items folder not empty, inspect all items found
foreach($Item in $puItems.Items)
{
if ($item.datetimereceived -le $PurgeBeforeDate) #compare email date to purge cutoff date
{
#Write-Host "Delete Item with date time received" $item.DateTimeReceived
[void]$item.Delete([Microsoft.Exchange.WebServices.Data.DeleteMode]::SoftDelete)
#Note: exchange email internal date is in format of : 5/31/2016 8:02:19 AM (get-date -format G)
} #end compare email date to purge cutoff date
} #end foreach $Item in $puItems.items
} #end if Sent Items folder not empty
$puItemView.Offset += $puItems.Items.Count
} #end do
while($puItems.MoreAvailable -eq $true)
#script done
Add-Content $errorLog $date": Email Service script finished."
exit
##############################################################################
And here are some sample email input files, where lines start with to:, subject:, and the first line of the message body starts with body:
Sample File 1:
to:sam.lemon#lemonade.org
subject:ews mail test
body:testing line 1
testing line 2
testing line 3
testing line 4
done testing
Sample File 2:
to:bill.hickock#wild.west.org
subject:ews mail test
body:testing line 1
testing line 2
testing line 3
testing line 4
done testing
With these two sample files, the first one would be sent to sam.lemon#lemonade.org as expected. But the second one would go to both sam.lemon#lemonade.org and bill.hickock#wild.west.org, which it should not, since the second file is only for bill.hickock#wild.west.org.
Thanks in advance!
Zistrosk
#BenH - Here's what I coded and what came out in white on the Powershell ISE console. No errors or warnings, just these info messages, and the email addressees were not cleared or removed, same symptoms of the problem persisted. Does my coding look right?
$EmailMessage.ToRecipients.Clear
MemberType : Method
OverloadDefinitions : {void Clear()}
TypeNameOfValue : System.Management.Automation.PSMethod
Value : void Clear()
Name : Clear
IsInstance : True
$EmailMessage.ToRecipients.Remove
MemberType : Method
OverloadDefinitions : {bool
Remove(Microsoft.Exchange.WebServices.Data.EmailAddress
emailAddress)}
TypeNameOfValue : System.Management.Automation.PSMethod
Value : bool
Remove(Microsoft.Exchange.WebServices.Data.EmailAddress
emailAddress)
Name : Remove
IsInstance : True
Note: added the following line above, thanks to #BenH, which fixes the issue and clears the ToRecipients array before the loop iterates and the next file is read, preventing an unwanted accumulation of TO addresses:
$EmailMessage.ToRecipients.Clear() ##BenH - clears ToRecipients before next file is processed so they don't accumulate

Layering multiple 'ForEach' statements to loop through a directory and perform actions on each file found

I've been working on this Powershell script for a good week now, and it almost works as expected.
Essentially, the script reaches into the specified directory which we have another script dropping .CSV files into, grabs the .CSV file(s) and pushes the information found into a Sharepoint list, well, that's the intention anyway. I've gotten the script to work perfectly if I manually specify the file, the issue I am having is actually getting all the .CSV files into a group, and then looping through each .CSV to pull the information out and push it into a Sharepoint list. Once done, it renames the file from .CSV to .ARCHIVED for another script to come in and re-locate after we're done with it.
I think I have, through selective (creative) troubleshooting, figured out what I am doing wrong, I just don't know how to proceed after identifying the issue.
I declare the string $Filecsv like so:
$Filecsv = get-childitem "Z:\" -recurse | where {$_.extension -eq ".csv"}
So, this reaches into my 'Z:\' directory, and pulls all the files with .CSV extension and combines them into a table...
ForEach ($items in $Filecsv) {
And this says for each item, perform logic...
foreach($row in $Filecsv)
The only problem is, when I call $Filecsv, it is returning the list of each .CSV file in the directory like such:
And as such, when I execute the bit of code that says 'put the information into my list', only the file name is added to my Sharepoint list....
Now, I can see what's going on here, it's pulling the 'Name' from the $Filecsv table, and pushing that up to Sharepoint, however, I am not sure how to re-construct my logic so that it operates as expected because as it exists now, it should (to me anyway) work as I think it does, but I am still new to Sharepoint and am certainly missing something here.
Below, is the full code, if it helps:
# Add SharePoint PowerShell Snapin which adds SharePoint specific cmdlets
Add-PSSnapin Microsoft.SharePoint.PowerShell -EA SilentlyContinue
#start the counter at 1 to track times script has looped
$iterations = 1
# set the location where the .CSV files will be pulled from and define the
# file extension we are concerned with
$filecsv = get-childitem "Z:\" -recurse | where {$_.extension -eq ".csv"}
# for each file found in the directory
ForEach ($items in $Filecsv) {
# check to see if files exist, if not exit cleanly
if ($Filecsv) {"File exists" + $Filecsv} else {exit}
# count the times we've looped through
"Iterations : $iterations"
# specify variables needed. The webURL should be the site URL, not including the list
# the listName should be the list name
$WebURL = "http://SHAREPOINTURL/"
$listName = "test"
# Get the SPWeb object and save it to a variable
$web = Get-SPWeb -identity $WebURL
# Get the SPList object to retrieve the list
$list = $web.Lists[$listName]
# START deletes all items. code shows the number of items in a list, then deletes all items
# If you don't want your script to delete items, then remove this
$site = new-object Microsoft.SharePoint.SPSite ( $WebURL )
$web = $site.OpenWeb()
"Web is : " + $web.Title
# Enter name of the List below instead of
$oList = $web.Lists["test"];
"List is :" + $oList.Title
"List Item Count: " + $oList.ItemCount
#delete existing contents and replace with new stuff
$collListItems = $oList.Items;
$count = $collListItems.Count - 1
for($intIndex = $count; $intIndex -gt -1; $intIndex--) {
"Deleting record: " + $intIndex
$collListItems.Delete($intIndex);
}
# END Deletes all items
# goes through the CSV file and performs action for each row
foreach($row in $Filecsv)
{
$newItem = $list.items.Add()
$item = $list.items.add()
# Check if cell value is not null in excel
if ($row."Name" -ne $null)
# Add item to sharepoint list. for this one, I had to use the internal column name.
#You don't always have to, but I had trouble with one SharePoint column, so I did
{$newItem["Name"] = $row."Name"}
else{$newItem["Name"] = $row."Not Provided"}
if ($row."Description" -ne $null)
{$newItem["Description"] = $row."Description"}
else{$newItem["Description"] = $row."No Description"}
if ($row."NetworkID" -ne $null)
{$newItem["Network ID"] = $row."NetworkID"}
else{$newItem["Network ID"] = $row."No NetworkID"}
if ($row."Nested" -ne $null)
{$newItem["Nested"] = $row."Nested"}
else{$newItem["Nested"] = $row."Not Nested"}
# Commit the update, then loop again until end of file
$newItem.Update()
}
# get the date and time from the system
$datetime = get-date -f MMddyy-hhmmtt
# rename the file
$NewName = $items.fullname -replace ".csv$","$datetime.csv.archived"
$Items.MoveTo($NewName)
# +1 the counter to count the number of files we've looped through
$iterations ++
}
a very cursory look would suggest that you need to use $items not $filecsv in your main loop.
essentially you are looping over the contents of the $filecsv collection, so you need to look at $items.
Your ForEach loops look redundant since they are both looping through a list of FileInfo objects. I think you want to find all the files, and for each file load it into memory and process it's contents. We'll go that route.
I have moved your SharePoint object creation out of the loop since I don't see any point to creating the object over and over for each file processed since it never references anything based on the file or it's contents. It simply makes the same object over, and over, and over.
# Add SharePoint PowerShell Snapin which adds SharePoint specific cmdlets
Add-PSSnapin Microsoft.SharePoint.PowerShell -EA SilentlyContinue
#start the counter at 1 to track times script has looped
$iterations = 1
# specify variables needed. The webURL should be the site URL, not including the list
# the listName should be the list name
#Setup SP object
$WebURL = "http://SHAREPOINTURL/"
$listName = "test"
# Get the SPWeb object and save it to a variable
$web = Get-SPWeb -identity $WebURL
# Get the SPList object to retrieve the list
$list = $web.Lists[$listName]
# START deletes all items. code shows the number of items in a list, then deletes all items
# If you don't want your script to delete items, then remove this
$site = new-object Microsoft.SharePoint.SPSite ( $WebURL )
$web = $site.OpenWeb()
"Web is : " + $web.Title
# Enter name of the List below instead of
$oList = $web.Lists["test"];
"List is : " + $oList.Title
"List Item Count: " + $oList.ItemCount
#delete existing contents and replace with new stuff
$collListItems = $oList.Items;
$count = $collListItems.Count - 1
for($intIndex = $count; $intIndex -gt -1; $intIndex--) {
"Deleting record: " + $intIndex
$collListItems.Delete($intIndex);
}
# END Deletes all items
Find all the CSV files, and start looping through the list of them. I removed the check to see if the file exists. You just pulled a directory listing to find these files, they really should exist.
# set the location where the .CSV files will be pulled from and define the
# file extension we are concerned with
$CSVList = get-childitem "Z:\" -recurse | where {$_.extension -eq ".csv"}
ForEach ($CSVFile in $CSVList) {
# count the times we've looped through
"Iterations : $iterations"
Now, this is different. It loads the CSV file, and processes each row in it as $row. I'm pretty sure this is what you intended to do from the start. I also changed it from If(Something -ne $null) to check for either null, or empty since either can actually exist and the later can cause you some issues. It's just a safer method in general.
foreach($row in (Import-CSV $CSVFile.FullName))
{
$newItem = $list.items.Add()
$item = $list.items.add()
# Check if cell value is not null in excel
if (![string]::IsNullOrEmpty($row."Name"))
# Add item to sharepoint list. for this one, I had to use the internal column name.
#You don't always have to, but I had trouble with one SharePoint column, so I did
{$newItem["Name"] = $row."Name"}
else{$newItem["Name"] = $row."Not Provided"}
if (![string]::IsNullOrEmpty($row."Description"))
{$newItem["Description"] = $row."Description"}
else{$newItem["Description"] = $row."No Description"}
if (![string]::IsNullOrEmpty($row."NetworkID"))
{$newItem["Network ID"] = $row."NetworkID"}
else{$newItem["Network ID"] = $row."No NetworkID"}
if (![string]::IsNullOrEmpty($row."Nested"))
{$newItem["Nested"] = $row."Nested"}
else{$newItem["Nested"] = $row."Not Nested"}
# Commit the update, then loop again until end of file
$newItem.Update()
}
I don't really understand why you are adding a new item twice, but if it works then more power to you. Then your bit to rename files when you're done with them (hey, this looks familiar):
# get the date and time from the system
$datetime = get-date -f MMddyy-hhmmtt
# rename the file
$NewName = $CSVFile.fullname -replace ".csv$","$datetime.csv.archived"
$CSVFile.MoveTo($NewName)
# +1 the counter to count the number of files we've looped through
$iterations ++
}
I did rename a few things to make them more indicative of what they represent ($Items to $CSVFile and what not). See if this works for you. If you have questions or concerns let me know.
Edit: Ok, to fix the loop trying to pull each item from the current folder we reference the FullName property of it. One line changed:
foreach($row in (Import-CSV $CSVFile.FullName))