I have a Google sheet where I have published a CSV to the web.
I am trying to use PowerShell to download the CSV.
After the CSV downloads I log the event in a the log file with datestamp.
I use Windows Task Scheduler to do this daily.
I have tried to use the code below, and it seems to work if I copy/paste it into PowerShell, but when I double-click to run it, I get errors.
$client = New-Object System.Net.WebClient
$client.DownloadFile(“https://docs.google.com/spreadsheets/d/e/2xxxxxxxxxxxxxxxxxxxxxxxx/pub?gid=xxxx&single=true&output=csv”,“C:\Users\xxxxxxxxxx\sent_ordes.txt")
function Get-TimeStamp {
return "[{0:MM/dd/yy} {0:HH:mm:ss}]" -f (Get-Date)
}
Write-Output "$(Get-TimeStamp) Sent Orders Powershell run " | Out-File
C:\Users\xxxxxxxxxxxxxx\Sent_Orders.log -Append
Related
I am working on a PowerShell script, which will pull files from an FTP site. The files are uploaded to the FTP site every hour so I need to download the most recent one. The code I currently have downloads all the files from today instead of just one file. How do I make it download only the most recent file?
Here is the code that I am currently using
$ftpPath = 'ftp://***.***.*.*'
$ftpUser = '******'
$ftpPass = '******'
$localPath = 'C:\Temp'
$Date = get-date -Format "ddMMyyyy"
$Files = 'File1', 'File2'
function Get-FtpDir ($url, $credentials)
{
$request = [Net.FtpWebRequest]::Create($url)
if ($credentials) { $request.Credentials = $credentials }
$request.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
(New-Object IO.StreamReader $request.GetResponse().GetResponseStream()) -split "`r`n"
}
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($ftpUser,$ftpPass)
$webclient.BaseAddress = $ftpPath
Foreach ( $item in $Files )
{
Get-FTPDir $ftpPath $webclient.Credentials |
? { $_ -Like $item+$Date+'*' } |
% {
$webClient.DownloadFile($_, (Join-Path $localPath $_))
}
}
It's not easy with the FtpWebRequest. For your task, you need to know file timestamps.
Unfortunately, there's no really reliable and efficient way to retrieve timestamps using features offered by FtpWebRequest/.NET framework/PowerShell as they do not support an FTP MLSD command. The MLSD command provides listing of remote directory in a standardized machine-readable format. The command and the format is standardized by RFC 3659.
Alternatives which you can use, that are supported by .NET framework:
ListDirectoryDetails method (an FTP LIST command) to retrieve details of all files in a directory and then you deal with FTP server specific format of the details (*nix format similar to ls *nix command is the most common, drawback is that the format may change over time, as for newer files "May 8 17:48" format is used and for older files "Oct 18 2009" format is used)
GetDateTimestamp method (an FTP MDTM command) to individually retrieve timestamps for each file. Advantage is that the response is standardized by RFC 3659 to YYYYMMDDHHMMSS[.sss]. Disadvantage is that you have to send a separate request for each file, what can be quite inefficient.
Some references:
C# class to parse WebRequestMethods.Ftp.ListDirectoryDetails FTP response
Parsing FtpWebRequest ListDirectoryDetails line
Retrieving creation date of file (FTP)
Alternatively, use a 3rd party FTP library that supports the MLSD command, and/or supports parsing of the proprietary listing format.
For example WinSCP .NET assembly supports both.
An example code:
# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Ftp
HostName = "example.com"
UserName = "user"
Password = "mypassword"
}
$session = New-Object WinSCP.Session
# Connect
$session.Open($sessionOptions)
# Get list of files in the directory
$directoryInfo = $session.ListDirectory($remotePath)
# Select the most recent file
$latest =
$directoryInfo.Files |
Where-Object { -Not $_.IsDirectory } |
Sort-Object LastWriteTime -Descending |
Select-Object -First 1
# Any file at all?
if ($latest -eq $Null)
{
Write-Host "No file found"
exit 1
}
# Download the selected file
$sourcePath = [WinSCP.RemotePath]::EscapeFileMask($remotePath + $latest.Name)
$session.GetFiles($sourcePath, $localPath).Check()
For a full code, see Downloading the most recent file (PowerShell).
(I'm the author of WinSCP)
I tried this, but i get an error:
Error: Exception calling "ListDirectory" with "1" argument(s): "Error listing directory '/path/'.
Could not retrieve directory listing
Can't open data connection for transfer of "/path/"
I read a lot about this problem on the internet, but could not find a solution which seemed fairly simple, and I am not a network setup wizard. So I choose a different approach. In our case the filename of the file which I want to automate the download for, has the date specified in it: backup_2018_08_03_020003_1048387.bak
So we can get the file by using mget *2018_08_03* in a command line ftp session.
Our backup procedure is run every morning at 01.00 AM, so we have a backup each day that we can fetch.
Of course it would have been prettier and nicer to have a script that fetched the latest backup file based on the backup file timestamps, just in case that something went wrong with the latest backup or the backup file naming format changes. The script is just a script to fetch the backup for internal development purposes so its not a big deal if it breaks. I will look into this later and check whether i can make a cleaner solution.
I made a batch script which just asks for todays backup file with the ordinary ftp command prompt scripting.
It is important to get the formatting of todays date right. It must match the formatting of the date in the filename correctly.
If you want to use the script you should replace the variables with your own information. You should also have write access to the directory where you run it from.
This is the script that I made:
#Echo Off
Set _FTPServerName=xxx.xxx.xx.xxx
Set _UserName=Username
Set _Password=Password
Set _LocalFolder=C:\Temp
Set _RemoteFolder="/path/"
Set _Filename=*%date:~-4,4%_%date:~-7,2%_%date:~-10,2%*
Set _ScriptFile=ftptempscript
:: Create script
>"%_ScriptFile%" Echo open %_FTPServerName%
>>"%_ScriptFile%" Echo %_UserName%
>>"%_ScriptFile%" Echo %_Password%
>>"%_ScriptFile%" Echo lcd %_LocalFolder%
>>"%_ScriptFile%" Echo cd %_RemoteFolder%
>>"%_ScriptFile%" Echo binary
>>"%_ScriptFile%" Echo mget -i %_Filename%
>>"%_ScriptFile%" Echo quit
:: Run script
ftp -s:"%_ScriptFile%"
del "%_ScriptFile%"
I am working on a PowerShell script, which will pull files from an FTP site. The files are uploaded to the FTP site every hour so I need to download the most recent one. The code I currently have downloads all the files from today instead of just one file. How do I make it download only the most recent file?
Here is the code that I am currently using
$ftpPath = 'ftp://***.***.*.*'
$ftpUser = '******'
$ftpPass = '******'
$localPath = 'C:\Temp'
$Date = get-date -Format "ddMMyyyy"
$Files = 'File1', 'File2'
function Get-FtpDir ($url, $credentials)
{
$request = [Net.FtpWebRequest]::Create($url)
if ($credentials) { $request.Credentials = $credentials }
$request.Method = [System.Net.WebRequestMethods+FTP]::ListDirectory
(New-Object IO.StreamReader $request.GetResponse().GetResponseStream()) -split "`r`n"
}
$webclient = New-Object System.Net.WebClient
$webclient.Credentials = New-Object System.Net.NetworkCredential($ftpUser,$ftpPass)
$webclient.BaseAddress = $ftpPath
Foreach ( $item in $Files )
{
Get-FTPDir $ftpPath $webclient.Credentials |
? { $_ -Like $item+$Date+'*' } |
% {
$webClient.DownloadFile($_, (Join-Path $localPath $_))
}
}
It's not easy with the FtpWebRequest. For your task, you need to know file timestamps.
Unfortunately, there's no really reliable and efficient way to retrieve timestamps using features offered by FtpWebRequest/.NET framework/PowerShell as they do not support an FTP MLSD command. The MLSD command provides listing of remote directory in a standardized machine-readable format. The command and the format is standardized by RFC 3659.
Alternatives which you can use, that are supported by .NET framework:
ListDirectoryDetails method (an FTP LIST command) to retrieve details of all files in a directory and then you deal with FTP server specific format of the details (*nix format similar to ls *nix command is the most common, drawback is that the format may change over time, as for newer files "May 8 17:48" format is used and for older files "Oct 18 2009" format is used)
GetDateTimestamp method (an FTP MDTM command) to individually retrieve timestamps for each file. Advantage is that the response is standardized by RFC 3659 to YYYYMMDDHHMMSS[.sss]. Disadvantage is that you have to send a separate request for each file, what can be quite inefficient.
Some references:
C# class to parse WebRequestMethods.Ftp.ListDirectoryDetails FTP response
Parsing FtpWebRequest ListDirectoryDetails line
Retrieving creation date of file (FTP)
Alternatively, use a 3rd party FTP library that supports the MLSD command, and/or supports parsing of the proprietary listing format.
For example WinSCP .NET assembly supports both.
An example code:
# Load WinSCP .NET assembly
Add-Type -Path "WinSCPnet.dll"
# Setup session options
$sessionOptions = New-Object WinSCP.SessionOptions -Property #{
Protocol = [WinSCP.Protocol]::Ftp
HostName = "example.com"
UserName = "user"
Password = "mypassword"
}
$session = New-Object WinSCP.Session
# Connect
$session.Open($sessionOptions)
# Get list of files in the directory
$directoryInfo = $session.ListDirectory($remotePath)
# Select the most recent file
$latest =
$directoryInfo.Files |
Where-Object { -Not $_.IsDirectory } |
Sort-Object LastWriteTime -Descending |
Select-Object -First 1
# Any file at all?
if ($latest -eq $Null)
{
Write-Host "No file found"
exit 1
}
# Download the selected file
$sourcePath = [WinSCP.RemotePath]::EscapeFileMask($remotePath + $latest.Name)
$session.GetFiles($sourcePath, $localPath).Check()
For a full code, see Downloading the most recent file (PowerShell).
(I'm the author of WinSCP)
I tried this, but i get an error:
Error: Exception calling "ListDirectory" with "1" argument(s): "Error listing directory '/path/'.
Could not retrieve directory listing
Can't open data connection for transfer of "/path/"
I read a lot about this problem on the internet, but could not find a solution which seemed fairly simple, and I am not a network setup wizard. So I choose a different approach. In our case the filename of the file which I want to automate the download for, has the date specified in it: backup_2018_08_03_020003_1048387.bak
So we can get the file by using mget *2018_08_03* in a command line ftp session.
Our backup procedure is run every morning at 01.00 AM, so we have a backup each day that we can fetch.
Of course it would have been prettier and nicer to have a script that fetched the latest backup file based on the backup file timestamps, just in case that something went wrong with the latest backup or the backup file naming format changes. The script is just a script to fetch the backup for internal development purposes so its not a big deal if it breaks. I will look into this later and check whether i can make a cleaner solution.
I made a batch script which just asks for todays backup file with the ordinary ftp command prompt scripting.
It is important to get the formatting of todays date right. It must match the formatting of the date in the filename correctly.
If you want to use the script you should replace the variables with your own information. You should also have write access to the directory where you run it from.
This is the script that I made:
#Echo Off
Set _FTPServerName=xxx.xxx.xx.xxx
Set _UserName=Username
Set _Password=Password
Set _LocalFolder=C:\Temp
Set _RemoteFolder="/path/"
Set _Filename=*%date:~-4,4%_%date:~-7,2%_%date:~-10,2%*
Set _ScriptFile=ftptempscript
:: Create script
>"%_ScriptFile%" Echo open %_FTPServerName%
>>"%_ScriptFile%" Echo %_UserName%
>>"%_ScriptFile%" Echo %_Password%
>>"%_ScriptFile%" Echo lcd %_LocalFolder%
>>"%_ScriptFile%" Echo cd %_RemoteFolder%
>>"%_ScriptFile%" Echo binary
>>"%_ScriptFile%" Echo mget -i %_Filename%
>>"%_ScriptFile%" Echo quit
:: Run script
ftp -s:"%_ScriptFile%"
del "%_ScriptFile%"
I am in a situation, when I would like to queue a few mailboxes to export (I don't want them to be processed at the same time) to PST files. I know, how to export them it with a command get-mailboxexportrequest, but when I do it, they almost instantly begin. Can I somehow queue another mailbox, so it would automatically start, when the previous one is completed?
I would do the following:
Build a powershell script which is checking if there is a running export happening (via Get-MailboxExportRequest) if that isn´t the case start to export x mailfiles you specify inside a CSV file
Use the Windows taskmanager on your Exchange Server to trigger that script and define a timeframe here how often and when your script should run
Once the powershell script runs, it should remove the exported mailfile from the CSV file and then quit
The next run from the powershell script via the taskmanager will then check if the current job is still ongoing, if it is, it should quit until its time to pick up the next entry from your list
Update:
As a starting point something like the following should be fine (untested but should give you a starting point):
# Get current Export Requests
$ExportStats = Get-MailboxExportRequest
#Check if there are completed questes
If ($ExportStats.Status -eq "Completed")
{
Write-Host "Export done"
Get-MailboxExportRequest -Status Completed -Name "$ObjectName-Export" | Remove-MailboxExportRequest -Confirm:$false
#Disable-Mailbox -identity "AD\$ObjectName"
# Create a new CSV file, which isn´t including the current export name we just marked as finish via above's section.
# CODE MISSING HERE!
# Now import our CSV list and proceed it
Import-CSV <Filepath of CSV file(\\server\folder\file.csv)> | ForEach-Object {
# Perform the export
New-MailboxExportRequest -Mailbox $_.MailboxAlias -FilePath $_.PSTpath
New-MailboxExportRequest -Mailbox $_.MailboxAlias -FilePath $_.ArchivePath
# Once done exit here, this will ensure we proceed only the first entry
Exit
}
}
elseif ($ExportStats.Status -eq "InProgress")
{
Write-Host "Export still ongoing"
}
I have a server health check script which i'm trying to get working by scheduled task.
The scheduled task has the following set for 'AddArguments"
Add Arguments: -NoLogo -ExecutionPolicy Bypass -File "C:\HealthCheck.ps1"
Everything in the server health portion of full script works fine to create the .csv report, except the last part, which does the CSV to excel conversion/save/close - I've not included the preceding code as it includes some confidential stuff, and i don't believe it's relevant.
When I run the script with the same ID, but from the GUI (not as a scheduled task) it works fine.
Note: The last part of the script definitely does launch excel briefly and performs the functions, and saves/closes it - i'm thinking the scheduled task isn't doing this because it's not supported by Microsoft?
I did find the following SpiceWorks post but the solution noted didn't resolve the issue for me in this case. That's where you create a DESKTOP folder under these paths depending on your version of Office (i'm using Office 2010 32-bit on Windows 7 x64 Pro)
C:\windows\system32\config\systemprofile
C:\windows\syswow64\config\systemprofile
Anyway, here's the code - Any help appreciated!
#Convert CSV to EXCEL, format, and save
#Create excel object
$xl = new-object -comobject excel.application
$xl.visible = $true
#Input
$Workbook = $xl.workbooks.open(“$Dir\Reports\SeverHealth-Results- $CurrentDate.csv”)
$worksheet = $workbook.worksheets.Item(1)
$xl.Rows.Item("2:2").Select()
$xl.ActiveWindow.FreezePanes = $true
$HeaderRow = $Worksheet.Range("A1:L1")
$HeaderRow.Font.Bold = $True
$HeaderRow.Font.Underline = $True
$range = $worksheet.UsedRange
$range.AutoFilter() | Out-Null
$range.EntireColumn.AutoFit() | Out-Null
$rowc = $WorkSheet.UsedRange.Rows.Count
$colc = $WorkSheet.UsedRange.Columns.Count
#Coloring
for ($z = 1; $z -le $rowc; $z++) {
$ActionReqCol = $worksheet.cells.item($z,7)
$ServerCol= $worksheet.cells.item($z,1)
if ($ActionReqCol.text -eq "YES") {
$ActionReqCol.interior.colorindex=3
$ACtionReqCol.font.colorindex=2
$ServerCol.interior.colorindex=3
$ServerCol.font.colorindex=2}}
#Save and close!
$EndDate = Get-Date
$EndDate = $EndDate.ToString('MM-dd-yyyy_hhmm')
$Worksheets = $Workbooks.worksheets
$xlFixedFormat = [Microsoft.Office.Interop.Excel.XlFileFormat]::xlWorkbookDefault
$Workbook.SaveAs($Dir + "\Reports\SeverHealth-Results-$EndDate.xls”, $XLFixedFormat)
$Workbook.Saved = $True
$xl.Quit()
Write the command to invoke the PowerShell with arguments in a batch file. I believe from the comments that you are already able to do this successfully. Configure the Task Scheduler to execute the batch file.
Other advantage of this is, you have reduced dependency. If later you want to make modifications to your command or alter arguments, then you will be able to do so without altering or even opening the Task Scheduler.
Update: #Kenny reported that running task scheduler's task with highest privilege resolved this. The script required elevated access and the same was provided by checking the check box in Task Scheduler to run the task with highest privilege.
I'm pretty new to Powershell and would like to output the result of a command to a file which has a timestamp. Currently I could get the date but I can't get the time. Here is what I have:
mstest /testcontainer:"C:\CodedUITests\CodedUISP.dll" /resultsfile:"C:\CodedUITests\TestResults\result $(get-date -f yyyy-MM-dd)$.trx"
How can I get the time as well?
After a quick google search I found this reference:
Get-Date -f yyyy-MM-dd-HH-mm-ss
I'm doing almost the same thing! Here's my code:
$outputFile = "$tempLocation\work\$($solutionName)_$((Get-Date -Format s).ToString().Replace('-','').Replace(':','')).trx"
Write-Verbose "$($MyInvocation.MyCommand.Name): Running MSTest.exe..."
Invoke-ExternalCommand MSTest.exe #(
"/testcontainer:$testContainer",
"/resultsfile:$outputFile")
```
Invoke-ExternalCommand is just an abstracted function that calls the mstest command. Doing this allows me to mock the call making Pester testing easier.
This is what the trc file is named: SolutionName_20151015T113206.trx