Manipulating a CSV with Powershell - powershell

At the moment I am working on a script to automate a process in IE to add Computer Names and their MACs so we can image them. The page has two fields one for MAC and one for a computer name then a add new button. I had to come to a pretty sloppy solution for avoiding a popup from the page by just quitting out of the com object after submitting.
I don't have much experience with Powershell yet and none with working with CSVs so I'm having a bit of trouble making this work. My goal is to have the script read two entries from a row fill out the correct field then submit it then move to the next row and repeat.
Right now what it does is fills out the fields with undefined in both fields, then submits and repeats.
EDIT: I have edited my code slightly just so it confirms what is trying to read.This is what the results look like. I believe #WalterMitty is on to something that something is wrong with $ie.document.getElementsByName lines, I just tried $ie.document.getElementById but that didn't fill out any fields. It seems it has no problem reading the CSV, but it does have a problem entering the information it reads into the fields properly.
This is an example of what the CSV would look like.
NewComputerName,NewMACAddress
ComputerName1,111122223333
ComputerName2,112233446677
ComputerName3,AAAABBBBCCCC
ComputerName4,AABBCCDDEEFF
This is what my code currently looks like.
cls
$URL = ""
$iterator = 1;
$csv = Get-Content C:\example1.csv
foreach($row in $csv)
{
#starts IE
$ie = new-object -ComObject "InternetExplorer.Application"
$ie.visible = $true
$ie.navigate($URL)
while($ie.Busy -eq $true) { start-sleep -Milliseconds 100 }
($ie.document.getElementsByName("mc_id") |select -first 1).value = $_.NewComputerName;
($ie.document.getElementsByName("mac_id") |select -first 1).value = $_.NewMACAddress;
$ie.document.forms | Select -First 1| % { $_.submit() };
$ie.quit()
$iterator++
write-host "$iterator new ID(s) added"
write-host $row.NewComputerName - $row.NewMACAddress
}

$URL = ""
$iterator = 1
# use Import-Csv for CSV files
$csv = Import-Csv "C:\example1.csv" -Delimiter ","
foreach($row in $csv) {
Write-Host "$iterator new ID(s) added"
#starts IE
$ie = New-Object -ComObject "InternetExplorer.Application"
$ie.Visible = $true
$ie.Navigate($URL)
while ($ie.Busy -eq $true) { Start-Sleep -Milliseconds 100 }
# $_ is not defined in foreach blocks, you have to use $row here
($ie.Document.getElementsByName("mc_id") | Select-Object -First 1).Value = $row.NewComputerName
($ie.Document.getElementsByName("mac_id") | Select-Object -First 1).Value = $row.NewMACAddress
$ie.Document.Forms | Select-Object -First 1 | ForEach-Object { $_.submit() }
$ie.Quit()
$iterator++
}

I'm having similar issues but I just found this may help you guys - I don't have 50 reputation to comment sorry :/....
I messed around with the -Path of the Import-CSV command but I just couldn’t make it work. Apparently this has nothing to do with the path of the CSV file. The Warlock posted this on his blog:
Long story short, the error came from having trailing blank columns in
my CSV. Import-Csv uses the first row in the CSV as names for the
columns (unless you specify otherwise) and when you have blank columns
(or at least multiple blank columns) it causes this error as it
doesn’t have a valid name for them.
Instead of changing the file, I changed my import command to include the headers as per Dale's comment and it worked perfectly:
$data = import-csv "C:\Sharepoint.csv" -header("Department","AD Group","Members","Notes")
The Warlock and Dale saved me lots of time, please stop by the Warlock’s blog and give them a big Thanks

Consider using Import-Csv and Invoke-WebRequest in combination, e.g. like this:
import-csv .\example.csv | %{ iwr http://someurl.local -body #{mc_id=$_.NewComputerName; mac_id=$_.NewMACAddress} -Method POST }
It will read the csv file, iterate over the records and create a application/x-www-form-urlencoded POST request with the values from each record.
When you use iwr (Invoke-WebRequest) and pass a hash table as the "body" it will act as if it is a form being submitted. The POST method will submit the form values as application/x-www-form-urlencode. Without the POST method it would submit the form as if it was a GET, i.e. pass the values in the url.
If you need authentication, session support etc. then read the documentation for Invoke-WebRequest.
Using IE to automate web requests is brittle and error prone.

Related

Powershell Array Output to html

Apologies if this is irrelevant but I'm new to powershell and I've been scratching my head on this for a few days on and off now. I'm trying to write a script that will output two columns of data to a html document. I've achieved most of it by learning through forums and testing different combinations.
The problem is although it gives me the result I need within powershell itself; it will not properly display the second column results for Net Log Level.
So the script looks at some folders and pulls the * value which is always three digits (this is the Site array). It then looks within each of these folders to the Output folder and grabs a Net Log Level node from a file inside there. The script is correctly listing the Sites but is only showing the last value for Net Log Level which is 2. You can see this in the screenshot above. I need this to take every value for each Site and display as appropriate. The image of the incorrect result is below. I need the result to be 1,4,2,2,2. Any help would be greatly appreciated!
function getSite {
Get-ChildItem C:\Scripts\ServiceInstalls\*\Output\'Config.exe.config' | foreach {
$Site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content $_
$NetLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']")
$NetLogLevel = $NetLogLevel.value
New-Object -TypeName System.Collections.ArrayList
$List1 += #([System.Collections.ArrayList]#($Site))
New-Object -TypeName System.Collections.ArrayList
$List2 += #([System.Collections.ArrayList]#($NetLogLevel))
}
$Results = #()
ForEach($Site in $List1){
$Results += [pscustomobject]#{
"Site ID" = $Site
"Net Log Level" = $NetLogLevel
}
}
$Results | ConvertTo-HTML -Property 'Site','Net Log Level' | Set-Content Output.html
Invoke-Item "Output.html"
}
getSite
Restructure your code as follows:
Get-ChildItem 'C:\Scripts\ServiceInstalls\*\Output\Config.exe.config' |
ForEach-Object {
$site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content -Raw $_.FullName
$netLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']").InnerText
# Construct *and output* a custom object for the file at hand.
[pscustomobject] #{
'Site ID' = $site
'Net Log Level' = $netLogLevel
}
} | # Pipe the stream of custom objects directly to ConvertTo-Html
ConvertTo-Html | # No need to specify -Property if you want to use all properties.
Set-Content Output.html
As for what you tried:
New-Object -TypeName System.Collections.ArrayList in effect does nothing: it creates an array-list instance but doesn't save it in a variable, causing it to be enumerated to the pipeline, and since there is nothing to enumerate, nothing happens.
There is no point in wrapping a [System.Collections.ArrayList] instance in #(...): its elements are enumerated and then collected in a regular [object[]] array - just use #(...) by itself.
Using += to "grow" an array is quite inefficient, because a new array must be allocated behind the scenes every time; often there is no need to explicitly create an array - e.g. if you can simply stream objects to another command via the pipeline, as shown above, or you can let PowerShell itself implicitly create an array for you by assigning the result of a pipeline or foreach loop as a whole to a variable - see this answer.
Also note that when you use +=, the result is invariably a regular [object[] array, even if the RHS is a different collection type such as ArrayList.
There are still cases where iteratively creating an array-like collection is necessary, but you then need to use the .Add() method of such a collection type in order to grow the collection efficiently - see this answer.
Instead of populating two separate lists, simply create the resulting objects in the first loop:
function getSite {
$Results = Get-ChildItem C:\Scripts\ServiceInstalls\*\Output\'Config.exe.config' | ForEach-Object {
$Site = $_.fullname.substring(27, 3)
[xml]$xmlRead = Get-Content $_
$NetLogLevel = $xmlRead.SelectSingleNode("//add[#key='Net Log Level']")
$NetLogLevel = $NetLogLevel.value
[pscustomobject]#{
"Site ID" = $Site
"Net Log Level" = $NetLogLevel
}
}
$Results | ConvertTo-HTML -Property 'Site', 'Net Log Level' | Set-Content Output.html
Invoke-Item "Output.html"
}
getSite

Powershell to automate IE | fill Paiement form issue

I'm just starting to use powershell scripts.
I am trying to automate a payment process (for personal use).
But when I enter my payment information via my script the information is not saved on the page (the data acts as a placeholder)
I tried to use a "focus()" before entering the values but it doesn't work :(
(maybe a bad use ?)
I had the idea to simulate an "ENTER" key press after entering my values but I feel like I'm misusing the syntax
Here is an example of my code:
$cardnumber = "4000 1000 2000 0000"
$nameoncard = "Alban Mysteriousguy"
$expirationdate = "10/77"
$ccv = "420"
$ie = New-Object -COMObject InternetExplorer.Application
$ie.visible = $true
$ie.Navigate(‘https://secure2.ldlc.com/fr-fr/DeliveryPayment’)
#Achat CB
while ($ie.Busy -eq $true) { Start-Sleep -Seconds 1; } #wait for browser idle
#notworking
$Carednumbervar = $ie.document.getElementsByName("CardNumber")
($Carednumbervar |Select-Object -first 1).value = $cardnumber;
#working
($ie.document.getElementsByName("OwnerName") |Select-Object -first 1).value = $nameoncard;
#notworking
($ie.document.getElementsByName("ExpirationDate") |Select-Object -first 1).value = $expirationdate;
#notworking
($ie.document.getElementsByName("Cryptogram") |Select-Object -first 1).value = $ccv;
#working
$payementButton = $ie.Document.getElementsByTagName('button') | Where-Object { ($_.ClassName -ceq 'button color2 maxi') }
$CGV = $ie.Document.getElementsByTagName('label') | Where-Object { ($_.for -ceq 'terms') }
$CGV.click
#$payementButton.click()
If anyone have an idea about how can I solve this issue or just a trick that I can try, you are welcome :)
Thanks by advance
PS: To test the script you need to create an account to Ldlc.com, add something in your cart and go to the payment page ( https://secure2.ldlc.com/fr-fr/DeliveryPayment for exemple) if needed I can upload a video of the behavior.
Edit: Here I made a youtube video to show my issue : youtu.be/pTbSWU_cudY
Thank you for the detailed code. I extracted the relevant html code and performed a simple test. I found that it works correctly. You need to cancel Enable Protected Mode in IE.
Open IE -> Tools -> Internet Options -> security tab -> uncheck option Enanle Protected Mode and apply
Something like this:

is there a simple way to output to xlsx?

I am trying to output a query from a DB to a xlsx but it takes so much time to do this because there about 20,000 records to process, is there a simpler way to do this?
I know there is a way to do it for csv but im trying to avoid that, because if the records had any comma is going to take it as a another column and that would mess with the info
this is my code
$xlsObj = New-Object -ComObject Excel.Application
$xlsObj.DisplayAlerts = $false
$xlsWb = $xlsobj.Workbooks.Add(1)
$xlsObj.Visible = 0 #(visible = 1 / 0 no visible)
$xlsSh = $xlsWb.Worksheets.Add([System.Reflection.Missing]::Value, $xlsWb.Worksheets.Item($xlsWb.Worksheets.Count))
$xlsSh.Name = "QueryResults"
$DataSetTable= $ds.Tables[0]
Write-Output "DATA SET TABLE" $DataSetTable
[Array] $getColumnNames = $DataSetTable.Columns | SELECT *
Write-Output "COLUMN NAMES" $DataSetTable.Rows[0]
[Int] $RowHeader = 1
foreach ($ColH in $getColumnNames)
{
$xlsSh.Cells.item(1, $RowHeader).font.bold = $true
$xlsSh.Cells.item(1, $RowHeader) = $ColH.ColumnName
Write-Output "Nombre de Columna"$ColH.ColumnName
$RowHeader++
}
[Int] $rowData = 2
[Int] $colData = 1
foreach ($rec in $DataSetTable.Rows)
{
foreach ($Coln in $getColumnNames)
{
$xlsSh.Cells.NumberFormat = "#"
$xlsSh.Cells.Item($rowData, $colData) = $rec.$($Coln.ColumnName).ToString()
$ColData++
}
$rowData++; $ColData = 1
}
$xlsRng = $xlsSH.usedRange
[void] $xlsRng.EntireColumn.AutoFit()
#Se elimina la pestaña Sheet1/Hoja1.
$xlsWb.Sheets(1).Delete() #Versión 02
$xlsFile = "directory of the file"
[void] $xlsObj.ActiveWorkbook.SaveAs($xlsFile)
$xlsObj.Quit()
Start-Sleep -Milliseconds 700
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsRng)) {''}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsSh)) {''}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsWb)) {''}
While ([System.Runtime.Interopservices.Marshal]::ReleaseComObject($xlsObj)) {''}
[gc]::collect() | Out-Null
[gc]::WaitForPendingFinalizers() | Out-Null
$oraConn.Close()
I'm trying to avoid [CSV files], because if the records had any comma is going to take it as a another column and that would mess with the info
That's only the case if you try to construct the output format manually. Builtin commands like Export-Csv and ConvertTo-Json will automatically quote the values as necessary:
PS C:\> $customObject = [pscustomobject]#{ID = 1; Name = "Solis, Heber"}
PS C:\> $customObject
ID Name
-- ----
1 Solis, Heber
PS C:\> $customObject |ConvertTo-Csv -NoTypeInformation
"ID","Name"
"1","Solis, Heber"
Notice, in the example above, how:
The string value assigned to $customObject.Name does not contain any quotation marks, but
In the output from ConvertTo-Csv we see values and headers clearly enclosed in quotation marks
PowerShell automatically enumerates the row data when you pipe a [DataTable] instance, so creating a CSV might (depending on the contents) be as simple as:
$ds.Tables[0] |Export-Csv table_out.csv -NoTypeInformation
What if you want TAB-separated values (or any other non-comma separator)?
The *-Csv commands come with a -Delimiter parameter to which you can pass a user-defined separator:
# This produces semicolon-separated values
$data |Export-Csv -Path output.csv -Delimiter ';'
I usually try and refrain from recommending specific modules libraries, but if you insist on writing to XSLX I'd suggest checking out ImportExcel (don't let the name fool you, it does more than import from excel, including exporting and formatting data from PowerShell -> XSLX)

Run command, extract a field, run a resultant command

Apologies if this is an insanely simple question, but I'm at something at a loss.
What I'm trying to do is take a command output - in this case from NetApp DFM:
dfm event list
ID Source Name Severity Timestamp
------- ------- ------------- ----------- ------------
1 332 volume-online Normal 20 Apr 10:16
2 443 volume-online Normal 20 Apr 10:17
3 3222 volume-online Normal 20 Apr 10:18
I have about 17,000 events - I want to delete them all by ID, by running:
dfm event delete <ID>
I know exactly how I'd do this on Unix (and used to, when this was our platform):
for i in `dfm event list | awk '{print $1}'`
do
dfm event delete $i
done
For bonus points - a 'grep' type criteria? I apologise in advance for the basic nature of the question - I've tried looking on Google for a suitable example, but haven't found anything.
I've made a start by:
dfm event list > dfmevent.txt
foreach ( $line in get-content dfmevent.txt ) {
echo $line
}
But I thought I would ask if there's a better way.
I don't have access to your environment to test but if you are just trying to get access to that first element which is the ID then that should be straight forward.
dfm event list | ForEach-Object{$_.Split(" ",2)[0]} | Where-Object{$_ -match '^\d+$'} | ForEach-Object{
#For Testing
Write-Host "Id: $_ will be deleted"
# Then do something
# dfm event delete $_
}
I'm sure the output is already delimited with new line so sending to file might be redundant.
We take each line and try and split it on the first space. Then pass the first element from that array. Next we ensure that element is indeed a number with a simple regex check. This will ensure that we only get numbers. I had thought about skipping the first two lines but this should work for other occurrences of text as well.
The last loop is for processing that ID. I left a Write-Host there for testing. Assuming you get the id's you are looking for you should just be able to uncomment out that last line with dfm event delete $_
Capturing the output of a DOS command into Powershell is a challenge.
Using a native snapin or module from NetApp would be easier.
might be worth checking out if that link helps
Otherwise, your method of writing to a text file and reading it back in is actually quite a good idea, this is one way of reading it back and pushing the data into the command you need.
$a = get-content dfmevent.txt
foreach ($i in $a) { if ($i.ReadCount -gt 2) { dfm event delete ($i.Substring(0,$i.IndexOf(" "))) } }
This will assign to the variable $result only
$a = get-content dfmevent.txt
$result = #()
foreach ($i in $a) { if ($i.ReadCount -gt 2) { $result += $i.Substring(0,$i.IndexOf(" "))} }
And if you did not want to write to a text file, you could use the .NET method of capturing the output directly
$ProcessInfo = New-Object System.Diagnostics.ProcessStartInfo
$ProcessInfo.FileName = "dfm"
$ProcessInfo.RedirectStandardOutput = $true
$ProcessInfo.UseShellExecute = $false
$ProcessInfo.Arguments = "event list"
$Process = New-Object System.Diagnostics.Process
$Process.StartInfo = $ProcessInfo
$Process.Start() | Out-Null
$Process.WaitForExit()
$output = $Process.StandardOutput.ReadToEnd()

Expressions are only allowed as the first element of a pipeline

I'm new at writing in powershell but this is what I'm trying to accomplish.
I want to compare the dates of the two excel files to determine if one is newer than the other.
I want to convert a file from csv to xls on a computer that doesn't have excel. Only if the statement above is true, the initial xls file was copied already.
I want to copy the newly converted xls file to another location
If the file is already open it will fail to copy so I want to send out an email alert on success or failure of this operation.
Here is the script that I'm having issues with. The error is "Expressions are only allowed as the first element of a pipeline." I know it's to do with the email operation but I'm at a loss as to how to write this out manually with all those variables included. There are probably more errors but I'm not seeing them now. Thanks for any help, I appreciate it!
$CSV = "C:filename.csv"
$LocalXLS = "C:\filename.xls"
$RemoteXLS = "D:\filename.xls"
$LocalDate = (Get-Item $LocalXLS).LASTWRITETIME
$RemoteDate = (Get-Item $RemoteXLS).LASTWRITETIME
$convert = "D:\CSV Converter\csvcnv.exe"
if ($LocalDate -eq $RemoteDate) {break}
else {
& $convert $CSV $LocalXLS
$FromAddress = "email#address.com"
$ToAddress = "email#address.com"
$MessageSubject = "vague subject"
$SendingServer = "mail.mail.com"
$SMTPMessage = New-Object System.Net.Mail.MailMessage $FromAddress, $ToAddress, $MessageSubject, $MessageBody
$SMTPClient = New-Object System.Net.Mail.SMTPClient $SendingServer
$SendEmailSuccess = $MessageBody = "The copy completed successfully!" | New-Object System.Net.Mail.SMTPClient mail.mail.com $SMTPMessage
$RenamedXLS = {$_.BaseName+(Get-Date -f yyyy-MM-dd)+$_.Extension}
Rename-Item -path $RemoteXLS -newname $RenamedXLS -force -erroraction silentlycontinue
If (!$error)
{ $SendEmailSuccess | copy-item $LocalXLS -destination $RemoteXLS -force }
Else
{$MessageBody = "The copy failed, please make sure the file is closed." | $SMTPClient.Send($SMTPMessage)}
}
You get this error when you are trying to execute an independent block of code from within a pipeline chain.
Just as a different example, imagine this code using jQuery:
$("div").not(".main").console.log(this)
Each dot (.) will chain the array into the next function. In the above function this breaks with console because it's not meant to have any values piped in. If we want to break from our chaining to execute some code (perhaps on objects in the chain - we can do so with each like this:
$("div").not(".main").each(function() {console.log(this)})
The solution is powershell is identical. If you want to run a script against each item in your chain individually, you can use ForEach-Object or it's alias (%).
Imagine you have the following function in Powershell:
$settings | ?{$_.Key -eq 'Environment' } | $_.Value = "Prod"
The last line cannot be executed because it is a script, but we can fix that with ForEach like this:
$settings | ?{$_.Key -eq 'Environment' } | %{ $_.Value = "Prod" }
This error basically happens when you use an expression on the receiving side of the pipeline when it cannot receive the objects from the pipeline.
You would get the error if you do something like this:
$a="test" | $a
or even this:
"test" | $a
I don't know why are trying to pipe everywhere. I would recommend you to learn basics about Powershell pipelining. You are approaching it wrong. Also, I think you can refer to the link below to see how to send mail, should be straight forward without the complications that you have added with the pipes : http://www.searchmarked.com/windows/how-to-send-an-email-using-a-windows-powershell-script.php