How to combine a template with a CSV file in Powershell - powershell

I want to combine a template that looks like this:
grant $privs
on $table
to $user;
With a CSV file that looks like this:
privs,table,user
ALL,Employees,DBA
READ,Employees,Analyst
"READ, WRITE", Employees, Application
ALL,Departments,DBA
READ,Departments,"Analyst, Application"
To produce an SQL script that looks like this:
grant ALL
on Employees
to DBA;
grant READ
on Employees
to Analyst;
grant READ, WRITE
on Employees
to Application;
grant ALL
on Departments
to DBA;
grant READ
on Departments
to Analyst, Application;
The template has three parameters that look like Powershell variables. The CSV file has enough data
to specify five copies of the template. In real life, it would be more like 200 copies.
I also want to be able to apply the same technique to a variety of CSV files, most of which
do not come from databases. And I want to use a variety of templates, most of which do not
generate SQL. For that reason, I want a technique that deals with plain text files, instead
of attaching to the database.
Note: I am asking this question so as to provide the community with an answer.

I have written a function, Expand-Csv, that does this. Here it is:
<# This function is a table driven template tool.
It generates output from a template and
a driver table. The template file contains plain
text and embedded variables. The driver table
(in a csv file) has one column for each variable,
and one row for each expansion to be generated.
12/12/2016
#>
function Expand-csv {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true)]
[string] $driver,
[Parameter(Mandatory=$true)]
[string] $template
)
Process
{
$pattern = (Get-Content $template) -join "`n"
Import-Csv $driver | % {
foreach ($p in $_.psobject.properties) {
Set-variable -name $p.name -value $p.value
}
$ExecutionContext.InvokeCommand.ExpandString($pattern)
}
}
}
In the case at hand, the call would look like this:
Expand-Csv grants.csv grants.tmplt > grants.sql
The output that would come to the console gets redirected to a file.
I've tried using this for generating scripts in Powershell itself,
but a word of caution is in order. ExpandString does some of the same processing
that Invoke-Command does, so you can get undesired consequences. I only use it
for the simplest kind of PS code, like a series of calls to some external app.

Related

Windows Powershell - trouble importing CSV and iterating

I'm new to Powershell (of course), and having troubles with a seemingly simple process. I have found a couple of examples that I think I am following, but they aren't working for me.
What I am trying to do: add a bunch of users to the local Windows OS, by reading from a CSV file (has names, usernames, passwords, etc).
My understanding is that the 'Import-CSV' cmdlet is supposed to return an object-like thing you can iterate over:
"The result of an Import-Csv command is a collection of strings that
form a table-like custom object."
When I perform that step, saving it to a variable, it seems that there is only ever 1 row present. And if I don't provide the "-Header" parameter, I get errors about a 'member is already present'... even if I include the header in the CSV file (my original file did not include a header row in the CSV file.)
I have tried various methods trying to get a Count of the imported CSV results, just trying to see what the data is, but I'm not having any luck. (MS Docs say you can use the Count property.)
MS Docs (https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/import-csv?view=powershell-7.2) say this about "Import-CSV":
Outputs
Object
This cmdlet returns the objects described by the content in the CSV
file.
...
Notes
Because the imported objects are CSV versions of the object type...
The result of an Import-Csv command is a collection of strings that
form a table-like custom object. Each row is a separate string, so you
can use the Count property of the object to count the table rows. The
columns are the properties of the object and items in the rows are the
property values.
An example of my input CSV file:
"ISA","LOG","Consulting & Other","Vendor","Isalog","alsdkjfalsdjflasdkfjalsdkfjlaksdjflkasdfj"
"Bry","Link","Bry Link","Vendor","Bry","asdkfjalsdjflaksdjflasdkjflaksdfj"
"Michael","Had","Premier Service Of Western","Vendor","Michael","alsdkfjalskdjflaksdjflaksdfjalksdfj"
Code of one example that I am testing:
param ($InputFile)
Write-Host "Provided input file: $InputFile"
$CSV = Import-CSV -Path $InputFile -Header 'FirstName', 'LastName', 'FirmName', 'Type', 'Username', 'Password'
foreach($LINE in $CSV)
{
$NewUser="$($LINE.USERNAME)"
$NewPass="$($LINE.PASSWORD)"
$SecurePass=ConvertTo-SecureString –AsPlainText -Force -String "$NewPass"
Write-Host "User = $NewUser"
#New-LocalUser -Name $NewUser -Password $SecurePass
}
And a screenshot of my script plus the run results:
Running on: Windows server 2019 datacenter.
Powershell version: 5.1
The ultimate answer was that the character encoding for the CSV file I was using as input was causing problems for Powershell. Specifically, the line-ending encoding.
My original file was created on a Mac. The line-ending enconding was 'Macintosh (CR)'. The files that worked OK were created on this Windows machine, and used the line-ending encoding = "Windows (CR LF)".
Thanks to Olaf who got me thinking about this issue and made me investigate that area further.

Export PC Name and Updates Needed from WSUS to CSV Using PowerShell

I'm trying to export a CSV file from WSUS using PowerShell containing a list of all computers that need updates and the titles or KBs of the updates each particular computer needs. Something like this...
Computer1, update1, update2
Computer2, update1, update3, update5
Computer3, update2, update4
I found this script on TechNet that returns the computer name and how many updates are needed, but it doesn't return the titles of the updates, and it may return all computers in WSUS, not just the ones that need updates (I'm in a test environment of only 1 computer right now).
Function Get-WSUSClientNeeded {
[cmdletbinding()]
Param (
[parameter(Mandatory=$true)]
[string]$WsusServer
)
#Load assemblies
[void][system.reflection.assembly]::LoadWithPartialName('Microsoft.UpdateServices.Administration')
#Connect to WSUS
$Global:wsus = [Microsoft.UpdateServices.Administration.AdminProxy]::getUpdateServer($WsusServer,$False,8530)
#Create Scope objects
$computerscope = New-Object Microsoft.UpdateServices.Administration.ComputerTargetScope
$updatescope = New-Object Microsoft.UpdateServices.Administration.UpdateScope
#Get Update Summary
$wsus.GetSummariesPerComputerTarget($updatescope,$computerscope) | ForEach {
New-Object PSObject -Property #{
ComputerName = ($wsus.GetComputerTarget([guid]$_.ComputerTargetId)).FullDomainName
NeededCount = ($_.DownloadedCount + $_.NotInstalledCount)
DownloadedCount = $_.DownloadedCount
NotApplicableCount = $_.NotApplicableCount
NotInstalledCount = $_.NotInstalledCount
InstalledCount = $_.InstalledCount
FailedCount = $_.FailedCount
}
}
}
Get-WSUSClientNeeded -WsusServer 'Server' | Select ComputerName, NeededCount
I'm very new to PowerShell, so any help would be greatly appreciated.
Thanks!
You're creating a custom-object from the results of the update summary. You're using piping and an inline loop. These are complicated and while you may be able to improve your scripts using them later, getting them to work in the first place is much easier if you use loops and variable assignments and arrays.
My suggestion of how to work through this is to
Split the work part of that into an actual loop. (`$wsus.Get.... piped through foreach and creating objects)
Add the results of your pull command (the object you create) to an array. You're creating an object then not doing anything with it.
Loop through the array and run commands against the elements. Apply filters or extract info as you wish.
Only pull the properties you want. Most Get- cmdlets include the -properties switch.
Use Get-Members to peek inside the objects returned by a command. It will tell you the properties and methods of the object.
Run commands like this at the command line in ISE, figure out which property names you want to extract.
$summaries = $wsus.GetSummariesPerComputerTarget($updatescope,$computerscope)
$summaries | get-members
loop through an array, pull out data, put it in another array
foreach (summary in $summaries) {
$props = $summary | select property1, property2
$AnswerArray += $props
}
Export to a csv, or use the array of answers in the next stage
$AnswerArray | export-csv -NoTypeInformation -NoClobber -Path c:\temp\answers.csv
Most of my PowerShell experience is with Active Directory, but I would have one script to extract data into a CSV, manipulate that in another script, or outside PowerShell with other tools like excel, and use it as input in a script that makes the changes I needed to make.
This turned out to be much easier than expected.
Get-WindowsUpdate -WindowsUpdate
Get-WindowsUpdate will return available updates from an online source. The -WindowsUpdate parameter will return the updates from WSUS. Make sure to import the PSWindowsUpdate module first.

Generate a SQL create table script using powershell

I want to use powershell to be able to quickly generate a SQL script with create table statements that are able to recreate all tables in an existing database. Only thing is that I want to tweak some options, such as turn identities off.
I just cannot figure out how to do this!
I have already come as far as to set a $server variable, and to set a variable called $tables to get all tables in that particular database.
Then I use a loop:
foreach ($table in in $tables)
{$table.Script()
}
This works, but I just don't know how to add scripting options, such as NoIdentities = $True
Can anyone help me out?
I once had to do a lot of repetitive type work, including generate SQL scripts for
every table in a database. So I developed a general purpose tool that is good for this type of work. I am including the tool, as is, and a sample run of the tool intended to
produce a series of grant commands for each table, and each category of database user.
My tool runs off of CSV files rather than off of the database directly. I found it fairly easy to generate CSV files and templates for a lot of different tasks. Your mileage may vary. Maybe you can start with this tool and adapt it to your needs.
Here is the tool, and a sample run.
<#
.SYNOPSIS
Generates multiple expansions of a template,
driven by data in a CSV file.
.DESCRIPTION
This function is a table driven template tool.
It generates output from a template and
a driver table. The template file contains plain
text and embedded variables. The driver table
(in a csv file) has one column for each variable,
and one row for each expansion to be generated.
#>
function Expand-csv {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true)]
[string] $driver,
[Parameter(Mandatory=$true)]
[string] $template
)
Process
{
$xp = (Get-Content $template) -join "`r`n"
Import-Csv $driver | % {
$_.psobject.properties | % {Set-variable -name $_.name -value $_.value}
$ExecutionContext.InvokeCommand.ExpandString($xp)
}
}
}
# Now do a sample run of Expand-csv
# then display inputs and output
Expand-csv grants.csv grants.tmplt > grants.sql
get-content grants.tmplt
import-csv grants.csv
get-content grants.sql
Here is the result of running the above:
PS> C:\Users\David\Software\Powershell\test\sample.ps1
grant $privs
on $table
to $user;
privs table user
----- ----- ----
ALL Employees DBA
READ Employees Analyst
READ, WRITE Employees Application
ALL Departments DBA
READ Departments Analyst, Application
grant ALL
on Employees
to DBA;
grant READ
on Employees
to Analyst;
grant READ, WRITE
on Employees
to Application;
grant ALL
on Departments
to DBA;
grant READ
on Departments
to Analyst, Application;
PS>
In real life, I have the tool defined in my $profile file, so that it's available whenever I'm in powershell.
I'm not sure how well this applies to your case. This doesn't address the exact situation you describe, but you may be able to adapt the technique.
If you're still looking for a solution, may I suggest this little script I've been using with a single table. You should be able to update it to support multiple tables fairly easily. Notice line "$scripter.Options.NoIdentities = $true;" below.
param
(
[string] $server,
[string] $database,
[string] $schema,
[string] $table
)
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") | Out-Null
$srv = New-Object "Microsoft.SqlServer.Management.SMO.Server" $server
$db = New-Object ("Microsoft.SqlServer.Management.SMO.Database")
$tbl = New-Object ("Microsoft.SqlServer.Management.SMO.Table")
$scripter = New-Object ("Microsoft.SqlServer.Management.SMO.Scripter") ($server)
$db = $srv.Databases[$database]
$tbl = $db.tables | Where-object {$_.schema -eq $schema-and$_.name -eq $table}
$scripter.Options.ScriptSchema = $true;
$scripter.Options.ScriptData = $false;
$scripter.Options.NoCommandTerminator = $false;
$scripter.Options.NoCollation = $true;
$scripter.Options.NoIdentities = $true;
$scripter.EnumScript($tbl)
Save it to "table.ps1" and execute from within PowerShell by passing param values:
& table.ps1 -server SERVER\INSTANCE -database MyDB -schema dbo -table MyTable
Let me know if it works.

Creat Another Independent Powershell Script through first Powershell Script

So I wanted to know how to make powershell Script A can create powershell Script B
The main goal is to have Script B be independent of Script A (for security purposes). I have Keys that I get with Script A and hash them.
I want to also have Script A create Script B, as Script B will do an action and will also be taken to other separate environments.
So how can I:
A: Make the new script via the first?
B: Have, $hash1 (hash of key1), pass on to Script B, So that when Script B runs in a separate VM or PC environment, it does not require Script A to use $hash1
Thanks
A: Powershell script is saved as text. You (=script A) can print text into file (for example scriptB.ps) and this file can be executed.
B: You can use "return statement" in script A. And if you don't need this hash, so you have to write your code to run even without this input parameter or use default parameter. And how? With if statements probably.
I created a tool that is like what you are describing except that it is more limited in scope, but perhaps more generic. I actually created it not to generate powershell scripts but to generate SQL scripts. I could use it to generate PS scripts, but of a very limited form, which would look like this:
& someFunctionorApp -arg11 -arg12 -arg13 ...
& someFunctionorApp -arg21 -arg22 -arg23 ...
and repeated perhaps a hundred times. The args all come from some data source or other, and I've just plugged generic names in for the sake of example. This generated script might do by brute force something that a clever scripter would do inside a loop, but so be it.
For my tool, the driver is data stored inside a CSV file. I can get CSV files from databases, from spreadsheets, and from simple PS scripts that capture information. Different CSV files have different headers, with different field names, and so on. The tool is generic. The other input to my tool is what I have called a template. A template, in this context, is just a text file with some embedded PS variables. The tool runs through the CSV file picking up actual values to store in place of the PS variables.
The output comes out on the console, but it's easily redirected to a file.
Just for grins, I have included the tool. If the tool is not to your liking, perhaps you can pick up a technique or two and adapt them.
<# This function is a table driven template tool.
It's a refinement of an earlier attempt.
It generates output from a template and
a driver table. The template file contains plain
text and embedded variables. The driver table
(in a csv file) has one column for each variable,
and one row for each expansion to be generated.
5/13/2015
#>
function Expand-csv {
[CmdletBinding()]
Param(
[Parameter(Mandatory=$true)]
[string] $driver,
[Parameter(Mandatory=$true)]
[string] $template
)
Process
{
$OFS = "`r`n"
$list = Import-Csv $driver
[string]$pattern = Get-Content $template
foreach ($item in $list) {
foreach ($key in $item.psobject.properties) {
Set-variable -name $key.name -value $key.value
}
$ExecutionContext.InvokeCommand.ExpandString($pattern)
}
}
}

common ways to pass state from cmdlet to cmdlet

I am creating my own set of cmdlets. They all need the same state data (like location of DB and credentials for connecting to DB). I assume this must be a common need and wonder what the common idiom for doing this is.
the obvious one is something like
$db = my-make-dbcreds db=xxx creds=yyyy ...
my-verb1 $db | my-verb2 $db -foo 42...
my-verb8 $db bar wiz
.....
but i was wondering about other ways. Can I silently pipe the state from one to another. I know I can do this if state is the only thing I pipe but these cmdlets return data
Can I set up global variables that I use if the user doesnt specify state in the command
Passing the information state through the pipe is a little lost on me. You could update your cmdlets to return objects that the next cmdlet will accept via ValueFromPipeline. When you mentioned
like location of DB and credentials for connecting to DB
the best this I could think that you want is....
SPLATTING!
Splatting is a method of passing a collection of parameter
values to a command as unit. Windows PowerShell associates
each value in the collection with a command parameter.
In its simplest form
$params = #{
Filter = "*.txt"
Path = "C:\temp"
}
Get-ChildItem #params
Create a hashtable of parameters and values and splat them to the command. The you can edit the table as the unique call to the cmdlet would allow.
$params.Path = "C:\eventemperor"
Get-ChildItem #params
I changed the path but left the filter the same. You also dont have to have everything in $params you splat and use other parameters in the same call.
It is just a matter of populating the variables as you see fit and changing them as the case requires.
Spewing on the pipeline
Pretty that is what it is actually called. If you use advanced function parameters you can chain properties from one cmdlet to the next if you really wanted to. FWIW I think splatting is better in your case but have a look at the following.
function One{
param(
[parameter(Mandatory=$true,
ValueFromPipeline=$True,
ValueFromPipelineByPropertyName=$true)]
[String[]]
$Pastry
)
write-host "You Must Like $Pastry"
Write-Output (New-Object -TypeName PSCustomObject -Property #{Pastry= $pastry})
# If you have at least PowerShell 3.0
# [pscustomobject]#{Pastry= $pastry}
}
Simple function that writes the variable $pastry to the console but also outputs an object for the next pipe. So running the command
"Eclairs" | One | One | Out-Null
We get the following output
You Must Like Eclairs
You Must Like Eclairs
We need to pipe to Out-Null at the end else you would get this.
Pastry
------
{Eclairs}
Perhaps not the best example but you should get the idea. If you wanted to extract information between the pipe calls you could use Tee-Object.
"Eclair" | One | Tee-Object -Variable oneresults | One | Out-Null
$oneresults
Consider Parameter Default Values
Revisiting this concept after trying to find a better way to pass SQL connection information between many function working against the same database. I am not sure if this is the best thing to do but it certainly simplifies thing for me.
The basic idea is to add a rule for your cmdlet or wildcard rule if your cmdlets share a naming convention. For instance I have a series of functions that interact with our ticketing system. They all start with Get-Task.... and all configured with SQL connection information.
$invokeSQLParameters = #{
ServerInstance = "serverName"
Username = $Credentials.UserName
Password = $Credentials.GetNetworkCredential().Password
}
$PSDefaultParameterValues.Add("New-Task*:Connection",$invokeSQLParameters)
$PSDefaultParameterValues.Add("Get-Task*:Connection",$invokeSQLParameters)
So now in my functions I have a parameter called Connection that will always be populated with $invokeSQLParameters as long as the above is done before the call. I still use splatting as well
Invoke-Sqlcmd -Query $getCommentsQuery #Connection
You can read up more about this at about_parameters_default_values