Retrieving SSIS 2012 environmentvariable.name via power shell - powershell

We are using power shell to deploy our 2012 SSIS packages and have environment variables on a SSIS 2012 Server. Now during project deployment I am attempting to loop through eachvariable in the environment variables collection (foreach($variable in $environment.Variables)). That is no problem. I can see "EnvironmentVariable[#Name = 'something']"....however attempting to retrieve the name ("something") from the variable via $variable.Name or $variable.Key doesn't work. I've tried looping through $environment.Variables.Keys and still nothing. My power shell skills are a little weak since I've been using NANT the past several years but is there something I'm just not seeing?
Thanks in advance,
Anthony
Adding snippet of existing power shell script. The bolded $variable.Name is not working within the CreateETLPackages task. There is a lot of setup and other scripts called from this scripts so I haven't included everything. When $variable.Name is returned in a debug statement it returns "EnvironmentVariable[#Name = 'something']" as I mentoned in my original post:
Task CreateSSISFolder -Depends CreateSSISCatalog {
if (!$script:SSISCanBeDeployed) { return }
# Create the project for the packages in the catalog
$catalog = $script:SSISCatalog
if ($catalog.Folders.Count -eq 0) {
Write-Host "Creating folder $SSISFolderName ..."
$script:SSISFolder = New-Object "Microsoft.SqlServer.Management.IntegrationServices.CatalogFolder" ($catalog, $SSISFolderName, "Folder for EDGE ETL packages")
$script:SSISFolder.Create()
Write-Host "... done"
} else {
Write-Host "SSIS folder $SSISFolderName already exists; skipping create"
}
}
Task CreateSSISEnvironment -Depends CreateSSISFolder {
if (!$script:SSISCanBeDeployed) { return }
# Create the environment in the project
$folder = $script:SSISFolder
$environment = $folder.Environments[$SSISEnvironmentName]
if ($environment -eq $null) {
# Create the environment
Write-Host "Creating environment $SSISEnvironmentName ..."
$environment = New-Object "Microsoft.SqlServer.Management.IntegrationServices.EnvironmentInfo" ($folder, $SSISEnvironmentName, "Environment to configure the SSIS packages")
$environment.Create()
Write-Host "... done"
# Now create the variables (Constructor args: variable name, type, default value, sensitivity, description)
$environment.Variables.Add("TestDatabase", [System.TypeCode]::String, "Data Source=$SSISServerName.TestDatabase;User ID=<USERNAME>;Password=<PASSWORD>;Initial Catalog=EdgeAviTrack;Provider=SQLNCLI11.1;Persist Security Info=True;Auto Translate=False;", $false, "Connection string for TestDatabase database")
$environment.Alter()
} else {
Write-Host "Environment $SSISEnvironmentName already exists; skipping create"
}
}
Task CreateETLPackages -Depends CreateSSISFolder, CreateSSISEnvironment {
if (!$script:SSISCanBeDeployed) { return }
# Get list of ETL .ispac files in the solution
$SSISProjects = GetListOfDeploymentFiles "*.ispac"
if ($SSISProjects -ne $null) {
$folder = $script:SSISFolder
$environment = $folder.Environments[$SSISEnvironmentName]
if ($folder -ne $null) {
foreach ($file in $SSISProjects) {
# Read the ispac file, and deploy it to the folder
[byte[]] $projectFile = [System.IO.File]::ReadAllBytes($file.FullName)
$nameParts = $file.Name.split(".")
$curProjectName = [string]::join(".", $nameParts[0..($nameParts.length - 2)])
Write-Debug "Deploying SSIS project $curProjectName"
$project = $folder.DeployProject($curProjectName, $projectFile)
if ($project.Status -ne "Success") {
Write-Error "SSIS packages did not deploy correctly!"
} else {
# Get the full information set, rather than the short version returned from DeployProject
$project = $folder.Projects[$curProjectName]
}
# Connect the project to the environment to stitch up all the connection strings
if ($project.References.Item($SSISEnvironmentName, ".") -eq $null) {
Write-Host "Adding environment reference to $SSISEnvironmentName ..."
$project.References.Add($SSISEnvironmentName)
$project.Alter()
Write-Host "... done"
}
# Connect all the project parameters to the environment variables
Write-Host "Adding connection string references to environment variables ..."
foreach($varialble in $environment.Variables) {
try {
$project.Parameters["CM." + **$varialble.Name** + ".ConnectionString"].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Referenced, **$variable.Name**)
}
catch {
Write-Debug "Unable to set connection string **$variable.Name** on SSIS project $curProjectName"
}
}
$project.Alter()
Write-Host "... done"
}
}
}
}

Ok I found my issue. Looks like I was trying I need to use the $($object.Name) to get what I need out. Appreciate those that reached out to for their help.
Thanks,
Anthony

Related

PowerShell: Loop through IIS 8+ server's custom log fields to check for existence of required fields

I'm trying to loop through a list of servers (IIS 8.5 mostly) and for each server see if certain custom field names exist for logs. I'm executing these commands from my local machine, using the "invoke-command" cmdlet. Currently, it just returns to the cursor instead of writing out if there is a Match or No Match.
srv.csv file contains:
- Host,IP
- srv1,10.0.0.2
- srv2,10.0.0.3
log_stig.csv file contains:
- LogFieldName,SourceType
- Connection,RequestHeader
- User-Agent,RequestHeader
- Content-Type,ResponseHeader
Assigns the values in srv.csv to $IISServers variable
try{
$IISServers = Import-Csv C:\Users\snappy\ps_script_resources\srv.csv
}
catch{
"The file 'srv.csv' is not available"
}
Assigns the values in log_stig.csv to $STIG_CustomFields variable
try{
$STIG_CustomFields = Import-Csv C:\Users\snappy\ps_script_resources\log_stig.csv
}catch{
"The file 'log_stig.csv' is not available"
}
Loop through each server and then loop through each customfield name checking if they exist on the server.
ForEach($IISServer in $IISServers){
$IISServerName = $($IISServer.Host)
try{
invoke-command -ComputerName $IISServerName -ScriptBlock{
$SiteLogFileCustom = Get-WebConfigurationProperty -PSPath 'MACHINE/WEBROOT/APPHOST' -
filter "system.applicationHost/sites/site/logFile/customFields" -Name 'Collection'
ForEach ($STIG_CustomField in $STIG_CustomFields){
write-output $STIG_CustomField.LogFieldName
write-output $SiteLogFileCustom.logFieldName
if($STIG_CustomField.LogFieldName -match $SiteLogFileCustom.logFieldName){
write-output "Match"
}
else{
write-output "No Match"
}
write-output "This ran"
}
}
}
catch{
"Invoke failed for "
}
}
The current code when I run from my local machine only returns to the cursor, I would expect that for every match, I would see either a Match or No Match message in my console. Any recommendations welcomed

How to get multiple Module configs auomatically to different folder using PowerShell?

I have Multiple folders, for Example, www contains Folders A, B, C and So on and this same structure exist in a different environment like INT, REG, DR, PROD. each contains different web.config.
So Here comes the challenge we need to provide an automated way to check-in each web.config.
Named like web_A_int.config, web_A_REG.config , Web_A_Prod.config, Web_A_DR.config and Web_B_int.config , Web_B_Reg_config and so on .
The environment is TFS (2015) Source Control and solution is .net based solution.
Please let me know if anything you might need to understand the question.
How to maintain this structure post build compilation. or How we can use proj or Powershell script to achieve it?
Desired Output:
INT--> www--> A, B,C Folders and each web.config placed inside respective folder.
REG--> www--> A, B, C --> with each web.config and each module level placed correctly.
One way to solve this is to use web transforms and create a web.{environment}.config file and then apply that transformation on deployment.
The Microsoft docs should get you started on how to use web transforms. I can provide you with some Powershell code on how to do this if that's how you decide to go.
I'm not familiar web server deployment in TFS, but here are some regular powershell functions you could use to build loops:
$environmentTypes = #('INT', 'REG', 'DR', 'PROD')
Function Get-WebConfig([String]$Path)
{
try {
$configContent = Get-Content \\path\to\config\file
}
catch {
Write-Debug "ERROR: Could not get content of $($Path)"
}
return $configContent
}
Function Create-FolderStructure() {
[cmdletBinding()]
param(
[Parameter(Mandatory)]
[ValidateSet('INT','REG','DR','PROD')]
[String]$EnvironmentType,
[Parameter(Mandatory)]
[String]$Server,
[Parameter(Mandatory)]
[String[]]$FolderNames,
[Parameter(Mandatory=$false)]
[Switch]$CheckServerUp
)
Begin {}
Process {
try {
if($CheckServerUp)
{
try {
Test-Connection -ComputerName $Server
}
catch {
Write-Debug "ERROR: Unable to test connection to $Server"
}
}
foreach($item in $FolderNames)
{
try {
New-Item -Path "\\$Server\c$\www\$item" -ItemType Directory
}
catch {
Write-Debug "ERROR: Unable to create folder $item"
}
try {
switch($EnvironmentType)
{
'INT'
{
$neededConfig = Get-WebConfig -Path \\path\to\your\intconfig
break
}
'REG'
{
$neededConfig = Get-WebConfig -Path \\path\to\your\regconfig
break
}
'DR'
{
$neededConfig = Get-WebConfig -Path \\path\to\your\drconfig
break
}
'PROD'
{
$neededConfig = Get-WebConfig -Path \\path\to\your\prodconfig
break
}
try {
New-Item -Path "\\$Server\c$\www\$item\Web_$EnvironmentType.config"
Set-Content -Path "\\$Server\c$\www\$item\Web_$EnvironmentType.config" -Value $neededConfig
}
catch {
Write-Debug "ERROR: Unable to create file"
}
}
}
catch {
Write-Debug "ERROR: Unable to get web config content"
}
}
}
catch {
Write-Debug "ERROR: Could not get content of $($Path)"
}
}
End{}
}

TFS 2015 no longer adds build number to Global List upon build complete?

In TFS 2015 new build system, did the functionality to automatically add build number to Global List (Build - Project Name) upon build complete removed?
Do I need to write a custom PowerShell task to accomplish this?
Note: XAML builds still add build number to Global List as it did before.
Since many features are still missing in the vNext build system, I've made a PowerShell script that do the Job.
In a near futur, I plan to update this script to support IntegratedIn field filling and to convert the script as a custom build task.
[CmdletBinding(SupportsShouldProcess=$false)]
param()
function Update-GlobalListXml
{
[CmdletBinding(SupportsShouldProcess=$false)]
param(
[xml]$globalListsDoc,
[parameter(Mandatory=$true)][string][ValidateNotNullOrEmpty()]$glName,
[parameter(Mandatory=$true)][string][ValidateNotNullOrEmpty()]$buildNumber
)
Write-Verbose "Checking whether '$glName' exists"
$buildList = $globalListsDoc.GLOBALLISTS.GLOBALLIST | Where-Object { $_.name -eq $glName }
if ($buildList -eq $null)
{
Write-Host "GlobalList '$glName' does not exist and will be created"
$globalLists = $globalListsDoc.GLOBALLISTS
if($globalLists.OuterXml -eq $null)
{
$newDoc = [xml]"<gl:GLOBALLISTS xmlns:gl="""http://schemas.microsoft.com/VisualStudio/2005/workitemtracking/globallists"""></gl:GLOBALLISTS>"
$globalLists = $newDoc.GLOBALLISTS
}
$globalList = $globalLists.OwnerDocument.CreateElement("GLOBALLIST")
$globalList.SetAttribute("name", $glName)
$buildList = $globalLists.AppendChild($globalList)
}
if(($buildList.LISTITEM | where-object { $_.value -eq $buildNumber }) -ne $null)
{
throw "The LISTITEM value: '$buildNumber' already exists in the GLOBALLIST: '$glName'"
}
Write-Host "Adding '$buildNumber' as a new LISTITEM in '$glName'"
$build = $buildList.OwnerDocument.CreateElement("LISTITEM")
$build.SetAttribute("value", $buildNumber)
$buildList.AppendChild($build) | out-null
return $buildList.OwnerDocument
}
function Invoke-GlobalListAPI()
{
[CmdletBinding(SupportsShouldProcess=$false)]
param(
[parameter(Mandatory=$true)][Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore]$wiStore,
[parameter(Mandatory=$true,ParameterSetName="Import")][switch]$import,
[parameter(Mandatory=$true,ParameterSetName="Import")][xml]$globalLists,
[parameter(ParameterSetName="Export")][switch]$export
)
try {
if($import)
{
$wiStore.ImportGlobalLists($globalLists.OuterXml) # Account must be explicitly in the Project Administrator Group
}
if($export)
{
return [xml]$wiStore.ExportGlobalLists()
}
}
catch [Microsoft.TeamFoundation.TeamFoundationServerException] {
Write-Error "An error has occured while exporting or importing GlobalList"
throw $_
}
}
function Get-WorkItemStore()
{
[CmdletBinding(SupportsShouldProcess=$false)]
param(
[parameter(Mandatory=$true)][string][ValidateNotNullOrEmpty()]$tpcUri,
[parameter(Mandatory=$true)][string][ValidateNotNullOrEmpty()]$agentWorker
)
# Loads client API binaries from agent folder
$clientDll = Join-Path $agentWorker "Microsoft.TeamFoundation.Client.dll"
$wiTDll = Join-Path $agentWorker "Microsoft.TeamFoundation.WorkItemTracking.Client.dll"
[System.Reflection.Assembly]::LoadFrom($clientDll) | Write-Verbose
[System.Reflection.Assembly]::LoadFrom($wiTDll) | Write-Verbose
try {
Write-Host "Connecting to $tpcUri"
$tfsTpc = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($tpcUri)
return $tfsTpc.GetService([Microsoft.TeamFoundation.WorkItemTracking.Client.WorkItemStore])
}
catch [Microsoft.TeamFoundation.TeamFoundationServerException] {
Write-Error "An error has occured while retrieving WorkItemStore"
throw $_
}
}
function Get-WITDataStore64
{
[CmdletBinding(SupportsShouldProcess=$false)]
param()
if($env:VS140COMNTOOLS -eq $null)
{
throw New-Object System.InvalidOperationException "Visual Studio 2015 must be installed on the build agent" # TODO: Change it by checking agent capabilities
}
$idePath = Join-Path (Split-Path -Parent $env:VS140COMNTOOLS) "IDE"
return Get-ChildItem -Recurse -Path $idePath -Filter "Microsoft.WITDataStore64.dll" | Select-Object -First 1 -ExpandProperty FullName
}
function Update-GlobalList
{
[CmdletBinding(SupportsShouldProcess=$false)]
param()
# Get environment variables
$tpcUri = $env:SYSTEM_TEAMFOUNDATIONCOLLECTIONURI
Write-Verbose "Team Project Collection Url: '$tpcUri'"
$teamProjectName = $env:SYSTEM_TEAMPROJECT
Write-Verbose "Team Project: '$teamProjectName'"
$buildNumber = $env:BUILD_BUILDNUMBER
Write-Verbose "Build Number: '$buildNumber'"
$agentHome = $env:AGENT_HOMEDIRECTORY
Write-Verbose "Agent home direrctory: '$agentHome'"
$globalListName = "Builds - $teamProjectName"
Write-Verbose "GlobalList name: '$teamProjectName'"
# Copy 'Microsoft.WITDataStore64.dll' from Visual Studio directory to AgentBin directory if it does not exist
$agentWorker = Join-Path $agentHome "agent\Worker"
$targetPath = Join-Path $agentWorker "Microsoft.WITDataStore64.dll" # Only compatible with x64 process #TODO use constant instead
if(-not (Test-Path $targetPath))
{
$wITDataStore64FilePath = Get-WITDataStore64
Write-Host "Copying $wITDataStore64FilePath to $targetPath"
Copy-Item $wITDataStore64FilePath $targetPath | Write-Verbose
}
$wiStore = Get-WorkItemStore -tpcUri $tpcUri -agentWorker $agentWorker
# Retrive GLOBALLISTS
$xmlDoc = Invoke-GlobalListAPI -export -wiStore $wiStore
$gls2 = Update-GlobalListXml -globalListsDoc $xmlDoc -glName $globalListName -buildNumber $buildNumber
Invoke-GlobalListAPI -import -globalLists $gls2 -wiStore $wiStore
}
Update-GlobalList
Here is the link of the Github repo, feedbacks are welcome => https://github.com/GregoryOtt/UpdateWiBuildNum/blob/master/Update-GlobalList.ps1
[disclaimer - I work on the new build system]
That global list on the workitem is a mechanism that dated back to the original release of TFS. It's one that sort of worked in that day and age (days of nightly builds, pre-CI and CD agility). It's starts to fall apart and doesn't show as proper relationships in TFS. I worked on WIT at that time and we needed a queryable mechanism and that's what we had (blame me :)
So, when we started a new build system, we didn't want to rebuild things and repeat the same mistakes. We're trying to take an agile, incremental approach to a better build system.
In the next sprint (88), we are starting work on proper links between builds and workitems and the WIT team is also doing work to make them more first class. The first thing you'll see is a link on the WIT form and that should hopefully make QU1 as well (at least parts of it).
We realize this does leave a few gaps but we are working to close them (gated and label sources being two others) and hopefully in a better way for a better long term product.
As far as a workaround goes, it should be possible to automate via powershell and our clients but we don't have anything canned for others to use.

Save file in PowerShell script access denied

I have a PowerShell script that is intended to modify a web config transform as a pre-build event in a build definition. I've gotten it working for the most part, however when it goes to save the updated file I am getting access denied.
Is there a way to give the right access, without opening a window as this is done via the TFS build agent?
Here is the script:
param(
[string]$buildTarget="Dev",
[string]$projectName="SalesTools"
)
$VerbosePreference = "continue"
Write-Verbose "Params: buildTarget = '$($buildTarget)', projectName = '$($projectName)'"
# Make sure path to source code directory is available
if (-not $Env:TF_BUILD_SOURCESDIRECTORY)
{
Write-Error ("TF_BUILD_SOURCESDIRECTORY environment variable is missing.")
exit 1
}
elseif (-not (Test-Path $Env:TF_BUILD_SOURCESDIRECTORY))
{
Write-Error "TF_BUILD_SOURCESDIRECTORY does not exist: $Env:TF_BUILD_SOURCESDIRECTORY"
exit 1
}
Write-Verbose "TF_BUILD_SOURCESDIRECTORY: $Env:TF_BUILD_SOURCESDIRECTORY"
$webConfig = "$($Env:TF_BUILD_SOURCESDIRECTORY)\$($buildTarget)\SalesTools.Web\$($projectName)\web.$($buildTarget).config"
#$webConfig = "$($Env:TF_BUILD_SOURCESDIRECTORY)\$($buildTarget)\SalesTools.Web\ARCTools\web.$($buildTarget).config"
Write-Verbose "File Path: $($webConfig)"
$doc = (gc $webConfig) -as [xml]
$versionNumber = $doc.SelectSingleNode('//appSettings/add[#key="versionNumber"]/#value').'#text'
Write-Verbose "Current Version Number: $($versionNumber)"
if (($versionNumber))
{
$versionInfo = $versionNumber.Split(".")
$versionIteration = $versionInfo[1]
$minorVersion = $versionInfo[2] -as [int]
$minorVersion = $minorVersion + 1
$currentIteration = Get-Iteration
$newVersionInfo = ("v: 1.$($currentIteration).$($minorVersion)")
}
else
{
Write-Error "Could not get version info from config."
exit 1
}
$doc.SelectSingleNode('//appSettings/add[#key="versionNumber"]/#value').'#text' = $newVersionInfo
$doc.Save($webConfig)
Before you read & update the web.config, try to change the "Read-Only" attribute of web.config file. Because by default, all the source files are "Read-Only".
Add this line before "$doc = ....":
attrib -R $webConfig /S

Powershell Script: prompt a file (by mask) and use that file in a command line

Disclaimer: I don't know enough about ps to accomplish this in a reasonable amount of time, so yes, I am asking someone else to do my dirty job.
I want to be able to run a web.config transformation without opening a command line.
I have following files in a folder:
web.config - actual web config
web.qa.config - web config transformation for qa env
web.production.config - web config transformation for production env
transform.ps1 - powershell script I want to use to run transformation
Here is what I want:
PS file shall enumerate current directory using .*\.(?<env>.*?)\.config and let me choose which <env> I am interested in generate web.config for. In my example I will be presented with two options: "qa", "production".
After I (user) select the environment (let's say it is "qa", selected environment is stored as $env, and corresponding filename will be stored as $transformation) script shall do following:
backup original web.config as web.config.bak
execute following command:
.
echo applying $transformation...
[ctt][1].exe source:web.config transformation:$transformation destination:web.config preservewhitespaces verbose
echo done.
ctt.exe is a tool based on XDT that runs web.config transformation from command line.
Okay, looks simple enough, I'll do your dirty job for you. ;)
Save the following as transform.ps1:
$environments = #()f
gci | %{if ($_ -match '.*\.(?<env>.*?)\.config') {$environments += $matches.env}}
Write-Host "`nEnvironments:"
for ($i = 0; $i -lt $environments.Length; $i++) {Write-Host "[$($i + 1)] $($environments[$i])"}
Write-Host
do {
$selection = [int](Read-Host "Please select an environment")
if ($selection -gt 0 -and $selection -le $environments.Length) {
$continue = $true
} else {
Write-Host "Invalid selection. Please enter the number of the environment you would like to select from the list."
}
} until ($continue)
$transformation = "web.$($environments[$selection - 1]).config"
if (Test-Path .\web.config) {
cpi .\web.config .\web.config.bak
} else {
Write-Warning "web.config does not exist. No backup will be created."
}
if ($?) {
Write-Host "`nApplying $transformation..."
[ctt][1].exe source:web.config transformation:$transformation destination:web.config preservewhitespaces verbose
Write-Host "Done.`n"
} else {
Write-Error "Failed to create a backup of web.config. Transformation aborted."
}