Deploying an SSIS package with dtutil - powershell

I am trying to deploy a package with dtutil, however I am getting an error that reads:
Option "-xxxx" is not valid.
This correlates to the SOURCESERVER pararmeter I am providing, which has a hypen in it.
$conString = "xxx-xxxx";
$dtUtilQueryToExecute = '/SOURCESERVER "' + $conString + '" /sql "my path" /exists';
$result = dtutil $dtUtilQueryToExecute
I think that dtutil is expecting a new parameter when it reaches the hyphen and cannot escape the hyphen even though it is in double/single quotation marks.
The reason why I am using dtutil is that I need to be able to deploy the same package to many other servers from one location.
So either I need to figure out how to escape the hyphen in the server name, or I need to use an alternative deployment method.
How do I escape the hyphen, or what alternative deployment method can I use?

I believe PowerShell is interpreting the hyphen in the variable. It looks like the wisdom of the internet suggests permutations like
$conString = 'xxx-xxxx'
$conString = "xxx`-xxxx"
Also, no trailing semicolon is required in PowerShell

As an alternative method why not use SMO with Powershell. That wayyou can create a script to deploy your package and also create foders, projects, environments and SQL Agent tasks.
Here is an example of something I would use:
$here = Split-Path -Parent $MyInvocation.MyCommand.Path
$target_server_name = 'target_server_name'
$source_server_name = 'source_server_name'
$target_database_name = 'target_database_name'
$source_database_name = 'source_database_name'
$environment_name = 'enviornment_name'
$folder_name = 'folder_name'
$project_name = 'project_name'
$ispac_name = 'ispac_name'
$variables = #{
email_alert = 'xx#xx.com'
email_error = 'xx#xx.com'
email_reply = 'noreply#xx.com'
smtp_server = 'outlook.xx.local'
tax_rate = '20'
}
function Write-Message ($msg) {
Write-Host ' [+] ' -ForegroundColor Yellow -NoNewline
Write-Host $msg
}
Write-Host 'Starting deployment' -ForegroundColor DarkGray
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo") | Out-Null
[System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.Management.IntegrationServices") | Out-Null
Write-Message ("Connecting to integration services on '$source_server_name'")
$conn_str = "Data Source={0};Initial Catalog=master;Integrated Security=SSPI;" -f $source_server_name
$conn = New-Object System.Data.SqlClient.SqlConnection $conn_str
$namespace = 'Microsoft.SqlServer.Management.IntegrationServices'
$intg_serv = New-Object "$namespace.IntegrationServices" $conn
$catalog = $intg_serv.Catalogs['SSISDB']
$folder = $catalog.Folders[$folder_name]
if(!$folder) {
Write-Message 'Creating folder ...'
$folder = New-Object "$($namespace).CatalogFolder" ($catalog, $Folder_name, $Folder_name)
$folder.Create()
} else {
Write-Message 'Folder found ...'
}
Write-Message 'Deploying project file ...'
$path = "$here\..\path_to_your_project\bin\Development\$($ispac_name).ispac"
if(Test-Path $path) {Copy-Item (Resolve-Path $path) $here}
[byte[]]$project_file = [System.IO.File]::ReadAllBytes("$here\crm_imports.ispac")
$dummy = $folder.DeployProject($project_name, $project_file)
$project = $folder.Projects[$project_name]
Write-Message 'Removing old environment ...'
$old = $folder.Environments | Select -ExpandProperty name
$old | % {$folder.Environments[$_].Drop()}
Write-Message "Creating environment '$envionment_name'"
$environment = New-Object "$namespace.EnvironmentInfo" ($folder, $environment_name, $environment_name)
$environment.Create()
$variables.GetEnumerator() | % {
$n = $_.Name
$v = $_.Value
Write-Message " -> adding environment variable $($n): $v"
$environment.Variables.Add($n,[System.TypeCode]::String, $v, $false, $n)
$environment.Alter()
$project.Packages | % {
if($_.Parameters[$n] -ne $null)
{
$_.Parameters[$n].Set([Microsoft.SqlServer.Management.IntegrationServices.ParameterInfo+ParameterValueType]::Referenced, $n)
$_.Alter()
}
}
}
$reference = $project.References[$environment.name, $folder.Name]
if(!$reference) {
$project.References.Add($environment.name, $folder.Name)
$project.Alter()
}
$environment_ref_id = $project.References[0].ReferenceId
$job_name = 'job_name'
$server = New-Object microsoft.sqlserver.management.smo.server $source_server_name
$Job_server = $server.JobServer
$job = $job_server.Jobs[$job_name]
if($job) {$job.Drop()}
Write-Message "Creating SQL agent job '$job_name'"
$job = New-Object Microsoft.SqlServer.Management.SMO.Agent.Job -ArgumentList $Job_server, $job_name
$job.Create()
$job.OwnerLoginName = 'sa'
$job.ApplyToTargetServer($source_server_name)
$job.Alter()
$job_step = New-Object Microsoft.SqlServer.Management.SMO.Agent.JobStep -ArgumentList $job, 'Run SSIS import_donations package'
$job_step.SubSystem = 'Ssis'
$job_step.command = ('/ISSERVER "\"\SSISDB\{3}\{4}\{5}.dtsx\"" ' +
'/SERVER {0} /ENVREFERENCE {1} ' +
'/Par "\"CM.source.ConnectionString\"";' +
'"\"Data Source={0};Initial Catalog={6};Integrated Security=SSPI;Application Name=my_app;\"" ' +
'/Par "\"CM.source.InitialCatalog\"";"\"{6}\"" ' +
'/Par "\"CM.source.ServerName\"";{0} ' +
'/Par "\"CM.target.ConnectionString\"";' +
'"\"Data Source={2};Initial Catalog={7};Integrated Security=SSPI;Application Name=my_app;\"" ' +
'/Par "\"CM.target.InitialCatalog\"";"\"{7}\"" ' +
'/Par "\"CM.target.ServerName\"";{2} ' +
'/Par "\"$ServerOption::LOGGING_LEVEL(Int16)\"";3 ' +
'/Par "\"$ServerOption::SYNCHRONIZED(Boolean)\"";True /CALLERINFO SQLAGENT /REPORTING E') -f $source_server_name, $reference.ReferenceId, $target_server_name, $folder_name, $project_name, $ispac_name, $source_database_name, $target_database_name
$job_step.Create()
$conn.Close()

Related

Comparing Server IDs in two tools via PowerShell script

I am making a PowerShell script that is supposed to retrieve and compare the server IDs in two tools that we are using - Octopus tool and MOSS (the idea is to check that all servers in Octopus are also registered in MOSS). The Octopus is accessed on PowerShell via MySQL query and the MOSS is accessed via API call. Currently I am able to retrieve successfully the sql query and format it to JSON to be able to be readable by MOSS. However, I am not aware as to how to make the check if the server IDs are also present in the MOSS. All that the script does is retrieve the IDs from the Octopus SQL and then parse them to JSON and make an empty call to MOSS. Would highly appreciate it if anyone knows how to make MOSS calls from PowerShell.
The current script is:
# Extract RDS and servers from Octopus
# Define log file path
$date = $(get-date).tostring()
$currentdate = get-date -format yyyy-MM-dd
$log_file_path = "C:\Program Files\test\logs\"+$currentdate+"_extract_rds_and_servers_from_octopus.log"
$errorlog_file_path = "C:\Program Files\test\logs\errors\errors.log"
# 0. Exclude Admin Users before getting the RDS licenses which need to be reported
#& ((Split-Path $MyInvocation.InvocationName) + "\exclude_users.ps1") -log_file_path $log_file_path -errorlog_file_path $errorlog_file_path
# 1. Extract ObjectID from Octopus API for current month for each RDP user
try {
$month = (Get-Date -UFormat "%Y%m")
$UrlHost = "https://octopus.mos-windows.eu01.stackit.cloud/api/workspace/55cd5c70-d188-4ac3-b946-f1afec8764ad/report/licensing/spla-usage-reseller?&payload[month_id]=$month&payload[with_itemized]=1&_format=json&_token=j8FE4wZDmBITewHUc7lyYeX9XVVjt3dqz0ID4S6A9KQjkMeKfO7_EcgV7Qshuuw1&_tenant=TlXcM&_language=en&payload[flat_structure]=1"
$HostResponse = Invoke-RestMethod -Uri $UrlHost -Method Get
$users = $HostResponse.itemized
$objectids = #()
foreach ($user in $users.PSObject.Properties) {
if ($user.Value.readable_label -eq "Windows Server Remote Desktop Services")
{
$objectid = $user.Value.object_id
$objectids += $objectid
}
}
}
catch {
$date+" - Error in Octopus API Call: "+$_ | Out-File -Append $errorlog_file_path
exit
}
# 2. Get access device ids from Octopus Database
Import-Module -Name "C:\Program Files\test\request_database.ps1" -ArgumentList $log_file_path, $errorlog_file_path -Verbose
$get_users_devices_query =
#"
select user_id, access_device_ids from oc_reporter.ws_installed_software where user_id is not null;
"#
$ui_ad = execute_db_query $get_users_devices_query
$access_device_ids = $users_devices.access_device_ids
$user_ids = $users_devices.user_id
# 3. Get all openstack server id from Octopus Database 
$get_access_device_server_ids_query =
#"
select id, lower(SUBSTRING_INDEX(SUBSTRING_INDEX(ref_id, "-", -6), "-", 5)) as server_id, ref_id, label from ws_device
where type_id = "vm" and operating_system like "%Windows%" and created > 1644820951;
"#
$ad_si = execute_db_query $get_access_device_server_ids_query
# 4. Map the users/objectids with access device ids and server ids
# Create array with UserID filtered by ObjectID and map each AccessDeviceID(s) to corresponding ServerID
try {
$filteredsi = #()
foreach ($userid in $ui_ad)
{
if ($objectids -contains $userid.user_id)
{
$filteredad = $userid
foreach ($id in $ad_si)
{
if ($filteredad.access_device_ids.split(',') -contains $id.id)
{
$filteredsi += [PSCustomObject]#{"userid" = $filteredad.user_id; "serverid" = $id.server_id}
}
}
}
}
}
catch {
$date+" - Error in Mapping userIDs/objectIDs/accessdeviceIDs/serverIDs: "+$_ | Out-File -Append $errorlog_file_path
exit
}
# Preparation for MOSS
# Create JSON contentblock with looped $filteredsi array
try {
$myArray = $filteredsi
$uniqueUsers = [System.Collections.ArrayList]::new()
for($i = 0; $i -lt $myArray.Count; $i++){
if(!$uniqueUsers.Contains($myArray[$i].userid)){
$uniqueUsers.Add($myArray[$i].userid)
}
}
$allMappings = [System.Collections.ArrayList]::new()
for($i = 0; $i -lt $uniqueUsers.Count; $i++){
$singleMapping = [PSCustomObject]#{id = $uniqueUsers[$i]; servers = $null}
$serverids = [System.Collections.ArrayList]::new()
for($j = 0; $j -lt $myArray.Count; $j++){
if($myArray[$j].userid -eq $uniqueUsers[$i]){
$serverids.Add($myArray[$j].serverid)
}
}
$singleMapping.servers = $serverids
$allMappings.Add($singleMapping)
}
$mosscontent = $allMappings | ConvertTo-Json
$mosscontent
}
catch {
$date+" - Error in creating content block for MOSS: "+$_ | Out-File -Append $errorlog_file_path
exit
}
# Create complete array including contentblock for MOSS API call
try {
$moss = #"
{
"type": "server.usage",
"data": {
"users": $mosscontent
}
}
"#
}
catch {
$date+" - Error in creating array for POST to MOSS: "+$_ | Out-File -Append $errorlog_file_path
exit
}
# 5. Call MOSS prod, dev and qa with the whole list of servers and users
# Authenticating to MOSS
$query_file_path_dev_pw = "~\Documents\MOSSDevEncryptedPassword_"
$query_file_path_qa_pw = "~\Documents\MOSSQaEncryptedPassword_"
$query_file_path_prod_pw = "~\Documents\MOSSProdEncryptedPassword_"
# Function to store credentials
function get_encrypted_content {
param (
[String] $file_path,
[String] $password
)
# Check if credentials file exis
if ( -Not (Test-Path -Path $file_path)) {
switch ($password) {
dev {
# Get credentials
Read-Host -Prompt "Enter password for mos-windows-us-dev-client-id" -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath $file_path
}
qa {
# Get credentials
Read-Host -Prompt "Enter password for mos-windows-us-qa-client-id" -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath $file_path
}
prod {
# Get credentials
Read-Host -Prompt "Enter password for mos-windows-us-prod-client-id" -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath $file_path
}
}
}
# Read credentials from file
$Encrypted_value = Get-Content -Path $file_path
# Decrypt credentials from file
return [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR((ConvertTo-SecureString $Encrypted_value)))
}
# Define username and password
$clientid_dev = "mos-windows-us-dev"
$clientid_qa = "mos-windows-us-qa"
$clientid_prod = "mos-windows-us-prod"
$dev_pass = get_encrypted_content $query_file_path_dev_pw "dev"
$qa_pass = get_encrypted_content $query_file_path_qa_pw "qa"
$prod_pass = get_encrypted_content $query_file_path_prod_pw "prod"
[System.Security.SecureString]$clientsecret_dev = ConvertTo-SecureString -String $dev_pass -AsPlainText -Force
[System.Security.SecureString]$clientsecret_qa = ConvertTo-SecureString -String $qa_pass -AsPlainText -Force
[System.Security.SecureString]$clientsecret_prod = ConvertTo-SecureString -String $prod_pass -AsPlainText -Force
#Prepare static variables
$MOSSToken_dev = 'https://auth.00.idp.eu01.stackit.cloud/oauth/token'
$MOSSToken_qa = 'https://auth.01.idp.eu01.stackit.cloud/oauth/token'
$MOSSToken_prod = 'https://auth.01.idp.eu01.stackit.cloud/oauth/token'
$MOSSUrl_dev = "https://stackit-service-mos-dev.apps.01.cf.eu01.stackit.cloud/v1/events"
$MOSSUrl_qa = "https://stackit-service-mos-qa.apps.01.cf.eu01.stackit.cloud/v1/events"
$MOSSUrl_prod = "https://stackit-service-mos.apps.01.cf.eu01.stackit.cloud/v1/events"
$body = #{grant_type='client_credentials'}
#Set function to get all customerinfo from all portals
function call_moss {
param (
[String] $clientid,
[SecureString] $clientsecret,
[String] $MOSSToken,
[String] $MOSSUrl
)
$cred = New-Object -typename System.Management.Automation.PSCredential -ArgumentList $clientid, $clientsecret
#Get Token from MOSS
$Response = Invoke-RestMethod -Uri $MOSSToken -Method Post -Credential $cred -Body $body -ContentType "application/x-www-form-urlencoded"
$Token = $Response.access_token
$Tokenfinal = "Bearer " + $Token
#Post Content to MOSS
Invoke-RestMethod -Uri $MOSSUrl -Method Post -Headers #{'Authorization' = $Tokenfinal } -Body $moss -ContentType "application/json"
}
#Call function to Call MOSS
try
{
Write-Host "Call to MOSS Dev.."
call_moss $clientid_dev $clientsecret_dev $MOSSToken_dev $MOSSUrl_dev
Write-Host "Call to MOSS QA.."
call_moss $clientid_qa $clientsecret_qa $MOSSToken_qa $MOSSUrl_qa
Write-Host "Call to MOSS Prod"
call_moss $clientid_prod $clientsecret_prod $MOSSToken_prod $MOSSUrl_prod
}
catch
{
$date+" - Error in calling MOSS: "+$_ | Out-File -Append $errorlog_file_path
exit
}
# 6. Create daily logs with users reported to MOSS
$date = $(get-date).tostring()
$log = foreach ($item in $filteredsi)
{
$item
}
echo $date $log | Out-file -Append $log_file_path
# delete logs older than 60 days
$limit = (Get-Date).AddDays(-60)
$path = "C:\Program Files\test\logs"
# Delete files older than the $limit.
Get-ChildItem -Path $path -Recurse -Force | Where-Object { !$_.PSIsContainer -and $_.CreationTime -lt $limit } | Remove-Item -Force
# Create activity file to check if script is working
$temp_file_path = "C:\Program Files\test\tempfile*"
if (Test-Path $temp_file_path)
{
Remove-Item $temp_file_path
}
[string]$filePath = "C:\Program Files\test\tempfile";
[string]$directory = [System.IO.Path]::GetDirectoryName($filePath);
[string]$strippedFileName = [System.IO.Path]::GetFileNameWithoutExtension($filePath);
[string]$extension = [System.IO.Path]::GetExtension($filePath);
[string]$newFileName = $strippedFileName + "_" + (Get-Date).ToString('MM-dd-yyyy') + $extension;
[string]$newFilePath = [System.IO.Path]::Combine($directory, $newFileName);
New-Item $newFilePath
Additional scripts that are being called:
-> request_database.ps1
# 1. Get path for log files
param(
[string]$log_file_path = "C:\Program Files\test\logs\request_database.log",
[string]$errorlog_file_path = "C:\Program Files\test\logs\request_database_errors.log"
)
# 2. Get credentials to access Octopus DB
try {
# Define username and password security string files
$mysql_user_file_path = "~\Documents\ue_"
$mysql_pass_file_path = "~\Documents\pe_"
# Functions
function get_encrypted_content {
param (
[String] $file_path,
[String] $user_or_pass
)
# Check if credentials file exist
if ( -Not (Test-Path -Path $file_path)) {
switch ($user_or_pass) {
msqu {
# Get credentials
Read-Host -Prompt "Please, enter MySQL username" -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath $file_path
}
msqp {
# Get credentials
Read-Host -Prompt "Please, enter MySQL password" -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath $file_path
}
}
}
# Read credentials from file
$Encrypted_value = Get-Content -Path $file_path
# Decrypt credentials from file
return [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR((ConvertTo-SecureString $Encrypted_value)))
}
# Define username and password
$my_sql_user = get_encrypted_content $mysql_user_file_path "msqu"
$my_sql_pass = get_encrypted_content $mysql_pass_file_path "msqp"
[System.Security.SecureString]$SecPwd = ConvertTo-SecureString -String $my_sql_pass -AsPlainText -Force
$Credential = new-object -typename System.Management.Automation.PSCredential -argumentlist #($my_sql_user,$SecPwd)
}
catch {
$(get-date).tostring() +" - Error in fetching SQL user/pwd: "+$_ | Out-File -Append $errorlog_file_path
exit
}
$error_string = ""
# 3. Execute DB query
# If you want the script to continue on error you have to provide $true as second parameter
# Example: execute_db_query $exclude_stackitadmin_users_query $true
function execute_db_query {
param (
[String] $query,
[bool] $continueOnError = $false
)
try {
# Query Octopus DB
Connect-MySqlServer -Credential $Credential -Server localhost
if ($continueOnError) {
$query_results = Invoke-MySqlQuery -Query $query
} else {
$query_results = Invoke-MySqlQuery -Query $query -ErrorAction Stop
}
Disconnect-MySqlServer
return $query_results
}
catch {
$error_message = $(get-date).tostring() + " - Error in DB Query 1: " + $_
$error_message | Out-File -Append $errorlog_file_path
Write-Error $error_message
exit
}
}
-> exclude_users.ps1 (probably not needed for the task but the overall script doesn't work without it)
# 1. Get path for log files or use default
param(
[string]$log_file_path = "C:\Program Files\test\logs\exclude_users.log",
[string]$errorlog_file_path = "C:\Program Files\test\logs\errors\exclude_users_errors.log"
)
Import-Module -Name "C:\Program Files\test\request_database.ps1" -ArgumentList $log_file_path, $errorlog_file_path -Verbose
# Exclude StackitAdmin users
$exclude_stackitadmin_users_query =
#"
update oc_reporter.ws_user as updated_user,
(
select id from oc_reporter.ws_user
where username = "StackITAdmin" and exclude_spla = "no"
) as us
set
exclude_spla = "yes",
exclude_spla_reason = "nh"
where updated_user.id = us.id;
"#
execute_db_query($exclude_stackitadmin_users_query)
# Exclude users on dev & qa environment
$exclude_users_on_dev_qa_query =
#"
update oc_reporter.ws_installed_software as updated_software,
(
select ws.access_device_ids, min(ws.access_device_labels), min(ws.user_label), excluded_users, min(ws.id) as id from oc_reporter.ws_user
join oc_reporter.ws_installed_software as ws
on ws.user_id = ws_user.id
join oc_reporter.ws_customer as wc
on wc.id = ws_user.customer_id
left join
(select access_device_ids, count(1) as excluded_users from oc_reporter.ws_user as wu
join oc_reporter.ws_customer as wc
on wc.id = wu.customer_id
join oc_reporter.ws_installed_software as ws
on ws.user_id = wu.id
where
(internal_id like "d-%" or internal_id like "q-%") and
locate(',', access_device_ids) = 0 and
ws.exclude_spla = "yes" and
ws.label = "Microsoft Remote Desktop Services" and
wu.username != "StackitAdmin"
group by access_device_ids, wu.exclude_spla) as servers
on servers.access_device_ids = ws.access_device_ids
where
ws.exclude_spla = "no" and
ws.label = "Microsoft Remote Desktop Services" and
(internal_id like "d-%" or internal_id like "q-%") and
locate(',', ws.access_device_ids) = 0
group by ws.access_device_ids
having (excluded_users = 1 or excluded_users is null)
) as us
set
exclude_spla = "yes",
exclude_spla_reason = "admin"
where updated_software.id = us.id;
"#
# run twice to exlude 2 users per vm
execute_db_query($exclude_users_on_dev_qa_query)
execute_db_query($exclude_users_on_dev_qa_query)
# Exclude users from our mos-windows-2 project
$exclude_users_from_our_projects =
#"
update oc_reporter.ws_installed_software as ins,
(
select ws.access_device_ids, min(ws.access_device_labels), min(ws.user_label), excluded_users, min(ws.id) as id, min(wu.id) from oc_reporter.ws_user as wu
join oc_reporter.ws_installed_software as ws
on ws.user_id = wu.id
join oc_reporter.ws_device as wd
on wd.id = ws.access_device_ids
left join (
select ws.access_device_ids, min(ws.id), min(wu.id), count(1) as excluded_users from oc_reporter.ws_user as wu
join oc_reporter.ws_installed_software as ws
on ws.user_id = wu.id
join oc_reporter.ws_device as wd
on wd.id = ws.access_device_ids
where
ws.exclude_spla = "yes" and
ws.label = "Microsoft Remote Desktop Services" and
LOCATE(',',access_device_ids) = 0 and
(
hkey like "%d57abb0200304506879bd8037f7a49cb%" or
hkey like "%fce60e2c938c49e4a37687492a45b652%" or
hkey like "%8eb91d45f25b45978b71abb0e06a0443%" or
hkey like "%66ad75e4ff624f7e940dc363549c8404%" or
hkey like "%351aa84fb9b54be896112b36ae15dd48%" or
hkey like "%64edd6c19e17417d86094e6a02610eed%"
) and
wu.username != "StackitAdmin"
group by ws.access_device_ids
) as excluded_ws
on
excluded_ws.access_device_ids = ws.access_device_ids
where
ws.exclude_spla = "no" and
ws.label = "Microsoft Remote Desktop Services" and
LOCATE(',',ws.access_device_ids) = 0 and
(
hkey like "%d57abb0200304506879bd8037f7a49cb%" or
hkey like "%fce60e2c938c49e4a37687492a45b652%" or
hkey like "%8eb91d45f25b45978b71abb0e06a0443%" or
hkey like "%66ad75e4ff624f7e940dc363549c8404%" or
hkey like "%351aa84fb9b54be896112b36ae15dd48%" or
hkey like "%64edd6c19e17417d86094e6a02610eed%"
) and
wu.domain not like "%HOP01%" and
wu.domain not like "%WSUS01%" and
wu.domain not like "%OCKMS%" and
wu.domain not like "%AZDVOP%"
group by ws.access_device_ids
having (excluded_users = 1 or excluded_users is null)
) as rds
set
exclude_spla = "yes",
exclude_spla_reason = "admin"
where ins.id = rds.id;
"#
# run twice to exlude 2 users per vm
execute_db_query($exclude_users_from_our_projects)
execute_db_query($exclude_users_from_our_projects)
The order was wrong, as well as lots of unneccessary elements causing errors and crashes.
Current working code is:
# Compare the data between the MOSS and the Octopus
# Define log file path
$date = $(get-date).tostring()
$currentdate = get-date -format yyyy-MM-dd
$log_file_path = "[FILE PATH]"
$errorlog_file_path = "[FILE PATH]"
# 1. Call MOSS (dev, qa, prod) to get the data for all servers created in the last 48 hours
# Authenticating to MOSS
$query_file_path_dev_pw = "[FILE PATH]"
$query_file_path_qa_pw = "[FILE PATH]"
$query_file_path_prod_pw = "[FILE PATH]"
# Function to store credentials
function get_encrypted_content {
param (
[String] $file_path,
[String] $password
)
# Check if credentials file exis
if ( -Not (Test-Path -Path $file_path)) {
switch ($password) {
dev {
# Get credentials
Read-Host -Prompt "Enter password for mos-windows-us-dev-client-id" -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath $file_path
}
qa {
# Get credentials
Read-Host -Prompt "Enter password for mos-windows-us-qa-client-id" -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath $file_path
}
prod {
# Get credentials
Read-Host -Prompt "Enter password for mos-windows-us-prod-client-id" -AsSecureString | ConvertFrom-SecureString | Out-File -FilePath $file_path
}
}
}
# Read credentials from file
$Encrypted_value = Get-Content -Path $file_path
# Decrypt credentials from file
return [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR((ConvertTo-SecureString $Encrypted_value)))
}
# Define username and password
$clientid_dev = "[USERNAME]"
$clientid_qa = "[USERNAME]"
$clientid_prod = "[USERNAME]"
$dev_pass = get_encrypted_content $query_file_path_dev_pw "dev"
$qa_pass = get_encrypted_content $query_file_path_qa_pw "qa"
$prod_pass = get_encrypted_content $query_file_path_prod_pw "prod"
[System.Security.SecureString]$clientsecret_dev = ConvertTo-SecureString -String $dev_pass -AsPlainText -Force
[System.Security.SecureString]$clientsecret_qa = ConvertTo-SecureString -String $qa_pass -AsPlainText -Force
[System.Security.SecureString]$clientsecret_prod = ConvertTo-SecureString -String $prod_pass -AsPlainText -Force
# Time variable
$date48h = ("{0:yyyy-MM-ddThh:mm:ss}" -f ((get-date).Addhours(-48))).split("T").split(":")
$date = $date48h[0]
$hour = $date48h[1]
$min = $date48h[2]
$sec = $date48h[3]
#Prepare static variables
$MOSSToken_dev = '[URL]'
$MOSSToken_qa = '[URL]'
$MOSSToken_prod = '[URL]'
$MOSSUrl_dev = "[URL]"
$MOSSUrl_qa = "[URL]"
$MOSSUrl_prod = "[URL]"
$body = #{grant_type='client_credentials'}
#Set function to get all customerinfo from all portals
function call_moss {
param (
[String] $clientid,
[SecureString] $clientsecret,
[String] $MOSSToken,
[String] $MOSSUrl
)
$cred = New-Object -typename System.Management.Automation.PSCredential -ArgumentList $clientid, $clientsecret
#Get Token from MOSS
$Response = Invoke-RestMethod -Uri $MOSSToken -Method Post -Credential $cred -Body $body -ContentType "application/x-www-form-urlencoded"
$Token = $Response.access_token
$Tokenfinal = "Bearer " + $Token
#Post Content to MOSS
Invoke-RestMethod -Uri $MOSSUrl -Method Get -Headers #{'Authorization' = $Tokenfinal } -ContentType "application/json"
}
#Call function to Call MOSS
try
{
Write-Host "Call to MOSS Dev.."
$get_moss_dev = call_moss $clientid_dev $clientsecret_dev $MOSSToken_dev $MOSSUrl_dev
Write-Host "Call to MOSS QA.."
$get_moss_qa = call_moss $clientid_qa $clientsecret_qa $MOSSToken_qa $MOSSUrl_qa
Write-Host "Call to MOSS Prod"
$get_moss_prod = call_moss $clientid_prod $clientsecret_prod $MOSSToken_prod $MOSSUrl_prod
}
catch
{
$date+" - Error in calling MOSS: "+$_ | Out-File -Append $errorlog_file_path
exit
}
$moss_dev_serverids = $get_moss_dev.items.id
$moss_qa_serverids = $get_moss_qa.items.id
$moss_prod_serverids = $get_moss_prod.items.id
$moss_serverid_arr = #($moss_dev_serverids, $moss_qa_serverids, $moss_prod_serverids)
# 2. Call Octopus to get the data for new servers created in the last 36 hours
Import-Module -Name "[FILE PATH]" -ArgumentList $log_file_path, $errorlog_file_path -Verbose
# Calculate timestamp
$DateTime = Get-Date #or any other command to get DateTime object
$CurrentUnixTime = ([DateTimeOffset]$DateTime).ToUnixTimeSeconds()
$queryTime = $CurrentUnixTime - (36 * 3600)
$get_new_servers_query_oc =
#"
select id, lower(SUBSTRING_INDEX(SUBSTRING_INDEX(ref_id, "-", -6), "-", 5)) as server_id, ref_id, label from oc_reporter.ws_device where type_id = "vm" and operating_system like "%Windows%" and created > $queryTime;
"#
$query = execute_db_query $get_new_servers_query_oc
$serverid_oc = $query.server_id
$serverid_oc_arr = #($serverid_oc)
# 3. Compare the properties in MOSS and Octopus
$unmatching_serverids = $serverid_oc_arr | Where {$moss_serverid_arr -NotContains $_}
$error_report = #($unmatching_serverids)
# Create daily logs with servers in Octopus that are unregistered in MOSS
$date = $(get-date).tostring()
$log = $error_report
echo $date $log | Out-file -Append $log_file_path
# delete logs older than 60 days
$limit = (Get-Date).AddDays(-60)
$path = "[FILE PATH]"
# Delete files older than the $limit.
Get-ChildItem -Path $path -Recurse -Force | Where-Object { !$_.PSIsContainer -and $_.CreationTime -lt $limit } | Remove-Item -Force
# 4. Generate summary with all errors and send a notification if there is an error. Schedule a task to check once per day.
If ($error_report.Count -eq 0) {
exit
}
else {
$JSONBody = [PSCustomObject][Ordered] #{
"type" = "MessageCard"
"title" = "Octopus Alerts"
"text" = "Servers located in Octopus, that are not registered in MOSS. <br>
Please check logs."
}
$TeamsMessageBody = ConvertTo-Json $JSONBody
$parameters = #{
"URI" = '[URL]'
"Method" = 'POST'
"Body" = $TeamsMessageBody
"ContentType" = 'application/json'
}
Invoke-RestMethod #parameters
}
# Create activity file to check if script is working
$temp_file_path = "[FILE PATH]"
if (Test-Path $temp_file_path)
{
Remove-Item $temp_file_path
}
[string]$filePath = "[FILE PATH]";
[string]$directory = [System.IO.Path]::GetDirectoryName($filePath);
[string]$strippedFileName = [System.IO.Path]::GetFileNameWithoutExtension($filePath);
[string]$extension = [System.IO.Path]::GetExtension($filePath);
[string]$newFileName = $strippedFileName + "_" + (Get-Date).ToString('MM-dd-yyyy') + $extension;
[string]$newFilePath = [System.IO.Path]::Combine($directory, $newFileName);
New-Item $newFilePath
Also, as already mentioned the exclude_users script was completely not needed. The only additionally included script is the request_database script.

PowerShell script ignores files with <whitespace>H

I inherited this script from my precursor, which is supposed to copy a bunch of files from one location into a JFrog Artifactory.
It workes fine except for files with followed by an H in the name.
For example Test My Client.exe will be copied correctly but Test My Host.exe will be ignored by the script.
Anybody has an idea what the reason for this strange behaviour is?
param
(
[string]$RootPath="D:\_temp\upload\",
[string]$Repo="repo-snapshot",
[string]$Product="SEARCHER",
[string]$Version="1.23.45678.9012",
[string]$ArtifactoryRoot
)
$AF_USER = "myUser"
$AF_PWD = "pA55VVoRd" #ConvertTo-SecureString "pA55VVoRd" -AsPlainText -Force
$CREDS = New-Object System.Management.Automation.PSCredential ($AF_USER, $AF_PWD)
$MajorMinor = $Version.Split('.')[0] + "." + $Version.Split('.')[1]
$BuildRevision = $Version.Split('.')[2] + "." + $Version.Split('.')[3]
$uri = "https://artifactory.test.com/artifactory/$Repo/TEST/$Product/$MajorMinor/$BuildRevision/$ArtifactoryRoot/"
$curl = "$PSScriptRoot\..\..\bin\ReleaseU\curl.exe"
Write-Host $MajorMinor
write-host $BuildRevision
$filesToUpload = Get-ChildItem $RootPath -Recurse -File
foreach($file in $filesToUpload)
{
$artifactoryAddPath = $file.FullName.Remove(0,$RootPath.Length).Replace('\','/')
$completeUrl = $uri+$artifactoryAddPath
write-host $completeUrl
$FileSize = (Get-Item $file.FullName).Length /1MB
Write-Host "Filesize: "$FileSize" MB"
$preTime = Get-Date
write-host "$curl --user $AF_USER`:$AF_PWD --upload-file " $file.FullName "$completeUrl --insecure"
. $curl --user $AF_USER`:$AF_PWD --upload-file $file.FullName $completeUrl --insecure
$postTime = Get-Date
Write-Host "Upload finished"
$timeDiff = $postTime - $preTime
$avgSpeed = $FileSize / $timeDiff.TotalSeconds
Write-Host -Object ("Upload speed is: {0:N2}MB/sec" -f ($avgSpeed));
}

Issue running an script block in powershell to get csv output

In powershell running the below code in a script block and having issue running it
foreach($list in $an){
Write-Host "on loop :" $loop
Write-Host "Checking on VM : "$list.vm
$checkdata = (([datetime]$currentdate) - ([datetime]$list.Created.ToString().split(" ")[0])).Days
Write-Host "Check data " $checkdata
IF($checkdata -gt 2){
Write-Host "on Dupe : " $dupe
#$datecreated = $sizeinGGB = $diff = $null | Out-Null
$datecreated = ($list.Created.ToString().split(" ")[0])
Write-Host $datecreated
$sizeinGGB = [INT]$list.SizeGB
Write-Host $sizeinGGB
$diff = ((([datetime]$currentdate) - ([datetime]$list.Created)).Days)
Write-Host $diff
$repo = [ordered] #{
vm = $list.vm
SNapshotname = $list.Name
DateCreated = $datecreated
Snapshotold = $diff
SizeinGB = $sizeinGGB
}
New-Object PSobject -Property $repo
$dupe ++
}ELSE{
Write-Host $list.vm " : Looks Good"
}
$loop ++
}
If I remove the $repo assignment statement my script is working good. If I put that on the script then script just stop execution on $repo block

Opening a SSRS project using Powershell

I have a report that is copied to a number of different servers. It is imported manually and the data source properties are altered to match the current server's specs. I would like to be able to automate the process by enabling users to open a the SSRS report and dynamically alter it's shared data source properties through PowerShell. I hope you could help. You may see reference below.
The script would accept an input parameter for servername, username and password. Also, the save my password must be ticked.
I couldn't believe I managed to create a script for this. You may make use of the script below as future reference. Comments are available for each part and anything that needs to be altered has a "here" keyword , ex. Your_database_name_here .
Import-Module SqlPs
#Input parameter to get Server\Instancename of your Datasource
$Servername = Read-Host "Please enter your Servername"
$Instancename = Read-Host "Please enter your Instancename. For default instance please press enter"
Write-host ""
if ($Instancename -eq ""){
$ServerInstance = $Servername
}
Else {
$ServerInstance = $Servername +"\"+ $InstanceName
}
#Setting up SSRS Target URL. This is the location where your reports would be deployed.
if ($Instancename -eq ""){
$ReportServerUri = "http://$Servername/ReportServer//ReportService2010.asmx?wsdl"
$TargetURL = "http://$Servername/Reports"
}
Else {
$ReportServerUri = "http://$Servername/ReportServer_$Instancename//ReportService2010.asmx?wsdl"
$TargetURL = "http://$Servername/Reports_$Instancename"
}
$global:proxy = New-WebServiceProxy -Uri $ReportServerUri -UseDefaultCreden
#We would make use of SQL Server Authentication for the reports shared datasource so you need to supply a username and password.
Write-Host " SQL Server Authentication:"
$Username = Read-Host " Username"
$Password = Read-Host -AsSecureString "Password"
$type = $Proxy.GetType().Namespace
$datatype = ($type + '.Property')
$property =New-Object ($datatype);
$property.Name = “NewFolder”
$property.Value = “NewFolder”
$numproperties = 1
$properties = New-Object ($datatype + '[]')$numproperties
$properties[0] = $property;
$newFolder = $proxy.CreateFolder("Reports”, “/”, $properties);
$newFolder = $proxy.CreateFolder("Data Sources”, “/”, $properties);
$Children =$proxy.ListChildren("/",$false)
$DBname = 'Your_Database_Name_Here'
# Creating Datasource through powershell
Write-Host " Creating Datasource ..."
$Name = "Name_Your_Datasource_here"
$Parent = "/Data Sources"
$ConnectString = "data source=$Servername\$Instancename;initial catalog=$DBname"
$type = $Proxy.GetType().Namespace
$DSDdatatype = ($type + '.DataSourceDefinition')
$DSD = new-object ($DSDdatatype)
if($DSD -eq $null){
Write-Error Failed to create data source definition object
}
$CredentialDataType = ($type + '.CredentialRetrievalEnum')
$Cred = new-object ($CredentialDataType)
$CredEnum = ($CredentialDataType).Integrated
$Cred.value__=1
$DSD.CredentialRetrieval =$Cred
$DSD.ConnectString = $ConnectString
$DSD.Enabled = $true
$DSD.EnabledSpecified = $false
$DSD.Extension = "SQL"
$DSD.ImpersonateUserSpecified = $false
$DSD.Prompt = $null
$DSD.WindowsCredentials = $false
$DSD.UserName = $Username
$DSD.Password = [Runtime.InteropServices.Marshal]::PtrToStringAuto([Runtime.InteropServices.Marshal]::SecureStringToBSTR($Password))
$newDSD = $proxy.CreateDataSource($Name,$Parent,$true,$DSD,$null)
#Deploying RLD files to Target URL
Write-Host " Deploying RDL files ..."
$stream = Get-Content 'D:\Your_RDL_path_here.rdl' -Encoding byte
$warnings =#();
$proxy.CreateCatalogItem("Report","Report_Name_here","/Reports",$true,$stream,$null,[ref]$warnings)
#Let's make use of the datasource we just created for your RDL files.
$Items = $global:proxy.listchildren("/Data Sources", $true)
foreach ($item in $items)
{
$DatasourceName = $item.Name
$DatasourcePath = $item.Path
}
$RDLS = $global:proxy.listchildren("/Reports", $true)
foreach ($rdl in $rdls)
{
$report = $rdl.path
$rep = $global:proxy.GetItemDataSources($report)
$rep | ForEach-Object {
$proxyNamespace = $_.GetType().Namespace
$constDatasource = New-Object ("$proxyNamespace.DataSource")
$constDatasource.Name = $DataSourceName
$constDatasource.Item = New-Object ("$proxyNamespace.DataSourceReference")
$constDatasource.Item.Reference = $DataSourcePath
$_.item = $constDatasource.Item
$global:proxy.SetItemDataSources($report, $_)
Write-Host "Changing datasource `"$($_.Name)`" to $($_.Item.Reference)"
}
}
#Open a IE browser to view the report.
$IE=new-object -com internetexplorer.application
$IE.navigate2($TargetURL)
$IE.visible=$true
Write-Host ""
Write-Host "You may now view the Reports through the open IE browser."
Write-Host -ForegroundColor Green "**STEP COMPLETED!"

Shortcut to add files into local sql database

I would like to be able to right click on a file(s) and "send-to" a local MSSQL database. The details are that I would like to store the file contents in "contents" column and the file name in the "filename" column ... how novel :)
*In most cases the file contents is HTML.
It seems like it should be possible through windows shell/SQL Shell using a shortcut to a command in the "shell:sendto" folder.
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') | Out-Null
$Server1 = New-Object ("Microsoft.SqlServer.Management.Smo.Server") 'SQLSERVER'
$Server1.databases["DB"].tables["Table"].rowcount
$RowCount = $server1.databases["DB"].tables["Table"].rowcount.ToString()
$TotalRecords = [int]$RowCount
$wc = New-Object system.net.WebClient
$url = ""
$files = #(Get-ChildItem c:\test\*.*)
"Number of files $($files.length)"
# Errors out when no files are found
if($files.length -lt 1) { return }
foreach($file1 in $files) {
# $txt = Get-Content($file1)
# $txt = $txt.Replace("'", "''")
# Write-Host $file1.name + " - - " + $Txt
$url1 = $url + $file1
Write-Host("URL is " + $url1)
$webpage = $wc.DownloadData($url1)
$string = [System.Text.Encoding]::ASCII.GetString($webpage)
$string = $string.Replace("'", "''")
Invoke-SqlCmd -ServerInstance SERVER -Query "Insert into DATABASE.dbo.Table(text,filename) Values ('$string','$file1')"}