Transfering the content of a text-file using JSON - powershell

I am trying to POST an associative array to a web-service.
I create the array and I use Invoke-RestMethod to POST it:
$LogData = #{
"MyProperty" = "bla bla";
"LogText" = "This is my Text."
}
Invoke-RestMethod -Method Post -Uri $URL -Body ( $LogData | ConvertTo-JSON -Compress ) -ContentType "application/json"
This is working fine.
Now I want the property LogText to be dynamical and not static like in the example above.
I want to take the content of a text-file and therefore I use the code as above, but with Get-Content:
$LogData = #{
"MyProperty" = "bla bla";
"LogText" = Get-Content -Path $LogFile
}
This does not work. The text does not seem to be a String. It seems to be an object. If I have a closer look using the command $LogData.LogText | ConvertTo-JSON, I get an output like:
I did try to use the following, but this did not change the situation:
Get-Content -Path $LogFile -Raw
( Get-Content -Path $LogFile -Raw ).ToString()
How can I solve this?
Thank you

Get-Content attaches some metadata to it's output, and ConvertTo-Json (in Windows PowerShell at least) serializes the attached metadata instead of treating the underlying string as just a string.
You can force ConvertTo-Json to treat them as strings by explicitly typing the containing array:
$LogData = #{
"MyProperty" = "bla bla";
"LogText" = [string[]]#( Get-Content -Path $LogFile )
}
For scalar string values, you can create a new string using "":
$LogData = #{
"MyProperty" = "bla bla";
"LogText" = "$( Get-Content -Path $LogFile -Raw )"
}

Using the following lines of code did the trick:
$LogData = #{
"MyProperty" = "bla bla";
"LogText" = "$( Get-Content -Path $LogFile -Raw )"
}
Invoke-RestMethod -Method Post -Uri $URL -Body ( $LogData | ConvertTo-JSON -Compress ) -ContentType "application/json; charset=utf-8"
Most important thing was adding charset=utf-8 to the ContentType. Without it the webservice did "sometimes" respond with "405 Method Not Allowed".

Related

Powershell - Loop through folder, Get Contents and post to SOAP

I am trying to loop through a folder, grab all files, read their contents then post each file content individually to SOAP.
This is how I would do it, but PowerShell returns an error.
Invoke-Webrequest : The input object cannot be bound to any parameters for the command either because the command does not take pipeline input or the input and its properties do not match any of the parameters that take pipeline input.
Below is my code:
$dataAPI = Get-ChildItem 'C:\Users\..\Output'
$uri = 'http://server-name.com:8080/name/name2'
ForEach ($Item in $dataAPI) {
Get-Content $Item.FullName | Invoke-Webrequest -Headers #{"Content-Type" = "text/xml;charset=UTF-8"; "SOAPAction" = "http://server-name.com:8080/name/name2"} -Method 'POST' -Body $dataAPI -Uri $uri -UseDefaultCredential
}
I am not really sure where I should place the Invoke-WebRequest...
Any help would be appreciated. Thanks.
Continuing from my comments,
Add switch -Raw to the Get-Content call to receive a single multiline string instead of an array of lines
Add switch -File to the Get-ChildItem call to ensure you will only deal with files in the loop, not directories too
Try
# if all files you need have a common extension, add `-Filter '*.xml'` to below line
# '*.xml' is just an example here..
$files = Get-ChildItem -Path 'C:\Users\Gabriel\Output' -File
$uri = 'http://server-name.com:8080/name/name2'
$header = #{"Content-Type" = "text/xml;charset=UTF-8"; "SOAPAction" = "http://server-name.com:8080/name/name2"}
foreach ($Item in $files) {
$content = Get-Content $Item.FullName -Raw
Invoke-Webrequest -Headers $header -Method 'POST' -Body $content -Uri $uri -UseDefaultCredential
}

How to escape non-ascii symbols in PowerShell 5.1?

I'm trying to send a file content to server:
$uri = ...
$headers = #{
...
"Content-Type" = "application/json"
}
[string] $content = Get-Content .\filename -Encoding utf8 -Raw
$body = #{
...
"content" = $content
} | ConvertTo-Json
$response = Invoke-WebRequest $uri -Method 'PUT' -Headers $headers -Body $body
But all of non-ascii symbols are changed to another similar symbols or question marks
How can I escape them?
I've read documentation and I know about parameter -EscapeHandling of cmdlet ConvertTo-Json, but it's available from PowerShell 6.2, I have only 5.1
As a result, I wrote a simple function:
function EscapeNonAscii([string] $s)
{
$sb = New-Object System.Text.StringBuilder;
for ([int] $i = 0; $i -lt $s.Length; $i++)
{
$c = $s[$i];
if ($c -gt 127)
{
$sb = $sb.Append("\u").Append(([int] $c).ToString("X").PadLeft(4, "0"));
}
else
{
$sb = $sb.Append($c);
}
}
return $sb.ToString();
}
And used it like this:
$updateFileResponse = Invoke-WebRequest $updateFileUri -Method 'PUT' -Headers $headers -Body (EscapeNonAscii $body)
It helped. For everybody who will google it in the future, it's a request to GitLab API Update existing file in repository
PS: I use PS as C# because I know it badly. If somebody knows how to rewrite this fragment better please let me know.
PPS: And also I know that StringBuilder.Append changes an existing object, but I add here assigning ($sb = $sb.Append($c) instead of simple $sb.Append($c)) because it prints every action to console. If you know how to fix it please let me know.

FreshAgent Get Asset Data

I am trying to fetch asset data from the Fresh client with Powershell. I am able to get any asset by typing it's name but I want to save some of the variables it returns so I can use it further.
$naam = Read-Host "Voer product naam in"
# Set global variables
$APIKey = 'Myapikey'
$EncodedCredentials = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes(("{0}:{1}" -f $APIKey,$null)))
$HTTPHeaders = #{ "X-ApiKey" = $APIKey}
$HTTPHeaders.Add('Authorization', ("Basic {0}" -f $EncodedCredentials))
$HTTPHeaders.Add('Content-Type', 'application/json')
$URL = 'https://helpdesk.company.nl/cmdb/items/list.json?field=name&q='+$naam
(Invoke-WebRequest -Method Get -Uri $URL -Headers $HTTPHeaders ).content
The following are some of the values that return after I run the above
{"config_items":[{"id":25000477949,"name":"SYS-MB1334","description":null,"ci_type_id":25000015988,"impact":1,"created_at":"2020-03-12T10:14:40+01:00","updated_at":"2020-04-24T16:42:42+02:00"
I would like to save the name and id variable for example
Unfortunately, the JSON you show is invalid.
Suppose the json returned from
$output = (Invoke-WebRequest -Method Get -Uri $URL -Headers $HTTPHeaders ).Content
looks like:
{"config_items":
[{"id":25000477949,"name":"SYS-MB1334","description":null,"ci_type_id":25000015988,"impact":1,"created_at":"2020-03-12T10:14:40+01:00","updated_at":"2020-04-24T16:42:42+02:00"},
{"id":12345678901,"name":"SYS-MB9876","description":null,"ci_type_id":12358745896,"impact":1,"created_at":"2020-03-12T10:14:40+01:00","updated_at":"2020-04-24T16:42:42+02:00"}]
}
Then you can collect the properties you need from the config_items using:
$result = ($output | ConvertFrom-Json).config_items |
Select-Object #{Name = 'Id'; Expression = {$_.id}},
#{Name = 'Name'; Expression = {$_.name}}
# output on screen
$result
# output to CSV file
$result | Export-Csv -Path 'X:\TheOutput.csv' -NoTypeInformation
Output on screen would look like
Id Name
-- ----
25000477949 SYS-MB1334
12345678901 SYS-MB9876
Hope that helps

Adding Powershell jobs to retrieve in parallel from paginated rest API

I've spent perhaps over 30 hours trying to add various methods to implement parallel processing to my API call. The most obvious would be to place the code into a job, but I'm having no luck. Any ideas?
Start-Transcript -Path "$PSScriptRoot\Errorlog.txt"
$WondeObjectsArray = Import-CSV $PSScriptRoot\WondeID.csv
$EndpointObjectArray = Import-CSV $PSScriptRoot\Endpoints.csv
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("REDACTED", "REDACTED")
# School loop
Foreach ($object in $WondeObjectsArray) {
$SchoolName = $object.School
$TrustName = $object.Trust
$WondeID = $object."Wonde ID"
# Create data structure
If(!(test-path "$PSScriptRoot\$TrustName"))
{New-Item -Path "$PSScriptRoot\$TrustName" -ItemType Directory}
If(!(test-path "$PSScriptRoot\$TrustName\$SchoolName"))
{New-Item -Path "$PSScriptRoot\$TrustName\$SchoolName" -ItemType Directory}
# Endpoint request loop
foreach($Eobject in $EndpointObjectArray){
$JsonName = $Eobject.JsonName
$Table = $Eobject.Table
$Method = $Eobject.Url_Method
# First response
$response = Invoke-RestMethod "https://api.wonde.com/v1.0/schools/$WondeID/$Table$Method&$Page" -Method 'GET' -Headers $headers -Body $body
$concat = $response.data
#Pagination loop
While ($response.meta.pagination.next){
$response = Invoke-RestMethod $response.meta.Pagination.next -Method 'GET' -Headers $headers -Body $Body
$concat = $concat + $response.data
} #pagination loop end
# Concatenate completed request
$concat | ConvertTo-Json | Out-File "$PSScriptRoot\$TrustName\$SchoolName\$JsonName.json"
} # Endpoint request loop end
} # School loop end
Stop-Transcript
Never mind, I figured it out. I turned the code that I wanted to run in parallel into a scriptblock, Passed it the parameters by using param () And in the start job line I included my variables in -ArgumentList.
My issue was actually that when you start a new job it changes the script file path to C:/, when I was passing $PSScriptRoot rather than the default file location.
Start-Transcript -Path "$PSScriptRoot\Errorlog.txt"
$PathRoot = $PSScriptRoot
$WondeObjectsArray = Import-CSV $PSScriptRoot\WondeID.csv
$EndpointObjectArray = Import-CSV $PSScriptRoot\Endpoints.csv
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("REDACTED", "REDACTED")
# School loop
Foreach ($object in $WondeObjectsArray) {
$SchoolName = $object.School
$TrustName = $object.Trust
$WondeID = $object."Wonde ID"
# Create data structure
If(!(test-path "$PSScriptRoot\$TrustName"))
{New-Item -Path "$PSScriptRoot\$TrustName" -ItemType Directory}
If(!(test-path "$PSScriptRoot\$TrustName\$SchoolName"))
{New-Item -Path "$PSScriptRoot\$TrustName\$SchoolName" -ItemType Directory}
# Endpoint request loop
foreach ($Eobject in $EndpointObjectArray){
$JsonName = $Eobject.JsonName
$Table = $Eobject.Table
$Method = $Eobject.Url_Method
$scriptblock = {
param ($WondeID, $Table, $Method, $headers, $trustName, $SchoolName, $JsonName, $PathRoot)
write-host "$TrustName : $SchoolName : $JsonName is complete!"
# First response
$response = Invoke-RestMethod "https://api.wonde.com/v1.0/schools/$WondeID/$Table$Method&$Page" -Method 'GET' -Headers $headers -Body $body
$concat = $response.data
#Pagination loop
While ($response.meta.pagination.next){
$response = Invoke-RestMethod $response.meta.Pagination.next -Method 'GET' -Headers $headers -Body $Body
$concat = $concat + $response.data
} #pagination loop end
# Concatenate completed request
$concat | ConvertTo-Json | Out-File "$PathRoot\$TrustName\$SchoolName\$JsonName.json"
}
Start-Job $scriptblock -Name "$SchoolName&$JsonName" -ArgumentList $WondeID, $Table, $Method, $headers, $trustName, $SchoolName, $JsonName, $PathRoot
} # Endpoint request loop end
Get-Job | Wait-Job | Receive-Job
Get-Job | Remove-Job
} # School loop end
Stop-Transcript

Make Invoke-WebRequest loop through each URL it finds

I'm new to PowerShell and I'm trying to make the Invoke-WebRequest cmdlet loop through each url the webscrape finds. All I have so far is this :
$site = Invoke-WebRequest -UseBasicParsing -Uri www.example.com/examples
$site.Links | Out-GridView
Any help would be appreciated!
Add your links to a comma separated list.
Split the list and loop each item.
Request each item.
As below:
$option = [System.StringSplitOptions]::RemoveEmptyEntries
$urlCollection = "link1,link2,link3"
$separator = ","
$urlList = $urlCollection.Split($separator, $option)
foreach ($url in $urlList) {
Invoke-WebRequest $url
# Give feedback on how far we are
Write-Host ("Initiated request for {0}" -f $url)
}