Invoke-WebRequest content ConvertFrom-Json: Conversion from JSON failed with error: Unexpected character encountered while parsing value (CR;LF) - powershell

Question

I successfully sent an webrequest to a Website and got a responde:
$Uri = "https://URLXYZ"
$Method = "POST"
$Header = #{
"Accept" = "*/*";
"Connection" = "keep-alive";
"Accept-Encoding" = "gzip, deflate, br";
"Content-Type" = "text/json; charset=""UTF-8"""
}
$Body = #"
{
"Items": [
{
"Type": "XX",
"Value": "YY",
}
],
"TypeId": XY,
"LiveConditions": []
}
"#
$webrequest = Invoke-WebRequest -Uri $Uri -Body $Body -Headers $Header -Method $Method
Now, I am trying to convert it from JSON without success:
$webrequest.Content | ConvertFrom-Json
ConvertFrom-Json: Conversion from JSON failed with error: Unexpected character encountered while parsing value: . Path '', line 0, position 0.
When I copy the output ($webrequest.Content) to Notepad++ I can see a carriage return (CR) and line feed (LF):
$webrequest.Content
[{"MailG":[{"DisplayName":{"7":"Worker","9":"Person"},"Mails":"max.mustermann#gmai.com;"}],"ResultCount":1,"Rows":[{"ElementGroups":[{"ObjectContainer":{"Id":55
6677889900,"UID":"1122334455","Info":[],"PreCalculatedSelectedProperties":{"11":"Mustermann","22":"","33":"StreetName","44":"","55":"max.mustermann#gmai.com","66":"","77":"Berlin","88":"","99":"Max Mustermann","00":"[phone]","111":"userid","222":"xyz","333":"company","444":"1122334455","555":"roomnumber","666":"Germany","777":"Team","888":"[phone]","999":"Max","000":""},"TID":5,"RuleConditionPartResults":{"1448925":false,"1448926":false,"1448927":false,"1448928":false,"1448929":false,"1448930":false,"1448931":false,"1448932":false,"1448933":false,"1448934":false,"1448935":false,"1448936":false,"1448937":false,"1448938":false,"1448939":false,"1448940":false,"1448941":false,"1448942":false},"Img":{"3714":["picture"]},"Parents":[],"Childs":[],"UpObjects":0,"Down":0,"LinkCount":0,"FootObject":{},"BoxIds":[],"DisplayValue":"Max Mustermann","Key":"1122334455"},"Columns":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17]}],"Blank":{},"Score":0.0,"SInd":0}],"Page":0,"TID":5}]
This is unexpected cause the CR;LF is e.g. in the middle of an email address.
If the response is longer, there are multiple CR;LF within the content of the webrequest.
The $webrequest.content is TypeName: System.String
If I manually remove the CR;CF in Notepad++ and PrettyPrint it is working:
[
{
"MailG": [
{
"DisplayName": {
"7": "Worker",
"9": "Person"
},
"Mails": "max.mustermann#gmai.com;"
}
],
"ResultCount": 1,
"Rows": [
{
"ElementGroups": [
{
"ObjectContainer": {
"Id": 556677889900,
"UID": "1122334455",
"Info": [],
"PreCalculatedSelectedProperties": {
"11": "Mustermann",
"22": "",
"33": "StreetName",
"44": "",
"55": "max.mustermann#gmai.com",
"66": "",
"77": "Berlin",
"88": "",
"99": "Max Mustermann",
"00": "[phone]",
"111": "userid",
"222": "xyz",
"333": "company",
"444": "1122334455",
"555": "roomnumber",
"666": "Germany",
"777": "Team",
"888": "[phone]",
"999": "Max",
"000": ""
},
"TID": 5,
"RuleConditionPartResults": {
"1448925": false,
"1448926": false,
"1448927": false,
"1448928": false,
"1448929": false,
"1448930": false,
"1448931": false,
"1448932": false,
"1448933": false,
"1448934": false,
"1448935": false,
"1448936": false,
"1448937": false,
"1448938": false,
"1448939": false,
"1448940": false,
"1448941": false,
"1448942": false
},
"Img": {
"3714": [
"picture"
]
},
"Parents": [],
"Childs": [],
"UpObjects": 0,
"Down": 0,
"LinkCount": 0,
"FootObject": {},
"BoxIds": [],
"DisplayValue": "Max Mustermann",
"Key": "1122334455"
},
"Columns": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16,
17
]
}
],
"Blank": {},
"Score": 0.0,
"SInd": 0
}
],
"Page": 0,
"TID": 5
}
]
I already tried to convert it or even replace the CR;LF it in PowerShell without success.
The only workaround I found is to write the content to a .txt-file and read it again.
$webrequest.Content | Out-File "C:\Temp\WebRequestTemp.txt"
$json = Get-Content "C:\Temp\WebRequestTemp.txt" | ConvertFrom-Json
Afterwards I was able to convert it from JSON and work with the data.
Why can I not directly convert it from JSON as usual?

Answer 1

To help you with a more detailed answer. It is useful to share the contents of the web response. But maybe the line below will solve your problem.
$jsonCorrected = [Text.Encoding]::UTF8.GetString([Text.Encoding]::GetEncoding(28591).GetBytes(($webrequest.Content)))

Answer 2

I had a similar issue, and the issue was there was an unknown invisible character at the beginning of the content block. This character was "U+FEFF".
I managed to get it to be replaced by doing $webrequest.Content -replace '\uFEFF' | ConvertFrom-Json
I hope this works for anyone else too.