I hope somebody can help me with this issue, i am trying to upload an SSRS report using powershell, however it totally loses the datasource reference after its uploaded.
I found some script online which changes the reference, but this would not work for what i need because its using the datasource name to become the reference, but in my scenario the datasource name could be something like Datasource1 but the reference could be /Data Sources/Translations
What i need to do is alter the reference in the bytearray before its uploaded.
This is the script im using so far, which is working for the upload.
#ReportName
$reportName = [System.IO.Path]::GetFileNameWithoutExtension($rdlFile);
write-host $reportName -ForegroundColor Green
#Upload File
try
{
#Get Report content in bytes
Write-Host "Getting file content of : $rdlFile"
$byteArray = gc $rdlFile.FullName -encoding Byte
$msg = "Total length: {0}" -f $byteArray.Length
Write-Host $msg
Write-Host "Uploading to: $reportFolder_Final"
$type = $ssrsProxy.GetType().Namespace
$datatype = ($type + '.Property')
$DescProp = New-Object($datatype)
$DescProp.Name = 'Description'
$DescProp.Value = ''
$HiddenProp = New-Object($datatype)
$HiddenProp.Name = 'Hidden'
$HiddenProp.Value = 'false'
$Properties = #($DescProp, $HiddenProp)
#Call Proxy to upload report
$warnings = $null
$Results = $ssrsProxy.CreateCatalogItem("Report",$reportName,$reportFolder_Final, $IsOverwriteReport,$byteArray,$Properties,[ref]$warnings)
If i try reading the XML in as a string and then converting it back into a bytearray before uploading it to the ssrs server the upload fails as it complains about the formatting. I had plans of reading it in as a string, altering the datasource reference, encoding it and then uploading but thats the part i need your help with doing.
Cheers
i have managed to suss this out, for anyone having similar issues i used the following code to read the data in from the XML file as a system.byte, converted it into a UTF8 string, made my changes to the references and then converted it back to a UTF8 Bytestream before uploading to the report server WDSL
$byteArray = gc $rdlFile.FullName -encoding Byte
$byteArray = [System.Text.Encoding]::UTF8.GetString($byteArray)
$byteArray = $byteArray -replace "<DataSourceReference>", "<DataSourceReference>/Data Sources/"
$byteArray = $byteArray -replace "<SharedDataSetReference>", "<SharedDataSetReference>/Datasets/"
$byteArray = [System.Text.Encoding]::UTF8.GetBytes($byteArray)
Related
I'm trying to convert a few thousand home videos to a smaller format. However, encoding the video changed the created and modified timestamp to today's date. I wrote a powershell script that successfully (somehow) worked by writing the original file's modified timestamp to the new file.
However, I couldn't find a way in powershell to modify the "Media created" timestamp in the file's details properties. Is there a way to add a routine that would either copy all of the metadata from the original file, or at least set the "media created" field to the modified date?
When I searched for file attributes, it looks like the only options are archive, hidden, etc. Attached is the powershell script that I made (please don't laugh too hard, haha). Thank you
$filepath1 = 'E:\ConvertedMedia\Ingest\' # directory with incorrect modified & create date
$filepath2 = "F:\Backup Photos 2020 and DATA\Data\Photos\Photos 2021\2021 Part1\Panasonic 3-2-21\A016\PRIVATE\PANA_GRP\001RAQAM\" # directory with correct date and same file name (except extension)
$destinationCodec = "*.mp4" # Keep * in front of extension
$sourceCodec = ".mov"
Get-ChildItem $filepath1 -File $destinationCodec | Foreach-Object { # change *.mp4 to the extension of the newly encoded files with the wrong date
$fileName = $_.Name # sets fileName variable (with extension)
$fileName # Optional used during testing- sends the file name to the console
$fileNameB = $_.BaseName # sets fileNameB variable to the filename without extension
$filename2 = "$filepath2" + "$fileNameB" + "$sourceCodec" # assembles filepath for source
$correctTime = (Get-Item $filename2).lastwritetime # used for testing - just shows the correct time in the output, can comment out
$correctTime # prints the correct time
$_.lastwritetime = (Get-Item $filename2).lastwritetime # modifies lastwritetime of filepath1 to match filepath2
$_.creationTime = (Get-Item $filename2).lastwritetime # modifies creation times to match lastwritetime (comment out if you need creation time to be the same)
}
Update:
I think I need to use Shell.Application, but I'm getting an error message "duplicate keys ' ' are not allowed in hash literals" and am not sure how to incorporate it into the original script.
I only need the "date modified" attribute to be the same as "lastwritetime." The other fields were added just for testing. I appreciate your help!
$tags = "people; snow; weather"
$cameraModel = "AG-CX10"
$cameraMaker = "Panasonic"
$mediaCreated = "2/16/1999 5:01 PM"
$com = (New-Object -ComObject Shell.Application).NameSpace('C:\Users\philip\Videos') #Not sure how to specify file type
$com.Items() | ForEach-Object {
New-Object -TypeName PSCustomObject -Property #{
Name = $com.GetDetailsOf($_,0) # lists current extended properties
Tags = $com.GetDetailsOf($_,18)
CameraModel = $com.GetDetailsOf($_,30)
CameraMaker = $com.GetDetailsOf($_,32)
MediaCreated = $com.GetDetailsOf($_,208)
$com.GetDetailsOf($_,18) = $tags # sets extended properties
$com.GetDetailsOf($_,30) = $cameraModel
$com.GetDetailsOf($_,32) = $cameraMaker
$com.GetDetailsOf($_,32) = $mediaCreated
}
}
Script Example
File Properties Window
I think your best option is to drive an external tool/library from Powershell rather than using the shell (not sure you can actually set values this way tbh).
Its definitely possible to use FFMpeg to set the Media Created metadata of a file like this:
ffmpeg -i input.MOV -metadata creation_time=2000-01-01T00:00:00.0000000+00:00 -codec copy output.MOV
This would copy input.MOV file to new file output.MOV and set the Media Created metadata on the new output.MOV. This is very inefficient - but it does work.
You can script ffmpeg something like the below. The script will currently output the FFMpeg commands to the screen, the commented out Start-Process line can be used to execute ffmpeg.
gci | where Extension -eq ".mov" | foreach {
$InputFilename = $_.FullName;
$OutputFilename = "$($InputFilename)-fixed.mov";
Write-Host "Reading $($_.Name). Created: $($_.CreationTime). Modifed: $($_.LastWriteTime)";
$timestamp = Get-Date -Date $_.CreationTime -Format O
Write-Host "ffmpeg -i $InputFilename -metadata creation_time=$timestamp -codec copy $OutputFilename"
# Start-Process -Wait -FilePath C:\ffmpeg\bin\ffmpeg.exe -ArgumentList #("-i $InputFilename -metadata creation_time=$timestamp -codec copy $($OutputFilename)")
}
How do I compare the output of Get-FileHash directly with the output of Properties.ContentMD5?
I'm putting together a PowerShell script that takes some local files from my system and copies them to an Azure Blob Storage Container.
The files change daily so I have added in a check to see if the file already exists in the container before uploading it.
I use Get-FileHash to read the local file:
$LocalFileHash = (Get-FileHash "D:\file.zip" -Algorithm MD5).Hash
Which results in $LocalFileHash holding this: 67BF2B6A3E6657054B4B86E137A12382
I use this code to get the checksum of the blob file already transferred to the container:
$BlobFile = "Path\To\file.zip"
$AZContext = New-AZStorageContext -StorageAccountName $StorageAccountName -SASToken "<token here>"
$RemoteBlobFile = Get-AzStorageBlob -Container $ContainerName -Context $AZContext -Blob $BlobFile -ErrorAction Ignore
if ($ExistingBlobFile) {
$cloudblob = [Microsoft.Azure.Storage.Blob.CloudBlockBlob]$RemoteBlobFile.ICloudBlob
$RemoteBlobHash = $cloudblob.Properties.ContentMD5
}
This value of $RemoteBlobHash is set to Z78raj5mVwVLS4bhN6Ejgg==
No problem, I thought, I'll just decrypt the Base64 string and compare:
$output = [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($RemoteBlobHash))
Which gives me g�+j>fWKK��7�#� so not directly comparable ☹
This question shows someone in a similar pickle but I don't think they were using Get-FileHash given the format of their local MD5 result.
Other things I've tried:
changing the System.Text.Encoding line above UTF8 to UTF16 & ASCII which changes the output but not to anything recognisable.
dabbling with GetBytes to see if that helped:
$output = [System.Text.Encoding]::UTF8.GetBytes([System.Text.Encoding]::UTF16.GetString([System.Convert]::FromBase64String($RemoteBlobHash)))
Note: Using md5sum to compare the local file and a downloaded copy of file.zip results in the same MD5 string as Get-FileHash: 67BF2B6A3E6657054B4B86E137A12382
Thank you in advance!
ContentMD5 is a base64 representation of the binary hash value, not the resulting hex string :)
$md5sum = [convert]::FromBase64String('Z78raj5mVwVLS4bhN6Ejgg==')
$hdhash = [BitConverter]::ToString($md5sum).Replace('-','')
Here we convert base64 -> binary -> hexadecimal
If you need to do it the other way around (ie. for obtaining a local file hash, then using that to search for blobs in Azure), you'll first need to split the hexadecimal string into byte-size chunks, then convert the resulting byte array to base64:
$hdhash = '67BF2B6A3E6657054B4B86E137A12382'
$bytes = [byte[]]::new($hdhash.Length / 2)
for($i = 0; $i -lt $bytes.Length; $i++){
$offset = $i * 2
$bytes[$i] = [convert]::ToByte($hdhash.Substring($offset,2), 16)
}
$md5sum = [convert]::ToBase64String($bytes)
I have a script that, in a nutshell, does the following:
copies required files to a temporary folder
compresses the files in the temporary folder to a .zip file
FTPs the .zip file to our FTP server
tidies up and deletes the temporary folder and .zip file
I have pinched the FTP code from a previous post:
Upload files with FTP using PowerShell
and modified it where necessary (keeping the basics in tact - I think).
The issue I have is that while the .zip file is being FTP'd the script doesn't wait until it is complete. It gets part way through, anywhere from 20Mb to 60Mb before it continues executing, tidies up and deletes the file it is transferring.
The temporary folder is always the same but the .zip filename varies depending on the date so I can't really reverse the order of operations.
Can anyone suggest how I might get the script to wait until the FTP process has completed, success or fail, before it moves on?
Cheers,
Andrew.
Edit: For those that asked....
function FTPtoServer ()
{
<#
What this function has to/should do:
- accept the right number of parameters,
minimum/mandatory: username, password, file
optional: proxy server address/port, proxy username and password
- check that the source file exists, then extract the filename.
- if a proxy is specified, set the appropriate parameters
- transmit the file
- if any errors occur, throw and return
#>
param(
[string]$sourcefile=$(throw 'A sourcefile is required, -sourcefile'), <#fully qualified zip file name#>
[string]$FTPUser =$(throw 'An FTP username is required, -ftpuser'),
[string]$FTPPass =$(throw 'An FTP password is required, -ftppass'),
#
[string]$proxyServer, #proxySocket?? it is an address and port
[string]$proxyUser,
[string]$proxyPass
)
#local variables
$FTPserver = "ftp://ftp.servername.com.au"
#check if the sourcefile exists, if not return/throw an error
# The sourcefile should contain the full path to the file.
if (-not (test-path $sourcefile)){
throw "the source file could not be located: $sourcefile"
}
# extract the filename from the sourcefile.
$filename = split-path -path $sourcefile -leaf
# create the FtpWebRequest and configure it
$ftp = [System.Net.FtpWebRequest]::Create("$FTPserver/$filename")
$ftp = [System.Net.FtpWebRequest]$ftp
$ftp.Method = [System.Net.WebRequestMethods+Ftp]::UploadFile
$ftp.Credentials = new-object System.Net.NetworkCredential($FTPUser,$FTPPass)
$ftp.UseBinary = $true
$ftp.UsePassive = $false
#proxy info
# ******** DANGER Will Robinson - this proxy config has not been
# tested and may not work.
if ($proxyServer){
$proxy = New-Object System.Net.WebProxy $proxyServer
if ($proxyUser -and $proxyPass){
$proxy.Credentials = new-object System.Net.NetworkCredential($proxyUser,$proxyPass)
}
$ftp.Proxy = $proxy
$ftp.UsePassive = $true #apparently, must usePassive if using proxy
}
#now we have checked and prepared everything, lets try and send the file.
# read in the file to upload as a byte array
try{
#work out how much we are sending
$content = [System.IO.File]::ReadAllBytes("$sourceFile")
$ftp.ContentLength = $content.Length
try {
# get the request stream, and write the bytes into it
$rs = $ftp.GetRequestStream()
$rs.Write($content, 0, $content.Length)
# be sure to clean up after ourselves
$rs.Close()
$rs.Dispose()
}
catch {
$errorMessage = "FTP failed. " + $_.exception.message
throw $errormessage
}
}
catch {
$errorMessage = "Unable to transmit file " + $sourceFile + "`r`n" + $_.exception.message
throw $errormessage
}
}
The above is in a separate file, but is called by the following:
try {
FTPtoServer -sourcefile $sourcefile -ftpuser $FTPUser -ftppass $FTPPass
}
catch {
$errorMessage = "FTPtoServer function failed with error: $_"
finishFail -failmessage $errorMessage
}
Cheers.
Found it.
I executed the FTP code above in isolation using a large file (~140Mb) and it threw the error; "The underlying connection was closed: An unexpected error occured on a receive."
I rebooted the FTP server, checked the user account etc etc.
I also tested the M$ FTP client with the same file and it transferred completely and correctly.
Anyway, I found this article: https://www.codeproject.com/Questions/597175/FileplusUploadplustoplusFTPplusserver which also has the error I received.
As it turns out, the Timeout value of FTPWebRequest is NOT -1 as in the doco but 100 seconds.
I checked my FTP logs and sure enough, time between logon and logoff was about 100 seconds.
I added the line: $ftp.Timeout = -1 to my code and first attempt transferred the entire file completely without error.
Previous transfers had worked as they fell below the 100 second timeout.
Many thanks for the posts and help.
I use an alternative oldschool method myself, it should work for you, and doesn't need any extra components on the server.
$ftp_user = "username"
$ftp_password = "password"
$ftp_address = "ftp.someserver.com"
$ftp_commands = #"
open $ftp_address
$ftp_user
$ftp_password
lcd c:\jobs\output
put estate_data_current.xml
bye
"#
set-content -encoding "ASCII" -path ftp_commands.txt -value $ftp_commands
ftp -s:ftp_commands.txt
I'm trying to experiment with Azure Functions. Basically my use case is calling the function with a GUID as GET Parameter, having the function download the WIX toolkit DLL and an MSI file, updating a parameter in the MSI file, and the returning that file to the caller of the function (as download prompt for example).
I'm mostly there, just need some help getting the download prompt/send to happen, my code so far:
$urlWix = "http://domain/wix.dll"
$outputWix = "$Env:TEMP\wix.dll"
Invoke-WebRequest -Uri $urlWix -OutFile $outputWix
try{Add-Type -Path $outputWix}catch{$Null}
$urlMSI = "http://domain/file.msi"
$outputFile = "$Env:TEMP\file.msi"
Invoke-WebRequest -Uri $urlMSI -OutFile $outputFile
$oDatabase = New-Object Microsoft.Deployment.WindowsInstaller.Database($outputFile,[Microsoft.Deployment.WindowsInstaller.DatabaseOpenMode]::Direct);
$sSQLQuery = "SELECT * FROM Property WHERE Property= 'MYPROPERTY'"
[Microsoft.Deployment.WindowsInstaller.View]$oView = $oDatabase.OpenView($sSQLQuery)
$oView.Execute()
$oRecord = $oView.Fetch()
$oRecord.SetString("Value","MyCustomValue")
$oView.Modify([Microsoft.Deployment.WindowsInstaller.ViewModifyMode]::Update,$oRecord)
$oView.Close();
$oDatabase.Dispose();
$file = get-item $outputFile
write-output $file
Unfortunately due to content type issues this is not possible in powershell. You can do this via a C#, F#, or Node (isRaw) function. The problem is that you need to specify headers via the JSON response format, which would convert any non-text data into a base64 string.
If you want to sent a text file via powershell it is possible:
$response = ConvertTo-JSON #{
Body="your file data";
Headers=#{
# unfortunately it seems functions does not support 'filename=...'
'Content-Disposition'='attachment';
# you would use application/octet-stream, but because it's converted to JSON you lose binary content
'Content-Type'='text/plain';
};
}
Out-File -Encoding Ascii -FilePath $res -inputObject $response
I'm new at writing in powershell but this is what I'm trying to accomplish.
I want to compare the dates of the two excel files to determine if one is newer than the other.
I want to convert a file from csv to xls on a computer that doesn't have excel. Only if the statement above is true, the initial xls file was copied already.
I want to copy the newly converted xls file to another location
If the file is already open it will fail to copy so I want to send out an email alert on success or failure of this operation.
Here is the script that I'm having issues with. The error is "Expressions are only allowed as the first element of a pipeline." I know it's to do with the email operation but I'm at a loss as to how to write this out manually with all those variables included. There are probably more errors but I'm not seeing them now. Thanks for any help, I appreciate it!
$CSV = "C:filename.csv"
$LocalXLS = "C:\filename.xls"
$RemoteXLS = "D:\filename.xls"
$LocalDate = (Get-Item $LocalXLS).LASTWRITETIME
$RemoteDate = (Get-Item $RemoteXLS).LASTWRITETIME
$convert = "D:\CSV Converter\csvcnv.exe"
if ($LocalDate -eq $RemoteDate) {break}
else {
& $convert $CSV $LocalXLS
$FromAddress = "email#address.com"
$ToAddress = "email#address.com"
$MessageSubject = "vague subject"
$SendingServer = "mail.mail.com"
$SMTPMessage = New-Object System.Net.Mail.MailMessage $FromAddress, $ToAddress, $MessageSubject, $MessageBody
$SMTPClient = New-Object System.Net.Mail.SMTPClient $SendingServer
$SendEmailSuccess = $MessageBody = "The copy completed successfully!" | New-Object System.Net.Mail.SMTPClient mail.mail.com $SMTPMessage
$RenamedXLS = {$_.BaseName+(Get-Date -f yyyy-MM-dd)+$_.Extension}
Rename-Item -path $RemoteXLS -newname $RenamedXLS -force -erroraction silentlycontinue
If (!$error)
{ $SendEmailSuccess | copy-item $LocalXLS -destination $RemoteXLS -force }
Else
{$MessageBody = "The copy failed, please make sure the file is closed." | $SMTPClient.Send($SMTPMessage)}
}
You get this error when you are trying to execute an independent block of code from within a pipeline chain.
Just as a different example, imagine this code using jQuery:
$("div").not(".main").console.log(this)
Each dot (.) will chain the array into the next function. In the above function this breaks with console because it's not meant to have any values piped in. If we want to break from our chaining to execute some code (perhaps on objects in the chain - we can do so with each like this:
$("div").not(".main").each(function() {console.log(this)})
The solution is powershell is identical. If you want to run a script against each item in your chain individually, you can use ForEach-Object or it's alias (%).
Imagine you have the following function in Powershell:
$settings | ?{$_.Key -eq 'Environment' } | $_.Value = "Prod"
The last line cannot be executed because it is a script, but we can fix that with ForEach like this:
$settings | ?{$_.Key -eq 'Environment' } | %{ $_.Value = "Prod" }
This error basically happens when you use an expression on the receiving side of the pipeline when it cannot receive the objects from the pipeline.
You would get the error if you do something like this:
$a="test" | $a
or even this:
"test" | $a
I don't know why are trying to pipe everywhere. I would recommend you to learn basics about Powershell pipelining. You are approaching it wrong. Also, I think you can refer to the link below to see how to send mail, should be straight forward without the complications that you have added with the pipes : http://www.searchmarked.com/windows/how-to-send-an-email-using-a-windows-powershell-script.php