I'm using terraform on windows, and would like to visualize the graphs using Graphviz. However, there is a difference between the encoding that is being output by Terraform vs what Graphviz expects. Ideally, I want to do the following:
terraform graph -draw-cycles | dot -Tsvg > output.svg
That doesn't work, because the output that is being given by terraform is in the wrong encoding. The following sequence works, but uses an intermediate file:
terraform graph -draw-cycles > output.tmp
Get-Content .\output.tmp | Set-Content -Encoding Ascii output2.tmp
dot -Tsvg output2.tmp > output.svg
rm output.tmp
rm output2.tmp
However, I would like to do this without intermediate files using piping. A statement such as
terraform graph -draw-cycles | Set-Content -Encoding Ascii -PassThru | dot -Tsvg > output.svg
doesn't work. The output from the terraform graph statement is text, and apparantly the Set-Content commandlet needs additional information (Path?):
Set-Content : The input object cannot be bound because it did not contain the information required to bind all mandatory parameters: Path
At line:1 char:32
+ ... rm graph -draw-cycles | Set-Content -Encoding Ascii -PassThru | dot - ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (digraph {:PSObject) [Set-Content], ParameterBindingException
+ FullyQualifiedErrorId : InputObjectMissingMandatory,Microsoft.PowerShell.Commands.SetContentCommand
Any suggestions?
After reading the article referred to by zett42, I saw a reference to CMD pipe different form Powershell pipe? in the comments.
The easiest solution for my specific case is simply executing the call with cmd:
cmd /c "terraform graph -draw-cycles | dot -Tsvg > output.svg"
Using the Invoke-WithEncoding would also work, but is a little more involved.
Related
I am pretty new to scripting and I have been stuck on this problem.
I want to extract a zip file and in that zip file there is a xml file called file.xml which I need to extract information from.
The file is huge and I only need to extract info between two tags.
<Report name="result\_many fail" summary="20" yes=19 no="1" finished=20>
I need to extract information between the tags name and finished and save it to a txt file.
The txt file should look like this:
name result_many fail, summary 20, yes 19 , no 1, finished 20
The problem is that it unzips to the right destination folder but it doesn't save anything into the result.txt file. My txt file is always empty.
This is my code:
echo "unzipping file..."
powershell "Expand-Archive C:\path_to_zip_file -Destinationpath C:\path_to_to_destination_folder;
$Path = “C:\path_to_to_destination_folder\file.xml;”
Select-Xml -XPath '//Report[#finished]').Node.InnerText;
Set-Content -Path 'C:\path_to_to_destination_folder\result.txt'
#echo "done"
Could someone help me out?
Leaving aside incidental aspects of your question (seemingly calling from cmd.exe with a broken attempt to pass a multi-line command string, invalid sample XML, lack of a file argument to Select-Xml, lack of input to the Set-Content cmdlet):
$path = 'C:\path_to_to_destination_folder\file.xml'
(Select-Xml '//Report[#finished]' $path).
Node.Attributes.ForEach({ $_.Name + ' ' + $_.Value }) -join ', ' |
Set-Content C:\path_to_to_destination_folder\result.txt
To call your PowerShell code from cmd.exe (a batch file) requires you to either pass it on a single line or to carefully spread it across multiple lines with line-continuations, using ^, based on the techniques described in this answer:
#echo off
echo unzipping file...
powershell -c Expand-Archive C:\path_to_zip_file -DestinationPath C:\path_to_to_destination_folder; ^
$path = 'C:\path_to_to_destination_folder\file.xml'; ^
(Select-Xml '//Report[#finished]' $path). ^
Node.Attributes.ForEach({ $_.Name + ' ' + $_.Value }) -join ', ' ^| ^
Set-Content C:\path_to_to_destination_folder\result.txt
echo done
If you need help with the fundamentals, here are some PowerShell learning resources:
A starting point is Learning PowerShell.
This series of articles, even though it predates PowerShell (Core) 6+, is a great, recipe-oriented introduction to PowerShell.
http://hyperpolyglot.org/shell juxtaposes the syntax of POSIX-compatible shells such as bash with that of cmd.exe and PowerShell in concise, tabular form.
I have an application log for which I am trying to write a batch file that will tail the log and return strings that contain "queue size" so that the updating queue size can be displayed. Basically the Windows equivalent of:
tail -f app.log | grep "queue size"
From what I've read I would need to use Windows powershell. I have devised the following script:
powershell -command Select-String -Path C:\logs\app.log -Pattern "queue size"
This gives me the following error:
Select-String : A positional parameter cannot be found that accepts
argument 'size'. At line:1 char:1
+ Select-String -Path C:\logs\app.log -Pattern queue size
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidArgument: (:) [Select-String], ParameterBindingException
+ FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.SelectStringCommand
Although this as it stands doesn't work, would it constantly update with the current logic?
No, the PowerShell command will not continue to read the log as it's being updated. PowerShell can't really handle this task, so you'd be better off grabbing a Windows port of the Unix tail command (e.g. from GnuWin32 or UnxUtils) and use it with the batch find command:
tail -f C:\path\to\app.log | find "queue size"
You need to wrap the command in double quotes and use single quotes for the pattern:
powershell -command "Select-String -Path C:\logs\app.log -Pattern 'queue size'"
this should do:
cat c:\path\to\app.log -tail 100 -wait | select-string "queue size"
cat is an alias for Get-Content...
The -wait parameter will make it wait for log updates.
I've tried a ton of different variations of this, but I can't get it to work. I am trying to run mysqldump to export a database (in this case called global).
PS C:\Users\Administrator> &"$mysqlpath\mysqldump.exe -u$mysqluser -p$mysqlpass --databases global | Out-File $env:TEMP\database_backup\global_$timestamp.sql -Encoding UTF8"
& : The term 'C:\Program Files\MySQL\MySQL Server 5.6\bin\mysqldump.exe -ubackup -pbackup_password --databases global | Out-File C:\Users\ADMINI~1\AppData\Local\Temp\2\database_backup\global_2013-12-11T11:47:28.sql -Encoding UTF8' is
not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or
if a path was included, verify that the path is correct and try again.
At line:1 char:2
+ &"$mysqlpath\mysqldump.exe -u$mysqluser -p$mysqlpass --databases global | Out-Fi ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (C:\Program File... -Encoding UTF8:String) [], CommandNotFoundException
+ FullyQualifiedErrorId : CommandNotFoundException
It looks like it is being exploded into the proper variable names, but I can't get it to run.
UPDATE: So that was the right answer, I had a : in the timestamp. My other problem was solved by putting the variables inside double quotes (")
You are telling Powershell to run an EXE named
&"$mysqlpath\mysqldump.exe -u$mysqluser -p$mysqlpass --databases global | Out-File $env:TEMP\database_backup\global_$timestamp.sql -Encoding UTF8"
So remove the quotes (") and try again. Like so,
& $mysqlpath\mysqldump.exe -u$mysqluser -p$mysqlpass --databases global | Out-File $env:TEMP\database_backup\global_$timestamp.sql -Encoding UTF8
I am trying to replicate the functionality of the cat command in Unix.
I would like to avoid solutions where I explicitly read both files into variables, concatenate the variables together, and then write out the concatenated variable.
Simply use the Get-Content and Set-Content cmdlets:
Get-Content inputFile1.txt, inputFile2.txt | Set-Content joinedFile.txt
You can concatenate more than two files with this style, too.
If the source files are named similarly, you can use wildcards:
Get-Content inputFile*.txt | Set-Content joinedFile.txt
Note 1: PowerShell 5 and older versions allowed this to be done more concisely using the aliases cat and sc for Get-Content and Set-Content respectively. However, these aliases are problematic because cat is a system command in *nix systems, and sc is a system command in Windows systems - therefore using them is not recommended, and in fact sc is no longer even defined as of PowerShell Core (v7). The PowerShell team recommends against using aliases in general.
Note 2: Be careful with wildcards - if you try to output to inputFiles.txt (or similar that matches the pattern), PowerShell will get into an infinite loop! (I just tested this.)
Note 3: Outputting to a file with > does not preserve character encoding! This is why using Set-Content is recommended.
Do not use >; it messes up the character encoding. Use:
Get-Content files.* | Set-Content newfile.file
In cmd, you can do this:
copy one.txt+two.txt+three.txt four.txt
In PowerShell this would be:
cmd /c copy one.txt+two.txt+three.txt four.txt
While the PowerShell way would be to use gc, the above will be pretty fast, especially for large files. And it can be used on on non-ASCII files too using the /B switch.
You could use the Add-Content cmdlet. Maybe it is a little faster than the other solutions, because I don't retrieve the content of the first file.
gc .\file2.txt| Add-Content -Path .\file1.txt
To concat files in command prompt it would be
type file1.txt file2.txt file3.txt > files.txt
PowerShell converts the type command to Get-Content, which means you will get an error when using the type command in PowerShell because the Get-Content command requires a comma separating the files. The same command in PowerShell would be
Get-Content file1.txt,file2.txt,file3.txt | Set-Content files.txt
I used:
Get-Content c:\FileToAppend_*.log | Out-File -FilePath C:\DestinationFile.log
-Encoding ASCII -Append
This appended fine. I added the ASCII encoding to remove the nul characters Notepad++ was showing without the explicit encoding.
If you need to order the files by specific parameter (e.g. date time):
gci *.log | sort LastWriteTime | % {$(Get-Content $_)} | Set-Content result.log
You can do something like:
get-content input_file1 > output_file
get-content input_file2 >> output_file
Where > is an alias for "out-file", and >> is an alias for "out-file -append".
Since most of the other replies often get the formatting wrong (due to the piping), the safest thing to do is as follows:
add-content $YourMasterFile -value (get-content $SomeAdditionalFile)
I know you wanted to avoid reading the content of $SomeAdditionalFile into a variable, but in order to save for example your newline formatting i do not think there is proper way to do it without.
A workaround would be to loop through your $SomeAdditionalFile line by line and piping that into your $YourMasterFile. However this is overly resource intensive.
To keep encoding and line endings:
Get-Content files.* -Raw | Set-Content newfile.file -NoNewline
Note: AFAIR, whose parameters aren't supported by old Powershells (<3? <4?)
I think the "powershell way" could be :
set-content destination.log -value (get-content c:\FileToAppend_*.log )
I'm trying to copy a list of files from a txt file and as a newbie, I'm having a hard time.
Here is a bit of the text file. The real file has no extra lines, but I had to do that to :
"D:\Shared\Customer Care\Customer Care Common\Customers Contracted\Customers Contracted\Fred 44705"
"D:\Shared\Customer Care\Customer Care Common\Customers Contracted\Customers Contracted\Johnson 47227"
"D:\Shared\Customer Care\Customer Care Common\Customers Contracted\Customers Contracted\Daniel 35434"
"D:\Shared\Customer Care\Customer Care Common\Customers Contracted\Customers Contracted\Frank, John 48273"
I've tried enclosing the filename string in double-quotes as well.
Here's the simple script I'm trying to use:
Get-Content c:\users\scripts\files-to-fix.txt | Foreach-Object {copy-item $_ d:\junk}
The error I'm getting is:
Copy-Item : Cannot find drive. A drive with the name ''D' does not
exist. At C:\users\mhyman\scripts\copyfiles.ps1:2 char:81
+ Get-Content c:\users\mhyman\scripts\files-to-fix.txt |
Foreach-Object {copy-item <<<< $_ d:\junk}
+ CategoryInfo : ObjectNotFound: ('D:String) [Copy-Item],
DriveNotFoundException
+ FullyQualifiedErrorId :
DriveNotFound,Microsoft.PowerShell.Commands.CopyItemCommand
I know this is simple, but I would really appreciate some help.
I think it is the surrounding quotes that are causing the problem ( as indicated by the error saying that a drive of name "D is not found. Try this:
get-content c:\users\scripts\files-to-fix.txt | %{ copy-item $_.trim('"') d:\junk}
Of course, if you can control the txt file, enter the list without the quotes.
By your tags and drive letters and backslashes it is clearly a Windows environment your working in and although I'm not a PowerShell scripter, I'm a better than most batch scipter and use a For / If conditioanla statement sicne it is shorter and you feed it your file instead of parsing out the file into reduudc commands on a line, so in your example:
for /F %%t in (the text file.txt) do copy /q %%t d:\junk
And then you go home and never worry about until the next morning
Does powershell have a runas ornative mode that can parse older, more proven and stable DOS commands ?