So I decided to start using PowerShell rather than Command Prompt. And I want to run curl. Very different output then discover that curl is an alias to Invoke-WebRequest in PowerShell.
Using PowerShell curl in the same way as real curl, I only get part of the content displayed.
I have seen that I can put the output of PowerShell curl into a variable and then use $variable.Content to display all of the content but that seems extra work over real curl.
Is there an option to show all of the content directly? I can't see one in the help.
Unlike the curl command line utility Invoke-WebRequest returns an object with various properties of which the content of the requested document is just one. You can get the content in a single statement by expanding the property like this:
Invoke-WebRequest 'http://www.example.org/' | Select-Object -Expand Content
or by getting the property value via dot-notation like this:
(Invoke-WebRequest 'http://www.example.org/').Content
Alternatively you could use the Windows port of curl:
& curl.exe 'http://www.example.org/'
Call the program with its extension to distinguish it from the alias curl for Invoke-WebRequest.
Well, if you are bothered with extra typing this is the shortest way to achieve that (well, at least I can think of):
(iwr google.tt).content
Something like this?
$res=Invoke-WebRequest "https://www.google.fr/"
#to view html of body
$res.ParsedHtml.body.innerHTML
#to view text of body
$res.ParsedHtml.body.innerText
Related
I'm trying to use Putty's plink.exe as part of a Powershell script, and am having trouble teeing the output.
Some of the commands invoke an interactive response (eg: entering password). Specifically, I'm testing against an Isilon.
Example code:
$command = '&"C:\Program Files\Putty\plink.exe" root#10.0.0.141 -pw "password" -t -batch "isi auth users create testuser --set-password"'
iex $command
Expected result:
I get a prompt password:
I enter the password
I get a prompt confirm:
I enter the password again
Command ends
If I try to tee the output, using iex $command | tee-object -variable result or even just redirect with iex $command *>test.log, the prompt text doesn't show up until after I've responded to it. While still technically functional, if you don't know exactly what prompt to expect, it's useless.
I've tried using Start-Transcript, but that doesn't capture the output at all. I've also tried using plink's -sshlog argument, but that logs way too much, in a less than readable format.
Is there any way to have stdout be unbuffered in the console, and also have it stored in a variable?
To answer some potential questions:
-This is to be run in an environment that doesn't allow modules, so can't use Posh-SSH.
-The Powershell version available isn't new enough to use the built-in openssh functionality.
This is all about redirecting streams.
When you use redirection, all outputs are redirected from the streams, and passed to be written to file. When you execute:
Write-Host "Some Text" *>out.txt
You don't see any output and it is all redirected to the file.
Key Note: Redirection works on a (simplification) line by line basis, as
the redirection works by writing to the file one line at a time.
Similarly, when you use Tee-Object, all outputs are redirected from the stream and down the pipeline. This is passed to the cmdlet Tee-Object. Tee-Object takes the input, and then writes that input to both the variable/file you want and to the screen. This happens After the input has been gathered and processed.
This means that both redirection and the Tee-Object commands work on a line by line basis. This makes sense both redirection and the Tee-Object commands work this way because it is hard to deal with things like deleting characters, moving around and editing text dynamically while trying to edit and maintain an open file at the same time. It is only designed for a one-way once the statement is complete, output.
In this case, when running it interactively, the password: prompt is written to the screen and you can respond.
When redirecting/Teeing the output, the password: text prompt is redirected, and buffered, awaiting your response. This makes sense because the statement has not completed yet. You don't want to send half a statement that could change, or half an object down the pipeline. It is only after you complete the statement (e.g. entering in the password + enter) that the whole statement is passed down the stream/pipeline. Once the whole statement is sent, then it is redirected/output Tee'd and can be displayed.
#Bill_Stewart is correct, in the sense that you should pick either an interactive prompt, or a fully automated solution.
Edit: To add some more information from comments.
If we use Tee-Object it relies on the Pipeline. Pipelines can only pass complete objects down the pipeline (e.g. complete strings inc. New Line). Pipelines have to interact with other commands like ForEach-Object or Select-Object, and they can't handle passing incomplete data to them. That's how the PowerShell console works, and you can't change it.
Similarly, redirection works line by line. The underlying reason why, I will explain why in a moment.
So, if you want to interact with it character by character, then you are dealing with streams. And if you want to deal with streams directly, it's 100 times more complicated because you can't use the convenience of the PowerShell console, you have to directly run of the process manually and handle all the input and output yourself.
To start, you have to manually launch the process. To do this we use the System.Diagnostics.Process class. The Pseudocode looks something like this:
$p = [System.Diagnostics.Process]::New()
$p.StartInfo.RedirectStandardOutput = $true
$p.StartInfo.RedirectStandardError = $true
$p.StartInfo.RedirectStandardInput = $true
$p.StartInfo.UseShellExecute = $false
#$p.StartInfo.CreateNoWindow = $true
$p.StartInfo.FileName = "plink.exe"
$p.StartInfo.Arguments = 'root#10.0.0.141 -pw "password" -t -batch "isi auth users create testuser --set-password"'
$p.EnableRaisingEvents = $true
....
We essentially create the process, specify that we are going to redirect the stdout (StartInfo.RedirectStandardOutput = $true), as well as the stdin to something else for us to handle. How do we know when to read the data? Well, the class has the Process.OutputDataReceived Event. You bind to this event to read in the additional data. But:
The OutputDataReceived event indicates that the associated Process has
written a line, terminating with a newline character, to its
redirected StandardOutput stream.
Remarks
So even the process class revolves around newlines for streaming data. This is why even redirects *> work on a line by line basis. PowerShell, and cmd, etc. all use the Process class as a basis to run processes. They all bind to this same event and methods to do their processing. Hence, why everything revolves around newlines and statement completions.
(big breath) So. You still want to interactively work with things one character at a time? well then you can't use the convenience of events. You will have to fall back to using a Stream Reader and directly binding to the Process.StandardOutput Property. Unfortunately this is where I stop, and say that to accomplish this
is beyond the scope of SO, and will require much more research to accomplish.
I used Invoke-WebRequest in Powershell to download a file without using the -OutFile parameter, and, from the documentation here, the file should've ended up in the directory I was in. However, there is nothing. Response was OK, no error was shown.
What could've happened to that file? Am I mistaken about how Invoke-WebRequest should work without an Out parameter?
Thanks!
Note: I know I can easily download the file using the parameter, but it's pretty big and I'd like to make sure it doesn't end up clogging disk space somewhere I don't need
From the linked docs:
By default, Invoke-WebRequest returns the results to the pipeline.
That is, in the absence of -OutFile no file is created.
(If you don't capture or redirect the output, it will print to the host (console).)
As techguy1029 notes in a comment, the current directory only comes into play if you do use -OutFile but specify a mere file name rather than a path.
As an aside: To-pipeline output is a response object of (a) type (derived from) WebResponseObject, whereas only the value of the response's body (the equivalent of property value .Content) is saved with -OutFile.
Lets talk about what the Microsoft documentation says for Invoke-WebRequest
"
-OutFile : Specifies the output file for which this cmdlet saves the response body. Enter a path and file name. If you omit the path, the
default is the current location. "
The Key word here is if a Path is omitted it will use the current path.
the -OutFile is a parameter of type String
The usage to save to current path would be
Invoke-webrequest "http://Test.com/test.pdf" -OutFile "Test.pdf"
else to have a custom path
Invoke-webrequest "http://Test.com/test.pdf" -OutFile "C:\Test\Test.Pdf"
I'm trying to run a program that will web scrape from Pastebin using PowerShell. I used the following code to do so:
Set Wshell = CreateObject("WScript.Shell")
Wshell.Run "%ComSpec% /c powershell & $result = Invoke-WebRequest
""https://pastebin.com/raw/wAhYB4UY"" & $result.content ", 0, True
$result.content will bring up everything I need from Pastebin. How can I transfer $result.content to a VBScript variable?
I know this is possible using the Exec() method as demonstrated here, but I can't use it because I want my code to stay hidden, which to my knowledge is not possible with Exec() (without having a window popping and closing)
I also don't want to use File I/O in Powershell because that can really complicate other things I want my program to do in the future; however, If absolutely no options are available, then I can use it.
EDIT: Some readers pointed out that my script only consists of running Powershell, so why not program my script in PowerShell? Well, not everything I am planning for this script to do can be done in PS. for example, I want my script to type some stuff outside of PS. I also want to wait until the user has pressed a certain key, in my case PrtSc (which will create a popup a message using MsgBox).
$Path = 'https://pastebin.com/raw/etc'
$Raw = Invoke-WebRequest $Path
$Raw.Content
What do you want to do with the data that using VBScript is the preferable tool?
I am currently trying the Powershell web server PoSH (http://poshserver.net/) for some administration reports. But i don't know how to format ouput.
From the start: i start the console with the default shortcut, with admin rights. I type Import-Module PoSHServer, then Start-PoSHServer. The web server starts, then i create a simple index.ps1 file, with just one line of code in the body section: $(command).
For example, i want to use the Get-Service Mpssvc command, but what i obtain is :
System.ServiceProcess.ServiceController
I try Get-Service MpsSvc | Select Name,Status. Output:
#{Name=MpsSvc; Status=Running}
Same thing for cmdlets Get-Process, i have an output with list of processes but it appears like this: System.Diagnostics.Process (AcroRd32) ...
However, some cmlets just like the Get-Date (used in the Posh demonstration web page) works fine and have a "normal" output.
I read the documentation, but there is no example which can help me for that.
How can i write powershell code to obtain a "clean" and console-like output?
I just downloaded and installed Posh-Server yesterday after reading this post.
If you want output to look like console inside a web-page you are probably looking at this from the wrong angle, you need to think string not console. Your code is supposed to be running inside of a here-string, in the example. So I got the hint here that the standard console formatter does not apply, posh-server will use whatever it wants to to turn your returned object into a STRING!. Your code output will get turned into a string using whatever formatter it deems applies unless you explicitly return a string - which the example script does correctly do. So try this on the console
get-process "power*" | out-string -width 80
And then try it in your posh-server script.
You probably really wanted this:
Get-Service MpsSvc | Select Name,Status | out-string -width 120
Hope that helps - I think the lack of examples in this project is a good thing because this is really a very simplistic web-server; lots of conceptual thinking required before you even start :) .
I am using powershell to automate some tasks related to checking out/merging in TFS. When I call
tf get * /recurse
I get a bunch of data scrolling by about the files that are getting checked out. The last line generated by this command (assuming its success) is one telling the checkin number. I would like to parse this out so it can be used later on in my script.
I know that I can do something like
$getOutput = tf get * /recurse
but then the output is suppressed entirely and I want the output of that command to be scrolled in realtime. I would basically like to grab everything that just got sent to the output buffer.
Try something like this:
tf get * /recurse | tee-Object -Variable getOutput
The tee-object in PowerShell 2.0 allows you to pipe results to two sources. If you leave the second source empty, the results go to the console.
ls | tee-object -filePath directoryListing.txt
This will write the directory listing to both the console and a text file.