I'm currently writing an project in Racket where in I need to communicate with a few (simulated) devices. I'm using the Racket pipes for this. Devices can read from the pipe, and write a generated answer on the pipe. The steward is also able to see the generated-answers on the pipe and write commands on the pipe.
Devices can write messages to the pipe by the following code:
File1:
(#%require "communication.rkt")
(write-string message output)
(flush-output output)
communication exports the input and output port:
File2:
(define-values (input output) (make-pipe #f))
The steward reads messages from the pipe:
File3:
(#%require "communication.rkt")
(if (char-ready? input)
(read-string 1 input)
When de steward tries to read a string from the pipe, 'char-ready' always returns #f.. But i'm sure that the devices write something on the pipe. Does anyone see my problem?
Thanks!
Related
Thanks to this great answer from #Przemyslaw Szufel, we can easily redirect stdio output into a file providing its path.
I wanted to know if it is possible to both redirect stio output to a file and still continue print it in the terminal. That is probably an easy feature to achieve but I could not find it!
And obviously not running the code twice :)
You need a Tee Stream such as https://github.com/fredrikekre/TeeStreams.jl/.
Unfortunately the interface of redirect_stdio does not accept abstract IO for its sink. It requires it to be an IOStream which is a concrete Julia type.
So it seems like there is no good way with the redirect_stdio via streams.
However redirect_stdio does accept sockets and named pipes and this is a way you could consider (yet too complex for this simple tasks - maybe someone finds a more elegant solution).
First you need to create a pipe server that will be running asynchronously in your REPL session (this is code for Windows for Linux you will probably need to remove \\.\pipe\ from the pipe name):
using Sockets
srv = Sockets.listen(raw"\\.\pipe\TeePipe")
mystdout = stdout
#async begin
open("log.txt", "w") do f
sock = accept(srv)
while !eof(sock)
line = readline(sock)
println(mystdout, line)
flush(mystdout)
println(f, line)
flush(f)
end
end
end
Now at the same REPL session you create the client:
teelogger = Sockets.connect(raw"\\.\pipe\TeePipe")
And here is a sample REPL session showing that this works:
julia> redirect_stdio(stdout=teelogger) do
println("Line1")
println("Line2")
#show 3+4
end
Line1
Line2
3 + 4 = 7
7
shell> more log.txt
Line1
Line2
3 + 4 = 7
I have a perl script for which I would like to write a few tests. If the perl script gets supplied a specific option, the user is allowed to paste or type out multi-line input and end their input using control-d.
These are the tests I want to write:
The script (when an interactive flag is supplied) waits for (multiple lines of) input on STDIN until the user hits control-d
[this test is already implemented - but provided for completeness] The script (when a flag is supplied that indicates a redirect/pipe) consumes all input on STDIN and does not wait for control-d
The script (when no input flag is provided [interactive or redirect]) does not wait for interactive user input
Test 1 toy example
I wrote a test for test 1 that confirms input was received (it works by the parent test script printing to the child's input handle and the child modifies and prints that input back out), but that test doesn't wait for an end of input signal (e.g. control-d) (which I don't know how to send anyway). So in other words, I can confirm it receives input, but I don't know how to confirm that it waits for the user to intentionally end the input entry. I.e. How do I know that the input consumption won't stop until the user types control-d?
Here's what I have so far for test 1. I wrote a 3rd little IO::Pipe::Consumer module to be able to send input to the child process that I'm testing and I wrote a toy example of the script that allows input on STDIN from a tty.
Here is a toy version of the script I'm testing:
>perl -e 'while(<STDIN>){print("*$_")}'
test
*test
^d
>
And here is the toy test code (that I want to improve) for the above script:
>perl -e '
use IO::Pipe::Consumer;
$obj = new IO::Pipe::Consumer;
$si = $obj->getSubroutineConsumer(
sub { while(<STDIN>) print("*$_") } }
);
print $si "test\n"
'
*test
>
I thought the parent would have to print an EOF (e.g. like what you get from "control-d") to end the input in the test, but the test ends immediately even though I'm not sending any such end-of-input character. I can see that it's getting, modifying, and printing the input. Is that sufficient to confirm that the script will wait for user input (and that the user will be able to intentionally end the input) or is there something else I should do to confirm it waits for all user input until the user intends to end it?
Test 2 - done
Test 3 toy - don't know how to write it...
Even if modified input spit back out is sufficient proof of "waiting for input" for test 1, I also wish to test that a script doesn't consume input on STDIN when no input option (interactive or redirect) is provided - but since it doesn't seem to wait even when I do send it input without an end-of-input signal, how would I test that the script wouldn't hang waiting for input? Note, the script has other options for consuming redirected or piped input, so my intent is specifically to know if it's waiting on input from the tty. All of the STDIN consumption options (whether from the tty or via redirect/pipe) are optional, which is why I want to write these tests.
My manual testing shows everything works as intended. I would just like some tests to assure that behavior for the future.
IO::Pipe::Consumer
I feel like the thing I'm missing is not relevant to IO::Pipe::Consumer, so WRT that, I'll just describe it instead of paste in 30 or so lines of code... All it does is it sets a pipe to the child's STDIN and gives that handle back to the parent for it to print to. I haven't put it on cpan. I'm just experimenting to see if I can use it to write these tests.
IO::Pipe::Consumer is basically the opposite of IO::Pipe::Producer (a module I published on cpan looong ago, say 2001-ish, when I was new to perl, or programming for that matter). The main difference, aside from swapping STDIN for STDOUT and Reader with Writer (and vice versa), is that the open is open(STDIN,"<",\${$stdin_pipe}).
I thought the parent would have to print an "end-of-input" (e.g. "control-d") character to end the input in the test,
Ctrl-D doesn't produce an "end of input character"; it causes the terminal to return EOF.
I don't know what IO::Pipe::Consumer is —it's not on CPAN— but I presume it creates a pipe. Exiting the program causes the writer end of the pipe to be closed and thus cause the reader end to return EOF.
is there something else I should do to confirm it waits for all user input until the user intends to end it?
<> reads until one of the following things happen:
A line feed is encountered (returning what was read including the line feed)
EOF is encountered (returning what was read up to an including the line feed)
An error is encountered (returning false).
You can confirm it waits by putting sleep statements between what you send. Be aware that buffering may interfere.
I'm trying to suppress output from a code section in a script (namely the network initialization from a Caffe network). I've tried wrapping the corresponding bit of code in an evalc command
[suppressed_output, var_output] = evalc('someFunction(input)');
But this doesn't work. I've still got loads of lines of (non-error) output from the network initialization that are clogging my logs (amidst all the wanted output printed via fprintf('') in the script). I think this happens because the corresponding function is writing to STDERR (instead of STDOUT?) - the first line it prints is this warning:
WARNING: Logging before InitGoogleLogging() is written to STDERR
... and then hundreds of lines of what it is doing follow, e.g.:
I0215 15:01:51.840272 28620 upgrade_proto.cpp:66] Attempting to upgrade input file specified using deprecated input fields: tmp-def.prototxt
I0215 15:01:51.840360 28620 upgrade_proto.cpp:69] Successfully upgraded file specified using deprecated input fields.
...
Can I somehow suppress the output to STDERR (without messing with the function content)? Ideally only locally for this specific function, since I'd still like to get potential error messages.
In case it is relevant:
I call myScript via matlab command line and its output written to a log (mlexec.log) with tee:
matlab -nodesktop -nosplash -display :1 -r "try, myScript; catch e, disp(getReport(e)), end, quit force" 2>&1| tee mlexec.log
The problem here is, that in the matlab command line call, the output from STDERR is streamed to STDOUT by this "command": 2>&1. Since the .cpp file seems to stream its output to STDERR (according to the Warning), it will be forwarded to STDOUT and eventually the log.
Streaming STDERR (2) to Nirvana with 2>NUL or a different log file (e.g. 2>mlexec.stderr.log) solves the problem.
I wanted to post this in a comment but it said I had to have 50 reputation (I have 49 now...)
I think this is what you're looking for
EDIT/UPDATE:
One thing you can do is enclose a section of your code with warning on/off statements as follows:
warning('off','all')
%your code here
warning('on','all')
This should stop any warnings being output to stderr from that section. I personally do not recommend this, it's good to know what you're doing that the MATLAB runtime does not like.
I am trying to write a simple server program in Ocaml that communicates with other processes via a socket. I currently have a problem that the strings the server reads (with input_line ic) do not seem to compare with other strings properly. For example, I run the server program and use telnet to connect to it, if I send "end" as a line, the server program trys to match with "end", but this doesn't work as expected. The function that handles communication is service (below), which is called to handle a client as part of a forking server (something like the double fork treatment here).
let service ic oc
try while true do
let instr = input_line ic in
match instr with
| "end" -> print_endline "matching end" (* this never runs *)
| _ -> output_string oc ((String.uppercase instr) ^ "\n") ; flush oc
done
with End_of_file -> print_endline "Input stream ended."
;;
In fact, if I do print_endline (string_of_bool ("end" = instr)) I always get false (even when I send "end" via telnet). To try and get some sense of what is going I printed out the result of different comparison operations between the let-binding and the try block:
print_endline instr ;
print_endline "end" ;
print_endline (string_of_bool ("end" = instr)) ;
print_endline (string_of_bool ("end" == instr)) ;
print_endline (string_of_int (String.compare "end" instr)) ;
When I send "end" the server now prints out
end
end
false
false
-1
I'm really lost as to what could be going on - I presume it must be something about getting the instr via reading from a socket, as usually you can compare strings just fine.
I don't think I actually had a question in all that background so here are a few variants that could work:
What am I doing wrong?
Why can't I test the input in order to take different actions?
Is this a bug in Ocaml?
Do you need the complete source to figure this out?
My guess is that there are carriage returns in the strings coming in from telnet. As I recall, the old protocols tend to send CRLF at the ends of lines.
You might try printing the string out using String.escaped.
It's pretty unlikely you're seeing a bug in OCaml.
I hope I'm doing something wrong, but it seems like kdb can't read data from named pipes (at least on Solaris). It blocks until they're written to but then returns none of the data that was written.
I can create a text file:
$ echo Mary had a little lamb > lamb.txt
and kdb will happily read it:
q) read0 `:/tmp/lamb.txt
enlist "Mary had a little lamb"
I can create a named pipe:
$ mkfifo lamb.pipe
and trying to read from it:
q) read0 `:/tmp/lamb.pipe
will cause kdb to block. Writing to the pipe:
$ cat lamb.txt > lamb.pipe
will cause kdb to return the empty list:
()
Can kdb read from named pipes? Should I just give up? I don't think it's a permissions thing (I tried setting -m 777 on my mkfifo command but that made no difference).
With release kdb+ v3.4 Q has support for named pipes: Depending on whether you want to implement a streaming algorithm or just read from the pipe use either .Q.fps or read1 on a fifo pipe:
To implement streaming you can do something like:
q).Q.fps[0N!]`:lamb.pipe
Then $ cat lamb.txt > lamb.pipe
will print
,"Mary had a little lamb"
in your q session. More meaningful algorithms can be implemented by replacing 0N! with an appropriate function.
To read the context of your file into a variable do:
q)h:hopen`:fifo://lamb.pipe
q)myText: `char$read1(h)
q)myText
"Mary had a little lamb\n"
See more about named pipes here.
when read0 fails, you can frequently fake it with system"cat ...". (i found this originally when trying to read stuff from /proc that also doesn't cooperate with read0.)
q)system"cat /tmp/lamb.pipe"
<blocks until you cat into the pipe in the other window>
"Mary had a little lamb"
q)
just be aware there's a reasonably high overhead (as such things go in q) for invoking system—it spawns a whole shell process just to run whatever your command is
you might also be able to do it directly with a custom C extension, probably calling read(2) directly…
The algorithm for read0 is not available to see what it is doing under the hood but, as far as I can tell, it expects a finite stream and not a continuous one; so it will will block until it receives an EOF signal.
Streaming from pipe is supported from v3.4
Details steps:
Check duplicated pipe file
rm -f /path/dataPipeFileName
Create named pipe
mkfifo /path/dataPipeFileName
Feed data
q).util.system[$1]; $1=command to fetch data > /path/dataPipeFileName &
Connect pipe using kdb .Q.fps
q).Q.fps[0N!]`$":/path/",dataPipeFileName;
Reference:
.Q.fps (streaming algorithm)
Syntax: .Q.fps[x;y] Where x is a unary function and y is a filepath
.Q.fs for pipes. (Since V3.4) Reads conveniently sized lumps of complete "\n" delimited records from a pipe and applies a function to each record. This enables you to implement a streaming algorithm to convert a large CSV file into an on-disk kdb+ database without holding the data in memory all at once.