Objective: To fetch webpage using package HTTP in julia:
Environment: CentOS7.4
Content of fetch.jl:
#! /opt/julia/julia-1.1.0/bin/julia
using HTTP, Cascadia, Gumbo;
function fetch()
url ="https://www.google.com";
#url ="https://en.wikipedia.org/Stack_Overflow";
#url ="https://stackoverflow.com/questions/tagged/julia";
parsedPage = parsehtml(String(HTTP.get(url)));
println(string(parsedPage));
end
fetch();
invoking julia>/home/julia/fetch.jl shows as follows:
...
ERROR: LoadError: IOError(Base.Error(connect: host is unreachable(EHOSTUNREACH)", -113), during request(https://www.google.com))
...
Observation:
a) These sites are opening in browser(Mozilla Firefox).
b) wget fetches the pages
c) Also, tried with i) switching off both firewall (using systemctl stop firewalld) and selinux (using setenforce 0).
ii) other urls. But, no success!
Please help me in connecting and fetching webpages using HTTP!
Your code seems to be valid.
Try connecting using plain sockets and see what happens:
julia> using Sockets
julia> t = connect("google.com", 80)
TCPSocket(Base.Libc.WindowsRawSocket(0x000000000000037c) open, 0 bytes waiting)
julia> println(t, "GET /")
julia> readline(t)
"HTTP/1.0 200 OK"
Related
Dancer2 is a PSGI web application framework, we can use the "plackup" tool (provided by Plack) for launching the application:
plackup -p 5000 bin/app.psgi
The web application created using Dancer2 script can be viewed on localhost and port 5000.
http://localhost:5000
We can then write a get request as follows:
get '/hello/:name' => sub {
return "Hi there " . route_parameters->get('name');
};
For GET requests to "/hello/...", the code block provided above gets executed.
If the application is down, for GET requests to "/hello/...", the following error is shown:
curl: (7) Failed connect to localhost:5000; Connection refused
How do we integrate Dancer2::Core::HTTP helpers for rendering HTTP status codes for Dancer2 so that the following status code is shown when the app is down? Similarly, there are other error handlers too, how to integrate them as well?
status_500 status_error, status_internal_server_error
I am using lua as module for nginx (openresty) to get files from remote host. My function:
function readfile(url)
local http = require ("socket.http")
if not http then
error("Couldn't open socket.http")
end
http.TIMEOUT = 5
local body, code = http.request(url)
if not body then
error("Couldn't read the remote file: " .. code)
end
return body
end
I have tested this code by using Siege. When I set the users more then 100 (for example), I catch this error:
2018/03/27 09:36:38 [info] 10#10: *91018 shutdown() failed (107: Socket not connected), client: 172.18.0.7, server: localhost
I have more errors when i set more users. What does it mean? Thank you for the help.
Don't use luasocket library with OpenResty. The resulting code would block on http.request().
I suppose that all nginx worker just blocked and it is the reason of these errors.
For you purpose you may use one of the libraries below:
lua-resty-http
lua-resty-http-simple
First is more fexible, allow to use secure transport.
Second has simpler API.
And both use internally nginx Lua cosocket API and are 100% nonblocking out of the box.
The lua-resty-http or lua-resty-http-simple do not work in the init_by_lua in http context.
it is fine to use it in the init context where blocking is not considered harmful.
I have the Telit LE920-EUG 4G LTE module. I am trying to execute GET and POST http requests to a remote server. Though the PDP context is activating properly and I have internet access on the SIM that I'm using, I can't seem to be able to connect to a remote server and execute HTTP requests (both POST and GET) from the module.
I have tried two ways, one through direct HTTP commands supported by the module(All commands mentioned in the LE9x0 AT command reference guide), the commands sequence for which is mentioned below, but +CME ERROR: 100 occurs, and it's same for every http command(AT#HHTPQRY, AT#HTTPRCV) that I try to execute.
AT#SGACT=1,1
#SGACT: 31.81.208.1
OK
AT#HTTPCFG=0,"httpbin.org",80,0,,,0,120,1
+CME ERROR: 100
//No configuration details
AT#HTTPCFG?
+CME ERROR: 100
AT#HTTPCFG=?
+CME ERROR: 100
I have also tried the GET and POST commands after socket dialing. The socket connects but they are not receiving any data from the server or posting anything onto the server, the connection closes with a NO CARRIER. The command sequence that I'm using is given below
//Socket Dial
AT#SD=1,0,80,www.m2msupport.net
CONNECT
//GET commands sequence
GET /m2msupport/http_get_test.php HTTP/1.1
Host:www.m2msupport.net
Connection:keep-alive
//Connection closes with No Response
NO CARRIER
//Socket info shows the bytes sent
at#si=1
#SI: 1,86,0,0,0
OK
I do not use a proxy.
Here is my code:
use JIRA::Client::Automated;
my $jira = JIRA::Client::Automated->new(https://myserver.com, "user", "password");
And the error response is:
Unable to GET /jira/rest/api/latest/issue/DCS-51191: 500 Can't connect
to myserver.com:443 Can't connect to myserver.com:443
Bad file descriptor at
C:/Users/Fred/applis_portables/Strawberry_Perl/perl/vendor/lib/LWP/Protocol/http.pm
line 47.
at createPage2.pl line 16.
Thank you for your help.
It seems that there is a self signed certificate on JIRA server. To bypass, I added following code:
my $jira_ua = $jira->ua();
$jira_ua->ssl_opts( verify_hostname => 0 );
The error doesn't look like a JIRA::Client::Automated error. It's generated by LWP::UserAgent and usually means exactly what is shown.
Do you have a self signed certificate on your server?
Did you try to open that URL in in your browser? https://myserver.com:443 (exactly as you provide it to the module).
Try using curl from your webserver:
curl -vvv https://myserver.com/jira/rest/api/latest/issue/DCS-51191
Maybe it's just a missing www. prefix in your server URL?
I am trying to run this example from the book Sinatra: Up and Running p. 46, but can't get it to work. Here's the program code:
require 'sinatra'
before do
content_type :txt
end
connections = []
get '/consume' do
stream(:keep_open) do |out|
# store connection for later on
connections << out
# remove connection when closed properly
out.callback { connections.delete(out) }
# remove connection when closed due to an error
out.errback do
logger.warn 'we just lost a connection!'
connections.delete(out)
end
end
end
get '/broadcast/:message' do
connections.each do |out|
out << "#{Time.now} -> #{params[:message]}" << "\n"
end
"Sent #{params[:message]} to all clients."
end
The instructions for testing the code are as follows:
It’s a little tricky to demonstrate the behavior in text, but a good demonstration would
be to start the application, then open a web browser and navigate to http://localhost:
4567/consume. Next, open a terminal and use cURL to send messages to the server.
$ curl http://localhost:4567/broadcast/hello
Sent hello to all clients.
If you look back at the web browser, you should see that the content of the page has
been updated with a time stamp and the message that you sent via the terminal. The
connection remains open, and the client continues to wait for further information from
the server.
When I follow these instruction, I get no errors, but the message "hello" does not appear in the browser. I am running Sinatra on with Webrick. Why is it not working?
Thanks!
UPDATE (Konstantin's Thin Suggestion)
I now start thin and perform the two steps described in the book and the OP. You can see that thin does indeed receive both requests. However, I am still not seeeing the output "hello" in the browser.
>rackup
>> Thin web server (v1.4.1 codename Chromeo)
>> Maximum connections set to 1024
>> Listening on 0.0.0.0:9292, CTRL+C to stop
127.0.0.1 - - [13/Aug/2012 12:48:03] "GET /consume HTTP/1.1" 200 - 0.0900
127.0.0.1 - - [13/Aug/2012 12:48:03] "GET /favicon.ico HTTP/1.1" 404 447 0.0000
127.0.0.1 - - [13/Aug/2012 12:49:02] "GET /broadcast/hello HTTP/1.1" 200 26 0.00
00
127.0.0.1 - - [13/Aug/2012 12:57:00] "GET /consume HTTP/1.1" 200 - 0.0000
Perhaps the mistake is in my configu.ru file:
require './streaming.rb'
run Sinatra::Application
Run Sinatra on Thin. :keep_open is not supported on Webrick. Make sure you're running Sinatra 1.3.3 or later.
I had the same problem. To speed up the response, I used
before do
content_type 'text/event-stream'
end
The second route has to be a POST:
post '/broadcast/:message' do
connections.each do |out|
out << "#{Time.now} -> #{params[:message]}" << "\n"
end
"Sent #{params[:message]} to all clients."
end
After that, you will also have to POST your message to the server:
curl -vX POST 127.0.0.1:4567/broadcast/hello