What is the best way to communicate between python 2 applications using callbacks? - callback

I have 2 independent python 2 applications running in the same linux (ubuntu) computer.
I want to send messages from one to another (bidirectional) and receives these messages inside a callback function.
Is it possible? Do you have any example as reference?
Thanks

There are different options available for communicating between python apps.
A simple one would be to use an API based on HTTP. Each application will expose an specific port and communication takes place by exchanging HTTP requests.
There are several frameworks that allow you to build it in few steps. For example, using Bottle:
In app1:
from bottle import route, run, request
#route('/action_1', method='POST')
def action_1_handler():
data = request.json
print(str(data))
# Do something with data
return {'success': True, 'data': {'some_data': 1}}
run(host='localhost', port=8080)
In app2:
import requests
r = requests.post("http://localhost:8080/action_1", json={'v1': 123, 'v2': 'foo'})
print r.status_code
# 200
data = r.json()
# {u'data': {u'some_data': 1}, u'success': True}
Note that if the action executed at app1 after receiving the HTTP request takes lot of time, this could result in a timeout error. In such a case, consider to run the action in another thread or use an alternative communication protocol (e.g. sockets, ZeroMQ Messaging Library).
Some related reads:
Basic Python client socket example
Communication between two python scripts
https://www.digitalocean.com/community/tutorials/how-to-work-with-the-zeromq-messaging-library

Related

Gatling Websocket react on message

Is it possible to write a gatling script which connects to WebSocket and then performs actions (e.g. new HTTP requests) when certain messages are received (preferably with out of the box support for STOMP messages, but I could probably workaround this).
In other words, "real clients" should be simulated as best as possible. The real clients (angular applications) would load data based on certain WebSocket messages.
I'm looking for something similar to (pseude code, does not work):
val scn = scenario("WebSocket")
.exec(http("Home").get("/"))
.pause(1)
.exec(session => session.set("id", "Steph" + session.userId))
.exec(http("Login").get("/room?username=${id}"))
.pause(1)
.exec(
ws("Connect WS")
.open("/room/chat?username=${id}")
// ---------------------------------------------------------------------
// Is it possible to trigger/exec something (http call, full scenario)
// as reaction to a STOMP/WebSocket message?
.onMessage(check(perform some check, maybe regex?).as("idFromPayload"))
.exec(http("STOMP reaction").get("/entity/${idFromPayload}"))
// ---------------------------------------------------------------------
)
.exec(ws("Close WS").close)
// ideally, closing the websocket should only be done once the full scenario is over
// (or never, until the script terminates in "forever" scenarios)
Is this currently possible? If not, is this planned for future versions of gatling?
To the best of my knowledge, this is not possible with Gatling.
I have since switched to k6 which supports writing and executing test scripts with this kind of logic/behavior.

How to Implement an Infrastructure for Automed IVR calls?

I need tips to build an infrastructe to send 1000 simultaneous voice calls (automated IVR calls with voicexml). Up to now i used asterisk with voiceglue but now i have performance issues.
The infrasturcture was like this:
the asterisk pulls request from queue
the queue consumer create a call file
when the call ends, call file is read and status is sent to the application server
To be honest, i am asking for tips to implement an infrastructure like callfire[1] or voxeo[2]?
[1]https://www.callfire.com/
[2]http://voxeo.com/
you can go with voxeo prophecy (http://voxeo.com/prophecy/) one of the good server which have the capability of making simultaneous voice calls
Note: The requirement which your are expecting to do will not only possible with voxeo prophecy it should also depend the web server like Tomcat, IIS e.t.c in case if you dealing with databases like Sql , Oracle e.t.c
Please do refer to know the architecture
http://www.alpensoftware.com/define_VoiceOverview.html
CallFire's API has a CreateBroadcast method where you can throw up an IVR using their XML in seconds. You can read up on the documentation here:
https://www.callfire.com/api-documentation/rest/version/1.1#!/broadcast
CallFire also offers a PHP-SDK, hosted on Github, with examples of how to do this. The SDK is minimal setup and allows you to easily tap into the APIs robust functionality. Version 1.1 can be found here, with instructions on how to get started: https://github.com/CallFire/CallFire-PHP-SDK
The method call might look something like this. Note the required dependencies.
<?php
use CallFire\Api\Rest\Request;
use CallFire\Api\Rest\Response;
require 'vendor/autoload.php';
$dialplan = <<<DIALPLAN
<dialplan><play type="tts">Congratulations! You have successfully configured a CallFire I V R.</play></dialplan>
DIALPLAN;
$client = CallFire\Api\Client::Rest("<api-login>", "<api-password>", "Broadcast");
$request = new Request\CreateBroadcast;
$request->setName('My CallFire Broadcast');
$request->setType('IVR');
$request->setFrom('15551231234'); // A valid Caller ID number
$request->setDialplanXml($dialplan);
$response = $client->CreateBroadcast($request);
$result = $client::response($response);
if($result instanceof Response\ResourceReference) {
// Success
}
You can read this:
http://www.voip-info.org/wiki/view/Asterisk+auto-dial+out
Main tip: you WILL have ALOT of issues. If you are not expert with at least 5 years development experience with asterisk, you have use already developed dialling cores or hire guru. There are no opensource core that can do more then 300 calls on single server.
You can't do 1000 calls on single asterisk in app developed by "just nice developer". It will just not work.
Task of create dialling core for 1000 calls is "rocket science" type task. It require very special dialling core, very special server/server tunning and very specialized dialler with pre-planning.
1000 calls will result 23Mbit to 80Mbit bandwidth usage with SMALL packets, even that single fact can result you be banned on your hosting and require linux network stack be tunned.
You can use ICTBroadcast REST API to integerate your application with reknown autodialer , please visit following link for more detail
http://www.ictbroadcast.com/news/using-rest-api-integerate-ictbroadcast--third-party-application-autodialer
ICTBroadcast is based on asterisk communication engine
I've already done this for phone validation and for phone message broadcasting using Asterisk and Freeswitch. I would go with Freeswitch and xmlrpc:
https://wiki.freeswitch.org/wiki/Freeswitch_XML-RPC

Efficiently retrieve IP address and status code

Just a practical question. I do need to retrieve the HTTP status code of a site as well as the IP address.
Given the fact I normally need to parse between 10k and 150k domains, I was wondering which is the most efficient method.
I've seen that using the urllib2.urlopen(site) attempts to download the entire file stream connected to the file. At the same time the urllibs2 doesn't offer a method to convert an hostname into an IP.
Given I'm interested only in the HEAD bit to collect information like the HTTP status code and the IP address of that specific server, what is the best way to operate?
SHould I try to use only the socket? Thanks
I think there is no one particular magic tool that will retrieve the HTTP status code of a site and the IP address.
For getting HTTP status code you should make a HEAD request using urllib2 or httplib or requests. Here's an example, taken from How do you send a HEAD HTTP request in Python 2?:
>>> import urllib2
>>> class HeadRequest(urllib2.Request):
... def get_method(self):
... return "HEAD"
...
>>> response = urllib2.urlopen(HeadRequest("http://google.com/index.html"))
An example, using requests:
>>> import requests
>>> requests.head('http://google.com').status_code
301
Also, you might want to take a look at grequests in order to speed things up with getting status codes from multiple pages.
GRequests allows you to use Requests with Gevent to make asyncronous
HTTP Requests easily.
For getting an IP address, you should use socket:
socket.gethostbyname_ex('google.com')
Also see these threads:
How do you send a HEAD HTTP request in Python 2?
How to resolve DNS in Python?
How do I get a website's IP address using Python 3.x?
Hope that helps.

wrapping socket with websocket

Is it possible to use a webserver with websockets as a wrapper to another server to pass messages from the "real server" to a web client and back?
Im curious of this as I have a game server written in Ada that has an OS-tied client. I would like to swap this client to a webclient based on Javascript, so that the game can be played in a normal browser. What can be done?
That is the purpose of websockify. It is designed to bridge between WebSocket clients and regular TCP servers. It was created as part of the noVNC which is an HTML5 VNC app that can connect to normal VNC servers. However, websockify is generic and there are now many other projects using it.
Disclaimer: I created websockify and noVNC.
You can easily accomplish this by using AWS:
http://libre.adacore.com/tools/aws/
There's support for websockets in AWS, and you can make use of it's excellent socket (AWS.Net) packages for normal socket support.
Websockets are, contrary to what some people believe, not pure sockets. The raw data is encapsulated and masked by the websocket protocol which isn't widely supported yet. That means an application which wasn't designed for it, you can't communicate with it directly via web sockets.
When you have an application which uses a protocol based on normal sockets, and you want to communicate with it with websockets, there are two options.
Either you use a websocket gateway which does the unpacking / packing of the websocket traffic and forwards it as pure socket traffic to the application. This has the advantage that you needn't modify the application, but it has the disadvantage that it also masks the real IP address of the client which might or might not be a problem for certain applications.
Or you implement websocket in your application directly. This can be done by having two different ports the server listens to, one for normal connections and one for websocket connections. Any data which is received or sent through the websocket-port is sent through your websocket implementation before sending / after receiving it, and otherwise processed by the same routines.
THe Kaazing HTML5 Gateway is a great way of bringing your TCP-based protocol to a web client. The Kaazing gateway basically takes your protocol running on top of TCP and converts it to WebSocket so you can access the protocol in the client. You would still need to write a JavaScript protocol library for the protocol that your back end uses. But if you can work with the protocol on top of TCP, then it's not hard to do it with JavaScript.
I used the following code in ruby to wrapp my sockets. The code was adapted from em-websocket-proxy. There might be some specifics for my project in it but generally switching remote_host and remote_port and connecting to localhost:3000 should set you up with a new connection to your server through a websocket.
require 'rubygems'
require 'em-websocket'
require 'sinatra/base'
require 'thin'
require 'haml'
require 'socket'
class App < Sinatra::Base
get '/' do
haml :index
end
end
class ServerConnection < EventMachine::Connection
def initialize(input, output)
super
#input = input
#output = output
#input_sid = #input.subscribe { |msg| send_data msg+ "\n" }
end
def receive_data(msg)
#output.push(msg)
end
def unbind
#input.unsubscribe(#input_sid)
end
end
# Configuration of server
options = {:remote_host => 'your-server', :remote_port => 4000}
EventMachine.run do
EventMachine::WebSocket.start(:host => '0.0.0.0', :port => 8080) do |ws|
ws.onopen {
output = EM::Channel.new
input = EM::Channel.new
output_sid = output.subscribe { |msg| ws.send msg; }
EventMachine::connect options[:remote_host], options[:remote_port], ServerConnection, input, output
ws.onmessage { |msg| input.push(msg)}
ws.onclose {
output.unsubscribe(output_sid)
}
}
end
App.run!({:port => 3000})
end
Enjoy! And ask if you have questions.

Nodejs Websocket Close Event Called...Eventually

I've been having some problems with the below code that I've pieced together. All the events work as advertised however, when a client drops off-line without first disconnecting the close event doesn't get call right away. If you give it a minute or so it will eventually get called. Also, I find if I continue to send data to the client it picks up a close event faster but never right away. Lastly, if the client gracefully disconnects, the end event is called just fine.
I understand this is related to the other listen events like upgrade and ondata.
I should also state that the client is an embedded device.
client http request:
GET /demo HTTP/1.1\r\n
Host: example.com\r\n
Upgrade: Websocket\r\n
Connection: Upgrade\r\n\r\n
//nodejs server (I'm using version 6.6)
var http = require('http');
var net = require('net');
var sys = require("util");
var srv = http.createServer(function (req, res){
});
srv.on('upgrade', function(req, socket, upgradeHead) {
socket.write('HTTP/1.1 101 Web Socket Protocol Handshake\r\n' +
'Upgrade: WebSocket\r\n' +
'Connection: Upgrade\r\n' +
'\r\n\r\n');
sys.puts('upgraded');
socket.ondata = function(data, start, end) {
socket.write(data.toString('utf8', start, end), 'utf8'); // echo back
};
socket.addListener('end', function () {
sys.puts('end'); //works fine
});
socket.addListener('close', function () {
sys.puts('close'); //eventually gets here
});
});
srv.listen(3400);
Can anyone suggest a solution to pickup an immediate close event? I am trying to keep this simple without use of modules. Thanks in advance.
close event will be called once TCP socket connection is closed by one or another end with few complications of rare cases when system "not realising" that socket been already closed, but this are rare cases. As WebSockets start from HTTP request server might just keep-alive till it timeouts the socket. That involves the delay.
In your case you are trying to perform handshake and then send data back and forth, but WebSockets are a bit more complex process than that.
The handshake process requires some security procedure to validate both ends (server and client) and it is HTTP compatible headers. But different draft versions supported by different platforms and browsers do implement it in a different manner so your implementation should take this in account as well and follow official documentation on WebSockets specification based on versions you need to support.
Then sending and receiving data via WebSockets is not pure string. Actual data sent over WebSockets protocol has data-framing layer, which involves adding header to each message you send. This header has details over message you sending, masking (from client to server), length and many other things. data-framing depends on version of WebSockets again, so implementations will vary slightly.
I would encourage to use existing libraries as they already implement everything you need in nice and clean manner, and have been used extensively across commercial projects.
As your client is embedded platform, and server I assume is node.js as well, it is easy to use same library on both ends.
Best suit here would be ws - actual pure WebSockets.
Socket.IO is not good for your case, as it is much more complex and heavy library that has multiple list of protocols support with fallbacks and have some abstraction that might be not what you are looking for.