Connect two machines in AKKA remotely ,connection refused - scala

I'm new to akka and wanted to connect two PC using akka remotely just to run some code in both as (2 actors). I had tried the example in akka doc. But what I really do is to add the 2 IP addresses into config file I always get this error?
First machine give me this error:
[info] [ERROR] [11/20/2018 13:58:48.833]
[ClusterSystem-akka.remote.default-remote-dispatcher-6]
[akka.remote.artery.Association(akka://ClusterSystem)] Outbound
control stream to [akka://ClusterSystem#192.168.1.2:2552] failed.
Restarting it. Handshake with [akka://ClusterSystem#192.168.1.2:2552]
did not complete within 20000 ms
(akka.remote.artery.OutboundHandshake$HandshakeTimeoutException:
Handshake with [akka://ClusterSystem#192.168.1.2:2552] did not
complete within 20000 ms)
And second machine:
Exception in thread "main"
akka.remote.RemoteTransportException: Failed to bind TCP to
[192.168.1.3:2552] due to: Bind failed because of
java.net.BindException: Cannot assign requested address: bind
Config file content :
akka {
actor {
provider = cluster
}
remote {
artery {
enabled = on
transport = tcp
canonical.hostname = "192.168.1.3"
canonical.port = 0
}
}
cluster {
seed-nodes = [
"akka://ClusterSystem#192.168.1.3:2552",
"akka://ClusterSystem#192.168.1.2:2552"]
# auto downing is NOT safe for production deployments.
# you may want to use it during development, read more about it in the docs.
auto-down-unreachable-after = 120s
}
}
# Enable metrics extension in akka-cluster-metrics.
akka.extensions=["akka.cluster.metrics.ClusterMetricsExtension"]
# Sigar native library extract location during tests.
# Note: use per-jvm-instance folder when running multiple jvm on one host.
akka.cluster.metrics.native-library-extract-folder=${user.dir}/target/native

First of all, you don't need to add cluster configuration for AKKA remoting. Both the PCs or nodes should be enabled remoting with a concrete port instead of "0" that way you know which port to connect.
Have below configurations
PC1
akka {
actor {
provider = remote
}
remote {
artery {
enabled = on
transport = tcp
canonical.hostname = "192.168.1.3"
canonical.port = 19000
}
}
}
PC2
akka {
actor {
provider = remote
}
remote {
artery {
enabled = on
transport = tcp
canonical.hostname = "192.168.1.4"
canonical.port = 18000
}
}
}
Use below actor path to connect any actor in remote from PC1 to PC2
akka://<PC2-ActorSystem>#192.168.1.4:18000/user/<actor deployed in PC2>
Use below actor path to connect from PC2 to PC1
akka://<PC2-ActorSystem>#192.168.1.3:19000/user/<actor deployed in PC1>
Port numbers and IP address are samples.

Related

Vault server deployment error, address already in use

I use latest version of docker image hashicorp/vault
i do all sptps from official documentation.
mount place config gile config.hcl in folder /vault/config/config.hcl
set environment variables :
VAULT_DEV_ROOT_TOKEN_ID: 'myroot_id'
VAULT_ADDR: "http://127.0.0.1:8200"
but containder are exited status , Error parsing listener configuration.
Error initializing listener of type tcp: listen tcp 127.0.0.1:8200: bind: address already in use
that is config file from official documentation
storage "raft" {
path = "./vault/data"
node_id = "node1"
}
listener "tcp" {
address = "127.0.0.1:8200"
tls_disable = "true"
}
api_addr = "http://127.0.0.1:8200"
cluster_addr = "https://127.0.0.1:8201"
ui = true
i try to change envidonment variables, ip adresses, ports, but no succes

How to make keepalived respond on failover on HAProxy?

I am using HAProxy as load balancer for my application and to make it highly available I am using keepalive service and floating ip address concept. But whenever my primary load balancer server gets down, by removing it from network or turning it off, my all services go down instead of making secondary load balancer server available.
My keepalived.conf for master server is,
global_defs
{
# Keepalived process identifier
lvs_id haproxy_DH
}
# Script used to check if HAProxy is running
vrrp_script check_haproxy
{
script "pidof haproxy"
interval 2
weight 2
}
# Virtual interface
vrrp_instance VI_01
{
state MASTER
interface eno16777984 #here eth0 is the name of network interface
virtual_router_id 51
priority 101
# The virtual ip address shared between the two loadbalancers
virtual_ipaddress {
172.16.231.162
}
track_script {
check_haproxy
}
}
For backup server it is like,
global_defs
{
# Keepalived process identifier
lvs_id haproxy_DH_passive
}
# Script used to check if HAProxy is running
vrrp_script check_haproxy
{
script "pidof haproxy"
interval 2
weight 2
}
# Virtual interface
vrrp_instance VI_01
{
state BACKUP
interface eno16777984 #here eth0 is the name of network interface
virtual_router_id 51
priority 100
# The virtual ip address shared between the two loadbalancers
virtual_ipaddress {
172.16.231.162
}
track_script {
check_haproxy
}
}
The virtual IP address is assigned and working when both load balancers are up. But whenever machine goes down, my service also goes down. I am using CentOS7, Please help.
Use this,
global_defs {
router_id ovp_vrrp
}
vrrp_script haproxy_check {
script "killall -0 haproxy"
interval 2
weight 2
}
vrrp_instance OCP_EXT {
interface ens192
virtual_router_id 51
priority 100
state MASTER
virtual_ipaddress {
10.19.114.231 dev ens192
}
track_script {
haproxy_check
}
authentication {
auth_type PASS
auth_pass 1cee4b6e-2cdc-48bf-83b2-01a96d1593e4
}
}
more info: read here, https://www.openshift.com/blog/haproxy-highly-available-keepalived

Gradle Liquibase SSH tunnel creating to Aurora Postgresql

need your help,
I need to connect to AWS Aurora Postgresql using liquibase, it's already configured for local machine, and works fine, but have issues with ssh configuration to it.
I'm using id 'org.hidetake.ssh' version '2.10.1', and id 'org.liquibase.gradle' version '2.0.4'
I'm able to run command directly on host machine, like getting date execute ('date') below, but have no idea why liquibase fails with
Unexpected error running Liquibase: liquibase.exception.DatabaseException: liquibase.exception.DatabaseException: Connection could not be created to jdbc:postgresql://xxxx.rds.amazonaws.com:5432/postgres with driver org.postgresql.Driver. The connection attempt failed.
here is my build.gradle setting:
ssh.settings {
knownHosts = allowAnyHosts
logging = 'stdout'
identity = file("${System.properties['user.home']}/myfolder/.ssh/id_rsa")}
remotes {
dev {
host = 'xxx.xxx.xxx.xxx'
port = 22
user = 'ec2-user'
identity = file("${System.properties['user.home']}/myfolder/.ssh/id_rsa")
}
}
ssh.run {
session(remotes.dev) {
forwardLocalPort port: 5432, hostPort: 5432
execute ('date')
liquibase {
activities {
main {
//changeLogFile changeLog
url 'jdbc:postgresql://xxxx.rds.amazonaws.com:5432/postgres'
username feedSqlUserDev
password feedSqlUserPasswordDev
logLevel 'debug'
}
}
}
}
}
Could you please help me with it, what am I doing wrong?
Also had to connect to SSH bastion host before running liquibase updates. My solution is based on https://github.com/int128/gradle-ssh-plugin/issues/246 answer by the plugin author.
Here is my setup:
ssh.settings {
knownHosts = allowAnyHosts
logging = 'stdout'
identity = file("${System.properties['user.home']}/.ssh/id_rsa")
}
remotes {
bastion {
host = '<hostname>'
user = '<username>'
}
}
liquibase {
activities {
main {
changeLogFile '...'
url 'jdbc:postgresql://localhost:5438/***'
username '***'
password '***'
driver 'org.postgresql.Driver'
}
}
}
task('sshTunnelStart') {
doFirst {
project.ext.ready = new CountDownLatch(1)
project.ext.done = new CountDownLatch(1)
Thread.start {
ssh.run {
session(remotes.bastion) {
forwardLocalPort port: 5438,
host: '<real db hostname>',
hostPort: 5432
project.ready.countDown()
project.done.await(5, TimeUnit.MINUTES) // liquibase update timeout
}
}
}
ready.await(10, TimeUnit.SECONDS) // start tunnel timeout
}
}
task('sshTunnelStop') {
doLast {
// teardown tunnel
project.done.countDown()
}
}
update.dependsOn(sshTunnelStart)
update.finalizedBy(sshTunnelStop)
Note that in liquibase config I use localhost:5438 as it is a local port forwarded to the remote. Later the same port is used in forwardLocalPort as a 'port' parameter. 'host' parameter is set to the remote database host, and 'hostPort' is accordingly the database port. The last part of the config adds dependencies between tasks to liquibase update and start/stop the tunnel.

akka cluster Connection refused: address is now > gated for [5000] ms

I have a cluster with 2 workers and 1 master. the cluster is implemented with Akka and Scala.
When i killed the worker and try to run it again with the following command:
java -Xms3500M -Xmx3500M -Dlog_file_name=worker1
"-Dconfig.file=F:\cluster\application.conf" -cp cluster.jar
knowmail.Worker worker1 2551
I get the following error:
Connection refused
Association with remote system
[akka.tcp://ClusterSystem#xxxxxx:2552] has failed, address is now
gated for [5000] ms. Reason: [As
kka.tcp://ClusterSystem#xxxx:2552]] Caused by: [Connection
refused: no further information: /xxxx:2552]
a configuration of cluster:
remote {
log-remote-lifecycle-events = off
log-received-messages = on
log-sent-messages = on
netty.tcp {
hostname = "xxxxxx"
port = 8888
bind-hostname = 0.0.0.0
bind-port = 8888
}
}
cluster {
seed-nodes = [
"akka.tcp://ClusterSystem#xxxxx:2551",
"akka.tcp://ClusterSystem#xxxxxx:2552"]
auto-down-unreachable-after = 20s
}
http.client.parsing.max-content-length = infinite
}
Did anyone encountered this error and solved it?
This happens when I start one of the seed nodes/workers before the other seed node has been started.
So one seed node is looking for the other and reports the following error of:
akka.tcp://ClusterSystem#10.5.2.10:2552] has failed, address is now
gated for [5000] ms.

Akka single point of failure

I want to create a system that will not have a single point of failure.
I was under the impression that routers are the tool for doing that but I'm not sure it works as I would expect.
This is the entry point of my program :
object Main extends App{
val system = ActorSystem("mySys", ConfigFactory.load("application"))
val router = system.actorOf(
ClusterRouterPool(RoundRobinPool(0), ClusterRouterPoolSettings(
totalInstances = 2, maxInstancesPerNode = 1,
allowLocalRoutees = false, useRole = Some("testActor"))).props(Props[TestActor]),
name = "testActors")
}
And this is the code for running the remote ActorSystem(so the router could deploy the TestActor code to the remote nodes):
object TestActor extends App{
val system = ActorSystem("mySys", ConfigFactory.load("application").getConfig("testactor1"))
case object PrintRouterPath
}
I'm running this twice, once with testactor1 and once with testactor2.
TestActor code:
class TestActor extends Actor with ActorLogging{
implicit val ExecutionContext = context.dispatcher
context.system.scheduler.schedule(10000 milliseconds, 30000 milliseconds,self, PrintRouterPath)
override def receive: Receive = {
case PrintRouterPath =>
log.info(s"router is on path ${context.parent}")
}
}
And application.conf
akka{
actor {
provider = "akka.cluster.ClusterActorRefProvider"
}
remote {
log-remote-lifecycle-events = off
netty.tcp {
hostname = "127.0.0.1"
port = 2552
}
}
cluster {
seed-nodes = [
"akka.tcp://mySys#127.0.0.1:2552"
"akka.tcp://mySys#127.0.0.1:2553"
"akka.tcp://mySys#127.0.0.1:2554"]
auto-down-unreachable-after = 20s
}
}
testactor1{
akka{
actor {
provider = "akka.cluster.ClusterActorRefProvider"
}
remote {
log-remote-lifecycle-events = off
netty.tcp {
hostname = "127.0.0.1"
port = 2554
}
}
cluster {
roles.1 = "testActor"
seed-nodes = [
"akka.tcp://mySys#127.0.0.1:2552"
"akka.tcp://mySys#127.0.0.1:2553"
"akka.tcp://mySys#127.0.0.1:2554"]
auto-down-unreachable-after = 20s
}
}
}
testactor2{
akka{
actor {
provider = "akka.cluster.ClusterActorRefProvider"
}
remote {
log-remote-lifecycle-events = off
netty.tcp {
hostname = "127.0.0.1"
port = 2553
}
}
cluster {
roles.1 = "testActor"
seed-nodes = [
"akka.tcp://mySys#127.0.0.1:2552"
"akka.tcp://mySys#127.0.0.1:2553"
"akka.tcp://mySys#127.0.0.1:2554"]
auto-down-unreachable-after = 20s
}
}
}
Now the problem is that when the process that started the router is killed, the actors that are running the code of TestActor, are not receiving any messages(the messages that the scheduler sends), I would have expect that the router will be deployed on another seed node in the cluster and the actors will be recovered. Is this possible? or is there any other way of implementing this flow and not having a single point of failure?
I think that, by deploying the router on only one node you are setting up a master-slave cluster, where the master is a single point of failure by definition.
From what I understand (looking at the docs), a router can be cluster-aware in the sense that it can deploy (pool mode) or lookup (group mode) routees on nodes in the cluster. The router itself will not react to failure by spawning somewhere else in the cluster.
I believe you have 2 options:
make use of multiple routers to make you system more fault-tolerant. Routees can either be shared (group mode) or not (pool mode) between routers.
make use of the Cluster Singleton pattern - which allows for a master-slave configuration where the master will be automatically re-spawned in case of failure. In relation to your example, note that this behaviour is achieved by having an actor (ClusterSingletonManager) deployed in each node. This actor has the purpose of working out if the chosen master needs to be respawned and where. None of this logic is in place in case of cluster-aware router like the one you setup.
You can find examples of multiple cluster setups in this Activator sample.
i tested two approaches, first using your code with ClusterRouterPool
Like you said when the process that started the router is killed, TestActor not receive more messages.
While reading the documentation and testing , if you change in application.conf :
`auto-down-unreachable-after = 20s`
for this
`auto-down-unreachable-after = off`
the TestActor keep receiving the messages, although in the log the following message appears(i don`t know how to put the log here, sorry):
[WARN] [01/30/2017 17:20:26.017] [mySys-akka.remote.default-remote-dispatcher-5] [akka.tcp://mySys#127.0.0.1:2554/system/endpointManager/reliableEndpointWriter-akka.tcp%3A%2F%2FmySys%40127.0.0.1%3A2552-0] Association with remote system [akka.tcp://mySys#127.0.0.1:2552] has failed, address is now gated for [5000] ms. Reason: [Association failed with [akka.tcp://mySys#127.0.0.1:2552]] Caused by: [Connection refused: /127.0.0.1:2552]
[INFO] [01/30/2017 17:20:29.860] [mySys-akka.actor.default-dispatcher-4] [akka.tcp://mySys#127.0.0.1:2554/remote/akka.tcp/mySys#127.0.0.1:2552/user/testActors/c1] router is on path Actor[akka.tcp://mySys#127.0.0.1:2552/user/testActors#-1120251475]
[WARN] [01/30/2017 17:20:32.016] [mySys-akka.remote.default-remote-dispatcher-5]
And in the case the MainApp is restarted the log works normally without warning or errors
MainApp Log :
[INFO] [01/30/2017 17:23:32.756] [mySys-akka.actor.default-dispatcher-2] [akka.cluster.Cluster(akka://mySys)] Cluster Node [akka.tcp://mySys#127.0.0.1:2552] - Welcome from [akka.tcp://mySys#127.0.0.1:2554]
TestActor Log:
INFO] [01/30/2017 17:23:21.958] [mySys-akka.actor.default-dispatcher-14] [akka.cluster.Cluster(akka://mySys)] Cluster Node [akka.tcp://mySys#127.0.0.1:2554] - New incarnation of existing member [Member(address = akka.tcp://mySys#127.0.0.1:2552, status = Up)] is trying to join. Existing will be removed from the cluster and then new member will be allowed to join.
[INFO] [01/30/2017 17:23:21.959] [mySys-akka.actor.default-dispatcher-14] [akka.cluster.Cluster(akka://mySys)] Cluster Node [akka.tcp://mySys#127.0.0.1:2554] - Marking unreachable node [akka.tcp://mySys#127.0.0.1:2552] as [Down]
[INFO] [01/30/2017 17:23:22.454] [mySys-akka.actor.default-dispatcher-2] [akka.cluster.Cluster(akka://mySys)] Cluster Node [akka.tcp://mySys#127.0.0.1:2554] - Leader can perform its duties again
[INFO] [01/30/2017 17:23:22.461] [mySys-akka.actor.default-dispatcher-2] [akka.cluster.Cluster(akka://mySys)] Cluster Node [akka.tcp://mySys#127.0.0.1:2554] - Leader is removing unreachable node [akka.tcp://mySys#127.0.0.1:2552]
[INFO] [01/30/2017 17:23:32.728] [mySys-akka.actor.default-dispatcher-4] [akka.cluster.Cluster(akka://mySys)] Cluster Node [akka.tcp://mySys#127.0.0.1:2554] - Node [akka.tcp://mySys#127.0.0.1:2552] is JOINING, roles []
[INFO] [01/30/2017 17:23:33.457] [mySys-akka.actor.default-dispatcher-14] [akka.cluster.Cluster(akka://mySys)] Cluster Node [akka.tcp://mySys#127.0.0.1:2554] - Leader is moving node [akka.tcp://mySys#127.0.0.1:2552] to [Up]
[INFO] [01/30/2017 17:23:37.925] [mySys-akka.actor.default-dispatcher-19] [akka.tcp://mySys#127.0.0.1:2554/remote/akka.tcp/mySys#127.0.0.1:2552/user/testActors/c1] router is on path Actor[akka.tcp://mySys#127.0.0.1:2552/user/testActors#-630150507]
The other approach is to use ClusterRouterGroup, because the routees are shared among the nodes of the cluster
Group - router that sends messages to the specified path using actor selection The routees can be shared among routers running on different nodes in the cluster. One example of a use case for this type of router is a service running on some backend nodes in the cluster and used by routers running on front-end nodes in the cluster.
Pool - router that creates routees as child actors and deploys them on remote nodes. Each router will have its own routee instances. For example, if you start a router on 3 nodes in a 10-node cluster, you will have 30 routees in total if the router is configured to use one instance per node. The routees created by the different routers will not be shared among the routers. One example of a use case for this type of router is a single master that coordinates jobs and delegates the actual work to routees running on other nodes in the cluster.
The Main App
object Main extends App {
val system = ActorSystem("mySys", ConfigFactory.load("application.conf"))
val routerGroup = system.actorOf(
ClusterRouterGroup(RoundRobinGroup(Nil), ClusterRouterGroupSettings(
totalInstances = 2, routeesPaths = List("/user/testActor"),
allowLocalRoutees = false, useRole = Some("testActor"))).props(),
name = "testActors")
}
you must start the TestActor in each remote node
object TestActor extends App{
val system = ActorSystem("mySys", ConfigFactory.load("application").getConfig("testactor1"))
system.actorOf(Props[TestActor],"testActor")
case object PrintRouterPath
}
http://doc.akka.io/docs/akka/2.4/scala/cluster-usage.html#Router_with_Group_of_Routees
The routee actors should be started as early as possible when starting the actor system, because the router will try to use them as soon as the member status is changed to 'Up'.
I hope it helps you