specifying public address of apache ignite node with zookeeper discovery - apache-zookeeper

I have 3 apache ignite nodes in 3 different datacenters.
datacenter-1
datacenter-2
datacenter-3
The nodes are visible externally(and to each other) as ignite-1-datacenter-1.myorg.com ,ignite-2-datacenter-2.myorg.com,
and ignite-3-datacenter-3.myorg.com.
When I start a node, say node-1 in datacenter-1, I want to inform zookeeper that the other 2 nodes should
connect to this node using the address ignite-1-datacenter-1.myorg.com.
How do I achieve that?
ZookeeperDiscoverySpi = new ZookeeperDiscoverySpi();
zk.setZKConnectionString("my-zookeeper-address");
IgniteConfiguratin cfg = new IgniteConfiguration();
cfg.setDiscovery(zk)
Ignite i = Ignition.start(cfg);

I believe that this line gives us some intel about addresses that the SPI reports to Zookeeper. It seems like it should enough to specify a reachable host name here. More information could be found in the documentation. Rough example for the ignite-1-datacenter-1.myorg.com could look like:
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="localHost" value="ignite-1-datacenter-1.myorg.com"/>
...
</bean>

The following code snippet worked for me.
ZookeeperDiscoverySpi = new ZookeeperDiscoverySpi();
zk.setZKConnectionString("my-zookeeper-address");
InetAddress local = InetAddres.getLocalHost();
Map<String, String> addresses = new HashMap<>();
String externalOrPublicNameForThisNode = "ignite-1-datacenter-1.myorg.com";
addresses.put(local.getHostAddress, externalOrPublicNameForThisNode );
AddressResolver resolver = new BasicAddressResolver(addresses);
TcpCommunicationSpi commSpi = new TcpCommunicationSpi();
commSpi.setAddressResolver(resolver);
IgniteConfiguration igniteConfig = new IgniteConfiguration();
igniteConfig.setCommunicationSpi(commSpi);
igniteConfig.setAddressResolver(resolver);
igniteConfig.setDiscovery(zk);
Ignite i = Ignition.start(igniteConfig);

Related

MqttBrowserClient fails to connect due to missing conack package

I am trying to make webapp over flutter which will connect to HIVE broker. I took the broker name from the official website, set the port number to 8000 just like mentioned there and still get the error message as below:
error is mqtt-client::NoConnectionException: The maximum allowed connection attempts ({1}) were exceeded. The broker is not responding to the connection request message (Missing Connection Acknowledgement?
I really have no clue how to proceed. Can someone please help?
Below is my code:
MqttBrowserClient mq = MqttBrowserClient(
'wss://broker.mqttdashboard.com:8000', '',
maxConnectionAttempts: 1);
/*
MqttBrowserClient mq = MqttBrowserClient('ws://test.mosquitto.org', 'client-1',
maxConnectionAttempts: 1);
*/
class mqttService {
Future<MqttBrowserClient?> connectToServer() async {
try {
final connMess = MqttConnectMessage()
.withClientIdentifier('clientz5tWzoydVL')
.authenticateAs('a14guguliye', 'z5tWzoydVL')
.withWillTopic('willtopic')
.withWillMessage('My Will message')
.startClean() // Non persistent session for testing
.withWillQos(MqttQos.atLeastOnce);
mq.port = 1883;
mq.keepAlivePeriod = 50;
mq.connectionMessage = connMess;
mq.websocketProtocols = MqttClientConstants.protocolsSingleDefault;
mq.onConnected = onConnected;
var status = await mq.connect();
return mq;
} catch (e) {
print("error is " + e.toString());
mq.disconnect();
return null;
}
}
}
That port 8000 may be open but the HiveMQ broker may not be listening.
Make sure that the broker is fully booted and binds to that IP:Port combo.
In the HiveMQ broker startup output, you should see something similar to:
Started Websocket Listener on address 0.0.0.0 and on port 8000
If needed, the HiveMQ Broker configuration documentation is here.
You can use the public HiveMQ MQTT Websocket demo client to test your connection to make sure it's not a local code issue.
As a last option, use Wireshark to monitor MQTT traffic with a filter of tcp.port == 8000 and mqtt

How to fix "IndexError: list index out of range" in micropython socket.getaddrinfo() method?

I am using micropython on my esp32. I want to send data from esp32 to AWS IoT MQTT Broker Endpoint.
But in micropython socket module, getaddrinfo(hostname, port) method return empty list every time. How can I solve it?
It is working when I use IP address instead of host name. But AWS MQTT broker endpoint has no static IP for its broker endpoint.
# My code:
sta_if = network.WLAN(network.STA_IF)
sta_if.active(True)
sta_if.connect("wifiname", "password")
KEY_PATH = "/619e3d582c-private.pem.key"
CERT_PATH = "/619e3d582c-certificate.pem"
with open(KEY_PATH, 'r') as f:
PVT_KEY = f.read()
with open(CERT_PATH, 'r') as f:
CERT_KEY = f.read()
client = MQTTClient(client_id="esp32_micropython_shafik",
server="xxxxxxxxxxx.iot.eu-west-1.amazonaws.com",
port = 8883,
keepalive = 10000,
ssl = True,
ssl_params = {
"cert": CERT_KEY,
"key": PVT_KEY,
"server_side":False
} )
def checkwifi():
while not sta_if.isconnected():
time.sleep_ms(500)
print(".")
sta_if.connect()
def publish():
while True:
checkwifi()
msg = b'hello shafik'
client.publish(b"weather", msg)
time.sleep(1)
print("type", type(PVT_KEY))
client.connect()
publish()`
I am getting continuously this error:
umqtt/simple.py in 57 line,
IndexError: list index out of range.
How can I solve it?
I have solved this problem just changing my wifi network.
Actually this problem occurred for my ISP network issue, that's why socket module didn't work properly for fetching IP address properly.

Configure log4j.properties for Kafka appender, error when parsing property bootstrap.servers

I want to add a Kafka appender to the audit-hdfs log in a Cloudera cluster.
I have successfully configured a log4j2.xml file with a Kafka appender, I need to convert this log4j2.xml into a log4j2.properties file in order to be able to merge it with the HDFS log log4j2.properties file. I am unable to do this because when I launch my dummy process with log4j2.properties instead of XML, I get an error.
I have tried writing the properties file in several different ways, always resulting in problems with the bootstrap.servers property
This is my properties file
filters = threshold
filter.threshold.type = ThresholdFilter
filter.threshold.level = ALL
appenders = console,kafka
appender.console.type = Console
appender.console.name = console
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = %m%n
appender.kafka.type = Kafka
appender.kafka.name = kafka
appender.kafka.layout.type = PatternLayout
appender.kafka.layout.pattern =%m%n
appender.kafka.property.type = Property
appender.kafka.property.bootstrap.servers = ip:port
appender.kafka.topic = cdh-audit-hdfs
Here the problem is in this line:
appender.kafka.property.bootstrap.servers = ip:port
I have tried the following, to no avail:
appender.kafka.property.bootstrap.servers = ip:port
appender.kafka.property.bootstrap\.servers = ip:port
appender.kafka.property.name = "bootstrap.servers"
appender.kafka.property.bootstrap.servers = ip:port
appender.kafka.property.key = "bootstrap.servers"
appender.kafka.property.value = ip:port
etc...
This is my dummy process:
package blabla
import org.apache.logging.log4j.LogManager
object dummy extends App{
val logger = LogManager.getLogger
val record = "...c"
while(true){
logger.info(record)
Thread.sleep(5000)
}
}
How do I need to configure my log4j2.properties in order to be able to define this property?
Im expecting this process to write the record in my Kafka topic but instead I get errors like this:
Exception in thread "main" org.apache.logging.log4j.core.config.ConfigurationException: No type attribute provided for component bootstrap
Exception in thread "main" org.apache.kafka.common.config.ConfigException: Missing required configuration "bootstrap.servers" which has no default value.
try this solution, it works for me :
appender.kafka.property.type=Property
appender.kafka.property.name=bootstrap.servers
appender.kafka.property.value=host:port

javax.security.sasl.SaslException: Authentication failed: the server presented no authentication mechanisms in Wildfly 10.1

I am new to EJBs, and I am trying to perform remote invocations on stateless and stateful beans that I have deployed on a pod in my project that is based on Wildfly 10.1 in the new OpenShift 3 (Origin). The code that I am using for initializing the client context looks like:
Properties clientProperties = new Properties();
clientProperties.put("remote.connectionprovider.create.options.org.xnio.Options.SSL_ENABLED", "false");
clientProperties.put("remote.connections", "default");
clientProperties.put("remote.connection.default.host", "localhost");
clientProperties.put("remote.connection.default.port", "8080");
clientProperties.put("remote.connection.default.username", "****");
clientProperties.put("remote.connection.default.password", "****"); clientProperties.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOANONYMOUS", "false");
clientProperties.put("remote.connection.default.connect.options.org.xnio.Options.SASL_POLICY_NOPLAINTEXT", "false");
EJBClientContext.setSelector(new ConfigBasedEJBClientContextSelector(new
PropertiesBasedEJBClientConfiguration(clientProperties)));
Properties contextProperties = new Properties();
contextProperties.put(Context.URL_PKG_PREFIXES, "org.jboss.ejb.client.naming");
contextProperties.put(Context.SECURITY_PRINCIPAL, "****"); //username
contextProperties.put(Context.SECURITY_CREDENTIALS, "****"); //password
Context context = new InitialContext(contextProperties);
String appName = "CloudEAR";
String moduleName = "CloudEjb";
String distinctName = "";
String beanName = "Calculator";
String qualifiedRemoteView = "cloudEJB.view.CalculatorRemote";
String lookupString = "ejb:" + appName + "/" + moduleName + "/" + distinctName + "/" + beanName + "!" + qualifiedRemoteView;
Calculator calculator = (CalculatorRemote) context.lookup(lookupString);
int sum = calculator.sum(10, 10);
And the error message that I get is:
WARN: Could not register a EJB receiver for connection to localhost:8080
javax.security.sasl.SaslException: Authentication failed: the server presented no authentication mechanisms
at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.handleEvent(ClientConnectionOpenListener.java:378)
at org.jboss.remoting3.remote.ClientConnectionOpenListener$Capabilities.handleEvent(ClientConnectionOpenListener.java:240)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.channels.TranslatingSuspendableChannel.handleReadable(TranslatingSuspendableChannel.java:198)
at org.xnio.channels.TranslatingSuspendableChannel$1.handleEvent(TranslatingSuspendableChannel.java:112)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.ChannelListeners$DelegatingChannelListener.handleEvent(ChannelListeners.java:1092)
at org.xnio.ChannelListeners.invokeChannelListener(ChannelListeners.java:92)
at org.xnio.conduits.ReadReadyHandler$ChannelListenerHandler.readReady(ReadReadyHandler.java:66)
at org.xnio.nio.NioSocketConduit.handleReady(NioSocketConduit.java:89)
at org.xnio.nio.WorkerThread.run(WorkerThread.java:567)
at ...asynchronous invocation...(Unknown Source)
at org.jboss.remoting3.EndpointImpl.doConnect(EndpointImpl.java:272)
at org.jboss.remoting3.EndpointImpl.connect(EndpointImpl.java:388)
Initially I tried using the "jboss-ejb-client.properties" file, but that wasn't even able to make the remote connection. Now I am manually creating and configuring the EJBClientContext, and at least is successfully connecting to the remote server, but the invocation filas because of authentication failures.
I remember that we used to solve this issue by removing the "security realm" argument in "standalone.xml" files in older versions of OpenShift; however I am not able to find that file in the new version anymore. I have been looking at concepts such as secrets, volumes etc. but I really don't have a clear understanding how this works. When I create a new secret and try to associate it with my pod, the new deployment procedure fails. I would really appreciate any help.

Jersey Grizzly REST service not visible outside localhost

I'm trying to write a REST service in java using Jersey and Glassfish Grizzly. I have a very simple case working internally, but can't seem to call on the server from an external address. I've tried using a variety of different pairs of machines with externally visible IP's, and tried specifying the actual IP address in the server instead of localhost, but nothing works. I'm somewhat loosely following the official user guide here. My resource:
package resources;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
#Path("/simpleREST")
public class SimpleRESTResource
{
#GET
#Produces("text/plain")
public String getMessage()
{
return "Message from server\n";
}
}
And the server:
import java.io.IOException;
import java.net.URI;
import javax.ws.rs.core.UriBuilder;
import org.glassfish.grizzly.http.server.HttpServer;
import com.sun.jersey.api.container.grizzly2.GrizzlyServerFactory;
import com.sun.jersey.api.core.PackagesResourceConfig;
import com.sun.jersey.api.core.ResourceConfig;
public class Main
{
public static final URI BASE_URI = UriBuilder.fromUri("http://localhost").port(9998).build();
public static void main(String[] args) throws IOException
{
System.out.println("Starting grizzly...");
ResourceConfig rc = new PackagesResourceConfig("resources");
HttpServer myServer = GrizzlyServerFactory.createHttpServer(BASE_URI, rc);
System.out.println(String.format("Jersey app started with WADL available at %s/application.wadl\n" +
"Try out %s/simpleREST\nHit enter to stop it...", BASE_URI, BASE_URI));
System.in.read();
myServer.stop();
}
}
On the same machine, I can successfully interact with the server using
curl -X GET localhost:9998/simpleREST
OR
curl -X GET [external numeric address]:9998/simpleREST
Many thanks in advance for any suggestions.
SOLUTION
I have fixed this problem by setting the server URI to http://0.0.0.0:9998 instead of localhost, 127.0.0.1, or the actual address.
To make a server IP adress visible outside of localhost, you must fist open the neccessary firewall ports(if you have one), or use "0.0.0.0" instead of "localhost" in order for the server to listen to all IP addresses and network adapters. Before testing it in your local network, try pinging your server device from your client device to check if there is an actual connection or if the devices are not connected at all.
With Jersey and Grizzly 2.30, it's simpler:
final ResourceConfig rc = new ResourceConfig().packages("com.rest");
HttpServer httpServer = Grizzly
HttpServerFactory.createHttpServer(URI.create("http://0.0.0.0:9998/api/"), rc);
or, you can try these codes below:
ResourceConfig rc = new PackagesResourceConfig("your-rest-packages");
HttpHandler handler = ContainerFactory.createContainer(HttpHandler.class, rc);
server = new HttpServer();
server.getServerConfiguration().addHttpHandler(handler);
//attach listeners
InetAddress localHost = InetAddress.getLocalHost();
String localHostAddr = localHost.getHostAddress();
NetworkListener localHostListener = new NetworkListener("localhost", localHostAddr, port);
server.addListener(localHostListener);
InetAddress loopback = InetAddress.getLoopbackAddress();
String loopbackAddr = loopback.getHostAddress();
NetworkListener loopbackListener = new NetworkListener("loopback", loopbackAddr, port);
now your server could both list to localhost and loopback