I try to check the Quartz Scheduler status with the code below, but the status return is confusing. Why after I shutdown the scheduler the isStarted status still true and after I re-start the scheduler the isShutDown status is true.
if (logger.isLoggable(Level.INFO)) {
logger.info("Before: Stand by: "
+ this.scheduler.isInStandbyMode() + ", Start: "
+ this.scheduler.isStarted() + ", Shutdown: "
+ this.scheduler.isShutdown());
}
this.scheduler.start();
if (logger.isLoggable(Level.INFO)) {
logger.info("After: Stand by: "
+ this.scheduler.isInStandbyMode() + ", Start: "
+ this.scheduler.isStarted() + ", Shutdown: "
+ this.scheduler.isShutdown());
}
//Shutdown scheduler
this.scheduler.shutdown(true);
if (logger.isLoggable(Level.INFO)) {
logger.info("Schedule stop: Stand by: "
+ this.scheduler.isInStandbyMode() + ", Start: "
+ this.scheduler.isStarted() + ", Shutdown: "
+ this.scheduler.isShutdown());
}
//Restart scheduler
this.scheduler.start();
if (logger.isLoggable(Level.INFO)) {
logger.info("schedule start: Stand by: "
+ this.scheduler.isInStandbyMode() + ", Start: "
+ this.scheduler.isStarted() + ", Shutdown: "
+ this.scheduler.isShutdown());
}
And the result return is
INFO: Before: Stand by: true, Start: false, Shutdown: false
INFO: After: Stand by: false, Start: true, Shutdown: false
INFO: Schedule stop: Stand by: true, Start: true, Shutdown: true
INFO: schedule start: Stand by: true, Start: true, Shutdown: true
While this may seem like a bug, it is in fact by design. The isStarted() and isShutdown() methods do not report the current status of the Quartz Scheduler, instead they are to only be used to determine if at some point in the past the Scheduler has been started or shutdown. Please see the following Javadoc:
/**
* Whether the scheduler has been started.
*
* <p>
* Note: This only reflects whether <code>{#link #start()}</code> has ever
* been called on this Scheduler, so it will return <code>true</code> even
* if the <code>Scheduler</code> is currently in standby mode or has been
* since shutdown.
* </p>
*
* #see #start()
* #see #isShutdown()
* #see #isInStandbyMode()
*/
boolean isStarted() throws SchedulerException;
Related
I want to execute a query on Flink SQL Table backed by kafka topic of secured kafka cluster. I'm able to execute the query programmatically but unable to do the same through Flink SQL client. I'm not sure on how to pass JAAS config (java.security.auth.login.config) and other system properties through Flink SQL client.
Flink SQL query programmatically
private static void simpleExec_auth() {
// Create the execution environment.
final EnvironmentSettings settings = EnvironmentSettings.newInstance()
.inStreamingMode()
.withBuiltInCatalogName(
"default_catalog")
.withBuiltInDatabaseName(
"default_database")
.build();
System.setProperty("java.security.auth.login.config","client_jaas.conf");
System.setProperty("sun.security.jgss.native", "true");
System.setProperty("sun.security.jgss.lib", "/usr/libexec/libgsswrap.so");
System.setProperty("javax.security.auth.useSubjectCredsOnly","false");
TableEnvironment tableEnvironment = TableEnvironment.create(settings);
String createQuery = "CREATE TABLE test_flink11 ( " + "`keyid` STRING, " + "`id` STRING, "
+ "`name` STRING, " + "`age` INT, " + "`color` STRING, " + "`rowtime` TIMESTAMP(3) METADATA FROM 'timestamp', " + "`proctime` AS PROCTIME(), " + "`address` STRING) " + "WITH ( "
+ "'connector' = 'kafka', "
+ "'topic' = 'test_flink10', "
+ "'scan.startup.mode' = 'latest-offset', "
+ "'properties.bootstrap.servers' = 'kafka01.nyc.com:9092', "
+ "'value.format' = 'avro-confluent', "
+ "'key.format' = 'avro-confluent', "
+ "'key.fields' = 'keyid', "
+ "'value.fields-include' = 'EXCEPT_KEY', "
+ "'properties.security.protocol' = 'SASL_PLAINTEXT', 'properties.sasl.kerberos.service.name' = 'kafka', 'properties.sasl.kerberos.kinit.cmd' = '/usr/local/bin/skinit --quiet', 'properties.sasl.mechanism' = 'GSSAPI', "
+ "'key.avro-confluent.schema-registry.url' = 'http://kafka-schema-registry:5037', "
+ "'key.avro-confluent.schema-registry.subject' = 'test_flink6', "
+ "'value.avro-confluent.schema-registry.url' = 'http://kafka-schema-registry:5037', "
+ "'value.avro-confluent.schema-registry.subject' = 'test_flink4')";
System.out.println(createQuery);
tableEnvironment.executeSql(createQuery);
TableResult result = tableEnvironment
.executeSql("SELECT name,rowtime FROM test_flink11");
result.print();
}
This is working fine.
Flink SQL query through SQL client
Running this giving the following error.
Flink SQL> CREATE TABLE test_flink11 (`keyid` STRING,`id` STRING,`name` STRING,`address` STRING,`age` INT,`color` STRING) WITH('connector' = 'kafka', 'topic' = 'test_flink10','scan.startup.mode' = 'earliest-offset','properties.bootstrap.servers' = 'kafka01.nyc.com:9092','value.format' = 'avro-confluent','key.format' = 'avro-confluent','key.fields' = 'keyid', 'value.avro-confluent.schema-registry.url' = 'http://kafka-schema-registry:5037', 'value.avro-confluent.schema-registry.subject' = 'test_flink4', 'value.fields-include' = 'EXCEPT_KEY', 'key.avro-confluent.schema-registry.url' = 'http://kafka-schema-registry:5037', 'key.avro-confluent.schema-registry.subject' = 'test_flink6', 'properties.security.protocol' = 'SASL_PLAINTEXT', 'properties.sasl.kerberos.service.name' = 'kafka', 'properties.sasl.kerberos.kinit.cmd' = '/usr/local/bin/skinit --quiet', 'properties.sasl.mechanism' = 'GSSAPI');
Flink SQL> select * from test_flink11;
[ERROR] Could not execute SQL statement. Reason:
java.lang.IllegalArgumentException: Could not find a 'KafkaClient' entry in the JAAS configuration. System property 'java.security.auth.login.config' is /tmp/jaas-6309821891889949793.conf
There is nothing in /tmp/jaas-6309821891889949793.conf except the following comment
# We are using this file as an workaround for the Kafka and ZK SASL implementation
# since they explicitly look for java.security.auth.login.config property
# Please do not edit/delete this file - See FLINK-3929
SQL client run command
bin/sql-client.sh embedded --jar flink-sql-connector-kafka_2.11-1.12.0.jar --jar flink-sql-avro-confluent-registry-1.12.0.jar
Flink cluster command
bin/start-cluster.sh
How to pass this java.security.auth.login.config and other system properties (that I'm setting in the above java code snippet), for SQL client?
flink-conf.yaml
security.kerberos.login.use-ticket-cache: true
security.kerberos.login.principal: XXXXX#HADOOP.COM
security.kerberos.login.use-ticket-cache: false
security.kerberos.login.keytab: /path/to/kafka.keytab
security.kerberos.login.principal: XXXX#HADOOP.COM
security.kerberos.login.contexts: Client,KafkaClient
I haven't really tested whether this solution is feasible, you can try it out, hope it will help you.
I am using scala to get information about my objects that are in S3 I am intersted in getting each object I have inside S3 Bucket his tag value so far I have accomplished this code to get me information about my objects but did not succeeded in getting its tagging value: I used this code in scala:
def retrieveObjectTags(keyName: String): Unit ={
try {
println("Listing objects")
val req: ListObjectsV2Request =
new ListObjectsV2Request().withBucketName(bucketName).withMaxKeys(2)
var result: ListObjectsV2Result = null
do {
result = client.listObjectsV2(req)
for (objectSummary <- result.getObjectSummaries) {
println(
" - " + objectSummary.getKey + " " + "(size = " + objectSummary.getSize +
")")
println(objectSummary.getETag)
}
println("Next Continuation Token : " + result.getNextContinuationToken)
req.setContinuationToken(result.getNextContinuationToken)
} while (result.isTruncated == true);
}catch {
case ase: AmazonServiceException => {
println(
"Caught an AmazonServiceException, " + "which means your request made it " +
"to Amazon S3, but was rejected with an error response " +
"for some reason.")
println("Error Message: " + ase.getMessage)
println("HTTP Status Code: " + ase.getStatusCode)
println("AWS Error Code: " + ase.getErrorCode)
println("Error Type: " + ase.getErrorType)
println("Request ID: " + ase.getRequestId)
}
case ace: AmazonClientException => {
println(
"Caught an AmazonClientException, " + "which means the client encountered " +
"an internal error while trying to communicate" +
" with S3, " +
"such as not being able to access the network.")
println("Error Message: " + ace.getMessage)
}
}
// val getTaggingRequest = new GetObjectTaggingRequest(bucketName,keyName)
// var getTagResult = client.getObjectTagging(getTaggingRequest)
//println(getTaggingRequest)
var tag: Tag = new Tag()
println("tag name:" + tag.getValue)
}
as for the remarked lines I have encounter a problem with it, what other way I can use to solve this problem?
I am using Orientdb 2.2.12.
This below is single source from where I am getting Connection instance
public static synchronized Connection getOrientDbConnection(String appName)
{
try {
// address of Db Server
OServerAdmin serverAdmin = new OServerAdmin
("remote:" + ip + ":" + port + "/"
+ appName).connect("root", "1234");
if (serverAdmin.existsDatabase())
{
serverAdmin.close();
Properties info = new Properties();
info.put("user", "root");
info.put("password", "1234");
Connection conn = (OrientJdbcConnection) DriverManager.getConnection("jdbc:orient:remote:" + ip + ":" + port + "/"
+ appName, info);
//System.out.println(" Database Connection instance returned for : " + appName + " for thread ID : " + Thread.currentThread().getId());
return conn;
}
else
{
// "Type_Of_Db", "Type_Of_Storage"
serverAdmin.createDatabase("GRAPH", "plocal");
serverAdmin.close();
OrientGraphFactory graphFactory = new OrientGraphFactory(
"remote:" + ip + ":" + port + "/" + appName);
/**
* ####################### Do This only Ones-Per-Database #####################
* 1. Create index with unique field
* #name = unique index will be created only one time for a particular database
*
*
*/
OrientGraphNoTx graph = graphFactory.getNoTx();
graph.createKeyIndex("name", Vertex.class, new Parameter<String, String>("type", "UNIQUE"));
// shutdown the graph which created Index immediately
graph.commit();
graph.shutdown();
// close all db pools
graphFactory.close();
try
{
Properties info = new Properties();
info.put("user", "root");
info.put("password", "pcp");
Connection conn = (OrientJdbcConnection) DriverManager.getConnection("jdbc:orient:remote:" +ip + ":" + port + "/" + appName, info);
//System.out.println(" Database created and Connection instance return for : " + appName + " for thread ID : " + Thread.currentThread().getId());
return conn;
} catch (Exception e) {
//System.out.println("Problem creating database or getting connection from database " + " for thread ID : " + Thread.currentThread().getId() + e.getMessage());
e.getMessage();
return null;
}
}
}
catch(Exception e)
{
e.getMessage();
}
return null;
}
My use-case is creating :-
vertexes
edges
update vertex, edges etc..
App.java
Connection conn = DB.getOrientDbConnection(String appName)
// create vertex
// create edges
conn.commit()
conn.close();
What problems I am facing ?
Even after program finishes and conn.close() done, JConsole showing Orientdb connection still active.
Name: OrientDB <- Asynch Client (/192.168.1.11:4424)
State: RUNNABLE
Total blocked: 0 Total waited: 136
Stack trace:
java.net.SocketInputStream.socketRead0(Native Method)
java.net.SocketInputStream.socketRead(Unknown Source)
java.net.SocketInputStream.read(Unknown Source)
java.net.SocketInputStream.read(Unknown Source)
java.io.BufferedInputStream.fill(Unknown Source)
java.io.BufferedInputStream.read(Unknown Source)
- locked java.io.BufferedInputStream#1780dcd
java.io.DataInputStream.readByte(Unknown Source)
com.orientechnologies.orient.enterprise.channel.binary.OChannelBinary.readByte(OChannelBinary.java:68)
com.orientechnologies.orient.client.binary.OChannelBinaryAsynchClient.beginResponse(OChannelBinaryAsynchClient.java:189)
com.orientechnologies.orient.client.binary.OAsynchChannelServiceThread.execute(OAsynchChannelServiceThread.java:51)
com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:77)
Is this a bug or I am doing something wrong?
If your server is running you will see that in JConsole.
I am using asmack 8-0.8.3.
I don't receive messages for changes of Presence from my contacts when they move to "available".
If one contact passes from "available" to "dnd", I do receive a message. But not in the other way around.
Contact passes: "available" --> "dnd" --> "available" --> "dnd"
I receive: Presence{dnd} Presence{dnd}
Whereas I expect to receive a Presence update {available} between the 2 dnd.
Since I receive presence updates except for "available" I suppose my listener works fine. Also I suppose I correctly subscribed to my contacts' presence...
private class FriendListener implements RosterListener {
public void entriesAdded(Collection<String> addresses) { }
public void entriesUpdated(Collection<String> addresses) { }
public void entriesDeleted(Collection<String> addresses) { }
public void presenceChanged(Presence presence) {
String fromUserID = StringUtils.parseBareAddress(presence.getFrom());
System.out.println(
"Presence changed: " + fromUserID +
" Presence=" + presence.toString() +
" Type=" + presence.getType().toString() +
" Mode=" + presence.getMode().toString()
);
mainCallback_.updatePresenceFriend(fromUserID, presence);
}
}
public void subscribe(String friendID, String friendName) {
Presence presence = new Presence(Presence.Type.subscribe);
connection.sendPacket(presence);
RosterPacket rosterPacket = new RosterPacket();
rosterPacket.setType(IQ.Type.SET);
Item item = new Item(friendID, friendName);
item.setItemType(RosterPacket.ItemType.both);
rosterPacket.addRosterItem(item);
connection.sendPacket(rosterPacket);
System.out.println("Send subscribe to " + friendID);
subscribedUsers.add(friendID);
}
I found the problem!
Actually there was a bug in the log of my Listener, this line:
System.out.println(
"Presence changed: " + fromUserID +
" Presence=" + presence.toString() +
" Type=" + presence.getType().toString() +
" Mode=" + presence.getMode().toString()
);
It made crashed when Presence.getMode()==null, so that I did not process the Presence message. But no coredump was showing in the logs, I guess because the listener is in another thread...
Changing the log by the following line solved the problem
System.out.println(
"Presence changed: " + fromUserID +
" Presence=" + presence.toString()
);
I have got the following problem. I encountered an error regarding the default parameters. I added a simple piece of code for an overload. Now I get in the new code (line 3: CreatePageSection(sItemID, "", null);) gives the error mentioned in the title.
I looked for the answer in the other topics, but I can't find the problem. Can someone help me?
The code is found here:
public void CreatePageSection(ref string sItemID)
{
CreatePageSection(sItemID, "", null);
}
public void CreatePageSection(ref string sItemID, ref string sFrameUrl, ref object vOnReadyState)
{
if (Strings.InStr(msPresentPageSections, "|" + sItemID + "|", 0) > 0) {
return;
}
msPresentPageSections = msPresentPageSections + sItemID + "|";
string writeHtml = "<div class=" + MConstants.QUOTE + "PageSection" + MConstants.QUOTE + " id=" + MConstants.QUOTE + "Section" + sItemID + "Div" + MConstants.QUOTE + " style=" + MConstants.QUOTE + "display: none;" + MConstants.QUOTE + ">";
this.WriteLine_Renamed(ref writeHtml);
//UPGRADE_WARNING: Couldn't resolve default property of object vOnReadyState. Click for more: 'ms-help://MS.VSCC.v90/dv_commoner/local/redirect.htm?keyword="6A50421D-15FE-4896-8A1B-2EC21E9037B2"'
//UPGRADE_NOTE: IsMissing() was changed to IsNothing_Renamed(). Click for more: 'ms-help://MS.VSCC.v90/dv_commoner/local/redirect.htm?keyword="8AE1CB93-37AB-439A-A4FF-BE3B6760BB23"'
writeHtml = " <iframe id=" + MConstants.QUOTE + sItemID + "Frame" + MConstants.QUOTE + " name=" + MConstants.QUOTE + sItemID + "Frame" + MConstants.QUOTE + " frameborder=" + MConstants.QUOTE + "0" + MConstants.QUOTE + (!string.IsNullOrEmpty(sFrameUrl) ? " src=" + MConstants.QUOTE + sFrameUrl + MConstants.QUOTE : "") + ((vOnReadyState == null) ? "" : " onreadystatechange=" + MConstants.QUOTE + Convert.ToString(vOnReadyState) + MConstants.QUOTE) + ">";
this.WriteLine_Renamed(ref writeHtml);
writeHtml = " </iframe>";
this.WriteLine_Renamed(ref writeHtml);
writeHtml = "</div>";
this.WriteLine_Renamed(ref writeHtml);
}
you must pass params by reference
public void CreatePageSection(ref string sItemID)
{
var missingString = String.Empty;
object missingObject = null;
CreatePageSection(ref sItemID, ref missingString, ref missingObject);
}
Since your are not manipulating sFrameUrl and vOnReadyState, remove the ref keyword from those parameters.
See: http://msdn.microsoft.com/en-us/library/14akc2c7(v=vs.71).aspx
hth
Mario