I have deployed 3 schema registry pods on a k8s cluster.
I am trying to find which schema registry pod is currently acting as a master.
I have figured it out one way to find master through schema registry logs. Here I have to check all pods logs manually.
Is there any command or configuration file in schema-registry to get the current master?
Get the master SR from ZK:
/usr/lib/zookeeper/bin/zkCli.sh
[zk: localhost:2181(CONNECTED) 1] get /schema_registry/schema_registry_master
{"host":"host name of current master schema registry","port":8081,"master_eligibility":true,"scheme":"http","version":1}
cZxid = ...
ctime = ...
mZxid = ...
mtime = ...
pZxid = ...
cversion = ...
dataVersion = ...
aclVersion = ...
ephemeralOwner = ...
dataLength = ...
numChildren = ...
Related
I've been using a PBS managed computing cluster for a few years now at my school. A few months ago I ran into this problem and they were never able to figure it out. When I submit jobs, they queue and then some run immediately. I believe that the jobs that should stay queued because of lack of resources die pretty much immediately. This happens intermittently depending on how many nodes I can use at one time. Sometimes I submit say 10 jobs, the first two will run, the next three will fail, then the next five will run.
I do not get either stdout or stderr files created for these failed jobs. The ones that do run do create these files. I get an email when these jobs die which I've attached here with some identifying information removed. Exit status -9 means "Could not create/open stdout stderr files" but I don't know how to fix that problem since it's so intermittent.
PBS Job Id: 11335.pearl.hpcc.XXX.edu
Job Name: mc1055
Exec host: m09/5
Aborted by PBS Server
Job cannot be executed
See Administrator for help
Exit_status=-9
resources_used.cput=00:00:00
resources_used.vmem=0kb
resources_used.walltime=00:00:02
resources_used.mem=0kb
resources_used.energy_used=0
req_information.task_count.0=1
req_information.lprocs.0=1
req_information.thread_usage_policy.0=allowthreads
req_information.hostlist.0=m09:ppn=1
req_information.task_usage.0.task.0={"task":{"cpu_list":"9","mem_list":"0","cores":0,"threads":1,"host":"m09"}}
Error_Path: CLUSTERNAME.hpcc.XXX.edu:/PATH/TOSCRIPT/run/mc1055.e11335
Output_Path: CLUSTERNAME.hpcc.XXX.edu:/PATH/TOSCRIPT/run/mc1055.o11335
I also looked at qstat -f right when a job failed and it's below. If I don't catch it right away it disappears from qstat.
Job Id: 11339.pearl.hpcc.XXX.edu
Job_Name = mc1059
Job_Owner = USERNAME#CLUSTERNAME.hpcc.XXX.edu
resources_used.cput = 00:00:00
resources_used.vmem = 0kb
resources_used.walltime = 00:00:00
resources_used.mem = 0kb
resources_used.energy_used = 0
job_state = C
queue = default
server = CLUSTERNAME.hpcc.XXX.edu
Account_Name = ADVISOR
Checkpoint = u
ctime = Mon Jan 4 20:02:25 2021
Error_Path = CLUSTERNAME.hpcc.XXX.edu/PATH/TOSCRIPT/mc1059.e11339
exec_host = m09/9
Hold_Types = n
Join_Path = n
Keep_Files = n
Mail_Points = a
mtime = Mon Jan 4 20:03:14 2021
Output_Path = CLUSTERNAME.hpcc.XXX.edu/PATH/TOSCRIPT/mc1059.o11339
Priority = 0
qtime = Mon Jan 4 20:02:25 2021
Rerunable = True
Resource_List.nodes = 1:ppn=1
Resource_List.walltime = 50:00:00
Resource_List.var = mkuuid:1e94a3e50dd44803bab2d3a7c2286ee2
Resource_List.nodect = 1
session_id = 0
Variable_List = PBS_O_QUEUE=largeq,
PBS_O_HOME=/PATH,PBS_O_LOGNAME=USERNAME,
PBS_O_PATH=lots of things
PBS_O_MAIL=/var/spool/mail/USERNAME,PBS_O_SHELL=/bin/bash,
PBS_O_LANG=en_US,KRB5CCNAME=FILE:/tmp/krb5cc_404112_hd4Yty,
PBS_O_WORKDIR=/PATH/TOSCRIPT/run,
PBS_O_HOST=CLUSTERNAME.hpcc.XXX.edu,
PBS_O_SERVER=CLUSTERNAME.hpcc.XXX.edu
euser = USERNAME
egroup = physics
queue_type = E
etime = Mon Jan 4 20:02:25 2021
exit_status = -9
submit_args = -l var=mkuuid:1e94a3e50dd44803bab2d3a7c2286ee2 -v KRB5CCNAME
/PATH/TOSCRIPT/run/tmp/montec_1059
start_time = Mon Jan 4 20:03:14 2021
start_count = 1
fault_tolerant = False
comp_time = Mon Jan 4 20:03:14 2021
job_radix = 0
total_runtime = 7.218811
submit_host = CLUSTERNAME.hpcc.XXX.edu
init_work_dir = /PATH/TOSCRIPT/run
request_version = 1
req_information.task_count.0 = 1
req_information.lprocs.0 = 1
req_information.thread_usage_policy.0 = allowthreads
req_information.hostlist.0 = m09:ppn=1
req_information.task_usage.0.task.0.cpu_list = 5
req_information.task_usage.0.task.0.mem_list = 1
req_information.task_usage.0.task.0.cores = 0
req_information.task_usage.0.task.0.threads = 1
req_information.task_usage.0.task.0.host = m09
Iam a beginner to Apache flume. Iam trying to pull data from a REST API and take it through flume and send to a kafka topic. But it is not working so far. The configuration I tried to use is shown below.There is a test GET API at localhost:8080/kafka/publish/ in the system. Iam trying to get data from this. The below configuration I tried pulling from flume documentation.
a1.sources = r1
a1.channels = c1
a1.sources.r1.type = http
a1.sources.r1.port = 8080
a1.sources.r1.channels = c1
a1.sources.r1.handler = org.apache.flume.source.http.JSONHandler
a1.sources.r1.handler.nickname = random props
a1.sources.r1.HttpConfiguration.sendServerVersion = false
a1.sources.r1.ServerConnector.idleTimeout = 300
a1.sinks.k1.channel = c1
a1.sinks.k1.type = org.apache.flume.sink.kafka.KafkaSink
a1.sinks.k1.kafka.topic = simple
a1.sinks.k1.kafka.bootstrap.servers = localhost:9092
a1.sinks.k1.kafka.flumeBatchSize = 20
a1.sinks.k1.kafka.producer.acks = 1
a1.sinks.k1.kafka.producer.linger.ms = 1
a1.sinks.k1.kafka.producer.compression.type = snappy
Can anyone help me solve this. What is the problem here?
The logs is added below
2020-12-03 11:16:17,696 (conf-file-poller-0) [WARN - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateConfigFilterSet(FlumeConfiguration.java:623)] Agent configuration for 'a1' has no configfilters.
2020-12-03 11:16:17,713 (conf-file-poller-0) [WARN - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:373)] Agent configuration for 'a1' does not contain any valid channels. Marking it as invalid.
2020-12-03 11:16:17,714 (conf-file-poller-0) [WARN - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:154)] Agent configuration invalid for agent 'a1'. It will be removed.
2020-12-03 11:16:17,715 (conf-file-poller-0) [INFO - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:163)] Post-validation flume configuration contains configuration for agents: []
2020-12-03 11:16:17,718 (conf-file-poller-0) [WARN - org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:139)] No configuration found for this host:a1
2020-12-03 11:16:17,730 (conf-file-poller-0) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:162)] Starting new configuration:{ sourceRunners:{} sinkRunners:{} channels:{} }
Iam new to flume and was trying my first experiment with it.Iam trying to read data from a file using fume and send it to a kafka topic.
The configuration is pulled from a tutorial website.The configuration is shown below.
a1.sources = r1
a1.sinks = sample
a1.channels = sample-channel
a1.sources.r1.type = exec
a1.sources.r1.command = tail -f \data.txt
a1.sources.r1.logStdErr = true
a1.channels.sample-channel.type = memory
a1.channels.sample-channel.capacity = 1000
a1.channels.sample-channel.transactionCapacity = 100
a1.sources.r1.channels = sample-channel
a1.sinks.sample.topic = sample
a1.sinks.sample.brokerList = 127.0.0.1:9092
a1.sinks.sample.requiredAcks = 1
a1.sinks.sample.batchSize = 20
a1.sinks.sample.channel = sample-channel
But this doesnot do anything.It isn't throwing any errors,but a few warnings. The log is shown below.
2020-12-03 12:01:17,265 (conf-file-poller-0) [WARN - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateConfigFilterSet(FlumeConfiguration.java:623)] Agent configuration for 'a1' has no configfilters.
2020-12-03 12:01:17,291 (conf-file-poller-0) [WARN - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:884)] Could not configure sink sample due to: Component has no type. Cannot configure. sample
org.apache.flume.conf.ConfigurationException: Component has no type. Cannot configure. sample
at org.apache.flume.conf.ComponentConfiguration.configure(ComponentConfiguration.java:76)
at org.apache.flume.conf.sink.SinkConfiguration.configure(SinkConfiguration.java:44)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.validateSinks(FlumeConfiguration.java:867)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.isValid(FlumeConfiguration.java:383)
at org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.access$000(FlumeConfiguration.java:228)
at org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:153)
at org.apache.flume.conf.FlumeConfiguration.<init>(FlumeConfiguration.java:133)
at org.apache.flume.node.PropertiesFileConfigurationProvider.getFlumeConfiguration(PropertiesFileConfigurationProvider.java:194)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:97)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:145)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:835)
How can I solve this?
As the error says, r1 has no type, so it doesn't know what to do with your source
You're missing
a1.sources.r1.type = exec
Personally, I'd suggest Filebeat or Telegraf over Flume for taking files to Kafka
I'm trying to pull data off my kafka topic and write it to HDFS, and appear to have my flume conf identical to what I've seen in several examples, but I can't seem to get around the below error. I can consume from the the topic through python, so I know I'm ok there. I'm on flume version 1.6.0 and java 9.0.1. What am I doing wrong to make it not accept the kafka topic?
09 Jul 2018 17:17:26,973 INFO [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadChannels:145) -Creating channels
09 Jul 2018 17:17:26,984 INFO [conf-file-poller-0] (org.apache.flume.channel.DefaultChannelFactory.create:42) - Creating instance of channel kafka_hdfs_channel type memory
09 Jul 2018 17:17:26,989 INFO [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadChannels:200) - Created channel kafka_hdfs_channel
09 Jul 2018 17:17:26,989 INFO [conf-file-poller-0] (org.apache.flume.source.DefaultSourceFactory.create:41) - Creating instance of source kafka_source, type org.apache.flume.source.kafka.KafkaSource
09 Jul 2018 17:17:26,993 ERROR [conf-file-poller-0] (org.apache.flume.node.AbstractConfigurationProvider.loadSources:361) - Source kafka_source has been removed due to an error during configuration
org.apache.flume.conf.ConfigurationException: Kafka topic must be specified.
at org.apache.flume.source.kafka.KafkaSource.configure(KafkaSource.java:180)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)
at org.apache.flume.node.AbstractConfigurationProvider.loadSources(AbstractConfigurationProvider.java:326)
at org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:97)
at org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:140)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:514)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:300)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
at java.base/java.lang.Thread.run(Thread.java:844)}
And here is my flume config:
agentCDIS.sources = kafka_source
agentCDIS.channels = kafka_hdfs_channel
agentCDIS.sinks = hdfs_sink
agentCDIS.sources.kafka_source.type = org.apache.flume.source.kafka.KafkaSource
agentCDIS.sources.kafka_source.kafka.bootstrap.servers = 10.4.3.61:9092, 10.4.3.62:9092, 10.4.3.63:9092
agentCDIS.sources.kafka_source.kafka.topic = test
agentCDIS.sources.kafka_source.kafka.consumer.group.id = cn_flume_group
agentCDIS.sources.kafka_source.channels = kafka_hdfs_channel
agentCDIS.sources.kafka_source.interceptors = i1
agentCDIS.sources.kafka_source.interceptors.i1.type = timestamp
agentCDIS.sources.kafka_source.kafka.consumer.timeout.ms = 1000
agentCDIS.channels.kafka_hdfs_channel.type = memory
agentCDIS.channels.kafka_hdfs_channel.capacity = 10000
agentCDIS.channels.kafka_hdfs_channel.transactionCapacity = 1000
agentCDIS.sinks.hdfs_sink.type = hdfs
agentCDIS.sinks.hdfs_sink.hdfs.path = hdfs://10.4.16.16:8020/user/cnelson/kafka/%{topic}/%y-%m-%d
agentCDIS.sinks.hdfs_sink.hdfs.rollInterval = 5
agentCDIS.sinks.hdfs_sink.hdfs.rollSize = 0
agentCDIS.sinks.hdfs_sink.fileType = DataStream
agentCDIS.sinks.hdfs_sink.channel = kafka_hdfs_channel
agentCDIS.sinks.loggerSink.type = logger
agentCDIS.sinks.loggerSink.kafka_hdfs_channel = memoryChannel
agentCDIS.channels.memoryChannel.type = memory
agentCDIS.channels.memoryChannel.capacity = 100
I went through the post and the config a few times and noticed - you've mentioned that you are using Flume's version 1.6 and as per the documentation, the properties are slightly different. Could you please try the following:
Instead of agentCDIS.sources.kafka_source.kafka.bootstrap.servers => try agentCDIS.sources.kafka_source.zookeeperConnect - the value for this property would be the zookeeper URI used by your Kafka cluster.
Instead of agentCDIS.sources.kafka_source.kafka.topic = test => try agentCDIS.sources.kafka_source.topic = test
Instead of agentCDIS.sources.kafka_source.kafka.consumer.group.id = cn_flume_group => try agentCDIS.sources.kafka_source.groupId = cn_flume_group
The above 3 properties that you've used in your config file were introduced from version 1.7.
I hope this helps!
Is it possible to read a znode with a space in it through Zookeeper CLI?
I've 2 values under regions ('us-west 1' and 'us-east')
[zk: localhost:2181(CONNECTED) 11] get /regions/
us-west 1 us-east
I can read 'us-east'.
[zk: localhost:2181(CONNECTED) 11] get /regions/us-east
null
cZxid = 0xa
ctime = Tue Jul 10 12:41:49 IST 2012
mZxid = 0xa
mtime = Tue Jul 10 12:41:49 IST 2012
pZxid = 0x1b
cversion = 9
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 9
But not 'us-west 1'
[zk: localhost:2181(CONNECTED) 11] get /regions/us-west 1
Node does not exist: /regions/us-west
I tried options like '%20', '\ ' , '+' etc.. for the space, but nothing worked.
Please try zookeepercli, as follows:
$ zookeepercli --servers srv1,srv2,srv3 -c create "/demo_only 1" "the value"
$ zookeepercli --servers srv1,srv2,srv3 -c get "/demo_only 1"
the value
zookeepercli is free and open source.
Disclaimer: I'm author of this tool.
It looks like you will not be able to do that from the ZK command-line client. The Zookeeper Java client (which is the one you are using, probably) separates commands (e.g. get) from their parameters (e.g. /regions/us-west 1 in your case) by parsing whitespace characters, as you can see in the code provided with the client (e.g. zookeeper-3.3.5\src\java\main\org\apache\zookeeper\ZooKeeperMain.java):
public boolean parseCommand( String cmdstring ) {
String[] args = cmdstring.split(" ");
if (args.length == 0){
return false;
}
command = args[0];
cmdArgs = Arrays.asList(args);
return true;
}
Since they are splitting by a " ", unless you discover a way to overcome that split call by some sort of escaping when calling the get command, you won't be able to retrieve those nodes using this client. The code above interprets your 1 in a get call as if it were the watch parameter, as you can see in the get command syntax:
get path [watch]
I recommend you to use a different character, like '_' for example, instead of the whitespace on the znodes naming. If that is not an option, you will either need to modify the ZK Java client yourself, or use another client.
There's an open issue in JIRA for this feature, but a workaround is to pass a command on the command-line instead of using the interactive console:
$ zkCli.sh -c get "/regions/us-west 1"