I am trying to add a counter in Flink as mentioned here, but the issue is that counter.inc() is returning void instead of Integer. Code for my Metric is given as below
private static class myMetric extends RichMapFunction<String,Integer> {
private Counter counter ;
#Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
this.getRuntimeContext().
getMetricGroup().
counter("countit");
}
#Override
public Integer map(String s) throws Exception {
return this.counter.inc();
}
}
It should work better if you assign a value to your counter:
this.counter = getRuntimeContext()
.getMetricGroup()
.counter("countit");
You may find the documentation helpful.
Related
I have a simple Solr query component as follows:
public class QueryPreprocessingComponent extends QueryComponent implements PluginInfoInitialized {
private static final Logger LOG = LoggerFactory.getLogger( QueryPreprocessingComponent.class );
private ExactMatchQueryProcessor exactMatchQueryProcessor;
public void init( PluginInfo info ) {
initializeProcessors(info);
}
private void initializeProcessors(PluginInfo info) {
List<PluginInfo> queryPreProcessors = info.getChildren("queryPreProcessors")
.get(0).getChildren("queryPreProcessor");
for (PluginInfo queryProcessor : queryPreProcessors) {
initializeProcessor(queryProcessor);
}
}
private void initializeProcessor(PluginInfo queryProcessor) {
QueryProcessorParam processorName = QueryProcessorParam.valueOf(queryProcessor.name);
switch(processorName) {
case ExactMatchQueryProcessor:
exactMatchQueryProcessor = new ExactMatchQueryProcessor(queryProcessor.initArgs);
LOG.info("ExactMatchQueryProcessor initialized...");
break;
default: throw new AssertionError();
}
}
#Override
public void prepare( ResponseBuilder rb ) throws IOException
{
if (exactMatchQueryProcessor != null) {
exactMatchQueryProcessor.modifyForExactMatch(rb);
}
}
#Override
public void process(ResponseBuilder rb) throws IOException
{
// do nothing - needed so we don't execute the query here.
return;
}
}
This works as expected functionally except when I use this in a distributed request, it has an issue with facets counts returned. It doubles the facet counts.
Note that I am not doing anything related to faceting in plugin. exactMatchQueryProcessor.modifyForExactMatch(rb); does a very minimal processing if the query is quoted otherwise it does nothing. Even if the incoming query is not quoted, facet count issue is there. Even if I comment everything inside prepare function, issue persists.
Note that this component is declared in as first-components in solrconfig.xml.
I resolved this issue by extending the class to SearchComponent instead of QueryComponent. It seems that SearchComponent sits at higher level of abstraction than QueryComponent and is useful when you want to work on a layer above shards.
I am using Apache Flink's sliding window with trigger. I would like to use a state variable in the trigger. However, I did not find any rich function corresponding to trigger function.
inputStream.keyBy(new KeySelector<Integer, String>() {
#Override
public String getKey(Integer i) throws Exception {
return i;
}
}).window(SlidingEventTimeWindows.of(Time.seconds(windowSize), Time.seconds(slideStep)))
.trigger(new Trigger<String, TimeWindow> {
#Override
public TriggerResult onElement(String str, long l, TimeWindow timeWindow, TriggerContext triggerContext) throws Exception {
return TriggerResult.CONTINUE;
}
#Override
public TriggerResult onProcessingTime(long l, TimeWindow timeWindow, TriggerContext triggerContext) throws Exception {
return TriggerResult.FIRE;
}
#Override
public TriggerResult onEventTime(long l, TimeWindow timeWindow, TriggerContext triggerContext) throws Exception {
return TriggerResult.FIRE;
}
#Override
public void clear(TimeWindow timeWindow, TriggerContext triggerContext) throws Exception {
})
.apply(new RichWindowFunction<String, String, String, TimeWindow>() {...}
So my question is, can we use state variables inside trigger block?
Yes, Triggers can be stateful. Look at CountTrigger for an example. You'll see that it keeps a count of how many elements have been added to the window in
private final ReducingStateDescriptor<Long> stateDesc =
new ReducingStateDescriptor<>("count", new Sum(), LongSerializer.INSTANCE);
It then uses that state in its onElement method
public TriggerResult onElement(Object element, long timestamp, W window, TriggerContext ctx)
throws Exception {
ReducingState<Long> count = ctx.getPartitionedState(stateDesc);
count.add(1L);
if (count.get() >= maxCount) {
count.clear();
return TriggerResult.FIRE;
}
return TriggerResult.CONTINUE;
}
and clears it in its clear method
public void clear(W window, TriggerContext ctx) throws Exception {
ctx.getPartitionedState(stateDesc).clear();
}
Keep in mind that if you are implementing a mergeable window type, then your Trigger will have to use state that can be merged, and then merge it when windows are merged. (Since you are using some flavor of sliding window, you can probably ignore this.)
I've seen a lot of examples of how to turn finite things like arrays or Iterables into Observables, but I'm not sure I understand how to make an Observable out of something live and effectively unbounded like an event receiver. I studied the RxJava2 docs and came up with this, using an Android LocationListener as an example.
Is there a simpler and/or more correct way to do this? I'm aware of the "RxBus" concept, but it seems like a way of clinging to the old event bus paradigm.
final Observable<Location> locationObservable = Observable.create(new ObservableOnSubscribe<Location>() {
final LocationManager mLocationManager = (LocationManager) getSystemService(LOCATION_SERVICE);
#Override
public void subscribe(final ObservableEmitter<Location> emitter) throws Exception {
final LocationListener listener = new LocationListener() {
#Override
public void onLocationChanged(final Location location) {
emitter.onNext(location);
}
#Override
public void onStatusChanged(final String s, final int i, final Bundle bundle) {
// TODO ???
}
#Override
public void onProviderEnabled(final String s) {
// TODO ???
}
#Override
public void onProviderDisabled(final String s) {
// TODO ???
}
};
mLocationManager.requestLocationUpdates(LocationManager.GPS_PROVIDER, 0, 0, listener);
emitter.setCancellable(new Cancellable() {
#Override
public void cancel() throws Exception {
mLocationManager.removeUpdates(listener);
}
});
emitter.setDisposable(new Disposable() {
private AtomicBoolean mDisposed;
#Override
public void dispose() {
if(mDisposed.compareAndSet(false, true)) {
mLocationManager.removeUpdates(listener);
}
}
#Override
public boolean isDisposed() {
return mDisposed.get();
}
});
}
});
using Observable.create() is indeed a correct way.
However, with RxJava2 the default way is to extend an Observable, you can see this answer for greater details.
some comments though regarding your implementation:
- there is no point setting both Cancellable and Disposable, as the later one will cancel/dispose the first one, you can see the difference between them here.
- I think it's best practice, to register cancellable/disposable before you start listening to update, in order to prevent weird edge cases races.
after toList operator performs, original Flowable<<\List>> converts to Single<<\List>>. and it turns out if I create Consumer to subscribe to Single,
the Consumer value type cannot be changed except Object?
#Override
public void loadBannerData(final ADFilterType adFilterType) {
remoteListDataSource.getBannerListData(adFilterType)
.flatMap(new Function<List<BannerBeanList.BannerBean>, Publisher<?>>() {
#Override
public Publisher<?> apply(List<BannerBeanList.BannerBean> bannerBeen) throws Exception {
return Flowable.fromIterable(bannerBeen);
}
})
.toList()
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
/******************************Consume Value Type**************************
.subscribe(new Consumer<List<BannerBeanList.BannerBean>>() {
#Override
public void accept(List<BannerBeanList.BannerBean> bannerBeens) throws Exception {
mainTabView.showMainBanner(bannerBeens);
}
});
*****************************************************************************/
}
From my comment: this happens because you have Publisher<?> instead of Publisher<BannerBeanList.BannerBean> in your code. Often IDEs can't infer types of lambdas or functional interfaces and you'll end up with ? or Object as their type when using some generate/convert refactoring function.
PLEASE HELP ME TO FIND THIS ANSWER
NULL POINTER EXCEPTION ERROR HELP ME
on these two lines
25,83
null pointer exception please correct my code
i created a object l with reference to class linkk
to access methods in the class
but while accessing the methods in the class with different conditions
it is showing null pointer exception
i didnt finish the code due to this error im struck at this point i cant move
further
this is my post
in overflow
i have read answers in overflow but this is the
first time fr me to post a question in overflow ,
today i created a ac nd posting this question please help me frnds
import java . util.Scanner;
class node
{
int i,q;
node next;
node prev;
}
class link{
public static void main(String args[])
{
linkk l = new linkk();
l.op();
int user=0;
while(user!=10)
{Scanner a=new Scanner(System.in);
if(user==1)
{
System.out.println("\nenter data\n");
l.create(a.nextInt());
}System.out.println("\n1.create link\n2.insert beginning\n3.insert middle\n4.insert end\n5.delete data\n6.reverse");
user=a.nextInt();
}
if(user==2)
l.insertbeg();
if(user==3)
l.insertmid();
if(user==4)
l.insertend();
if(user==5)
l.del();
if(user==6)
l.reverse();
if(user==7)
l.display();
}
}
class linkk
{
node temp4;
int ch,add,cnt=0,t=0,b;
node p= new node();
node q;
node last;
node first=null;
public boolean isEmpty()
{
return first == null;
}
public void insertbeg()
{
}
public void insertmid()
{
}
public void insertend()
{
}
public void del()
{
}
public void reverse()
{
}
public void display()
{
}
public void create(int val)
{
first.i=val;
first.next=null;
cnt++;
}
public void ob()
{
}
public void op()
{
}
}
Your defined first as null as node first=null; and you are trying to access i with first object using first.i=val; by calling l.create(a.nextInt());.
You should initialize first as below:
node first = new node();//and then access i of it and so on.