Kapacitor Join not Performing Full Outer Join - kapacitor

I have a TICK script (shown below) with two queries, both performing groupBy on the same tag. The script then joins the two queries on that tag and specifies a full outer join with a fill of 'null'. However Kapacitor seems to be treating it as a inner join as shown by the stats (also below). The queries emit 33 and 32 points each and the join emits 32. Shouldn't a full outer join emit at least as many points as the query with the greater point count (33)? When I |log() the queries I'm able to identify the record that is dropped by the join - it was emitted by one query and not the other.
Any suggestions on how to further troubleshoot this?
TICK script:
var raw_event = batch
|query('''select rsum from jsx.autogen.raw_event''')
.period(1m)
.every(1m)
.offset(1h)
.align()
.groupBy('location')
var event_latency = batch
|query('''select rsum from jsx.autogen.event_latency''')
.period(1m)
.every(1m)
.offset(1h)
.align()
.groupBy('location', 'glocation')
raw_event
|join(event_latency)
.fill('null')
.as('raw_event','event_latency')
.on('location')
.streamName('join_stream')
|log()
Stats:
"node-stats": {
"batch0": {
"avg_exec_time_ns": 0,
"collected": 65,
"emitted": 0
},
"join4": {
"avg_exec_time_ns": 11523,
"collected": 65,
"emitted": 32
},
"log5": {
"avg_exec_time_ns": 0,
"collected": 32,
"emitted": 0
},
"query1": {
"avg_exec_time_ns": 0,
"batches_queried": 33,
"collected": 33,
"emitted": 33,
"points_queried": 33,
"query_errors": 0
},
"query2": {
"avg_exec_time_ns": 0,
"batches_queried": 32,
"collected": 32,
"emitted": 32,
"points_queried": 32,
"query_errors": 0
}

Related

Waiting for stream to stop before returning List

So I have a StreamSubscription that correctly reads data received from my device.
StreamSubscription<List<int>> _rxSubscription;
List<List<int>> _response = new List<List<int>>();
_rxSubscription = _ble.subscribeToCharacteristic(_rx).listen((event) {
_response.add(event);
});
writeToDevice(List<int> command) async {
_rxSubscription.resume();
await _ble.writeCharacteristicWithoutResponse(_tx, value: command);
return _response;
}
Here is how the device responds:
I/flutter ( 5506): [[0, 65, 0, 0], [0, 0, 104, 220, 69, 91, 240, 63, 0, 0, 240, 100, 112, 209, 240, 63, 0, 0, 176, 19], [133, 233, 240, 63, 0, 0, 104, 126, 172, 204, 240, 63, 0, 0, 96, 250, 189, 193, 240, 63]]
This data is continuously streamed when any write operation is performed and can vary in length depending on the 'command' meaning there can be n number of packets which are sent back. Right now after running the function, it returns an empty list of lists since _response is returned before the device starts responding.
What I'm trying to do: Get the _response to return only after all the data is collected and the device stops responding, ie. the Stream stops streaming. How would I do this in dart? I don't want to use a timer, since the data might take longer or shorter.
(Currently there is no implementation in the packets which might indicate end of line of end of message)
Vendor Instructions:
What I'm trying to do: Get the _response to return only after all the data is collected and the device stops responding, ie. the Stream stops streaming.
Depending on the listener's needs, this may be simplified by using a Future. Is this a broadcast stream?

custom encoder for Data

I got a series of data as bytes, I tried to encode it
let encoder = JSONEncoder()
encoder.dataEncodingStrategy = .deferredToData
let encodedData = try encoder.encode(data)
let data = encodedData.compactMap { $0 }
my data is looks like that:
[ 91,91,48, 44, 49, 57, 51, ..., 49, 44, 48, 93, 93]
It works, but the outcome is not what I expected (the numbers inside the data.element are different), then I tried to change the encoding strategy.
encoder64.dataEncodingStrategy = .base64
let data = encodedData.compactMap { $0 }
[91, 34, 65, 77, 72, ..., 65, 61, 61, 34, 93]
Then I have a different results, but still not is the what that I expected.
there is another custom encoding strategy, could you please give me some example of this custom strategy that I can test another forms to compare the results.
the example data that I expected is like :
[ 0, 193, 193, 193, 193, 72, 20, 193, ..., 255, 91, 0]
the 0 in the beginning and in the end is so important to me.
Thank you so much
The output is correct.
The JSONEncoder creates a JSON string from the data and compactMap maps each character to its UInt8 value
91 is [
91 is [
48 is 0
44 is ,
49 is 1
57 is 9
51 is 3
...
49 is 1
44 is ,
48 is 0
93 is ]
93 is ]
And consider the different representations: 91hex is 145dec

Array prefix(while:) looping different number of times for same logic. (unable to comprehend)

Here is the shot of the playground where I am trying to print all the even numbers until you reach an odd one in 3 ways and I don't understand why there are different number of times it looped
From my understanding it should loop only 7 times because there are 6 even numbers in the starting and the 7th time there is an odd number which must terminate the loop but we have gotten 8 times in one case ! Why ?
Also quite literally the second and third way are same but still there is a difference in the number of times the loop was executed.
This, I don't understand.
===== EDIT =======
I was asked for text form of the code so here it is
import UIKit
// First way
var arr = [34, 92, 84, 78, 90, 42, 5, 89, 64, 32, 30].prefix { num in
return num.isMultiple(of: 2)
}
print(arr)
// Second way
var arr2 = [34, 92, 84, 78, 90, 42, 5, 89, 64, 32, 30].prefix(while: {$0.isMultiple(of: 2)})
print(arr2)
//Third way (almost second way but directly prinitng this time)
print([34, 92, 84, 78, 90, 42, 5, 89, 64, 32, 30]
.prefix(while: {$0.isMultiple(of: 2)}))
The message of "(x times)" isn't really saying how many times a line is executed in a loop.
Think of it like this, sometimes the playground want to output multiple messages on the same line, because two things are being evaluated on that line. This happen the most often in loops, where in each iteration a new thing is evaluated, and Xcode wants to display that on the same line, but can't, so can only display "(2 times)".
You can also see this happen in other places, like closures that are on the same line:
let x = Optional.some(1).map { $0 + 1 }.map { String($0) } // (3 times)
Here, the 3 things that are evaluated are:
x
$0 + 1
String($0)
Another example:
func f() { print("x");print("y") } // (2 times)
f()
Normally, each print statement will have some text displayed on the right, won't they? But since they are on the same line, Xcode can only show "(2 times)".
The same thing happens in this case:
var arr2 = [34, 92, 84, 78, 90, 42, 5, 89, 64, 32, 30].prefix(while: {$0.isMultiple(of: 2)})
There are 8 things to evaluate. The closure result is evaluated 7 times, plus once for the value of arr2.
If you put the variable declaration and the closure on separate lines, then Xcode can show the value of the variable on a separate line, which is why the number is one less in the two other cases.
To illustrate this graphically:
You are forcing Xcode to combine the two boxed messages into one single line. Xcode can't usefully fit all that text onto one single line, so the best it can do is "(8 times)".
I believe these are just artifacts of how playgrounds work. With a little re-formatting I get "7 times" in all three variations:
let data = [34, 92, 84, 78, 90, 42, 5, 89, 64, 32, 30]
let arr = data.prefix {
$0.isMultiple(of: 2)
}
print(arr)
let arr2 = data.prefix(while: {
$0.isMultiple(of: 2)
})
print(arr2)
print(data.prefix(while: {
$0.isMultiple(of: 2)
}))

Why connects geom_line not to next point when using in gganimate?

When I have this data frame
library(ggplot)
library(gganimate)
data <- tribble(
~year, ~num,
1950, 56,
1951, 59,
1952, 64,
1953, 76,
1954, 69,
1955, 74,
1956, 78,
1957, 98,
1958, 85,
1959, 88,
1960, 91,
1961, 87,
1962, 99,
1963, 104
)
and want to make an animated line plot with gganimate:
ggplot(data, aes(year, num))+geom_point()+geom_line()+transition_reveal(year, num)
I get a diagram, in which points and lines are drawn in the wrong sequence.
What is the reason for this and how can I correct it?
In
transition_reveal()
the first argument (id) regards the group aesthetic (which you don't have). I found that just using id = 1 for a single time series works.
The second argument (along) should be your x aesthetic (in your case the year).
Try:
ggplot(data, aes(year, num))+
geom_point()+
geom_line()+
transition_reveal(1, year)

Difference between "enqueue" and "dequeue"

Can somebody please explain the main differences? I don't have a clear knowledge about these functions in programming for any language.
Some of the basic data structures in programming languages such as C and C++ are stacks and queues.
The stack data structure follows the "First In Last Out" policy (FILO) where the first element inserted or "pushed" into a stack is the last element that is removed or "popped" from the stack.
Similarly, a queue data structure follows a "First In First Out" policy (as in the case of a normal queue when we stand in line at the counter), where the first element is pushed into the queue or "Enqueued" and the same element when it has to be removed from the queue is "Dequeued".
This is quite similar to push and pop in a stack, but the terms enqueue and dequeue avoid confusion as to whether the data structure in use is a stack or a queue.
Class coders has a simple program to demonstrate the enqueue and dequeue process. You could check it out for reference.
http://classcoders.blogspot.in/2012/01/enque-and-deque-in-c.html
Enqueue and Dequeue tend to be operations on a queue, a data structure that does exactly what it sounds like it does.
You enqueue items at one end and dequeue at the other, just like a line of people queuing up for tickets to the latest Taylor Swift concert (I was originally going to say Billy Joel but that would date me severely).
There are variations of queues such as double-ended ones where you can enqueue and dequeue at either end but the vast majority would be the simpler form:
+---+---+---+
enqueue -> | 3 | 2 | 1 | -> dequeue
+---+---+---+
That diagram shows a queue where you've enqueued the numbers 1, 2 and 3 in that order, without yet dequeuing any.
By way of example, here's some Python code that shows a simplistic queue in action, with enqueue and dequeue functions. Were it more serious code, it would be implemented as a class but it should be enough to illustrate the workings:
import random
def enqueue(lst, itm):
lst.append(itm) # Just add item to end of list.
return lst # And return list (for consistency with dequeue).
def dequeue(lst):
itm = lst[0] # Grab the first item in list.
lst = lst[1:] # Change list to remove first item.
return (itm, lst) # Then return item and new list.
# Test harness. Start with empty queue.
myList = []
# Enqueue or dequeue a bit, with latter having probability of 10%.
for _ in range(15):
if random.randint(0, 9) == 0 and len(myList) > 0:
(itm, myList) = dequeue(myList)
print(f"Dequeued {itm} to give {myList}")
else:
itm = 10 * random.randint(1, 9)
myList = enqueue(myList, itm)
print(f"Enqueued {itm} to give {myList}")
# Now dequeue remainder of list.
print("========")
while len(myList) > 0:
(itm, myList) = dequeue(myList)
print(f"Dequeued {itm} to give {myList}")
A sample run of that shows it in operation:
Enqueued 70 to give [70]
Enqueued 20 to give [70, 20]
Enqueued 40 to give [70, 20, 40]
Enqueued 50 to give [70, 20, 40, 50]
Dequeued 70 to give [20, 40, 50]
Enqueued 20 to give [20, 40, 50, 20]
Enqueued 30 to give [20, 40, 50, 20, 30]
Enqueued 20 to give [20, 40, 50, 20, 30, 20]
Enqueued 70 to give [20, 40, 50, 20, 30, 20, 70]
Enqueued 20 to give [20, 40, 50, 20, 30, 20, 70, 20]
Enqueued 20 to give [20, 40, 50, 20, 30, 20, 70, 20, 20]
Dequeued 20 to give [40, 50, 20, 30, 20, 70, 20, 20]
Enqueued 80 to give [40, 50, 20, 30, 20, 70, 20, 20, 80]
Dequeued 40 to give [50, 20, 30, 20, 70, 20, 20, 80]
Enqueued 90 to give [50, 20, 30, 20, 70, 20, 20, 80, 90]
========
Dequeued 50 to give [20, 30, 20, 70, 20, 20, 80, 90]
Dequeued 20 to give [30, 20, 70, 20, 20, 80, 90]
Dequeued 30 to give [20, 70, 20, 20, 80, 90]
Dequeued 20 to give [70, 20, 20, 80, 90]
Dequeued 70 to give [20, 20, 80, 90]
Dequeued 20 to give [20, 80, 90]
Dequeued 20 to give [80, 90]
Dequeued 80 to give [90]
Dequeued 90 to give []
These are terms usually used when describing a "FIFO" queue, that is "first in, first out". This works like a line. You decide to go to the movies. There is a long line to buy tickets, you decide to get into the queue to buy tickets, that is "Enqueue". at some point you are at the front of the line, and you get to buy a ticket, at which point you leave the line, that is "Dequeue".
A queue is a certain 2-sided data structure. You can add new elements on one side, and remove elements from the other side (as opposed to a stack that has only one side). Enqueue means to add an element, dequeue to remove an element. Please have a look here.
Enqueue means to add an element, dequeue to remove an element.
var stackInput= []; // First stack
var stackOutput= []; // Second stack
// For enqueue, just push the item into the first stack
function enqueue(stackInput, item) {
return stackInput.push(item);
}
function dequeue(stackInput, stackOutput) {
// Reverse the stack such that the first element of the output stack is the
// last element of the input stack. After that, pop the top of the output to
// get the first element that was ever pushed into the input stack
if (stackOutput.length <= 0) {
while(stackInput.length > 0) {
var elementToOutput = stackInput.pop();
stackOutput.push(elementToOutput);
}
}
return stackOutput.pop();
}
In my opinion one of the worst chosen word's to describe the process, as it is not related to anything in real-life or similar. In general the word "queue" is very bad as if pronounced, it sounds like the English character "q". See the inefficiency here?
enqueue: to place something into a queue; to add an element to the tail of a queue;
dequeue to take something out of a queue; to remove the first available element from the head of a queue
source: https://www.thefreedictionary.com