Get count(null) as zero in Grafana - InfluxDB data source - grafana

Is it possible to set the value of count to zero when the result to which count is applied is null.
SELECT count(status) FROM ( SELECT last("P-status") AS "status" FROM "autogen"."Pl" WHERE ("Platform" = 'Database-plat' AND "P-status" = 'ERROR') AND time >= now() - 1m GROUP BY time(500ms), "Node" fill(0) )
In this case if the inner query returns null (for all the Node), count doesnt give any value , since fill will be ignored. I need the value to be zero, so that if i have to perform any other operation on the returned result, it can be done.
If possible, how can it be done?

I know this is an old question, but as it still is very relevant and I have been struggling with this problem for over a year now, I'd like to answer to this unanswered question with the current status of this issue according to my research:
The Github issues here and here imply that this problem is known since 2016 and not really understood by the contributers as a problem, as there are questionable rationales for the implementation (like "it's not a bug, it's a feature, because ambiguities of multiple series") that can easily be answered with special rules for unique series identification, but there has not been much activity any more despite heavy interest of the user community. Another point is that they have published version 2.x, which relies more on their new querying language (Flux), so it is very likely they have more or less practically abandoned the 1.x branch with InfluxQL (maybe except for QL backwards compatibility in 2.x and some minor updates, not sure).
Meanwhile I updated Grafana several times, but I had to stick with InfluxDB 1.x for a couple of reasons and the Flux support changed at some point (deprecated Flux plugin, but Flux included in standard InfluxDB plugin, but latter doesn't really work), so that Flux in Grafana is basically not working any more for a while now. I hoped for a better handling of the counting problem there, but now I'm out of luck regarding counting anything in InfluxDB reliably. I even tried some tricks with sum() function, fancy grouping and dummy values that I need to resubtract again and whatnot, but it always boiled down to the same conclusion: InfluxDB can do a lot, but counting just doesn't work.
It's very unsatisfying, but there doesn't seem to be a way to achieve the "eager" goal of counting data points without a system of bloated queries, excessive use of strange rules, dummy values and an insecurity that any query might break any time or break if you need to query only a specific time frame (where any dummy value workaround might not work). And regarding the priority given, this might not be fixed in the near future.

Related

Firebase analytics - Unity - time spent on a level

is there any possibility to get exact time spent on a certain level in a game via firebase analytics? Thank you so much 🙏
I tried to use logEvents.
The best way to do so would be measuring the time on the level within your codebase, then have a very dedicated event for level completion, in which you would pass the time spent on the level.
Let's get to details. I will use Kotlin as an example, but it should be obvious what I'm doing here and you can see more language examples here.
firebaseAnalytics.setUserProperty("user_id", userId)
firebaseAnalytics.logEvent("level_completed") {
param("name", levelName)
param("difficulty", difficulty)
param("subscription_status", subscriptionStatus)
param("minutes", minutesSpentOnLevel)
param("score", score)
}
Now see how I have a bunch of parameters with the event? These parameters are important since they will allow you to conduct a more thorough and robust analysis later on, answer more questions. Like, Hey, what is the most difficult level? Do people still have troubles on it when the game difficulty is lower? How many times has this level been rage-quit or lost (for that you'd likely need a level_started event). What about our paid players, are they having similar troubles on this level as well? How many people have ragequit the game on this level and never played again? That would likely be easier answer with sql at this point, taking the latest value of the level name for the level_started, grouped by the user_id. Or, you could also have levelName as a UserProperty as well as the EventProperty, then it would be somewhat trivial to answer in the default analytics interface.
Note that you're limited in the number of event parameters you can send per event. The total number of unique parameter names is limited too. As well as the number of unique event names you're allowed to have. In our case, the event name would be level_completed. See the limits here.
Because of those limitations, it's important to name your event properties in somewhat generic way so that you would be able to efficiently reuse them elsewhere. For this reason, I named minutes and not something like minutes_spent_on_the_level. You could then reuse this property to send the minutes the player spent actively playing, minutes the player spent idling, minutes the player spent on any info page, minutes they spent choosing their upgrades, etc. Same idea about having name property rather than level_name. Could as well be id.
You need to carefully and thoughtfully stuff your event with event properties. I normally have a wrapper around the firebase sdk, in which I would enrich events with dimensions that I always want to be there, like the user_id or subscription_status to not have to add them manually every time I send an event. I also usually have some more adequate logging there Firebase Analytics default logging is completely awful. I also have some sanitizing there, lowercasing all values unless I'm passing something case-sensitive like base64 values, making sure I don't have double spaces (so replacing \s+ with " " (space)), maybe also adding the user's local timestamp as another parameter. The latter is very helpful to indicate time-cheating users, especially if your game is an idler.
Good. We're halfway there :) Bear with me.
Now You need to go to firebase and register your eps (event parameters) into cds (custom dimensions and metrics). If you don't register your eps, they won't be counted towards the global cd limit count (it's about 50 custom dimensions and 50 custom metrics). You register the cds in the Custom Definitions section of FB.
Now you need to know whether this is a dimension or a metric, as well as the scope of your dimension. It's much easier than it sounds. The rule of thumb is: if you want to be able to run mathematical aggregation functions on your dimension, then it's a metric. Otherwise - it's a dimension. So:
firebaseAnalytics.setUserProperty("user_id", userId) <-- dimension
param("name", levelName) <-- dimension
param("difficulty", difficulty) <-- dimension (or can be a metric, depends)
param("subscription_status", subscriptionStatus) <-- dimension (can be a metric too, but even less likely)
param("minutes", minutesSpentOnLevel) <-- metric
param("score", score) <-- metric
Now another important thing to understand is the scope. Because Firebase and GA4 are still, essentially just in Beta being actively worked on, you only have user or hit scope for the dimensions and only hit for the metrics. The scope basically just indicates how the value persists. In my example, we only need the user_id as a user-scoped cd. Because user_id is the user-level dimension, it is set separately form the logEvent function. Although I suspect you can do it there too. Haven't tried tho.
Now, we're almost there.
Finally, you don't want to use Firebase to look at your data. It's horrible at data presentation. It's good at debugging though. Cuz that's what it was intended for initially. Because of how horrible it is, it's always advised to link it to GA4. Now GA4 will allow you to look at the Firebase values much more efficiently. Note that you will likely need to re-register your custom dimensions from Firebase in GA4. Because GA4 is capable of getting multiple data streams, of which firebase would be just one data source. But GA4's CDs limits are very close to Firebase's. Ok, let's be frank. GA4's data model is almost exactly copied from that of Firebase's. But GA4 has a much better analytics capabilities.
Good, you've moved to GA4. Now, GA4 is a very raw not-officially-beta product as well as Firebase Analytics. Because of that, it's advised to first change your data retention to 12 months and only use the explorer for analysis, pretty much ignoring the pre-generated reports. They are just not very reliable at this point.
Finally, you may find it easier to just use SQL to get your analysis done. For that, you can easily copy your data from GA4 to a sandbox instance of BQ. It's very easy to do.This is the best, most reliable known method of using GA4 at this moment. I mean, advanced analysts do the export into BQ, then ETL the data from BQ into a proper storage like Snowflake or even s3, or Aurora, or whatever you prefer and then on top of that, use a proper BI tool like Looker, PowerBI, Tableau, etc. A lot of people just stay in BQ though, it's fine. Lots of BI tools have BQ connectors, it's just BQ gets expensive quickly if you do a lot of analysis.
Whew, I hope you'll enjoy analyzing your game's data. Data-driven decisions rock in games. Well... They rock everywhere, to be honest.

OrientDB Lightweight Edges with Ridbags?

Recently I've come across a use case where it made a lot of sense to use lightweight edges. It made for much faster queries when checking whether two vertices are related as part of the select conditional.
That said, because I operate in a highly concurrent environment, I've run into some conflicts (OConcurrentModificationException). I got past this by setting the conflict strategy to auto-merge for that particular class.
In investigating further, I came across this article on concurrency when adding edges: http://orientdb.com/docs/2.1/Concurrency.html#concurrency-when-adding-edges
It recommends using RID Bags for situations where edges change very frequently, and has the neat advantage of not incrementing the version each time an edge is added/removed. Sounds great, but I can't get it to work.
I've tried adding the -DridBag.embeddedToSbtreeBonsaiThreshold=-1 to my client, with no effect. I then went into my code and added:
OGlobalConfiguration.RID_BAG_EMBEDDED_TO_SBTREEBONSAI_THRESHOLD.setValue(-1);
I finally also tried adding the -DridBag.embeddedToSbtreeBonsaiThreshold=-1 to my orientdb server (in server.sh).Still no effect. Each time the edges get updated, the version gets incremented (which is how I assume I can tell that its not working properly).
Does anybody have thoughts about how lightweight edges might work with ridbags (or not work for that matter)?
Thanks!

How to avoid hardcoding sliding window duration in Drools and pass time-window as argument.

I have started using drools a week ago.
I need to calculate average of a metric over a window-duration, say 4s. Below code-snippet of Drools will do this job.
... over window:time(4s) ...
However, I want to take this value as input to a rule with the value taken from control-panel UI where someone, say the customer, can specify the window duration.
I tried many options, including the one below, but that doesn't compile.
... over window:time($SlidingWindowDuration)
Googled for hours, but there is little documentation available on this subject.
Any clues in this regard would be of great help to me.
The length of a sliding window:time cannot be set dynamically. (I think this is so because dynamic lengths would make it impossible to infer an expiration offset for the automatic removal of obsolete events.)
Note that if this length can be set by the user before the engine is started and remains constant afterwards, you can insert the duration into the rule text, compile on the fly (only the rules that need last-minute editing) and execute.
To be absolutely dynamic, you'll have to implement the "window" mechanism explicitly. Make the timestamp an attribute of the event and set it explicitly: then you can base reasoning on timestamp differences, retract old events explicitly and compute the average over all that's left using a simple accumulate CE.

ExecuteGet do not return the updated value from couchbase server

We use couchbase server 2.2.0 in our production application.
In the last couple of month , when we ExecuteGet on a value, we get the previous value instead of the updated one.
Example scenario (all the actions are synchronized):
UserBalance_12345 is 500.
1. ExecuteSet("UserBalance_12345",1000)
2. ExecuteGet("UserBalance_12345"); -> The result is 500 instead of 1000.
This scenario occur very rarely but still happen and make damage to us because people say their balance is not updated immidietly.
How can such a scenario happen in couchbase?
Do you have a solution for this scenario?
A long time ago, I've also expirienced this issue. Sometimes it returned incorrect values. There are also some forum topics about this. But in my case that data wasn't so important and that issue raised very rarely, so I actually don't tried to figure out what causes it. And this issue is related .net sdk, because i.e. in nodejs sdk with js callbacks it never happend to me.
But I have sevral ideas that may help you.
1) Try to add to your ExecuteStore persist parameter:
PersistTo persistTo = PersistTo.One;
var result = _Client.ExecuteStore(StoreMode.Set, key, value, persistTo);
2) If you store user balance in separate key (just like in question) and it's type fits to ulong you can try to use Increment or ExecuteDecrement function instead of ExecuteStore. Increment/decrement functions are usally work better than store and they are atomic.

Find First and First Difference in Progress 4GL

I'm not clear about below queries and curious to know what is the different between them even though both retrieves same results. (Database used sports2000).
FOR EACH Customer WHERE State = "NH",
FIRST Order OF Customer:
DISPLAY Customer.Cust-Num NAME Order-Num Order-Date.
END.
FOR EACH Customer WHERE State = "NH":
FIND FIRST Order OF Customer NO-ERROR.
IF AVAILABLE Order THEN
DISPLAY Customer.Cust-Num NAME Order-Num Order-Date.
END.
Please explain me
Regards
Suga
As AquaAlex says your first snippet is a join (the "," part of the syntax makes it a join) and has all of the pros and cons he mentions. There is, however, a significant additional "con" -- the join is being made with FIRST and FOR ... FIRST should never be used.
FOR LAST - Query, giving wrong result
It will eventually bite you in the butt.
FIND FIRST is not much better.
The fundamental problem with both statements is that they imply that there is an order which your desired record is the FIRST instance of. But no part of the statement specifies that order. So in the event that there is more than one record that satisfies the query you have no idea which record you will actually get. That might be ok if the only reason that you are doing this is to probe to see if there is one or more records and you have no intention of actually using the record buffer. But if that is the case then CAN-FIND() would be a better statement to be using.
There is a myth that FIND FIRST is supposedly faster. If you believe this, or know someone who does, I urge you to test it. It is not true. It is true that in the case where FIND returns a large set of records adding FIRST is faster -- but that is not apples to apples. That is throwing away the bushel after randomly grabbing an apple. And if you code like that your apple now has magical properties which will lead to impossible to cure bugs.
OF is also problematic. OF implies a WHERE clause based on the compiler guessing that fields with the same name in both tables and which are part of a unique index can be used to join the tables. That may seem reasonable, and perhaps it is, but it obscures the code and makes the maintenance programmer's job much more difficult. It makes a good demo but should never be used in real life.
Your first statement is a join statement, which means less network traffic. And you will only receive records where both the customer and order record exist so do not need to do any further checks. (MORE EFFICIENT)
The second statement will retrieve each customer and then for each customer found it will do a find on order. Because there may not be an order you need to do an additional statement (If Available) as well. This is a less efficient way to retrieve the records and will result in much more unwanted network traffic and more statements being executed.