Maximum close_inactive time for filebeat harvesters - elastic-stack

I was wondering what was the longest time possible to set in that close_inactive field, because I want it to be the longest as possible (since we cannot set it as infinite by any means). Their documentation does not precise it, they just put examples like 2h and 5m.

There is no limit to the close_inactive time.
You can see it in FileBeat git
In addition, if you like,You can search the variable CloseInactive in this repository and see it for your self.

Related

Delete Time series in prometheus without API

I am monitoring a instance and changed its target IP. now when I graph it in grafana, there is 2 lines(with different color) showing with the tail of first line the head of the second line.
My goal is to remove the first line and just show the updated, second line.
My attempt is to adjust the time frame in grafana which works but it will affect all the instances that are not changed.
My second attempt is to remove the time-series in prometheus but the API was not enabled and restarting would cause a hiccup in the prometheus system (which is not good in monitoring).
It also said here that time-series can only be deleted via API but this is 2018. I was wondering if it is now possible to remove time-series without API.
No, the only way to remove time series is using the API
Yes, restarting would cause a hiccup, but let's be practical: the downtime is really very small.

Is there a way to force the output of a Stat panel in Grafana to only show values in days

I'm building a Grafana dashboard with some Stat panels that show average, minimum, and maximum time values (see below) for specific fields in my database. I'm storing the data in seconds and setting the value's units to seconds after which the panel displays the time in weeks, days, hours, etc. For the sake of consistency, I would like everything to be shown in days but I haven't been able to find a way to force units for the output value. If it's possible to do this, could someone please point me to some docs, or something that could show me which configurations to make in my panels?
So far, I've tried (without success):
Configure each panel to use units of days
The result of this was that everything showed up in years, etc.
Configure each panel to create a new field by performing a binary calculation where I converted from seconds to days and then I updated the units to be days
The result was that the values were not changed at all -> instead of showing X days, or whatever, it just showed the value in seconds without the units. I'm not sure what I messed up there, but it didn’t change anything.
I found this link that discusses setting a time range for queries
This didn’t end up being useful for what I was trying to do because it was actually geared towards changing the query to a specific date range rather than the output.
I looked through the transformations documentation, stat panel documentation, and a few other panel documentation pages in an effort to see if there was any information on how to do it but I was unable to find anything on forcing the output value to use a specific unit.
Edit:
So I kept messing around with the dashboard and got a solution that works - i.e. it's a "good-enough" solution (see below) - but, now, I'm curious if it's possible to show the units along with the value without it converting it to some other unit. Does anyone have any ideas about this?
One thing to note is that the data for this image is different than the data for the previous one so I'm expecting an inexact conversion to days.
You can use a custom unit. It is a bit tricky to enter the unit in the UI because of the automatic selection but if you enter i.e. "days" (without the quotes) in the Unit field ans instead of leving the field with tab or mouse click use the scrollbar of the combobox and select the last entry "Custom Unit: days".
Hope it helps. And for the record: I am using Garafana 7.1.4

How do we change the "precision:ms" setting in the Grafana Query Inspector?

I have an InfluxDB database with only x11 data points in it. These data are not displaying correctly (or at least as I would expect) in Grafana when the time between them is shorter than 1ms.
If I insert data points 1 ms apart, then everything works as expected and I see all x11 points at the correct times, as shown below.:
However, if I delete these points and upload new ones but this time one point per 100 μs, then although the data displays correctly in InfluxDB, in Grafana I see only two points in my graph:
It seems like the data is being rounded/binned to the nearest millisecond, an that this is related to the “precision=ms” setting in the query here:
but I cannot find any way to change this setting. What is the correct way to fix this?
You can't configure Grafana to support different time precision for the InfluxDB. It is hardcoded in the source code: https://github.com/grafana/grafana/blob/36fd746c5df1438f27aa33fc74b24be77debc7ff/public/app/plugins/datasource/influxdb/datasource.ts#L364 (It may need to be fixed in multiple places of the source, not only in this one.)
So the correct way to fix it is to code it, which is of course not in the scope of this question.

Can you calculate active users using time series

My atomist client exposes metrics on commands that are run. Each command is a metric with a username element as well a status element.
I've been scraping this data for months without resetting the counts.
My requirement is to show the number of active users over a time period. i.e 1h, 1d, 7d and 30d in Grafana.
The original query was:
count(count({Username=~".+"}) by (Username))
this is an issue because I dont clear the metrics so its always a count since inception.
I then tried this:
count(max_over_time(help_command{job=“Application
Name”,Username=~“.+“}[1w]) -
max_over_time(help_command{job=“Application name”,Username=~“.+“}[1w]
offset 1w) > 0)
which works but only for one command I have about 50 other commands that need to be added to that count.
I tried the:
"{__name__=~".+_command",job="app name"}[1w] offset 1w"
but this is obviously very expensive (timeout in browser) and has issues with integrating max_over_time which doesn't support it.
Any help, am I using the metric in the wrong way. Is there a better way to query... my only option at the moment is the count (format working above for each command)
Thanks in advance.
To start, I will point out a number of issues with your approach.
First, the Prometheus documentation recommends against using arbitrarily large sets of values for labels (as your usernames are). As you can see (based on your experience with the query timing out) they're not entirely wrong to advise against it.
Second, Prometheus may not be the right tool for analytics (such as active users). Partly due to the above, partly because it is inherently limited by the fact that it samples the metrics (which does not appear to be an issue in your case, but may turn out to be).
Third, you collect separate metrics per command (i.e. help_command, foo_command) instead of a single metric with the command name as label (i.e. command_usage{commmand="help"}, command_usage{commmand="foo"})
To get back to your question though, you don't need the max_over_time, you can simply write your query as:
count by(__name__)(
(
{__name__=~".+_command",job=“Application Name”}
-
{__name__=~".+_command",job=“Application name”} offset 1w
) > 0
)
This only works though because you say that whatever exports the counts never resets them. If this is simply because that exporter never restarted and when it will the counts will drop to zero, then you'd need to use increase instead of minus and you'd run into the exact same performance issues as with max_over_time.
count by(__name__)(
increase({__name__=~".+_command",job=“Application Name”}[1w]) > 0
)

How to decrease the number of issues?

When I mark as false positive or resolved some issues, the total number of issues is not decremented. Is that behavior normal? If so, is there a way to decrease this number according to the number of marked issues?
I'm using SonarQube 4.5.4 (LTS).
This behavior is normal.
When you resolve (mark False Positive, Won't Fix, or Fixed) an issue the numbers in your dashboard won't reflect that resolution until after the next analysis. That's because those dashboard numbers are Metrics, and metrics aren't updated on the fly - only during analysis.
However, you should see the counts on the Issues page update immediately. Those numbers are calculated on the fly based on the issue set that matches the current query.