What are static vs dynamic labels in Grafana Loki? - grafana

The Loki documentation states the following:
Try to keep values bounded to as small a set as possible. We don’t have perfect guidance as to what Loki can handle, but think single digits, or maybe 10’s of values for a dynamic label. This is less critical for static labels. For example, if you have 1,000 hosts in your environment it’s going to be just fine to have a host label with 1,000 values.
From my understanding, each combination of labels in Loki will create a new stream. So request=GET,host=my-host-1 will write to a different file than request=POST,host=my-host-1.
But what is a dynamic vs a static label? Searching "loki static vs dynamic labels" does not yield useful results.
My best guess is--if we describe labels as key, value pairs--then static labels are ones where the key is fixed but the value can be anything & dynamic labels are ones where both the key & value can be anything.

I think that the only fundamental difference between static and dynamic labels is that, it's common to have unbounded or infinite label values for the second one. This uncontrolled fact can lead to a big issue for Loki. If you have a limited and controlled number of values, dynamic labels are not worst than static labels. Anyway, always have in mind the following rule: "don’t add a label for something until you know you need it!".
You can find a lot of valuable info at:
How labels in Loki can make log queries faster and easier

Related

Is there a good way to set dynamic labels for k8s resources?

I'm attempting to add some recommended labels to several k8s resources, and I can't see a good way to add labels for things that would change frequently, in this case "app.kubernetes.io/instance" and "app.kubernetes.io/version". Instance seems like a label that should change every time a resource is deployed, and version seems like it should change when a new version is released, by git release or similar. I know that I could write a script to generate these values and interpolate them, but that's a lot of overhead for what seems like a common task. I'm stuck using Kustomize, so I can't just use Helm and have whatever variables I want. Is there a more straightforward way to apply labels like these?
Kustomize's commonLabels transformer is a common way to handle this, sometimes via a component. It really depends on your overall layout.

Set start-time for histogram sample

The usecase
We got multiple changelogs stored in the database, and want to create a histogram monitoring the duration between changes.
The problem
There doesn't seem to be a way to set the start time of a Historgram.Timer, e.g we want to set it to lastUpdated given the current changelog.
Avenues of approach
1 Subclassing Histogram
Should work. However the java-lib use protected/package-private extensively, thus making it hard without copying large portions of the library.
2 Using reflection
After a Histogram.Timer is created it should be possible to use reflection to set the start field. The field is marked as private final, and thus a SecurityManager could stop us in some environments.
Ideas?
Neither of the solutions seems like the correct way to go, and I suspect that I'm overlooking a simpler solution (but could find anything at SO or google). We're using grafana to visualize our metrics, if thats at all helpful in this scenario.
You don't need to subclass Histogram, as you don't need to use Histogram.Timer only because your histogram is measuring times.
Simply call myHistogram.observe(System.now() - lastUpdated) every time you record a new change in the database.

Make a Class Schedule Report

How can I make my Crystal Report look like the attached image? I have had no success creating it with a crosstab.
The short answer is that Crystal Reports isn't really equipped to handle the format you're dealing with. And here's why:
Let's assume for a moment you've already figured out how to interpret your query into something usable. Since we aren't using a Cross Table, the best you could hope for would be setting a Details section for each individual time slot and arranging a large number of formulas into a grid shape:
The problem is that every Formula would need to be unique; interpreting whether there is a Class at that Time and Date, and which Class it is. There would be up to 168 of those formulas and you'd have to manually go in and modify each one to check for their own unique combination of Date and Time. Which defeats the whole purpose of using a computer - to make repeated tasks easier.
Plus you'll have difficulty with the formatting: You'd need to program every "cell" to use a unique set of colors based on the displayed Class. That part is technically doable, but there's no way to "merge the cells" when classes last longer than a half hour. You'd end up with something like this:
So don't torture yourself trying to make this happen in Crystal. Even with all the time and effort it would take to formulate the grid, there's no good way to make it look like your screenshot.
That said, it looks as though you managed to put a schedule together in Excel. Is there any reason you can't use Excel instead? It's a much more powerful tool, and a cursory Google search suggests it can handle queries as well.

Do having multiple labels for a node in Neo4j make any sense?

Following this post from Neo4j's google group I have to say that I don't see any benefits when using this multiple-label-thing but rather, on the contrary, IMHO it just adds complexity for what a uniqueness constraint is. It could also tempt the user to introduce inheritance into the data model which would cause frustration since that's not possible at all...
Labels have not the notion of just representing a type, they are rather roles which are viable in different contexts.
So in one role, certain attributes and relationships of a node might matter and in another role (label) a different set (that might intersect with the first one).
We stayed away from inheritance as it opens a new can of worms, and we favor composition. So you'd rather compose a node whole as the sum of its parts. You can also mimic an inheritance by also attaching the "super"-types as labels to the child elements in your hierarchy.
Node labels can also be used to separate subgraphs in a larger graph, e.g. label the proteins that are active in human pathways and phylo pathways with those labels. So you can quickly select a part of the graph that you're interested in.
Those separate subgraphs can also come from different domains, like geo,social,catalogue,supplier that are combined in a single graph.
And multiple labels also make sense to separate "technical" namespaces of your graph that are used to represent "in-graph-indexes" from your "domain"-labels.
Regarding uniqueness - all uniqueness constraints for the existing labels and properties on your nodes are enforced at the same time. If they cannot be resolved on insert or update the operation will fail.

Complex URL handling conception

I'm currently struggling at a complex URL handling concept question. The application have a product property database table/collection with all the different product types (i.e. categories, colors, manufacturers, materials, etc.).
{_id:1,alias:"mercedes-benz",type:"brand"},
{_id:2,alias:"suv-cars",type:"category"},
{_id:3,alias:"cars",type:"category"},
{_‌​id:4,alias:"toyota",type:"manufacturer"},
{_id:5,alias:"red",type:"color"},
{_id:6,alias:"yellow",type:"color"},
{_id:7,alias:"bmw",type:"manufacturer"},
{_id:8,alias:"leather",type:"material"}
...
Now the mission is to handle URL requests in the style below in every(!) possible order to retrieve the included product properties. The only allowed character is the dash (settled SEO requirement, some properties also can include dashes by themselve - i think also an important point - i.e. the category "suv-cars" or the manufacturer "mercedes-benz"):
http:\\www.example.com\{category}-{color}-{manufacturer}-{material}
http:\\www.example.com\{color}-{manufacturer}
http:\\www.example.com\{color}-{category}-{material}-{manufacturer}
http:\\www.example.com\{category}-{color}-nonexistingproperty-{manufacturer}
http:\\www.example.com\{color}-{category}-{manufacturer}
http:\\www.example.com\{manufacturer}
http:\\www.example.com\{manufacturer}-{category}-{color}-{material}
http:\\www.example.com\{category}
http:\\www.example.com\{manufacturer}-nonexistingproperty-{category}-{color}-{material}
http:\\www.example.com\{color}-crap-{manufacturer}
...
...so: every order of the properties should be allowed! The result have to be the information about the used properties per URL-Request (BTW yes, the duplicate content will be fixed by redirects and a predefined schema). The "nonexistingproperties"/"crap" are possible and just should be ignored.
UPDATE:
Idea 1: One way i'm thinking about the question is to split the query string by dashes and analyze them value by value, the problem: At the two or three or more word combinations at some properties there are too many different combinations and variations so a loooot of queries which kills this idea i think..
Idea 2: The other way is to build a (in my opinion) too large Alias/URL-Table with all of the different combinations, but i think that's just an ugly workaround. There are about 15.000 of different properties so the count of the aliases in the different sort orders is killing this idea.
Idea 3: It's your turn! Thanks for your mind and your time.
While your question is a bit broad, below are some ideas. There isn't a single awesome answer unless you find a free or commercial engine for this that works exactly the way you want.
The way I thought about your problem was to consider the URL as a list of keywords.
use Lucene as a keyword/tag system. It's good at the types of searches you suggest you want, including phrases, stems, etc.
store and index the data in DB of choice, but pull the keywords into memory and build a bit index of all keywords vs items. Iterate through the keyword table producing weighted results. If order of keywords matters, you'll also need make a pass through the result set to weight based on word order. These types of searches always need to cap their result set quickly in order to return results quickly.
cache the results like crazy from working matches, and give precedence to results that users seem to click on the most for a given URL.
attack the database by using tag indexes in MongoDB. You'd still need to merge and weight results. Very intensive and not likely a good use of DB resources.
read some of the academic papers on keyword searches. It's a popular topic.
build a table of words that have dashes in them, and normalize/convert those before running your queries
always check for full exact matches first
The only way this may work, if you restrict all property values to be unique. So, you make a set of categories+colors+manufacturers, etc. All values have to be unique. This will allow you to find to what property the value belongs.
The data structure for this should be fairly simple:
{_id:ValueOfTheProperty, Property:TypeOfProperty}
Here are some possible samples:
{ _id: Red, Property: Color }
{ _id: Green, Property: Color }
{ _id: Boots, Property: Category }
{ _id: Shoes, Property: Category }
...
This way, the order does not matter, and you are able to convert them in a single pass to a map:
{ Color: Red, Category: Boots }
Though, I predict some problems with ambigous names here.