I'm having trouble understanding how to create a dashboard time series plot to display a single key/value from a Loki log which is in JSON format.
eg:
here is my query in the Explorer:
{job="railsdevlogs"}|json
which returns log lines such as:
{"date":"2022-01-05T21:27:21.895Z","pool":{"Pool Size":50,"Current":5,"Active":1,"Idle":4,"Dead":0,"Timeout":"5 sec"},"puma":{"Started At":"2022-01-05T20:35:26Z","Max Threads":16,"Pool Capacity":16,"Running":1,"Backlog":0,"IO Handles":15,"File Handles":2,"Socket Handles":4,"Server Log Size":46750072},"process":[{"Name":"ruby.exe","Process ID":656,"Threads":11,"Working Set":150728704,"Virtual Size":288079872},{"Name":"mysqld.exe","Process ID":4836,"Threads":3,"Working Set":360448,"Virtual Size":4445065216},{"Name":"mysqld.exe","Process ID":5808,"Threads":49,"Working Set":69906432,"Virtual Size":4924059648},{"Name":"aaaaa.exe","Process ID":14460,"Threads":18,"Working Set":49565696,"Virtual Size":5478469632},{"Name":"bbbbb.exe","Process ID":9584,"Threads":14,"Working Set":35012608,"Virtual Size":4496551936},{"Name":"ccccc.exe","Process ID":11944,"Threads":14,"Working Set":29609984,"Virtual Size":4481880064}],"gc":{"count":242,"heap_allocated_pages":1277,"heap_sorted_length":1279,"heap_allocatable_pages":9,"heap_available_slots":869213,"heap_live_slots":464541,"heap_free_slots":404672,"heap_final_slots":0,"heap_marked_slots":411311,"heap_swept_slots":457903,"heap_eden_pages":1268,"heap_tomb_pages":9,"total_allocated_pages":1278,"total_freed_pages":1,"total_allocated_objects":74364715,"total_freed_objects":73900174,"malloc_increase_bytes":640096,"malloc_increase_bytes_limit":16777216,"minor_gc_count":131,"major_gc_count":111,"remembered_wb_unprotected_objects":57031,"remembered_wb_unprotected_objects_limit":114062,"old_objects":349257,"old_objects_limit":698512,"oldmalloc_increase_bytes":640288,"oldmalloc_increase_bytes_limit":16777216},"os":{"System Name":"xxxxx","Description":"","Organization":"","Operating System":"Microsoft Windows 10 Enterprise LTSC","OS Version":"10.0.17763","OS Serial Number":"xxxxx-xxxxx-xxxxx-xxxxx","System Time":"2022-01-05T16:27:22.000-05:00","System Time Zone":-300,"Last Boot Time":"2021-12-15T23:26:38.000-05:00","System Drive":"C:","Total Physical Memory":34204393472,"Free Physical Memory":20056260608,"Total Virtual Memory":39304667136,"Free Virtual Memory":13915041792,"Number of Processes":307,"Number of Users":2,"volumes":[{"Drive":"C:\\","Type":"NTFS","Total Space":1023563264000,"Free Space":681182343168,"Block Size":4096}]},"symbol":{"size":28106},"stats_collection_time":387}
using |json will automatically create dynamic labels for all the key/values in the json log line:
gc_count = 123
os_Free_Virtual_Memory = 456789
etc.
Now I would like to plot one of these values in a grafana time series plot, but I am struggling to understand how to isolate one dynamic label and plot it.
Perhaps I'm using |json incorrectly. The documentation and examples I have read so far shows how to filter the logs using the dynamic labels, but I dont need that since I want to plot every log line.
thanks
I think this should help https://grafana.com/go/observabilitycon/2020/keynote-what-is-observability/ if you go to minute 41.
There's an example which is very similar to what you're trying to achieve.
Your query should look something like:
quantile_over_time(0,99, {job="railsdevlogs"}
| json
| unwrap gc_count [1m]}
by (job)
I have a one hdf5 format file Data File containing the molecular dynamics simulation data. For quick inspection, the h5ls tool is handy. For example:
h5ls -d xaa.h5/particles/lipids/positions/time | less
now my question is based on the comment I received on the data format! What attributes are missing according the hdf5 specifications and metadata in group?
Are you trying to get the value of the Time attribute from a dataset? If so, you need to use h5dump, not h5ls. And, the attributes are attached to each dataset, so you have to include the dataset name on the path. Finally, attribute names are case sensitive; Time != time. Here is the required command for dataset_0000 (repeat for 0001 thru 0074):
h5dump -d /particles/lipids/positions/dataset_0000/Time xaa.h5
You can also get attributes with Python code. Simple example below:
import h5py
with h5py.File('xaa.h5','r') as h5f:
for ds, h5obj in h5f['/particles/lipids/positions'].items():
print(f'For dataset={ds}; Time={h5obj.attrs["Time"]}')
I have the handle of a StateFlow input from a previous find result. I'm trying to get the port number for that StateFlow input. I've been using the sf('get',[handle#],[field name]) command for other fields like 'data.name', 'data.scope', etc. For example, given the image below if I issue the command, sf('get',301,'data.name') the result 'ABS_Vehicle_Speed' is returned. However I can't seem to figure out the command to get the current port number, 4, in this example. Can anyone help me with this?
sname='Sheet1';
startingColumn='A';
nextRow='1';
filename = 'C:\Users\vvb\Documents\New\ltp.xlsx';
ltp_hist_uper=hist(ltp_uper,10);
ltp_hist_lower=hist(ltp_lower,10);
ltp_full=[ltp_hist_uper ltp_hist_lower]; % contatinate both uper and lower vectors
range=sprintf('%s%d',startingColumn,nextRow);
writematrix(ltp_full,filename,sname,range);
I have download this code for extracting features from and audio file and store those features in matrix in Excel but writematrix command is throwing an error i.e. Invalid parameter name: Sheet1.
The arguments for 'writematrix' function following the initial two are in name-value pair combination. In this case, you should be able to proceed with your requirement by using the following:
writematrix(ltp_full, filename,'Sheet', sname, 'Range',range);
I have written a macro for ImageJ/FIJI to deconvolve my confocal microscopy images and run the "3D Object Counter" plugin. The macro successfully runs all required commands and saves all required data in the specified places.
However, I have found that the 3D-OC autothreshold (as shown in the plugin dialog box) is to stringent resulting in objects being lost or divided.
To remedy this I would like to reduce the autothreshold by a predetermined function something similar to what was done here (from:How to get threshold value used by auto threshold Plugin) which resulted in this code:
setAutoThreshold();
getThreshold(lower,upper);
v=setThreshold(lower,upper*0.5);
run("3D Objects Counter", "threshold="v" slice=10 min.=400 max.=20971520 objects statistics summary");
The idea was to call the AutoThreshold values, modify them and set them to a variable. However when these lines are run the following error is returned:
Number or numeric function expected in line 3.
v=<setThreshold>(lower,upper*0.5);
And if the variable is inserted directly into the threshold key for run(3D-OC) the following msg is encountered:
Numeric value expected in run() function
Key:"threshold"
Value or variable name:"setThreshold(lower,upper*0.5"
Any suggestions or help on how to designate the 3D-OC threshold value as a variable as described would be greatly appreciated (as would any work arounds of course :) ).
Cheers
Edit: After testing Jan's response below (which works perfectly), it appears I need to call the threshold set by the 3D-OC plugin. Anyone know how to do this?
The getThreshold(lower, upper) function returns the lower and upper threshold levels in the provided variables. There is no need to assign any value to a new variable, and as you observed, setThreshold does not have any return value.
Instead, you can use the value(s) returned from getThreshold and use them as parameters in the run method (in the correct way, by string concatenation, see here):
setAutoThreshold();
getThreshold(lower, v);
run("3D Objects Counter", "threshold=" + v + " slice=10 min.=400 max.=20971520 objects statistics summary");
Alternatively, you can use &v in the second parameter to avoid string concatenation in the last line (see the documentation for the run() macro function):
run("3D Objects Counter", "threshold=&v slice=10 min.=400 max.=20971520 objects statistics summary");
You might have to use the lower instead of the upper threshold value, depending on whether you count bright or dark objects.