I am using collectd and influx for monitoring a cluster of 200 cores.
I would like to create a gauge in grafana which is adding all load_shortterm value of all hosts to see the total usage of the cluster.
My structure looks like this :
name: load_shortterm
time host metric type value
---- ---- ------ ---- -----
1601891780201909599 cpu007.cluster load 0
1601891790145618383 cpu001.cluster load 2
1601891790163106767 cpu002.cluster load 0.03
1601891790167701326 cpu009.cluster load 0
So I want to have a request which will answer 2.03 in this case.
I don't understand how to get last values for each host and sum it. I tried this :
select sum(*) from load_shortterm where "host" =~ /^*.cluster/
But it returns a sum of all values.
Can you please help me ?
Thanks,
RB
SUM all LAST values with subquery, e.g.:
SELECT SUM(last)
FROM (
SELECT LAST(value)
FROM load_shortterm
WHERE "host" =~ /^*.cluster/
)
Thanks for the reply, the subquery returns the last value and not last values for each host :
> select last(value) from load_shortterm where "host" =~ /^*.cluster/
name: load_shortterm
time last
---- ----
1605863555866457205 0
With the full query it is the same result, it seems it does not take all last values but only the last one :
> select sum(last) from (select last(value) from load_shortterm where "host" =~ /^*.cluster/)
name: load_shortterm
time last
---- ----
1605863555866457205 0
Related
Related to PowerShell 5.1
I was playing around with XML to show how to handle conflicting namespaces. Here's the example I created:
<Employees>
<ms:Employee id='1' xmlns:ms="MicrosoftEmployees">
<FirstName>Bill</FirstName>
<LastName>Gates</LastName>
</ms:Employee>
<ms:Employee id='2' xmlns:ms="MicrosoftEmployees">
<FirstName>Paul</FirstName>
<LastName>Allen</LastName>
</ms:Employee>
<ap:Employee id='1' xmlns:ap="AppleEmployees">
<Name>Steve Jobs</Name>
</ap:Employee>
<ap:Employee id='2' xmlns:ap="AppleEmployees">
<Name>Steve Wozniak </Name>
</ap:Employee>
</Employees>
The scenario might be combining data from two different companies.
PowerShell demonstration program:
cls
$filename = "c:\XMLClass\IntroSamples\Sample05_Simpler_Namespace.xml"
[xml]$xmlDoc = Get-Content $filename
$xmlDoc.Employees.Employee[0]
$xmlDoc.Employees.Employee[1]
$xmlDoc.Employees.Employee[2]
$xmlDoc.Employees.Employee[3]
Output:
id ms FirstName LastName
-- -- --------- --------
1 MicrosoftEmployees Bill Gates
2 MicrosoftEmployees Paul Allen
1
2
Is there anyway to get a more logical output?
It seems like PowerShell locks into the first schema it sees for the Employee element, then cannot show the Name element of the Apple employees. This actually makes sense, but I was just checking to see if there is something fancier to handle this that I might be missing.
I know I could use SelectSingleNodes and XPath, but was just trying to see if and how PowerShell could handle this "out of the box".
If I reverse the code:
$xmlDoc.Employees.Employee[2]
$xmlDoc.Employees.Employee[3]
$xmlDoc.Employees.Employee[1]
$xmlDoc.Employees.Employee[0]
Then the output is:
id ap Name
-- -- ----
1 AppleEmployees Steve Jobs
2 AppleEmployees Steve Wozniak
1 ms:Employee
2 ms:Employee
Use format list to see all the properties. Format-table doesn't handle different sets of properties well.
$xmldoc.employees.employee | format-list
id : 1
ms : MicrosoftEmployees
FirstName : Bill
LastName : Gates
id : 2
ms : MicrosoftEmployees
FirstName : Paul
LastName : Allen
id : 1
ap : AppleEmployees
Name : Steve Jobs
id : 2
ap : AppleEmployees
Name : Steve Wozniak
I am using Postgres FTS to search a field in a table. The only issue is for some reason the below issue is happening.
store=# select name from service where to_tsvector(name) ## to_tsquery('find:*') = true;
name
--------------
Finding Nora
(1 row)
store=# select name from service where to_tsvector(name) ## to_tsquery('findi:*') = true;
name
------
(0 rows)
store=# select name from service where to_tsvector(name) ## to_tsquery('findi:*') = true;
How come when searching using the query findi:*,the result doesnt show?
In my PG 12.2 with default text search configuration I have:
# select to_tsvector('Finding Nora');
to_tsvector
-------------------
'find':1 'nora':2
(1 row)
# select to_tsquery('findi:*');
to_tsquery
------------
'findi':*
(1 row)
I understand that because there is no lexeme findi in the default dictionary, the query does not find any match.
Loading flat file to postgres table.I need to do few transformations while reading the file and load it.
Like
-->Check for characters, if it is present, default some value. Reg_Exp can be used in oracle. How the functions can be called in below syntax?
-->TO_DATE function from text format
-->Check for Null and defaulting some value
-->Trim functions
-->Only few columns from source file should be loaded
-->Defaulting values, say for instance, source file has only 3 columns. But we need to load 4 columns. One column should be defaulted with some value
LOAD CSV
FROM 'filename'
INTO postgresql://role#host:port/database_name?tablename
TARGET COLUMNS
(
alphanm,alphnumnn,nmrc,dte
)
WITH truncate,
skip header = 0,
fields optionally enclosed by '"',
fields escaped by double-quote,
fields terminated by '|',
batch rows = 100,
batch size = 1MB,
batch concurrency = 64
SET work_mem to '32 MB', maintenance_work_mem to '64 MB';
Kindly help me, how this can be accomplished used pgloader?
Thanks
Here's a self-contained test case for pgloader that reproduces your use-case, as best as I could understand it:
/*
Sorry pgloader version "3.3.2" compiled with SBCL 1.2.8-1.el7 Doing kind
of POC, to implement in real time work. Sample data from file:
raj|B|0.5|20170101|ABCD Need to load only first,second,third and fourth
column; Table has three column, third column should be defaulted with some
value. Table structure: A B C-numeric D-date E-(Need to add default value)
*/
LOAD CSV
FROM inline
(
alphanm,
alphnumnn,
nmrc,
dte [date format 'YYYYMMDD'],
other
)
INTO postgresql:///pgloader?so.raja
(
alphanm,
alphnumnn,
nmrc,
dte,
col text using "constant value"
)
WITH truncate,
fields optionally enclosed by '"',
fields escaped by double-quote,
fields terminated by '|'
SET work_mem to '12MB',
standard_conforming_strings to 'on'
BEFORE LOAD DO
$$ drop table if exists so.raja; $$,
$$ create table so.raja (
alphanm text,
alphnumnn text,
nmrc numeric,
dte date,
col text
);
$$;
raj|B|0.5|20170101|ABCD
Now here's the extract from running the pgloader command:
$ pgloader 41287414.load
2017-08-15T12:35:10.258000+02:00 LOG Main logs in '/private/tmp/pgloader/pgloader.log'
2017-08-15T12:35:10.261000+02:00 LOG Data errors in '/private/tmp/pgloader/'
2017-08-15T12:35:10.261000+02:00 LOG Parsing commands from file #P"/Users/dim/dev/temp/pgloader-issues/stackoverflow/41287414.load"
2017-08-15T12:35:10.422000+02:00 LOG report summary reset
table name read imported errors total time
----------------------- --------- --------- --------- --------------
fetch 0 0 0 0.007s
before load 2 2 0 0.016s
----------------------- --------- --------- --------- --------------
so.raja 1 1 0 0.019s
----------------------- --------- --------- --------- --------------
Files Processed 1 1 0 0.021s
COPY Threads Completion 2 2 0 0.038s
----------------------- --------- --------- --------- --------------
Total import time 1 1 0 0.426s
And here's the content of the target table when the command is done:
$ psql -q -d pgloader -c 'table so.raja'
alphanm │ alphnumnn │ nmrc │ dte │ col
═════════╪═══════════╪══════╪════════════╪════════════════
raj │ B │ 0.5 │ 2017-01-01 │ constant value
(1 row)
I am trying to get a list of security groups. (Successful - Using Get-EC2SecurityGroup)
Get a list of the specific IPPermissions associated with each security group. ( Successful - Using (Get-EC2SecurityGroup).IpPermissions )
Only return results where the FromPort = "xxx" ( Unsuccessful - Not sure how to access the FromPort property that is returned in the result list )
Ultimately what I am trying to accomplish is:
Get a list of existing security groups, and loop through each group.
While looping through each group, call the IpPermissions, and look for the specific FromPort "xxx".
If the FromPort is a match, record the other properties: (FromPort, IpProtocol, IpRanges, ToPort, UserIdGroupPairs)
Problem I am having
I am not sure how to do a loop using the amazon objects
I cant seem to access the properties even though they appear to be named and have values.
I have tried using -Filter with many different iterations, with no success.
The documentation seems self-referencing, and the examples I have run across dont get down to this level of detail.
Results returned from (Get-EC2SecurityGroup).IpPermissions
FromPort : 123
IpProtocol : tcp
IpRanges : {0.0.0.0/0}
ToPort : 123
UserIdGroupPairs : {}
Here's an example that does as you've described:
Filters security group objects by FromPort
Of the matched security groups, output IpProtocol, IpRanges, ToPort, and UserIdGroupPairs.
Code:
# Example using port 22
PS C:\> $port = 22
PS C:\> Get-EC2SecurityGroup |
? { $_.IpPermissions.FromPort -eq $port } |
% { $_.IpPermissions } |
Select -property IpProtocol, IpRanges, ToPort, UserIdGroupPairs
Output:
IpProtocol IpRanges ToPort UserIdGroupPairs
---------- -------- ------ ----------------
tcp {0.0.0.0/0} 22 {}
... ... ... ...
Firstly, I have the data below:
*********************************************************************
TEST CASE
*********************************************************************
test results for the last 1 run(s)
TEST TITLE GROUP PRIO R-STAT R-TIME VERDICT VERDICT-TEXT
------------------------- ---------------------------------------- -------- ---- ------ ------------ -------- --------------------------------------------------
TESTCASE1 TC1 ABC 1 PASS 00:00:09.572 PASS nothing
TESTCASE2 TC2 DEF 2 PASS 00:00:01.650 PASS nothing
----------------------------------------------------------------------------------------------------
NUMBER OF : 2
NUMBER OF TC: 2
*********************************************************************
VERDICT: PASS
*********************************************************************
I would like to perform lines:
TESTCASE1 TC1 ABC 1 PASS 00:00:09.572 PASS nothing
TESTCASE2 TC2 DEF 2 PASS 00:00:01.650 PASS nothing
and split first lines and last lines.
How can I do that?
ADDITIONAL:
Sorry for too late reply, i want to split
*********************************************************************
TEST CASE
*********************************************************************
test results for the last 1 run(s)
TEST TITLE GROUP PRIO R-STAT R-TIME VERDICT VERDICT-TEXT
------------------------- ---------------------------------------- -------- ---- ------ ------------ -------- --------------------------------------------------
just handled two lines
TESTCASE1 TC1 ABC 1 PASS 00:00:09.572 PASS nothing
TESTCASE2 TC2 DEF 2 PASS 00:00:01.650 PASS nothing
i use while loop to check line by line, the problem is it will be looped all of lines whereas i want two lines only.
You can try this to get first and last line from file:
cat filename | grep "^TESTCASE" | sed -n '1p;$p'