I am new to linux environment, but due to my new project I am learning and I love it, (coming from vxwork era). Now my question is How can I filter I2c address from this command "i2cdetect" using bash script. I have heard that I can use either SED or AWKD to scan and search text.
so basically I want to get each i2c address, like these 0c, 5b, 5c, UU, 6e, 6f.
I thank you any clue or help I get.
root#plnx_aarch64:/# i2cdetect -y -r 3
0 1 2 3 4 5 6 7 8 9 a b c d e f
00: -- -- -- -- -- -- -- -- -- 0c -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- 5b 5c -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- 6e 6f
70: UU -- -- -- -- -- -- --
I wrote a function to do this. I'm not sure if there is a better way, but this does seem to work.
import re
def get_addresses(i2cdetect_output):
''' Takes output from i2cdetect and extracts the addresses
for the entries.
'''
# Get the rows, minus the first one
i2cdetect_rows = i2cdetect_output.split('\r\n')[1:]
i2cdetect_matrix = []
# Add the rows to the matrix without the numbers and colon at the beginning
for row in i2cdetect_rows:
i2cdetect_matrix.append(filter(None, re.split(' +', row))[1:])
# Add spaces to the first and last rows to make regularly shaped matrix
i2cdetect_matrix[0] = [' ', ' ', ' '] + i2cdetect_matrix[0]
i2cdetect_matrix[7] = i2cdetect_matrix[7][:-1] + [' ', ' ', ' ', ' ', ' ', ' ', ' ', ' ']
# Make a list of the addresses present
address_list = []
for i in range(len(i2cdetect_matrix)):
for j in range(len(i2cdetect_matrix[i])):
if i2cdetect_matrix[i][j] not in (' ', '--', 'UU'):
address_list.append(str(i) + str(format(j, 'x')))
if i2cdetect_matrix[i][j] == 'UU':
address_list.append('UU')
return address_list
Related
When I run the following code examples: I get different results (Shown below). Please explain why I get a match in the second example? I must be missing some understanding.
print ##version
-- First example behaves as expected
declare #strSearch nvarchar(10) = 'x (20) ';
declare #definition nvarchar(100) = 'x (200000000) ';
print charindex(#strSearch, #definition);
go
-- Second example does not behave as expected
declare #strSearch nvarchar(10) = 'xrchar (20) ';
declare #definition nvarchar(100) = 'xrchar (200000000) ';
print charindex(#strSearch, #definition);
Results below: (0 'not found' and 1 'found'):
Microsoft SQL Server 2008 (SP3) - 10.0.5538.0 (X64) Apr 3 2015
14:50:02 Copyright (c) 1988-2008 Microsoft Corporation Enterprise
Edition (64-bit) on Windows NT 6.1 (Build 7601: Service Pack 1)
(VM)
0 1
The reason you get 1 in the second example is that you are setting a value to your #strSearch that is too long for the variable definition, so sql server trims it to fit the length of the variable (instead of raising a "string or binary data would be truncated" error).
You can see it if you alter the script a little bit:
First example:
declare #strSearch nvarchar(10) = 'x (20) ';
declare #definition nvarchar(100) = 'x (200000000) ';
SELECT #strSearch as '#strSearch',
#definition as '#definition',
charindex(#strSearch, #definition) as 'charindex';
results:
#strSearch #definition charindex
x (20) x (200000000) 0
Second example:
declare #strSearch nvarchar(10) = 'xrchar (20) ';
declare #definition nvarchar(100) = 'xrchar (200000000) ';
SELECT #strSearch as '#strSearch',
#definition as '#definition',
charindex(#strSearch, #definition) as 'charindex';
results:
#strSearch #definition charindex
xrchar (20 xrchar (200000000) 1
You can see a live demo on rextester (Turns out that in 2014 that behavior still continues).
I know how to do a left, right and substring in T-SQL, but I'm having difficulty extracting just the name of the person below since the length of the name are not the same. Any ideas or syntax that I can use to extract just the name? Thanks
Data Value:
581;#Jackson, Daniel H; 501;#Sims, Katy L; 606;#Lawrence, Jennifer O
You can get the length of the name dynamically using PATINDEX, but it assumes that the Names are ALWAYS formatted the same way.
Here is an example of a TSQL Select that will give select the first names from the data you supplied:
CREATE TABLE #Temp (
ID INT NOT NULL,
FullName VARCHAR(100)
)
INSERT #Temp VALUES (581, 'Jackson, Daniel H')
INSERT #Temp VALUES (606, 'Lawrence, Jennifer O')
SELECT ID, FullName , SUBSTRING(FullName, PATINDEX('%, % _', FullName) + 2,
PATINDEX('% _', FullName) - PATINDEX('%, %', FullName) - 2) FirstName
FROM #Temp
DROP TABLE #Temp
I use a function to split up CSV-Strings:
create function Do_Split
(#InputString NVARCHAR(4000)
,#Delimiter NVARCHAR(50) = ';')
RETURNS #Items TABLE (Item NVARCHAR(4000)) AS
BEGIN --Function
IF (#Delimiter = ' ')
BEGIN
SET #Delimiter = ';'
SET #InputString = REPLACE(#InputString, ' ', #Delimiter)
END;
IF (#Delimiter IS NULL OR #Delimiter = '') SET #Delimiter = ';';
DECLARE #Item NVARCHAR(4000)
DECLARE #ItemList NVARCHAR(4000)
DECLARE #DelimIndex INT
SET #ItemList = #InputString;
SET #DelimIndex = CHARINDEX(#Delimiter, #ItemList, 0);
WHILE (#DelimIndex != 0)
BEGIN
SET #Item = SUBSTRING(#ItemList, 0, #DelimIndex);
INSERT INTO #Items VALUES (#Item);
-- Set #ItemList = #ItemList minus one less item
SET #ItemList = SUBSTRING(#ItemList, #DelimIndex+1, LEN(#ItemList)-#DelimIndex);
SET #DelimIndex = CHARINDEX(#Delimiter, #ItemList, 0);
END; -- End WHILE
IF #Item IS NOT NULL -- At least one delimiter was encountered in #InputString
BEGIN
SET #Item = #ItemList;
INSERT INTO #Items VALUES (#Item);
END;
ELSE -- No delimiters were encountered in #InputString, so just return #InputString
BEGIN
INSERT INTO #Items VALUES (#InputString);
END;
RETURN
END -- End Function
go
Usage:
SELECT * FROM Do_Split('581;#Jackson, Daniel H; 501;#Sims, Katy L; 606;#Lawrence, Jennifer O',';');
Result:
581
#Jackson, Daniel H
501
#Sims, Katy L
606
#Lawrence, Jennifer O
It isn't clear how your data looks like, because your example concatenates multiple values in one row.
Case 1
Assume that there're 2 columns id and full_name in your_table. The semicolon ; was added by you intentionally to distinguish columns. In this case, you can obtain the value of full name using the function RIGHT. The length would be the length of full_name minus 1 which excludes the #.
--
-- id full_name
-- -- ---------
-- 581 #Jackson, Daniel H
-- 501 #Sims, Katy L
-- 606 #Lawrence, Jennifer O
--
SELECT RIGHT(full_name, LEN(full_name) - 1) AS full_name
FROM your_table;
Case 2
If the above solution is not suitable, let's discuss another case. Assume that there's 1 column content in your_table and 3 rows. The semicolon ; was inside the value of content and need to be treated explicitly. In this case, you can obtain the value of full name using the function SUBSTRING. The full name will begin after just after the index of char # and this index can be obtained using CHARINDEX (Transact-SQL):
--
-- content
-- -------
-- 581;#Jackson, Daniel H
-- 501;#Sims, Katy L
-- 606;#Lawrence, Jennifer O
--
SELECT SUBSTRING(content, CHARINDEX('#', content) + 1, 1000) AS full_name
FROM your_table;
I call this procedure without any problem :
SELECT SP_getGlobalVariable('current_user_id')::INT ;
If I call it inside another stored procedure like this :
-- Get current User Id --
SELECT SP_getGlobalVariable('current_user_id')::INT INTO __currentUserId ;
I have this error :
ERROR : Syntax error near "current_user_id"
LINE 25: SELECT SP_getGlobalVariable('current_user_id')::INT INTO _...
It is probably a stupid syntax error... If someone can help a poor MySQL user starting using PG !
EDIT 1 :
SELECT version();
PostgreSQL 9.3.3 on i686-pc-linux-gnu, compiled by gcc-4.4.real (Debian 4.4.5-8) 4.4.5, 32-bit
This is the result of the version() call.
CREATE OR REPLACE FUNCTION SP_getGlobalVariable (__variableName VARCHAR(64))
RETURNS VARCHAR
AS
'
DECLARE
BEGIN
RETURN (SELECT value FROM tmp_variables
WHERE name = __variableName) ;
END ;
'
LANGUAGE 'plpgsql';
The procedure to get the global variable in the temporary table.*
EDIT 2 :
CREATE OR REPLACE FUNCTION SP_insertSite (__siteName VARCHAR(70), __siteDescription TEXT, __siteLatitude NUMERIC, __siteLongitude NUMERIC)
RETURNS INT
AS
'
DECLARE
-- Variables
__right_edge_father INT ;
__left_edge_father INT ;
__depth_father INT ;
__nbrSites INT ;
__insertId INT ;
__currentUserId INT ;
BEGIN
-- Check if tree is empty --
SELECT COUNT(*) INTO __nbrSites
FROM site ;
-- Get current User Id --
SELECT SP_getGlobalVariable('current_user_id')::INT INTO __currentUserId ;
IF __nbrSites = 0 THEN
INSERT INTO site (site_name, site_description, site_latitude, site_longitude, site_left_edge, site_right_edge, site_depth, site_create_dt, site_create_user_id)
VALUES(__siteName, __siteDescription, 0.0, 0.0, 1, 2, 0, LOCALTIMESTAMP, __currentUserId)
RETURNING site_id INTO __insertId;
ELSE
-- Get father edges --
SELECT site_left_edge, site_right_edge, site_depth
INTO __left_edge_father, __right_edge_father, __depth_father
FROM site
WHERE site_id = (SELECT site_id FROM site WHERE site_depth = 0 LIMIT 1) ;
-- Updates left edges --
UPDATE site
SET site_left_edge = site_left_edge + 2
WHERE site_left_edge >= __right_edge_father ;
-- Updates right edges --
UPDATE site
SET site_right_edge = site_right_edge + 2
WHERE site_right_edge >= __right_edge_father ;
-- Insert new node --
INSERT INTO site (site_name, site_description, site_latitude, site_longitude, site_left_edge, site_right_edge, site_depth, site_create_dt, site_create_user_id)
VALUES(__siteName, __siteDescription, __siteLatitude, __siteLongitude, __right_edge_father, __right_edge_father+1, __depth_father+1, LOCALTIMESTAMP, __currentUserId)
RETURNING site_id INTO __insertId;
END IF ;
RETURN __insertId ;
END ;
'
LANGUAGE plpgsql;
The problem is that you are using single quotes to encapsulate the function body, and so, when you try to add single quotes inside the body, you need to escape it, for instance:
CREATE OR REPLACE FUNCTION SP_insertSite (...)
RETURNS INT
AS
'
DECLARE
...
-- Get current User Id --
SELECT SP_getGlobalVariable(''current_user_id'')::INT INTO __currentUserId ;
...
RETURN __insertId ;
END ;
'
LANGUAGE plpgsql;
That would work nice. But I recommend you to use dollar-quoting and so you don't have to worry with single-quote escaping inside the function body, as the following:
CREATE OR REPLACE FUNCTION SP_insertSite (...)
RETURNS INT
AS
$$
DECLARE
...
-- Get current User Id --
SELECT SP_getGlobalVariable('current_user_id')::INT INTO __currentUserId ;
...
RETURN __insertId ;
END ;
$$
LANGUAGE plpgsql;
I'd recommend you to read PostgreSQL docs about the usage of dollar-quoting in PL/pgSQL (the entire document, not just this sub-section is also interesting to read).
We are evaluating using PostgreSQL to implement a multitenant database,
Currently we are running some tests on single-database-multiple-schema model
(basically, all tenants have the same set of database objects under then own schema within the same database).
The application will maintain a connection pool that will be shared among all tenants/schemas.
e.g. If the database has 500 tenants/schemas and each tenants has 200 tables/views,
the total number of tables/views will be 500 * 200 = 100,000.
Since the connection pool will be used by all tenants, eventually each connection will hit all the tables/views.
In our tests, when the connection hits more views, we found the memory usage of the backend process increases quite fast and most of them are private memory.
Those memory will be hold until the connection is closed.
We have a test case that one backend process uses more the 30GB memory and eventually get an out of memory error.
To help understand the issue, I wrote code to create a simplified test cases
- MTDB_destroy: used to clear tenant schemas
- MTDB_Initialize: used to create a multitenant DB
- MTDB_RunTests: simplified test case, basically select from all tenant views one by one.
The tests I've done was on PostgreSQL 9.0.3 on CentOS 5.4
To make sure I have a clean environment, I re-created database cluster and leave majority configurations as default,
(the only thing I HAVE to change is to increase "max_locks_per_transaction" since MTDB_destroy needs to drop many objects.)
This is what I do to reproduce the issue:
create a new database
create the three functions using the code attached
connect to the new created db and run the initialize scripts
-- Initialize
select MTDB_Initialize('tenant', 100, 100, true);
-- not sure if vacuum analyze is useful here, I just run it
vacuum analyze;
-- check the tables/views created
select table_schema, table_type, count(*) from information_schema.tables where table_schema like 'tenant%' group by table_schema, table_type order by table_schema, table_type;
open another connection to the new created db and run the test scripts
-- get backend process id for current connection
SELECT pg_backend_pid();
-- open a linux console and run ps -p and watch VIRT, RES and SHR
-- run tests
select MTDB_RunTests('tenant', 1);
Observations:
when the connection for running tests was first created,
VIRT = 182MB, RES = 6240K, SHR=4648K
after run the tests once, (took 175 seconds)
VIRT = 1661MB RES = 1.5GB SHR = 55MB
re-run the test again (took 167 seconds)
VIRT = 1661MB RES = 1.5GB SHR = 55MB
re-run the test again (took 165 seconds)
VIRT = 1661MB RES = 1.5GB SHR = 55MB
as we scale up the number of tables, the memory usages go up in the tests too.
Can anyone help explain what's happening here?
Is there a way we can control memory usage of PostgreSQL backend process?
Thanks.
Samuel
-- MTDB_destroy
create or replace function MTDB_destroy (schemaNamePrefix varchar(100))
returns int as $$
declare
curs1 cursor(prefix varchar) is select schema_name from information_schema.schemata where schema_name like prefix || '%';
schemaName varchar(100);
count integer;
begin
count := 0;
open curs1(schemaNamePrefix);
loop
fetch curs1 into schemaName;
if not found then exit; end if;
count := count + 1;
execute 'drop schema ' || schemaName || ' cascade;';
end loop;
close curs1;
return count;
end $$ language plpgsql;
-- MTDB_Initialize
create or replace function MTDB_Initialize (schemaNamePrefix varchar(100), numberOfSchemas integer, numberOfTablesPerSchema integer, createViewForEachTable boolean)
returns integer as $$
declare
currentSchemaId integer;
currentTableId integer;
currentSchemaName varchar(100);
currentTableName varchar(100);
currentViewName varchar(100);
count integer;
begin
-- clear
perform MTDB_Destroy(schemaNamePrefix);
count := 0;
currentSchemaId := 1;
loop
currentSchemaName := schemaNamePrefix || ltrim(currentSchemaId::varchar(10));
execute 'create schema ' || currentSchemaName;
currentTableId := 1;
loop
currentTableName := currentSchemaName || '.' || 'table' || ltrim(currentTableId::varchar(10));
execute 'create table ' || currentTableName || ' (f1 integer, f2 integer, f3 varchar(100), f4 varchar(100), f5 varchar(100), f6 varchar(100), f7 boolean, f8 boolean, f9 integer, f10 integer)';
if (createViewForEachTable = true) then
currentViewName := currentSchemaName || '.' || 'view' || ltrim(currentTableId::varchar(10));
execute 'create view ' || currentViewName || ' as ' ||
'select t1.* from ' || currentTableName || ' t1 ' ||
' inner join ' || currentTableName || ' t2 on (t1.f1 = t2.f1) ' ||
' inner join ' || currentTableName || ' t3 on (t2.f2 = t3.f2) ' ||
' inner join ' || currentTableName || ' t4 on (t3.f3 = t4.f3) ' ||
' inner join ' || currentTableName || ' t5 on (t4.f4 = t5.f4) ' ||
' inner join ' || currentTableName || ' t6 on (t5.f5 = t6.f5) ' ||
' inner join ' || currentTableName || ' t7 on (t6.f6 = t7.f6) ' ||
' inner join ' || currentTableName || ' t8 on (t7.f7 = t8.f7) ' ||
' inner join ' || currentTableName || ' t9 on (t8.f8 = t9.f8) ' ||
' inner join ' || currentTableName || ' t10 on (t9.f9 = t10.f9) ';
end if;
currentTableId := currentTableId + 1;
count := count + 1;
if (currentTableId > numberOfTablesPerSchema) then exit; end if;
end loop;
currentSchemaId := currentSchemaId + 1;
if (currentSchemaId > numberOfSchemas) then exit; end if;
end loop;
return count;
END $$ language plpgsql;
-- MTDB_RunTests
create or replace function MTDB_RunTests(schemaNamePrefix varchar(100), rounds integer)
returns integer as $$
declare
curs1 cursor(prefix varchar) is select table_schema || '.' || table_name from information_schema.tables where table_schema like prefix || '%' and table_type = 'VIEW';
currentViewName varchar(100);
count integer;
begin
count := 0;
loop
rounds := rounds - 1;
if (rounds < 0) then exit; end if;
open curs1(schemaNamePrefix);
loop
fetch curs1 into currentViewName;
if not found then exit; end if;
execute 'select * from ' || currentViewName;
count := count + 1;
end loop;
close curs1;
end loop;
return count;
end $$ language plpgsql;
Are these connections idle in transaction or just idle? Sounds like unfinished transactions are holding onto memory, or maybe you've got a memory leak or something.
For people who see this thread when searching around (as i did), I found what appeared to be the same problem in a different context. Idle processes slowly consuming more and more memory until the OOM killer takes them out (causing periodic DB crashes).
We traced the problem back to really long running PHP scripts which kept one connection open for a long time. We were able to get the memory under control by periodically closing the connection and re-connecting.
From what i've read postgres does a lot of caching so if you have one session hitting a lot of different tables/queries this cache data can continue to grow and grow.
-Ken
Part 1: In his article, "Dynamic Search Conditions in T-SQL...for SQL 2005 and Earlier", Erland Sommarskog gives an example of how to use dynamic sql with sp_executesql.
http://www.sommarskog.se/dyn-search-2005.html#sp_executesql
SELECT #sql = -- 19
'SELECT o.OrderID, o.OrderDate, od.UnitPrice, od.Quantity, -- 20
c.CustomerID, c.CompanyName, c.Address, c.City, -- 21
c.Region, c.PostalCode, c.Country, c.Phone, -- 22
p.ProductID, p.ProductName, p.UnitsInStock, -- 23
p.UnitsOnOrder -- 24
FROM dbo.Orders o -- 25
JOIN dbo.[Order Details] od ON o.OrderID = od.OrderID -- 26
JOIN dbo.Customers c ON o.CustomerID = c.CustomerID -- 27
JOIN dbo.Products p ON p.ProductID = od.ProductID -- 28
WHERE 1 = 1' -- 29
-- 30
IF #orderid IS NOT NULL -- 31
SELECT #sql = #sql + ' AND o.OrderID = #xorderid' + -- 32
' AND od.OrderID = #xorderid' -- 33
-- 34
IF #fromdate IS NOT NULL -- 35
SELECT #sql = #sql + ' AND o.OrderDate >= #xfromdate' -- 36
etc...
So, as you build your dynamic sql statement, it makes sense if you have to run just one sp_executesql for your #sql variable.
However, let's suppose you've built your #sql, and returned the filtered records you want returned, but you also want a COUNT of the records returned.
What would be the best way to go about doing this?
Would you have to declare another variable, #sql_2, whose build would be nearly identical to #sql, except the SELECT statement in #sql_2 would do a SELECT COUNT(*)... instead of a SELECT col1, col2, col3?
Or is there a better approach to take?
String the SQL statements together separated by semicolons. Here is a working example that returns the tables in your database that start with the letter "A" and the count.
First the simple version. This returns 2 result sets, the second one being the count.
declare #findTables nvarchar(256)
set #findTables = N'A%'
declare #sql nvarchar(max)
set #sql = N'set nocount on; '+
'select * from sys.tables where name like '''+#findTables+''';'+
'select ##RowCount as [RowCount];';
execute sp_executesql #sql
Now a version where a variable gets valued with the count for when you need to use it later in the stored proc.
declare #findTables nvarchar(256)
set #findTables = N'A%'
declare #sql nvarchar(max)
declare #ParmDefinition nvarchar(500);
declare #rowCount int
set #sql = N'set nocount on;
select * from sys.tables where name like #findTablesParm;
select #rowCountParm = ##rowcount;
select #rowCountParm as [RowCount];';
SET #ParmDefinition = N'#findTablesParm nvarchar(256),
#rowCountParm int OUTPUT';
execute sp_executesql #sql,
#ParmDefinition,
#findTablesParm=#findTables,
#rowCountParm=#rowCount OUTPUT
After this runs you should see 2 result sets, the second one will contain the RowCount and the variable #rowCount will also contain the row count.