Find string in a large text with postgresql - postgresql

I have a column called data in a PostgreSQL table that contains the following:
-----BEGIN HEADER-----
TYPE = PKCS#10
SERIAL = xxxxx
NOTBEFORE = Thu Sep 9 12:37:43 2010 UTC
LOA = 10
ROLE = xxxxx
RA = xxxxx
REQUEST_AUTH_USERID = CN=xxxxxxx
SCEP_TID = xxxxx
ARCHIVED_AFTER = Mon Nov 30 17:41:40 2015
_AFTER = Tue Jan 26 09:26:14 2016
-----END HEADER-----
-----BEGIN CERTIFICATE REQUEST-----
MIICWTCCAUECAQAwFjEUMBIGA1UEAwwLQ09QUzAwMDEzMDAwggEiMA0GCSqGSIb3
.
.
.
.
bV0eG4rlMOgTPv6mqb9HHKFqi3dsDzZKfXyoAsLOyOkj+AWXmAfXG8enT4uqBJJf
AsrUuJyTwzvmfdcEgYxokI6FU/nAjgQpmLkuVrE=
-----END CERTIFICATE REQUEST-----
As we want to move to a different application, I only need the part beginning with -----BEGIN CERTIFICATE REQUEST-----.
I tried to extract this string with substring and position in a SQL query but even if I try to find the possition of this string with:
SELECT position('-----BEGIN CERTIFICATE REQUEST-----' in (SELECT data from openca2.request where req_key = 3874))
I get a NULL value. I only can find examples where a part of a single string is extracted.
I don't know if the spaces are a problem. The part within the beginning and end header are not always the same.
Can this be done with a SQL query?

SELECT substr(
data,
position('-----BEGIN CERTIFICATE REQUEST-----' IN data)
)
FROM openca2.request
WHERE req_key = 3874;

How about:
SELECT split_part(data, '-----BEGIN CERTIFICATE REQUEST-----', 2)
FROM openca2.request
WHERE req_key = 3874;
If you want to keep '-----BEGIN CERTIFICATE REQUEST-----' then use '-----END HEADER-----' instead.

Thanks for your response. Both solutions are working.
I just have notices when I tried your queries that I was searching for the wrong req_key ..... and where I have the right req_key my solution works also
SELECT substring(data, position('-----BEGIN CERTIFICATE REQUEST-----' in (SELECT data from openca2.request where req_key = 3873)) +0 ,50000) from openca2.request where req_key = 3873
But yours look better as it is shorter :)
So this question is answered. THX again to you both.

Related

windows command line csr file certreq.exe output bad format on Subject ,Issue to, and issue by

Windows command line CSR file
`certreq.exe -new C:\Certs\template.inf C:\Certs\mydomain.txt.csr`
returns:
bad format on Subject ,Issue to, and issue by this format
The template.inf has
Subject = "CN=mydomain.com, OU=IT, O=My Company, L=my city, S=my state, C=US
I get this on
Issue to: CN=mydomain.com, OU=IT, O=My Company, L=my city, S=my state, C=US in 1 line.
same for
issue by: CN=mydomain.com, OU=IT, O=My Company, L=my city, S=my state, C=US
also
Subject : CN=mydomain.com, OU=IT, O=My Company, L=my city, S=my state, C=US
With the IIS server certificates mmc. I get the below, the desired result
CN = mydomain.com
OU = IT
O = My Company
L = my city
S = my state
C = US
All the documentation says to put it all in 1 line in the template file.
Can some one please advice?
Take the following *.inf as source template to generate a web server certificate:
[Version]
Signature="$Windows NT$"
[Strings]
szOID_SAN = "2.5.29.17"
szOID_ENHANCED_KEY_USAGE = "2.5.29.37"
szOID_PKIX_KP_SERVER_AUTH = "1.3.6.1.5.5.7.3.1"
[NewRequest]
Exportable=False
SMIME=False
Hashalgorithm=SHA256
KeyLength=2048
KeySpec = AT_KEYEXCHANGE
KeyUsage = "CERT_DIGITAL_SIGNATURE_KEY_USAGE | CERT_KEY_ENCIPHERMENT_KEY_USAGE"
RequestType=PKCS10; PKCS10 or CMC
Subject="CN=mydomain.com, OU=IT, O=My Company, L=my city, S=my state, C=US"
ProviderName="Microsoft Software Key Storage Provider"
MachineKeySet=True
[Extensions]
%szOID_SAN%="{text}"
_continue_ = "DNS=mydomain.com&"
%szOID_ENHANCED_KEY_USAGE%="{text}"
_continue_ = "%szOID_PKIX_KP_SERVER_AUTH%,"
I used your subject and for better compatibility with browsers I also added the CN value as DNS Host Name to the subject alternative name.
As MachineKeySet is set to True the following command must be run with administrative privileges:
& certreq.exe -New ".\Template.inf" "CertificateSigningRequest.p10"

How to fetch doctype eg: address or tax rule

I want to fetch the doctype. How do I do this? I want to add a separate column which will give doctype such as sales order, purchase order etc. The first line gives me error what query should be fired. Please help I am new to ERP Next.
SELECT
AD.ref_doctype AS “Doctype:Link/User:120”,
AD.name AS “Doc#:Link/Doctype:120”,
AD.owner AS “Created By:Link/User:120”,
AD.modified AS “Modified On:Date:120”
FROM tabAddress AS AD
WHERE
DATEDIFF(now(),AD.modified) BETWEEN 1 AND 30
UNION ALL
SELECT
TR.name AS “Doc#:Link/Doctype:120”,
TR.owner AS “Created By:Link/User:120”,
TR.modified AS “Modified On:Date:120”
FROM tabTax Rule AS TR
WHERE
DATEDIFF(now(),TR.modified) BETWEEN 1 AND 30
UNION ALL
SELECT
IT.name AS “Doc#:Link/Doctype:120”,
IT.owner AS “Created By:Link/User:120”,
IT.modified AS “Modified On:Date:120”
FROM tabItem AS IT
WHERE
DATEDIFF(now(),IT.modified) BETWEEN 1 AND 30
It isn't completely clear to me what you mean by docType field.
Are you wanting a result like this?
Doctype:Link/User:120|Doc#:Link/Doctype:120|Created By:Link/User:120|Modified On:Date:120|
---------------------|---------------------|------------------------|--------------------|
Email Account |Jobs |Administrator | 2019-12-04 06:07:55|
Email Account |Notifications |Administrator | 2019-12-01 05:25:53|
Email Account |Replies |Administrator | 2019-12-01 05:25:53|
Email Account |Sales |Administrator | 2019-12-04 06:07:55|
Email Account |Support |Administrator | 2019-12-04 06:07:55|
Here's the select :
set #docType = "Email Account";
SELECT
#tabDocType AS `Doctype:Link/User:120`,
AD.name AS `Doc#:Link/Doctype:120`,
AD.owner AS `Created By:Link/User:120`,
AD.modified AS `Modified On:Date:120`
FROM `tabEmail Account` AS AD
Note the backticks on the field aliases! All these have different meanings in SQL:
"
'
`
The last one, backtick, is used to refer to database entities. You were trying to use “Doctype:Link/User:120” with double quotes, which declare plain text. Using backtick converts the alias into a db entity which can be referred to from elsewhere.
MariaDb doesn't allow the use of variables as table names directly, but you can do it using prepared statements, like this:
set #docType = "Email Account";
set #tabDocType = CONCAT('tab', #docType);
SET #sql_text = concat('
SELECT
"', #docType, '" AS `Doctype:Link/User:120`
, AD.name AS `Doc#:Link/Doctype:120`
, AD.owner AS `Created By:Link/User:120`
, AD.modified AS `Modified On:Date:120`
FROM `', #tabDocType, '` as AD;
');
PREPARE stmt FROM #sql_text;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
The table name is now also specified by a variable, created from concatenation of 'tab' with the docType declared before.
You get the same result as above but -- you avoid accidentally changing the table name in one place but not in the other when editing some time in the future.
to fetch doctype name you have to give the linked doctype name, For example,
select
IT.name as "IT No:Link/IT:120"

T-SQL replace function missing some occurrences with nvarchar(max)

Given the following T-SQL code:
declare #lf char(1) set #lf = char(10);
declare #cr char(1) set #cr = char(13);
select replace(isnull(note,''), #cr+#lf,#lf) from T
are there circumstances where not every occurrence of #cr+#lf in the note column would be replaced with #lf?
I am trying to troubleshoot a situation where that is indeed happening.
The note column is defined as nvarchar(max). The documentation says:
If string_expression is not of type varchar(max) or nvarchar(max), REPLACE truncates the return value at 8,000 bytes. To return values greater than 8,000 bytes, string_expression must be explicitly cast to a large-value data type.
If I understand that correctly, there's no need to cast because note is already of the proper datatype to allow for return values greater than 8000 bytes.
I thought maybe the isnull function was not returning nvarchar(max) but the documentation says it returns the type of the value being tested:
... Return Types
Returns the same type as check_expression.
And the returned value isn't being truncated; it's just that some crlf pairs are being overlooked.
I must be overlooking something.
declare #t table( notes nvarchar(max));
insert #t(notes)
values
(
'From: kkkkkkk#aaaaaaaaa.com <kkkkkkk#aaaaaaaaa.com>
Sent: Monday, May 00, 0008 00:55 PM
To: Jan Zzzz <sZzzz#dddd.dd.com>
Subject: RE: [Secure Message] aaaaaaaaa ABC ddddddddddddd--XXXXX-X
Hi Jan,
The ddddddddddddd is valid for one year. I have attached the Zzzzzzz Rrrrrrrr which you will find behind the blank cover page and ddddddddddddd form. Please let me know if this is what you need.
Best Regards,
Yyyyyy
Kkkkkkkk Kkkkkk, ABC, DEF
ABC Mmmmmmmm
P 800.007.0000 ext 000 | F 600.000.0070
Electronic mail is not considered to be a secure form of communication for Protected Health Information. If you are concerned about privacy, please call aaaaaaaaa directly at 0-800-007-0000. If you would like to review our privacy policy, it can be found on our website: www.ddddddddddddd.com.
This email, including any attached files, may contain confidential and privileged information for the sole use of the intended recipient(s). Any review, use, distribution or disclosure by others is strictly prohibited. If you are not the addressee indicated in this message (or authorized to receive information for the recipient), please contact the sender by reply e-mail and delete all copies of this message (including any attachments).
From: Jan Zzzz <sZzzz#dddd.dd.com>
Sent: Monday, May 00, 0008 8:56 AM
To: Kkkkkkkk Kkkkkk <kkkkkkk#aaaaaaaaa.com>; Jan Zzzz <sZzzz#dddd.dd.com>
Subject: Re: [Secure Message] aaaaaaaaa ABC ddddddddddddd--XXXXX-X
Hi, this expired, I need a copy of the aaaaaaa aaaa so I can submit my aaaaaaa aaa aaaaaaaa. Thank you. SZzzz
On 0/00/0008 8:00 AM, Jan Zzzz wrote:
Thank you for the dddddddd, I am mmmmmmm mmm today.
On 0/0/0008 6:05 PM, Kkkkkkkk Kkkkkk wrote:
[Secure Message] aaaaaaaaa ABC ddddddddddddd--XXXXX-X
Kkkkkkkk Kkkkkk has sent you a secure message.
Subject: aaaaaaaaa ABC ddddddddddddd--XXXXX-X
From: Kkkkkkkk Kkkkkk <kkkkkkk#aaaaaaaaa.com>
To: Jan Zzzz <sZzzz#dddd.dd.com>
Expires: May 00, 0008
View Secure Message
Note: If you have any concerns about the authenticity of this message please contact the original sender directly.'
)
select notes from #t;
select replace(notes, char(13),'') from #t;
You would be left with #cr+#lf in the SELECT if #cr+#cr+#lf occurs in the note, unless you need #cr when it occurs on its own you are probably better off doing:
declare #cr char(1) set #cr = char(13);
select replace(isnull(note,''), #cr,'') from T
if its just a single line, this should do it, remove them all and put it back
set #note = replace(replace(isnull(#note,''),#cr,''),#lf,'')+#lf . //or whatever line endings you want
if its multi line try something like this
declare #note as nvarchar(max)
declare #lf char(1) set #lf = char(10);
declare #cr char(1) set #cr = char(13);
set #note = 'A'+char(10)+char(13)+char(10)+char(13)+char(10)+char(13)+char(10)+char(13)+'A'+char(10)+char(13)
set #note = replace(isnull(#note,''),#cr,'')
--not sure if you want to keep all the user lf's but if you want only one try this?
if (patindex(isnull(#note,''),#lf+#lf) >= 0)
begin
set #note = replace(isnull(#note,''),#lf+#lf,#lf)
end
select #note
select cast(#note as VARBINARY(100))
select len(#note)
Every circumstance will be replaced, but you might be creating some CrLfs via your replace. Please see the example below and how to mitigate it.
DECLARE #Cr CHAR(1)=CHAR(13)
DECLARE #Lf CHAR(1)=CHAR(10)
DECLARE #CrLf CHAR(2)=CHAR(13)+CHAR(10)
DECLARE #NoteTbl TABLE(Note NVARCHAR(MAX))
INSERT INTO #NoteTbl (Note) SELECT #Cr + #CrLf
--example can result in CrLF being created
SELECT [NewNote],LEN([NewNote]) FROM (SELECT replace(isnull(note,''), #CrLf,#lf) AS [NewNote] FROM #NoteTbl) AS a
--Option 1: Replace all Cr with nothing; this is effectively the same as replacing CrLf with Lf
SELECT [NewNote],LEN([NewNote]) FROM (SELECT replace(isnull(note,''), #Cr,'') AS [NewNote] FROM #NoteTbl) AS a
--Option 2: insert the notes into a table and loop until CrLf is gone, this might be useful if you need to do multiple different data scrubs
DECLARE #NotesCleaned TABLE(Note NVARCHAR(MAX))
INSERT INTO #NotesCleaned (Note) SELECT Note FROM #NoteTbl
WHILE EXISTS(
SELECT * FROM #NotesCleaned WHERE Note Like '%' + #CrLf + '%'
)
BEGIN
UPDATE #NotesCleaned SET Note=replace(isnull(note,''), #CrLf,#lf)
END
SELECT Note,LEN(Note) FROM #NotesCleaned
I believe I may have found a partial answer. In SSMS:
Tools->Options->SQL Server->Results to Grid
[ x ] Retain CR/LF on copy or save
will actually restore the CR that your call to replace() has removed.

TSQL PIVOT Function: Invalid column name error and creating calculated column

I'm pretty new to the PIVOT function and I have been trying to figure this out for the past day and a half so I thought I would create an account after lurking for so long and just ask.
I have a table with the layout as follows:
AsOfDt AcctNum MntYr Dt Category Count
4/15/2015 12345 Jan-15 1/18/2015 Registered User 1
4/15/2015 12346 Feb-15 2/7/2015 New Registration User 1
4/15/2015 12347 Jan-15 1/27/2015 Unique Account 1
4/15/2015 12348 Jan-15 1/24/2015 Registered User 1
This is the end result I am trying to achieve
MntYr Account Population New Registration User Registered User Unique Account
Jan 2015 330984 12 26212 26311
Feb 2015 331897 2953 58702 58894
Mar 2015 343561 950 29498 29638
Apr 2015 343181 675 8845 8916
Grand Total 1349623 4590 123257 123759
Here is the Query that I currently have built:
WITH BaseQuery AS (
SELECT
MntYr
,Category
,[Count]
FROM [dbo].[rpt_gen_WebPortal_TestingData]
)
SELECT [MntYr]
,'Account Population'
,'Unique Account'
,'Registered User'
,'New Registration User'
FROM BaseQuery
pivot (sum([count]) for MntYr
in ("Jan 2015", "Feb 2015", "Mar 2015", "Apr 2015" )
) AS Pivoting
My first question:
I am getting an error for my MntYr column in the second SELECT statement, "Invalid column name 'MntYr'." I really don't understand why this is throwing an error. What am I doing wrong with trying to pull that column when I explicitly name it in my BaseQuery pull?
My second question:
I would also like to create a calculated field based upon the percentage of (Unique Account / Account Population), but I'm not quite sure how to go about calculated fields in a PIVOT function. Any ideas on how to get started with this one?
Any and all help would be much appreciated!
Thanks.
Your pivot clause was wrong. You also don't need a CTE. Try this:
SELECT
MntYr
,[Account Population]
,[Unique Account]
,[Registered User]
,[New Registration User]
,case
when isnull([Account Population],0) = 0 then 0
else 100 * [Unique Account] / [Account Population]
end Pct
FROM (
SELECT
MntYr
,Category
,[Count]
FROM [dbo].[rpt_gen_WebPortal_TestingData]
) BaseQuery
pivot (sum([Count]) for Category
in ([Account Population]
,[Unique Account]
,[Registered User]
,[New Registration User] )
) AS Pivoting

postgres search in column with escapes

Is there a way to do something like a grep for "site" from the following select (so that only "site=*" is returned from thedata)?
rr=# select thename,encode(thedata, 'escape') from
management_data.datas limit 2;
thename | thedata
-----------------------------------------------------------------------------------
Alexander | #
+
| #Fri Mar 15 14:58:18 PDT 2014
+
| BUs=ALL
+
| site=33$36$354$380$357$360$36$353$36$38$39$34$31$355
+
Anthony | #
+
| #Mon Jan 05 13:33:00 PST 2015
+
| mem=12000
+
| site=50$5$1$50
+
|
Given test data round-tripped successfully:
WITH somerow(name, blobofgarble) AS (
SELECT
TEXT 'Alexander',
BYTEA E'#\n#Fri Mar 15 14:58:18 PDT 2014\nBUs=ALL\nsite=33$36$354$380$357$360$36$353$36$38$39$34$31$355\n'
)
SELECT name, encode(blobofgarble, 'escape') FROM somerow;
Now, I can't possibly imagine why you'd store this information as bytea not a text field, but ... well, I guess there must be a reason. I'm going to rely on the simplifying assumption that the data, when escaped, can be treated as fairly sanely formed text, since otherwise the whole concept of "lines" is garbage and your question makes no sense.
With that assumption it's possible to use regexp_split_to_table to split on newlines, getting somewhat more sanely formed data:
WITH (...)
SELECT name, garblepart FROM somerow, regexp_split_to_table(encode(blobofgarble, 'escape'), E'\n') AS garblepart;
name | garblepart
-----------+------------------------------------------------------
Alexander | #
Alexander | #Fri Mar 15 14:58:18 PDT 2014
Alexander | BUs=ALL
Alexander | site=33$36$354$380$357$360$36$353$36$38$39$34$31$355
Alexander |
(5 rows)
(this is an implicit LATERAL query, so it'll only work in PostgreSQL 9.3 and above).
Now you can use pretty ordinary operations to find the row of interest and extract the desired part, in this case with some more pattern matching:
WITH (...)
SELECT
name, substring(garblepart from '=(.*$)')
FROM somerow,
regexp_split_to_table(encode(blobofgarble, 'escape'), E'\n') AS garblepart
WHERE garblepart LIKE 'site=%';
name | substring
-----------+-------------------------------------------------
Alexander | 33$36$354$380$357$360$36$353$36$38$39$34$31$355
(1 row)
Now go fix your schema so that you store your data sanely and don't have to do this.