I need to extract text between the last occurrence of a word called "-----BEGIN CERTIFICATE-----" and the last occurrence of a word called "-----END CERTIFICATE-----".
Input :
some other data above this line
-----BEGIN CERTIFICATE-----
a
b
c
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
d
e
f
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
g
h
i
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
j
k
l
-----END CERTIFICATE-----
some other data below this
Expected output :
-----BEGIN CERTIFICATE-----
j
k
l
-----END CERTIFICATE-----
Output should include the -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----
My Approach :
I have tried the below command but it is giving me the output without -----BEGIN CERTIFICATE----- and -----END CERTIFICATE-----
openssl s_client -showcerts -connect localhost:443 </dev/null | sed -n 'H; /^-----BEGIN CERTIFICATE-----/h; ${g;p;}' |sed -e '1d' -e '/-----END CERTIFICATE-----/q' |sed '$ d' > mycertfile.pem
output from above command :
j
k
l
When I hear "get the last XYZ" I think "reverse the file and take the first XYZ": reverse the input, and have sed quit when the first "BEGIN CERT" appears, then re-reverse the data:
openssl ... | tac | sed '/BEGIN CERT/q' | tac
Since there's stuff after the last ---END line, try this variation:
openssl ... | tac | sed -n '/---END/,/---BEGIN/ p; /---BEGIN/ q' | tac
That only prints the ranges of lines bounded by the cert markers, and quits after the first one.
Another approach: use perl to read in the whole input, use a regex to capture all the BEGIN/END blocks, and print the last one.
openssl ... | perl -0777 -nE '
#certs = m/^-----BEGIN .+?^-----END .+?$/gms;
say $certs[-1];
'
Related
The following command works for SHA1: csum -h SHA1 (FileName).txt > (FileName_chksum).txt. How to create a similar file using the SHA256 algorithm in AIX?
You can use the openssl command from the openssl.base package; it has a dgst sub-command that will generate a SHA256 hash of the file:
openssl dgst -sha256 filename.txt > filename_sha256.txt
By default, it will print in the following format:
SHA256(filename.txt)= hash-string-here
The csum command prints in a slightly different format:
hash-string-here filename.txt
... so you may want to rearrange the output of openssl based on your specific needs for the filename_sha256.txt file.
If you only want the hashed string itself in the new file, you could use awk:
openssl dgst -sha256 filename.txt | awk '{print $2}' > filename_sha256.txt
I have the command openssl dgst -sha256 -binary _your_file_path_ | openssl enc -base64 I use in terminal to get an output for a jar file that matches what AWS Lambda uses to hash.
I want to program that in Java, but I am having trouble understanding exactly what is going on in that line, so that I can go through each step in my code. Obviously, there is mode than just hashing in SHA256, because when I do that the output does not match.
Could someone help explain the steps that line is completing in a simple way for me?
You need to break the command down to understand what is going on.
The first part of the command:
openssl dgst -sha256 -binary <file> gives you a SHA256 binary checksum for the file.
The second part of the command:
openssl enc -base64 encodes the SHA256 binary checksum to Base64.
So to replicate in Java, you just need to carry out those same steps:
Calculate a SHA256 binary checksum.
Base64 encode the SHA256 binary checksum.
Without you posting the command you used to try and get a SHA256 checksum separately to the command you did post, I'm guessing the reason you were probably getting a different hash is because by default a checksum seems to output in hexadecimal.
See my example below and how the results are completely different.
# Hexadecimal
$ openssl dgst -sha256 data.csv
SHA256(data.csv)= 114811b0b8998cb9853a5379598021410feddf69bb2ee7b7145d052a7e9b5d45
# Binary (note the usage of the -binary flag)
$ openssl dgst -sha256 -binary data.csv
H:SyY!Ai.]*~]E
If you then Base64 encode the hexadecimal checksum above, and the binary one, you'll also get two completely different results, as you can see below.
# Hexadecimal
$ printf 114811b0b8998cb9853a5379598021410feddf69bb2ee7b7145d052a7e9b5d45 | openssl enc -base64
MTE0ODExYjBiODk5OGNiOTg1M2E1Mzc5NTk4MDIxNDEwZmVkZGY2OWJiMmVlN2I3
MTQ1ZDA1MmE3ZTliNWQ0NQ==
# Binary
$ printf 'H:SyY!Ai.]*~]E' | openssl enc -base64
SDpTeVkhQWkuXSp+XUU=
For those, who TLDR. To get the same result as in this cat FILENAME.js | openssl dgst -sha256 -binary | openssl base64 -A command you should do the following conversions:
1) your content -> sha256 (you'll get the hexadecimal number, not a text)
2) hexadecimal -> binary
3) binary -> base64
I have to convert RSA private key to one-line to store in the password manager (Passwordstate). I used tr -d '\n' < id_rsa to convert to single line and cat id_rsa.line | sed -e "s/-----BEGIN RSA PRIVATE KEY-----/&\n/" -e "s/\S\{64\}/&\n/g" to convert back to original multi-line.
Conversion back to multi-line worked on Ubuntu, but not on Mac. Why this doesn't work on Macbook
Please try:
LF=$'\\\x0A'
cat id_rsa.line | sed -e "s/-----BEGIN RSA PRIVATE KEY-----/&${LF}/" -e "s/-----END RSA PRIVATE KEY-----/${LF}&${LF}/" | sed -e "s/[^[:blank:]]\{64\}/&${LF}/g"
or
LF=$'\\\x0A'
cat id_rsa.line | sed -e "s/-----BEGIN RSA PRIVATE KEY-----/&${LF}/" -e "s/-----END RSA PRIVATE KEY-----/${LF}&${LF}/" | fold -w 64
Non-GNU sed does not interpret "\n" in the replacement as a newline.
As a workaround, you can assign a variable to a newline and embed it in the replacement.
Note that I've kept the UUC for readability :P.
Building on top of tshiono's answer. Wrote a shell script to achieve the same:
#!/bin/sh
# Convert single-line certificate to multi-line certificate
echo "$1" > secret
LF=$'\\\x0A'; cat secret | sed -e "s/-----BEGIN RSA PRIVATE KEY-----/&${LF}/" -e "s/-----END RSA PRIVATE KEY-----/${LF}&${LF}/" | sed -e "s/[^[:blank:]]\{64\}/&${LF}/g" > secret-temp
x=`echo "-----BEGIN RSA PRIVATE KEY-----"; cat secret-temp; echo "-----END RSA PRIVATE KEY-----"`
echo "$x" > secret.pem
echo "\n\n## Private Key ##\n\n"
cat secret.pem
When I try to SHA512 some file using openssl I got the output file contents starts with something like "SHA512(in.txt)= 090c..."
I tried the different options -r, -binary with the command
Here is the command I'm using
openssl dgst -sha512 -out out.txt in.txt
The question is: How can I got the file contains only the hash, without that starting note?
You can only print the second column using awk, if the file name doesn't contain spaces:
openssl dgst -sha512 -out in.txt | awk '{print $2}' > out.txt
Or (looks like not cross-platform) you can try either pipe or reading from stdin:
openssl dgst -sha512 -out out.txt < in.txt
cat in.txt | openssl dgst -sha512 -out out.txt
This works for me (Mac OS X).
The default delimiter of awk is a space character, and the accepted answer will not work if there are spaces in the filename. You can override the default delimiter with the -F flag (field separator) to = , but that would also not work if there happens to be an equal space in the filename. Printing the last column using the default delimiter should work for all of those edge cases. The $NF awk variable stores the number of fields and can be used directly to print the last column, which should always be the hash.
openssl dgst -sha512 -out in.txt | awk '{print $NF}' > out.txt
https://linux.die.net/man/1/awk
Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 8 years ago.
Improve this question
I am new to perl so this might be very basic, but i am finding no way around this problem. I am trying to sign my data which has to be send over https connection using my ssh private key(id_rsa). I am not able to do so using perl and i am at it for days. Please someone show me a possible way to do so. If any more info is needed please ask me. Thanks in advance
My code as asked is
#!/usr/bin/perl
use File::Slurp qw(read_file);
use Crypt::OpenSSL::RSA;
use MIME::Base64 qw( decode_base64 encode_base64 );
my $keystring = read_file( 'id_rsa' );
my $privatekey = Crypt::OpenSSL::RSA->new_private_key($keystring);
$privatekey->use_pkcs1_padding();
my $datatosign = "hello";
my $signature = $privatekey->sign($datatosign);
my $base64 = encode_base64($signature);
print "$base64";
The error coming on running it is RSA.xs:178: OpenSSL error: unsupported encryption at test.pl line 7.
NOTE: As discussed in the comments, I am using a passphrase protected id_rsa.
So, leaving out the perl, and just doing this by hand with openssl
Checking the key:
ssh-keygen -t rsa -f test_id
openssl rsa -in test_id -check
RSA key ok
So we do have a 'valid' RSA private key there, which we should be able to use for encrpyting.
Generating a comparison key
However if you use openssl to generate a key pair
openssl genrsa -out openssl_gen_rsa
openssl rsa -in openssl_gen_rsa -pubout -out openssl_gen_rsa.out
Looks like the command you need is:
openssl rsautl -inkey test_id -in test_file.txt -encrypt -out test_file.enc
(That encrypts with the private key, so you'd decrypt with the public key - that's actually pretty similar to signing - normally you wouldn't do this).
The problem is - if you compare your generated -public- keys they don't match. test_id.pub looks like (Yes, this is the real one, and no, I did only use it for testing!)
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKfjRD+Gb5EE+SgOy7eoT0siQaAqfSY7KI2wkdrdygnJ+ccW/uMCtCVPhpz00u3EW2Gz1WI DteLKppjvUem1lKb8Tt2EWBQGyFOYKp44r3AJZgTcxLeDdqSUoiPsjWf1aUqy2Z1fBgtG+QOa7bpA8km6CbsORYX/TVg4B6vvdkkH K8WcmzBBF3rGsTCM3VXPp56bPoMCbwCsXvIjejmq+JdGHyxUmCxe1PrPyvmoYX3OUqpFBYIjeLWGDI9EXS6jA/r7viIAxdllvulPg IJ+4mdYzKN+T1ME0X0c+ZdFTMdeUnB9/TZmJr1j8Q/4SQm+3J9CiwtVXKxdkDsDObkcDp root#raspberrypi
Where my generated key file looks like:
-----BEGIN PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDG8i33AuEp1wqbJgkEsnOmQim3
QT76B/oxsVGfJEDX3h4A6CD+ypBbfhhIn0GlfHanYvcGlpOJIlk3fzspbZNeoPJS
T4a0zQ0z8uJkugl8utyl9WR4tpgBRmzXZ42T/f4QSNqjDxUidRp5zPnXs9aRDtWb
XptswiGL3eVHMpbSnwIDAQAB
-----END PUBLIC KEY-----
I can use my generated pair (encrypting using public key, decrypting using private):
openssl rsautl -inkey openssl_gen_rsa.out -pubin -in test_file.txt -encrypt -out test_file.enc2
openssl rsautl -inkey openssl_gen_rsa -in test_file.enc2 -decrypt
This works.
Turning your OpenSSH private key into an RSA public key
So if we run your ssh private key through openssl:
openssl rsa -in test_id -pubout -out test_id.openssl.pub
We get:
-----BEGIN PUBLIC KEY-----
MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAyn40Q/hm+RBPkoDsu3qE
9LIkGgKn0mOyiNsJHa3coJyfnHFv7jArQlT4ac9NLtxFths9ViA7XiyqaY71HptZ
Sm/E7dhFgUBshTmCqeOK9wCWYE3MS3g3aklKIj7I1n9WlKstmdXwYLRvkDmu26QP
JJugm7DkWF/01YOAer73ZJByvFnJswQRd6xrEwjN1Vz6eemz6DAm8ArF7yI3o5qv
iXRh8sVJgsXtT6z8r5qGF9zlKqRQWCI3i1hgyPRF0uowP6+74iAMXZZb7pT4CCfu
JnWMyjfk9TBNF9HPmXRUzHXlJwff02Zia9Y/EP+EkJvtyfQosLVVysXZA7Azm5HA
6QIDAQAB
-----END PUBLIC KEY-----
Which isn't even a remotely similar encoding to the id_rsa.pub file - and I think therefore why you have this problem.
openssl rsautl -inkey test_id.openssl.pub -pubin -in test_file.txt -encrypt -out test_file.openssl.pub.enc
Now works, and you can decrypt it using your test_id private key:
openssl rsautl -inkey test_id -in test_file.openssl.pub.enc -decrypt
So yes - for some reason the public key generated by ssh-keygen isn't the same format as openssl is expecting, so it's not going to work.
using an openssh key to verify
Anyway, back to your original example - using the private key to generate a signature:
openssl rsautl -sign -inkey test_id -in test_file.txt -out test_file.sig
openssl rsautl -verify -in test_file.sig -inkey test_id.openssl.pub -pubin
But the verify step won't work using the generated ssh key.
So does this answer your question as to whether it's possible? I'm afraid I don't know how the ssh public key differs from a 'normal' rsa public key.
I suspect your code therefore doesn't even need to be as complicated as it - I don't think you need to pkcs pad your key file.
E.g.
#!/usr/bin/perl
use strict;
use warnings;
use File::Slurp qw(read_file);
use Crypt::OpenSSL::RSA;
use MIME::Base64 qw( decode_base64 encode_base64 );
my $keystring = read_file ('test_id');
my $privatekey = Crypt::OpenSSL::RSA->new_private_key($keystring);
my $datatosign = "hello";
my $signature = $privatekey->sign($datatosign);
my $base64 = encode_base64($signature);
print "Signature:\n";
print "$base64";
my $public_key_text = $privatekey-> get_public_key_string();
print "Public Key:\n";
print $public_key_text;
my $rsa_pub = Crypt::OpenSSL::RSA->new_public_key ( $public_key_text );
print "Signed correctly\n" if ($rsa_pub->verify($datatosign, decode_base64($base64)));
This seems to work. (note lack of pkcs padding line).
Encrypted private key (passphrase)
Following from comments:
If your id_rsa has a passphrase set you'll have something like this:
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,B44716076DD8B7D2B5E909BB8F70B48B
You can decrypt this by hand using openssl again:
openssl rsa -in test_id.enc
Enter pass phrase for test_id.enc:
Or:
openssl rsa -in test_id.enc -passin pass:testpass
I'm less sure how you do this using Crypt::OpenSSL::RSA - there doesn't seem to be any option to specify a passphrase to a private key.
I think you therefore need to use Crypt::CBC to decrypt the private key first. I can't test this, because I've got a load of dependencies to install.
A bit of googling suggests that you might be able to use Crypt::PK::RSA instead.
And as an alernative - use IPC::Open2 and start an openssl process to do the things without using libraries at all.
e.g.
my $keystring = `openssl rsa -passin pass:testpass -in test_id.enc`;
Not very nice, but it'll work. (and test it does actually work)