I have a x509 .der certificate that I need to concatenate to a .bin package. The problem is that if I use
cat mycert.der >> package.bin
some of the characters in the certificate are changed. Is there a way to export the certificate in a .bin file using openssl or something? I am using Windows powershell to run commands.
The redirection operator in PowerShell (> or >>) messes up your binary data, because it applies some encoding based on $OutputEncoding. Piping between Get-Content and Set-/Add-Content does not modify your data.
So you can use
Get-Content mycert.der -Raw | Add-Content package.bin -NoNewline
to append your certificate to your binary as binary data. You need -Raw so that PowerShell will preserve any CR/LF bytes and you need -NoNewline to prevent PowerShell from adding an own CR/LF at the end.
maybe because of windows powershell cat replaces line ending \n into \r\n, can you try to copy file instead
otherwise to convert certificates
from man openssl and man x509
...
Convert a certificate from PEM to DER format:
openssl x509 -in cert.pem -inform PEM -out cert.der -outform DER
Convert a certificate to a certificate request:
openssl x509 -x509toreq -in cert.pem -out req.pem -signkey key.pem
Convert a certificate request into a self signed certificate using extensions for a CA:
openssl x509 -req -in careq.pem -extfile openssl.cnf -extensions v3_ca \
-signkey key.pem -out cacert.pem
...
Related
so I am learning X.509 certificates and I got a certificate from a random site with openssl, now it is in PEM format, and for example when I open the PEM in a text viewer, I can see it in it's pure PEM view, and I know how to decode it in openssl to see what it actually says.
But when I convert it into a DER format with this
openssl x509 -in cert.pem -out cert.der -outform DER
I get a DER file and when I view it in a text editor I don't get the binary view.
Is there a way to see the binary view of DER?
Thank you in advance,
Dubbed_Kiwi
Use the following:
openssl x509 -inform der -in CERTIFICATE.der -text -noout
I use the following command in Powershell to sign a base64 encoded string. It is reading currently from a file. Can I also let it directly take it from a variable?
openssl dgst -sha256 -sign jwtRS256.key -binary $payload | openssl enc -base64 -
It works if I use the following:
openssl dgst -sha256 -sign jwtRS256.key -binary payload.b64 | openssl enc -base64 -A
Maybe it is very simple or it is not possible what I try to achieve.
This line is part of some steps that I try to follow to sign a concatenate of header.payload for JWT geneartion by using openssl.
I would like to know how to sign it with an email digital certificate and then use CURL to send this signed email message.
Below is an example of a simple TEXT/PLAIN email message that I would like to sign.
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0 (Created with SublimeText 3)
Date: Fri, 01 Jan 2021 23:59:59 -0700
Message-ID: <aade2ef2-960f-4e0d-94e4-d100f5270d28#initech.com>
Subject: TPS Report #1001
From: "Michael # Initech" <michael#initech.com>
To: "Peter # Initech" <peter#initech.com>
User-Agent: CURL/7.75.0
Good morning.
This is a message in plain text format.
It contains no inline images or attachments.
Each line is no more than 76 characters in length.
Please reach out if you have any questions.
Thank you,
Michael Bolton
SENIOR DEVELOPER
michael#initech.com
P | 801.555.1234
I place the entire message in a file called singlepart--text-plain.eml and then I use CURL to send it.
curl --verbose -ssl smtps://secure.email.com:465 --login-options AUTH=PLAIN --user michael#initech.com:Letmein --mail-from michael#initech.com --mail-rcpt peter#initech.com --mail-rcpt-allowfails --upload-file singlepart--text-plain.eml
Bam. That is all it takes.
Now the following link describes how to sign mime messages with OPENSSL.
https://www.misterpki.com/openssl-smime/
openssl smime -sign -in singlepart--text-plain.eml -text -out signed-singlepart--text-plain.eml -signer michael-digital-signature.pem
RFC 8551 Section 3.5, 3.5.1 & 3.5.2 shows limited examples on how to structure the message and my question is how to correctly do this.
https://www.rfc-editor.org/rfc/rfc8551#section-3.5
What I don't know is, what am I signing?
The entire content?
Just the literal body of the message?
Is this supposed to be a multipart message containing the body of text unchanged and a digital signature as a base64 attachment?
Or is it supposed to contain modified/signed text within a single part just like text/plain?
As you know, RFC documents are not for the faint of heart.
Only the message body is signed. That is everything after the blank line that comes after the headers.
Headers cannot be included because they can be changed (existing ones modified, or new ones added) by mail servers and mail clients on their way to the recipient.
However, there are approaches to include some important headers (like To and Subject) by copying them to the body, but they're not widely supported (yet?).
openssl smime knows how to handle the S/MIME message correctly, so its input is the complete message (singlepart--text-plain.eml in your example).
Here's a load of useful openssl smime examples which I've copied from the page http://openssl.cs.utah.edu/docs/apps/smime.html which, much to my regret, seems to be gone:
Create a cleartext signed message:
openssl smime -sign -in message.txt -text -out mail.msg \
-signer mycert.pem
Create an opaque signed message:
openssl smime -sign -in message.txt -text -out mail.msg -nodetach \
-signer mycert.pem
Create a signed message, include some additional certificates and read the private key from another file:
openssl smime -sign -in in.txt -text -out mail.msg \
-signer mycert.pem -inkey mykey.pem -certfile mycerts.pem
Create a signed message with two signers:
openssl smime -sign -in message.txt -text -out mail.msg \
-signer mycert.pem -signer othercert.pem
Send a signed message under Unix directly to sendmail, including headers:
openssl smime -sign -in in.txt -text -signer mycert.pem \
-from steve#openssl.org -to someone#somewhere \
-subject "Signed message" | sendmail someone#somewhere
Verify a message and extract the signer's certificate if successful:
openssl smime -verify -in mail.msg -signer user.pem -out signedtext.txt
Send encrypted mail using triple DES:
openssl smime -encrypt -in in.txt -from steve#openssl.org \
-to someone#somewhere -subject "Encrypted message" \
-des3 user.pem -out mail.msg
Sign and encrypt mail:
openssl smime -sign -in ml.txt -signer my.pem -text \
| openssl smime -encrypt -out mail.msg \
-from steve#openssl.org -to someone#somewhere \
-subject "Signed and Encrypted message" -des3 user.pem
Note: the encryption command does not include the -text option because the message being encrypted already has MIME headers.
Decrypt mail:
openssl smime -decrypt -in mail.msg -recip mycert.pem -inkey key.pem
The output from Netscape form signing is a PKCS#7 structure with the detached signature format. You can use this program to verify the signature by line wrapping the base64 encoded structure and surrounding it with:
-----BEGIN PKCS7-----
-----END PKCS7-----
and using the command:
openssl smime -verify -inform PEM -in signature.pem -content content.txt
Alternatively you can base64 decode the signature and use:
openssl smime -verify -inform DER -in signature.der -content content.txt
Create an encrypted message using 128 bit Camellia:
openssl smime -encrypt -in plain.txt -camellia128 -out mail.msg cert.pem
Add a signer to an existing message:
openssl smime -resign -in mail.msg -signer newsign.pem -out mail2.msg
This is a recap starting from the main purpose of the question.
RFC-2311 - Section 3.4.3.3 Sample multipart/signed Message - I simply could not get this implementation to work with openssl. For some reason, it corrupts the message.
Content-Type: multipart/signed;
protocol="application/pkcs7-signature";
micalg=sha1; boundary=boundary42
--boundary42
Content-Type: text/plain
This is a clear-signed message.
--boundary42
Content-Type: application/pkcs7-signature; name=smime.p7s
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7s
ghyHhHUujhJhjH77n8HHGTrfvbnj756tbB9HG4VQpfyF467GhIGfHfYT6
4VQpfyF467GhIGfHfYT6jH77n8HHGghyHhHUujhJh756tbB9HGTrfvbnj
n8HHGTrfvhJhjH776tbB9HG4VQbnj7567GhIGfHfYT6ghyHhHUujpfyF4
7GhIGfHfYT64VQbnj756
--boundary42--
RFC-2311 - Section 3.4.2 Signing Using application/pkcs7-mime and SignedData - This implementation is veritably working in Outlook365 that I have tested.
I believe the reason signed-data works best is, if the entire contents are encoded to base64, it's less likely to be messed with during the email transfer process.
It takes your plain text message and embeds your digital signature into it creating an smime.p7m attachment that contains both. This is the file that contains all of it in base64 format.
S/MIME email clients will automatically verify, decode and display to you as plain text again. It works great!
Content-Type: application/pkcs7-mime; smime-type=signed-data;
name=smime.p7m
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=smime.p7m
567GhIGfHfYT6ghyHhHUujpfyF4f8HHGTrfvhJhjH776tbB9HG4VQbnj7
77n8HHGT9HG4VQpfyF467GhIGfHfYT6rfvbnj756tbBghyHhHUujhJhjH
HUujhJh4VQpfyF467GhIGfHfYGTrfvbnjT6jH7756tbB9H7n8HHGghyHh
6YT64V0GhIGfHfQbnj75
Step 1. Generate a Certificate Signing Request and Private Key for the email address you want with openssl. Copy Private Key to openssl\bin directory.
Step 2. Use Certificate Signing Request to obtain a DigiCert S/MIME Class 1 Certificate (DV) from https://www.sslstore.com
Step 3. Download certificate files to disk and look for My_CA_Bundle.ca-bundle file. Copy this file to openssl\bin directory.
Optional Step. Create PKCS #12 from the 3 files My_Private.key My_Certificate.crt My_CA_Bundle.ca-bundle if you feel like importing this certificate into Windows Certificate Manager.
openssl pkcs12 -export -out My_Digital_Signature_Bundle.p12 -inkey My_Private.key -in My_Certificate.crt -certfile My_CA_Bundle.ca-bundle
Signing the plain text message
Step 1. Create a text file called text-message.eml, add the string This is an opaque-signed message. and copy it to openssl\bin directory. This file is the literal plain text message you want to sign. No headers. Just your message.
Now we are going to use the 3 files My_Private.key My_Certificate.crt My_CA_Bundle.ca-bundle to sign a plain text message which will then be output to signed-message.eml file.
openssl smime -sign -in text-message.eml -inkey My_Private.key -signer My_Certificate.crt -certfile My_CA_Bundle.ca-bundle -nodetach -text -out signed-message.eml -to "<admin#example.com>" -from "<client#example.com>" -subject "A Digitally Signed Message"
In the above openssl command, you see the -nodetach which creates a .p7m base64 fusion of the message and the signature into one blob.
signed-message.eml
What used to be just This is an opaque-signed message. has now transformed into:
To: <client#example.com>
From: <admin#example.com>
Subject: A Digitally Signed Message
MIME-Version: 1.0
Content-Disposition: attachment; filename="smime.p7m"
Content-Type: application/x-pkcs7-mime; smime-type=signed-data; name="smime.p7m"
Content-Transfer-Encoding: base64
MIISuwYJKoZIhvcNAQcCoIISrDCCEqgCAQExDzANBglghkgBZQMEAgEFADBKBgkq
hkiG9w0BBwGgPQQ7Q29udGVudC1UeXBlOiB0ZXh0L3BsYWluDQoNClRoaXMgaXMg
YSBjbGVhci1zaWduZWQgbWVzc2FnZS6ggg+1MIIGTjCCBTagAwIBAgIQBK55YGZm
kBq5xX+mbFvczTANBgkqhkiG9w0BAQsFADBlMQswCQYDVQQGEwJVUzEVMBMGA1UE
ChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3d3cuZGlnaWNlcnQuY29tMSQwIgYD
VQQDExtEaWdpQ2VydCBBc3N1cmVkIElEIFJvb3QgQ0EwHhcNMTMxMTA1MTIwMDAw
WhcNMjg...
From this point forward, the plain text message is embedded in the base64 content together with public certificates. Even the slightest alteration to this base64 message will invalidate the entire message and fail verification. That means nothing; no extra words, no spaces, no dots. At all. Period.
Verify Digitally Signed Message with openssl
The final step is to verify the recently digitally signed message.
openssl smime -verify -in signed-message.eml -CAfile My_CA_Bundle.ca-bundle -binary -signer signer-certificate-extracted.crt -out signed-content-extracted.txt
openssl only needs the public My_CA_Bundle.ca-bundle file to do this verification. If successful, openssl will output the signer-certificate-extracted.crt and lastly the signed-content-extracted.txt that was digitally signed; the message intact, if you would.
The screenshot below illustrates a successful digital signature verification.
Send digitally signed email with CURL
curl --verbose -ssl smtps://secure.email.com:465 --login-options AUTH=PLAIN --user admin#example.com:Letmein --mail-from admin#example.com --mail-rcpt client#example.com --mail-rcpt-allowfails --upload-file signed-message.eml
I have the command openssl dgst -sha256 -binary _your_file_path_ | openssl enc -base64 I use in terminal to get an output for a jar file that matches what AWS Lambda uses to hash.
I want to program that in Java, but I am having trouble understanding exactly what is going on in that line, so that I can go through each step in my code. Obviously, there is mode than just hashing in SHA256, because when I do that the output does not match.
Could someone help explain the steps that line is completing in a simple way for me?
You need to break the command down to understand what is going on.
The first part of the command:
openssl dgst -sha256 -binary <file> gives you a SHA256 binary checksum for the file.
The second part of the command:
openssl enc -base64 encodes the SHA256 binary checksum to Base64.
So to replicate in Java, you just need to carry out those same steps:
Calculate a SHA256 binary checksum.
Base64 encode the SHA256 binary checksum.
Without you posting the command you used to try and get a SHA256 checksum separately to the command you did post, I'm guessing the reason you were probably getting a different hash is because by default a checksum seems to output in hexadecimal.
See my example below and how the results are completely different.
# Hexadecimal
$ openssl dgst -sha256 data.csv
SHA256(data.csv)= 114811b0b8998cb9853a5379598021410feddf69bb2ee7b7145d052a7e9b5d45
# Binary (note the usage of the -binary flag)
$ openssl dgst -sha256 -binary data.csv
H:SyY!Ai.]*~]E
If you then Base64 encode the hexadecimal checksum above, and the binary one, you'll also get two completely different results, as you can see below.
# Hexadecimal
$ printf 114811b0b8998cb9853a5379598021410feddf69bb2ee7b7145d052a7e9b5d45 | openssl enc -base64
MTE0ODExYjBiODk5OGNiOTg1M2E1Mzc5NTk4MDIxNDEwZmVkZGY2OWJiMmVlN2I3
MTQ1ZDA1MmE3ZTliNWQ0NQ==
# Binary
$ printf 'H:SyY!Ai.]*~]E' | openssl enc -base64
SDpTeVkhQWkuXSp+XUU=
For those, who TLDR. To get the same result as in this cat FILENAME.js | openssl dgst -sha256 -binary | openssl base64 -A command you should do the following conversions:
1) your content -> sha256 (you'll get the hexadecimal number, not a text)
2) hexadecimal -> binary
3) binary -> base64
I am used to generate keys with OpenSSL via shell, this way:
my $ssk="a29bRwoA73xZGeBc";
openssl req -new -newkey rsa:2048 -nodes -keyout "/etc/ssl/$ssk.key" -out "/etc/ssl/$ssk.csr"
openssl x509 -req -days 365 -in "/etc/ssl/$ssk.csr" -signkey "/etc/ssl/$ssk.key" -out "/etc/ssl/$ssk.crt"
Now I need to perform this action via PERL, but I have no idea if the below code can at least implement the 1st line:
my $ssk="a29bRwoA73xZGeBc";
# I didn't use Crypt::OpenSSL::Random, as I didn't found where it comes to action
use Crypt::OpenSSL::RSA;
my $rsa=Crypt::OpenSSL::RSA->generate_key(2048);
my $rsa_pub=Crypt::OpenSSL::RSA->new_public_key($rsa->get_public_key_x509_string());
my $rsa_prv=Crypt::OpenSSL::RSA->new_private_key($rsa->get_private_key_string());
my $cipher=$rsa->encrypt($rsa_pub);
my $plain=$rsa->encrypt($cipher);
my $sigt=$rsa_prv->sign($plain); # This seems to be part of the 2nd line in shell code, but maybe I am wrong.
unless($rsa->verify($plain,$sigt)) { die "Failed to create keys"; }
# Here is the issue:
&SaveFile("/etc/ssl/$ssk.key",$plain,0400); # Is it correct?
&SaveFile("/etc/ssl/$ssk.csr",$cipher,0400); # How about this?
# How do I proceed to implement the 2nd line in the shell code?
Note that the part of Crypt::OpenSSL::Random and saving to key files is not docmented...
Thanks a lot