I need to access some XML from the internet using MATLAB, and to get it I need to download the security certificate from the website manually and use the following function with the downloaded certificate;
% Title: inportcert
% Author: MathWorks Support Team
% Code Version: 1.0
% Type: MATLAB soruce code https://www.mathworks.com/matlabcentral/answers/uploaded_files/1142/importcert.m
function importcert(filename)
if (nargin == 0)
% If no certificate specified show open file dialog to select
[filename,path] = uigetfile({'*.cer;*.crt','Certificates
(*.cer,*.crt)'},'Select Certificate');
if (filename==0), return, end
filename = fullfile(path,filename);
end
% Determine Java keytool location and cacerts location
keytool = fullfile(matlabroot,'sys','java','jre',computer('arch'),'jre','bin','keytool');
cacerts = fullfile(matlabroot,'sys','java','jre',computer('arch'),'jre','lib','security','cacerts');
% Create backup of cacerts
if (~exist([cacerts '.org'],'file'))
% copyfile(cacerts,[cacerts '.org'])
end
% Construct and execute keytool
command = sprintf('"%s" -import -file "%s" -keystore "%s" -storepass
changeit -noprompt',keytool,filename,cacerts);
dos(command);
which will allow me to use the following command;
xmlread('https://finds.org.uk/database/images/index/format/xml');
These certificates expire and I would not be able to manually change the certificates at that point so I was wondering if;
A) Can I automatically download the certificate using MATLAB instead of doing it manually
B) Just ignore the certificate completely, security isn't an issue.
C) Preferably do A or B without the need for Admin rights
NOTE: It seems that you need to go through this process if you're using a Windows-based system, using MacOSX or Linux you will not need to import the certificates and can access the page without getting an SSL Handshake error.
Related
I would like to create a Private Key and a CSR, submit the CSR to a Certificate Authority, retrieve the certificate once issued, and have the Private Key and Certificate as separate PEM files suitable for use in non-Microsoft applications (they are generally web servers). I'd like to avoid using Java Keytool or OpenSSL to generate keys and certificate signing requests in Windows PowerShell on Windows Server 2016. The CSRs will be submitted to a Microsoft Active Directory Certificate Services.
OpenSSL and Java are not (and won't be) installed on the computers requiring certificates. As the certificates are for non-Microsoft applications, I also want to avoid using the Certificate Store on the computers. I don't mind using "certreq" to actually submit the completed CSR and retrieve the resulting certificate once approved.
I have some code, based on C# Export Private/Public RSA key from RSACryptoServiceProvider to PEM string, which will extract the private key from an X509Certificate2. So far, as an experiment, I have used this successfully with a PKCS12 keystore (where the key and CSR were created with Keytool).
Inspired by Automate the process of creating a private key, a CSR and a final Signed Certificate in .NET Core I knocked together the following, but ran out of inspiration, and didn't really know what I was doing. How do I complete the process of submitting the CSR to the CA (or outputting the CSR as a file for using with certreq)?
[int]$KeyLength = 2048
$ComputerName = "jon"
$Domain = "domain.local"
[string]$DistinguishedName = "CN=$($ComputerName).$($Domain),OU=Unit,O=Org,C=GB"
$HashAlgo = [System.Security.Cryptography.HashAlgorithmName]::SHA256
$RSASigPadding = [System.Security.Cryptography.RSASignaturePadding]::Pkcs1
$RSAKey = [System.Security.Cryptography.RSA]::Create($KeyLength)
$Certificate = [System.Security.Cryptography.X509Certificates.CertificateRequest]::new($DistinguishedName,$RSAKey,$HashAlgo,$RSASigPadding)
# Add Basic Constraints
$BasicConstraints = [System.Security.Cryptography.X509Certificates.X509BasicConstraintsExtension]::new($false,$false,0,$false)
$BCExtension = [System.Security.Cryptography.X509Certificates.X509Extension]::new($BasicConstraints,$false)
$Certificate.CertificateExtensions.Add($BCExtension)
# Add Subject Key Identifier extension
$SubjectKeyIdentifier = [System.Security.Cryptography.X509Certificates.X509SubjectKeyIdentifierExtension]::new($Certificate.PublicKey,$false)
$SKIExtension = [System.Security.Cryptography.X509Certificates.X509Extension]::new($SubjectKeyIdentifier,$false)
$Certificate.CertificateExtensions.Add($SKIExtension)
# Add Key Usage
$KeyUsageFlags = [System.Security.Cryptography.X509Certificates.X509KeyUsageFlags]::DigitalSignature -bor [System.Security.Cryptography.X509Certificates.X509KeyUsageFlags]::KeyEncipherment
$KeyUsage = [System.Security.Cryptography.X509Certificates.X509KeyUsageExtension]::new($KeyUsageFlags,$true)
$KUExtension = [System.Security.Cryptography.X509Certificates.X509Extension]::new($KeyUsage,$true)
$Certificate.CertificateExtensions.Add($KUExtension)
# Add EKU
$ServerAuthentication = [System.Security.Cryptography.Oid]::New("Server Authentication")
$EKUOidCollection = [System.Security.Cryptography.OidCollection]::new()
$EKUOidCollection.Add($ServerAuthentication) | out-null # this outputs 0
$EnhancedKeyUsage = [System.Security.Cryptography.X509Certificates.X509EnhancedKeyUsageExtension]::new($EKUOidCollection,$false)
$EKUExtension = [System.Security.Cryptography.X509Certificates.X509Extension]::new($EnhancedKeyUsage,$false)
$Certificate.CertificateExtensions.Add($EKUExtension)
# Add SAN
$SubjectAlternateNameBuilder = [System.Security.Cryptography.X509Certificates.SubjectAlternativeNameBuilder]::new()
$SubjectAlternateNameBuilder.AddDnsName("$($ComputerName).$($Domain)")
$Certificate.CertificateExtensions.Add($SubjectAlternateNameBuilder.Build())
Previously we have one code signing cert, every thing working fine. Recently cert expired so we taken new cert from Comodo and released new build with that cert.
So here is the problem: Package Family Name Changed with that new cert, so I am not able to upgrade old installed apps with this new build packaged with new cert.
Need solution for this to maintain Package Family Name as constant or for new cert's also we need to provide upgrade for older apps.
How can we achieve this?
Error msg while upgrading package from PowerShell:
Add-AppxPackage : Deployment failed with HRESULT: 0x80073CF3, Package failed
updates, dependency or conflict validation.
Windows cannot install package df70dbc9-455c-4c32-b052-7ac2943630b7_1.0.193.1_x64__qbsrcgy0j364g
because a different package df70dbc9-455c-4c32-b052-7ac2943630b7_1.0.0.191_x64__hs446qhh7vdt4
with the same name is already installed. Remove package
df70dbc9-455c-4c32-b052-7ac2943630b7_1.0.0.191_x64__hs446qhh7vdt4 before
installing.
NOTE: For additional information, look for [ActivityId]
b0deec37-ac10-0001-81fd-deb010acd101 in the Event Log or use the command line
Get-AppxLog -ActivityID b0deec37-ac10-0001-81fd-deb010acd101
At C:\Users\\Desktop\\myappName_1.0.193.1_x64_Test\Add-AppDevPackage.ps1:388 char:13
+ Add-AppxPackage -Path $DeveloperPackagePath.FullName -DependencyPath ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : WriteError: (C:\Users\myuser....193.1_x64.appx:String) [Add-AppxPackage], IOException
+ FullyQualifiedErrorId : DeploymentError,Microsoft.Windows.Appx.PackageManager.Commands.AddAppxPackageCommand
Error: Could not install the app.
The Package Family Name (PFN) suffix (in your case hs446qhh7vdt4) is a hash of the certificate's subject (AKA subject name):
certutil -dump foo.pfx
Enter PFX password:
================ Certificate 0 ================
================ Begin Nesting Level 1 ================
Element 0:
Serial Number: xxxxxxxxxxxxxxxxx
Issuer: CN=Microsoft, O=Contoso, L=Redmond, S=Washington, C=US
NotBefore: 11/1/2016 12:00 AM
NotAfter: 11/1/2017 12:00 AM
Subject: CN=Microsoft, O=Contoso, L=Redmond, S=Washington, C=US <== THIS IS HASHED
If you make sure the new cert you generate has the same subject, you'll get the same PFN. Note that you might not be able to generate store certs from within Visual Studio (at the time of writing, it can't parse complex subjects like the one above with multiple 'parts' like CN=X, O=Y). In that case you'll have to create your own, but it must comply with the store validations.
Luckily, there's a simple command that generates the exact certificate you need. Open a Visual Studio developer prompt and run (one line):
makecert -sv foo.pvk -n "CN=Contoso, O=Contoso, L=Redmond, S=Washington, C=US"
foo.cer -b 11/01/2016 -e 11/01/2017 -r -cy end -a sha256 -eku 1.3.6.1.5.5.7.3.3
Make sure to replace the validity dates (no more than a year apart!) as well as the subject (taken from your previous cert using certutil -dump). The names of the output cert (cer) and private key (pvk) are meaningless. That command will generate foo.pvk and foo.cer, which you will then be able to combine to a pfx like so:
PVK2PFX -pvk foo.pvk -spc foo.cer -pfx foo.pfx
Another option for advanced generation
In case you have more advanced cert requirements, you should be able to use certreq (haven't tested it though). Create a file named cert.inf with the following contents:
[Version]
Signature = "$Windows NT$"
[Strings]
szOID_ENHANCED_KEY_USAGE = "2.5.29.37"
szOID_CODE_SIGNING = "1.3.6.1.5.5.7.3.3"
szOID_BASIC_CONSTRAINTS2 = "2.5.29.19"
[NewRequest]
Subject = "CN=Contoso, O=Contoso, L=Redmond, S=Washington, C=US"
Exportable = true
HashAlgorithm = Sha256
KeyLength = 2048
RequestType = Cert
ValidityPeriod = "Years"
ValidityPeriodUnits = "1"
[Extensions]
%szOID_ENHANCED_KEY_USAGE% = "{text}%szOID_CODE_SIGNING%"
%szOID_BASIC_CONSTRAINTS2% = "{text}"
Replace the subject and validity period, and adjust any advanced settings you need per the docs (or more likely found on the web). Then do the following:
certreq -new cert.inf cert.cer
Double-click the resulting cert.cer and install it to the Trusted Root Certificate Authorities store (either user or machine).
certreq -accept -user cert.cer OR certreq -accept -machine cert.cer (depending on the store you picked in the previous step).
Go to the Personal store in the cert manager (user or machine scope, depending on what you picked above) and find the cert you just installed. Double-click it and copy the serial number from the details tab (I encountered some voodoo here where the cert would only show up after a long time, or after I installed a different cert (with a different subject name).
certutil -exportpfx -p "YOUR_PFX_PASS" my SERIAL_NUMBER foo.pfx (replace the password and the serial number with their actual values)
You should now have a valid store pfx.
Yet another option for even more advanced generation
Use OpenSSL. Pretty sure it can do all the above and more, but I haven't tried it personally so you'll have to figure it out - and hopefully share here once you do!
I need to use a AsyncHttpClient in my play application to make POST() call on https protocol. I am using play 2.2.x version.
I tried #Esfand's code mentioned in below question's first answer but it is failing for java.io.FileNotFoundException.
How to use Play WS with SSL?
Here is what I did :
1) I used keytool command to generate keyStore on my C drive .. something like this
keytool -import -trustcacerts -alias mypublickey -file "" -keystore "c:\mypublicstore.jks”
It stores file on C drive with name mypublicstore.jks" (note the double quotes at the end)
2) Then I am using following code to read that
KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
KeyStore keyStore = KeyStore.getInstance("JKS");
InputStream inputStream = new FileInputStream("C:\\mypublicstore.jks\"");
String certPass = Play.application().configuration().getString("certificate.password");
keyStore.load(inputStream, certPass.toCharArray());
keyManagerFactory.init(keyStore, certPass.toCharArray());
But it is failing at line 3 of the code. It can not find the file on the system. I tried reading another file on the same location and it works fine then why it can not read jks" type of file? Am I doing something wrong here?
In FileInputStream I should give keystore path or certificate path?
Thanks for your help in advance.
I found out that AHC can not be used with play 2.2.x version. I need to upgrade to 2.3.x version.
How can I find out where my OpenSSL installation is looking for installed (trusted) certificates?
It is sometimes /etc/ssl/cert, but I have a new system and it is not working with this path.
The default path where certificates are looked up might be different on each platform. You can lookup your system configuration using the following command:
$ openssl version -d
OPENSSLDIR: "/etc/pki/tls"
This C snippet, compiled against OpenSSL, will tell you:
#include <stdlib.h>
#include <stdio.h>
#include <openssl/x509.h>
int main()
{
const char *dir;
dir = getenv(X509_get_default_cert_dir_env());
if (!dir)
dir = X509_get_default_cert_dir();
puts(dir);
return 0;
}
The path you are looking for is the "Directory for OpenSSL files". As #tnbt answered, openssl version -d (or -a) gives you the path to this directory. OpenSSL looks here for a file named cert.pem and a subdirectory certs/. Certificates it finds there are treated as trusted by openssl s_client and openssl verify (source: the article, What certificate authorities does OpenSSL recognize?).
% openssl version -d
OPENSSLDIR: "/opt/local/etc/openssl"
% ls -l /opt/local/etc/openssl/cert*
lrwxr-xr-x 1 root admin 40 29 Nov 02:05 /opt/local/etc/openssl/cert.pem -> /opt/local/share/curl/curl-ca-bundle.crt
% head -10 /opt/local/etc/openssl/cert.pem
##
## Bundle of CA Root Certificates
##
## Certificate data from Mozilla as of: Fri Nov 24 08:00:26 2017 GMT
##
## This is a bundle of X.509 certificates of public Certificate Authorities
## (CA). These were automatically extracted from Mozilla's root certificates
## file (certdata.txt). This file can be found in the mozilla source tree:
## https://hg.mozilla.org/releases/mozilla-release/raw-file/default/security/nss/lib/ckfw/builtins/certdata.txt
##
...[rest of file omitted]...
It turns out that the installer which installed OpenSSL on my system also installed cert.pem as a symlink to a bundle of Certificate Authority certificates from the tool cUrl . Those in turn came from Mozilla.
You might have nothing installed in this file or directory, or you might have a different set of certificates. This will affect which server certificates OpenSSL verifies.
OpenSSL commands like s_client support, I think since version 1.1, options -no-CAfile and -no-CApath. These let you ignore the certificates in this file and directory respectively, for the duration of one command. (I can't reproduce this because I am still using version 1.0.2, and it lacks those options.)
How can I find out, where my openssl installed is looking for installed certificates (trusted)?
You can't. OpenSSL trusts nothing by default, and it does not go looking for certs. You have to instruct it what to trust. There's even a FAQ topic covering it: Why does <SSL program> fail with a certificate verify error?:
This problem is usually indicated by log messages saying something
like "unable to get local issuer certificate" or "self signed
certificate". When a certificate is verified its root CA must be
"trusted" by OpenSSL this typically means that the CA certificate must
be placed in a directory or file and the relevant program configured
to read it. The OpenSSL program 'verify' behaves in a similar way and
issues similar error messages: check the verify(1) program manual page
for more information.
Caf's answer is kind of correct, but OpenSSL does not use it and there's nothing there...
$ grep -R X509_get_default_cert_dir *
...
crypto/x509/x509_def.c:const char *X509_get_default_cert_dir(void)
...
In the above, notice it does not hit on anything in the apps/ directory. apps/ is where all the OpenSSL samples and utilities are, like openssl req, openssl rsa, openssl dsa, openssl x509, openssl sign, openssl verify, etc.
Then:
$ cat crypto/x509/x509_def.c
...
const char *X509_get_default_cert_dir(void)
{ return(X509_CERT_DIR); }
...
$ grep -R X509_CERT_DIR *
crypto/cryptlib.h:#define X509_CERT_DIR OPENSSLDIR "/certs"
And finally:
$ ls /usr/local/ssl/certs/
$
Like I said, its not used and there's nothing there.
I'd like to check programatically if a file has been digitally signed or not.
For the moment, I found a rather obscure Microsoft code, that doesn't compile...
Any idea on the subject?
An external tool with command line would also be great, by the way.
The important missing part of the answer mentioning signtool is:
Yes, with the well known signtool.exe you can also find out, if a file is signed. No need to download another tool!
E.g. with the simple line:
signtool verify /pa myfile.exe
if %ERRORLEVEL% GEQ 1 echo This file is not signed.
(For verbose output, add a /v after /pa.)
One may ask: Why this is important? I just sign the files (again) which shall be signed and it works.
My objective is to keep builds clean, and don't sign files a second time because not only the date is changed, but the is binary different after that.
Business example:
My client has a streamlined automated "dev ops" kind build and post build process. There are multiple sources for different file sets, and at the end all is build, tested and bundled to distribution- and for that some files have to be signed. To guarantee that some files don't leave the unit without being signed, we used to sign all important files found on the media, even if they were already signed.
But this hasn´t been clean enough ! Generally:
If we sign a file again, which is already signed, the file date and binary fingerprint changes, and the file looses comparability with it's sources, if it was simply copied.
(At least if you sign with a timestamp, which we always do and I think is highly recommended.)
This is a severe quality loss, because this file is no longer identical to it's predecessors although the file itself has not changed.
If we sign a file again, this also could be a fault when it is a third party file which shouldn't be signed by our company.
You can avoid both by making the signing itself conditional depending on the return code of the preceding signtool verify call mentioned.
Download Sigcheck and use the following command.
sigcheck.exe -a -u -e
An example of a signed dll
File version: 0.0.0.0
Strong Name: Signed
An example of an unsigned dll
File version: 0.0.0.0
Strong Name: Unsigned
Sigcheck is a command-line utility that shows file version number. Good Luck
I found another option (pure .NET code) on the web here.
The code is very simple and works.
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Threading.Tasks;
internal class Program
{
private static void Main(string[] args)
{
string filePath = args[0];
if (!File.Exists(filePath))
{
Console.WriteLine("File not found");
return;
}
X509Certificate2 theCertificate;
try
{
X509Certificate theSigner = X509Certificate.CreateFromSignedFile(filePath);
theCertificate = new X509Certificate2(theSigner);
}
catch (Exception ex)
{
Console.WriteLine("No digital signature found: " + ex.Message);
return;
}
bool chainIsValid = false;
/*
*
* This section will check that the certificate is from a trusted authority IE
* not self-signed.
*
*/
var theCertificateChain = new X509Chain();
theCertificateChain.ChainPolicy.RevocationFlag = X509RevocationFlag.ExcludeRoot;
/*
*
* Using .Online here means that the validation WILL CALL OUT TO THE INTERNET
* to check the revocation status of the certificate. Change to .Offline if you
* don't want that to happen.
*/
theCertificateChain.ChainPolicy.RevocationMode = X509RevocationMode.Online;
theCertificateChain.ChainPolicy.UrlRetrievalTimeout = new TimeSpan(0, 1, 0);
theCertificateChain.ChainPolicy.VerificationFlags = X509VerificationFlags.NoFlag;
chainIsValid = theCertificateChain.Build(theCertificate);
if (chainIsValid)
{
Console.WriteLine("Publisher Information : " + theCertificate.SubjectName.Name);
Console.WriteLine("Valid From: " + theCertificate.GetEffectiveDateString());
Console.WriteLine("Valid To: " + theCertificate.GetExpirationDateString());
Console.WriteLine("Issued By: " + theCertificate.Issuer);
}
else
{
Console.WriteLine("Chain Not Valid (certificate is self-signed)");
}
}
}
Since PowerShell 5.1, you can use Get-AuthenticodeSignature to verify the signature of a binary or a PowerShell script.
> Get-AuthenticodeSignature -FilePath .\MyFile.exe
SignerCertificate Status Path
----------------- ------ ----
A59E92E31475F813DDAF41C3CCBC8B78 Valid MyFile.exe
Or
> (Get-AuthenticodeSignature -FilePath .\MyFile.exe).Status
Valid
If you need an external tool, you can use signtool.exe. It is part of the Windows SDK, it takes command line arguments, and you can find out more about it here, http://msdn.microsoft.com/en-us/library/aa387764.aspx
Also you can try to use npm package sign-check for that purposes.
This package implements WinVerifyTrust API and has simple usage:
npm install -g sign-check
sign-check 'path/to/file'
Select the <*>.exe rightclick >properties. if the file is signed then you will get this tab on the property windows of that file.