Sign PDF using an external service and iText - itext

Question

I have this scenario.
I have an application that generates a PDF, and that needs to be signed.
We have not the certificates to sign the document, because they're in a HSM, and the only way we could make use of the certificates is using a webservice.
This webservice, offers two options, send the PDF document, and it returns a signed pdf, or send a hash that will be signed.
The first option, is not viable, because the PDF is signed without a timestamp (this is a very important requisite), so the second option is chosen.
This is our code, first, we get the signature appearance, and calculate the hash:
PdfReader reader = new PdfReader(Base64.decode(pdfB64));
reader.setAppendable(true);
baos = new ByteArrayOutputStream();
PdfStamper stamper = PdfStamper.createSignature(reader, baos, '\0', null, true);
appearance = stamper.getSignatureAppearance();
appearance.setCrypto(null, chain, null, PdfSignatureAppearance.SELF_SIGNED);
appearance.setVisibleSignature("Representant");
cal = Calendar.getInstance();
PdfDictionary dic = new PdfDictionary();
dic.put(PdfName.TYPE, PdfName.SIG);
dic.put(PdfName.FILTER, PdfName.ADOBE_PPKLITE);
dic.put(PdfName.SUBFILTER, new PdfName("adbe.pkcs7.detached"));
dic.put(PdfName.M, new PdfDate(cal));
appearance.setCryptoDictionary(dic);
HashMap<PdfName, Integer> exc = new HashMap<PdfName, Integer>();
exc.put(PdfName.CONTENTS, Integer.valueOf(reservedSpace.intValue() * 2 + 2));
appearance.setCertificationLevel(1);
appearance.preClose(exc);
AbstractChecksum checksum = JacksumAPI.getChecksumInstance("sha1");
checksum.reset();
checksum.update(Utils.streamToByteArray(appearance.getRangeStream()));
hash = checksum.getByteArray();
In this point, we have the hash code of the document. Then we send the hash to the webservice, and we get the signed hash code.
Finally, we put the signed hash to the PDF:
byte[] paddedSig = new byte[reservedSpace.intValue()];
System.arraycopy(signedHash, 0, paddedSig, 0, signedHash.length);
PdfDictionary dic = new PdfDictionary();
dic.put(PdfName.CONTENTS, new PdfString(paddedSig).setHexWriting(true));
appearance.close(dic);
byte[] pdf = baos.toByteArray();
In this point, we get a PDF signed, but with an invalid signature. Adobe says that "Document has been altered or corrupted since it was signed".
I think that we make something wrong in the process, and we don't know exactly what could be.
We appreciate help on this, or an alternative way to do that.
Thanks.
EDITED
As suggested by mkl, I have followed the 4.3.3 section of this book Digital Signatures for PDF documents, and my code now what that follows:
The first part, when we calculate the hash:
PdfReader reader = new PdfReader(Base64.decode(pdfB64));
reader.setAppendable(true);
baos = new ByteArrayOutputStream();
PdfStamper stamper = PdfStamper.createSignature(reader, baos, '\0');
appearance = stamper.getSignatureAppearance();
appearance.setReason("Test");
appearance.setLocation("A casa de la caputeta");
appearance.setVisibleSignature("TMAQ-TSR[0].Pagina1[0].DadesSignatura[0].Representant[0]");
appearance.setCertificate(chain[0]);
PdfSignature dic = new PdfSignature(PdfName.ADOBE_PPKLITE, PdfName.ADBE_PKCS7_DETACHED);
dic.setReason(appearance.getReason());
dic.setLocation(appearance.getLocation());
dic.setContact(appearance.getContact());
dic.setDate(new PdfDate(appearance.getSignDate()));
appearance.setCryptoDictionary(dic);
HashMap<PdfName, Integer> exc = new HashMap<PdfName, Integer>();
exc.put(PdfName.CONTENTS, new Integer(reservedSpace.intValue() * 2 + 2));
appearance.preClose(exc);
ExternalDigest externalDigest = new ExternalDigest()
{
public MessageDigest getMessageDigest(String hashAlgorithm) throws GeneralSecurityException
{
return DigestAlgorithms.getMessageDigest(hashAlgorithm, null);
}
};
sgn = new PdfPKCS7(null, chain, "SHA256", null, externalDigest, false);
InputStream data = appearance.getRangeStream();
hash = DigestAlgorithms.digest(data, externalDigest.getMessageDigest("SHA256"));
cal = Calendar.getInstance();
byte[] sh = sgn.getAuthenticatedAttributeBytes(hash, cal, null, null, CryptoStandard.CMS);
sh = MessageDigest.getInstance("SHA256", "BC").digest(sh);
hashPdf = new String(Base64.encode(sh));
And in the second part, we get the signed hash, and we put that into the PDF:
sgn.setExternalDigest(Base64.decode(hashSignat), null, "RSA");
byte[] encodedSign = sgn.getEncodedPKCS7(hash, cal, null, null, null, CryptoStandard.CMS);
byte[] paddedSig = new byte[reservedSpace.intValue()];
System.arraycopy(encodedSign, 0, paddedSig, 0, encodedSign.length);
PdfDictionary dic2 = new PdfDictionary();
dic2.put(PdfName.CONTENTS, new PdfString(paddedSig).setHexWriting(true));
appearance.close(dic2);
byte[] pdf = baos.toByteArray();
Now, Adobe raises a Internal Cryptographic library error. Error Code: 0x2726, when we try to validate the signature.

Answer 1

If the web service returned a mere signed hash
In this point, we have the hash code of the document. Then we send the hash to the webservice, and we get the signed hash code.
Finally, we put the signed hash to the PDF:
If the webservice merely returns a signed hash, then your PDF signature is incorrect: You set the signature SubFilter to adbe.pkcs7.detached. This implies that the signature Contents have to contain a full-blown PKCS#7 signature container, not merely a signed hash.
You might want to download Digital Signatures for PDF documents, A White Paper by Bruno Lowagie (iText Software) on creating and verifying digital PDF signatures using iText. It especially contains a section "4.3 Client/server architectures for signing" which should encompass your use cases.
But the web service returns a full-fledged CMS signature container
Following to the explanation above, the OP started using code from section 4.3.3 of the above-mentioned white paper which is intended for signing using externally generated signed hashes. As this also resulted in a signed document Adobe Reader was not happy with, he provided a sample document created with this new code.
Analysis of the sample showed that the CMS signature container embedded in the document contained another CMS signature container where there should have been the signature bytes (the signed hash) for the signed attributes:
2417 13: SEQUENCE {
2419 9: OBJECT IDENTIFIER rsaEncryption (1 2 840 113549 1 1 1)
2430 0: NULL
: }
2432 5387: OCTET STRING, encapsulates {
2436 NDEF: SEQUENCE {
2438 9: OBJECT IDENTIFIER signedData (1 2 840 113549 1 7 2)
2449 NDEF: [0] {
2451 NDEF: SEQUENCE {
(The OCTET STRING following the signature algorithm should contain the signature bytes and not embed another SignedData structure.)
This indicates that the web service indeed already returns a full-fledged CMS container.
For such a scenario the original code looked quite ok. The issue might be due to a detail like the use of a wrong hashing algorithm (the original code hashed using SHA1).
A possible issue: BER encoding
The CMS signature container from the web service embedded in the CMS container generated by iText from the first sample provided by the OP hints at a possible issue: Looking at the ASN.1 dump above the sizes of the outer structures in the embedded CMS container are often NDEF.
This indicates that these outer structures are created using the less strict BER (Basic encoding Rules), not the more strict DER (Distinguished Encoding Rules) because the BER option to start a structure without stating its size is forbidden in DER.
The CMS specification (RFC 3852) referenced from the PDF specification does allow any BER encoding for the outer structures of the container, the PDF specification on the other hand requires:
the value of Contents shall be a DER-encoded PKCS#7 binary data
object containing the signature. The PKCS#7 object shall conform to RFC3852 Cryptographic Message Syntax.
Strictly speaking, therefore, signature containers embedded in PDFs are required to be DER encoded all over.
As far as I know no PDF signature validator rejects such signatures as long as the signature container DER-encodes certain pivotal elements. Concerning future tools such signatures are a possible point of failure, though.

Answer 2

After much debugging, we finally found the problem.
For some mysterious reason, the method that generates the hash of the document, was executed twice, invalidating the first hash (which we use to send to the service).
After a refactoring work of the code, the original code worked correctly.
Very thanks to all people that help me, especially mkl.