Itext 7.1.6 Out of memory error while merging documents - itext

When a collate is created from multiple documents, java.lang.OutOfMemoryError: Java heap space error is coming in the server and the application goes down.
Below is a sample code snippet,
PdfReader objReader = new PdfReader(new ByteArrayInputStream(content));
PdfDocument srcPdfDocument = new PdfDocument(objReader);
Document srcDocument = new Document(srcPdfDocument);
WriterProperties wp = new WriterProperties();
wp.setPdfVersion(PdfVersion.PDF_1_7);
PdfDocument destPdfDoc = new PdfDocument(new PdfWriter(baos,wp));
Document destDocument = new Document(destPdfDoc);
PdfMerger merger = new PdfMerger(destPdfDoc, true,true);
merger.merge(srcPdfDocument, 1, srcPdfDocument.getNumberOfPages());
//finally block
finally{
if(srcPdfDocument != null && !srcPdfDocument.isClosed()) {
srcPdfDocument.close();
}
if(srcDocument != null) {
srcDocument.close();
}
if(destPdfDoc != null && !destPdfDoc.isClosed()) {
destPdfDoc.close();
}
if(destDocument != null) {
destDocument.close();
}
}
If its large documents (file size in GBs) or documents with corrupted tag structure are collated (Error in server log -com.itextpdf.kernel.pdf.tagging.ParentTreeHandler Corrupted tag structure: encountered invalid marked content reference - it doesn't refer to any page or any mcid. This content reference will be ignored), the out of memory error is thrown.
Is there any way we can collate documents without keeping the bytes in the memory
using Itext 7. Please note, using Itext 5 (PdfCopy) the functionality works fine without issues.

a bit late reply.
I had the same problem itext 7.2.1.
I solved it by adding.
flushCopiedObjects()
Merging a 6000 page, 400 MiB file, works fine.
PdfDocument pdf = new PdfDocument(new PdfWriter(outfile));
PdfMerger merger = new PdfMerger(pdf);
for ( String file : listFilesToAdd ) {
PdfDocument fileAdd = new PdfDocument(new PdfReader(file));
merger.merge(fileAdd , 1, fileAdd.getNumberOfPages());
pdf.flushCopiedObjects(fileAdd);
fileAdd.close();
}
pdf.close();

Related

iText7: com.itextpdf.kernel.PdfException: Dictionary doesn't have supported font data

I try to generate a toc(table of content) for my pdf, and I want to get some strings which look like chapter title in xxx.pdf using ITextExtractionStrategy. But I got com.itextpdf.kernel.PdfException when I am running a test.
Here is my code:
#org.junit.Test
public void test() throws IOException {
ByteArrayOutputStream baos = new ByteArrayOutputStream();
PdfDocument pdfDoc = new PdfDocument(new PdfReader("src/test/resources/template/xxx.pdf"),
new PdfWriter(baos));
pdfDoc.addNewPage(1);
Document document = new Document(pdfDoc);
// when add this code, throw com.itextpdf.kernel.PdfException: Dictionary doesn't have supported font data.
Paragraph title = new Paragraph(new Text("index"))
.setTextAlignment(TextAlignment.CENTER);
document.add(title);
SimpleTextExtractionStrategy extractionStrategy = new SimpleTextExtractionStrategy();
for (int i = 1; i < pdfDoc.getNumberOfPages(); i++) {
PdfPage page = pdfDoc.getPage(i);
PdfCanvasProcessor parser = new PdfCanvasProcessor(extractionStrategy);
parser.processPageContent(page);
}
...
document.close();
pdfDoc.close();
new FileOutputStream("./yyy.pdf").write(baos.toByteArray());
}
Here is the output:
com.itextpdf.kernel.PdfException: Dictionary doesn't have supported font data.
at com.itextpdf.kernel.font.PdfFontFactory.createFont(PdfFontFactory.java:123)
at com.itextpdf.kernel.pdf.canvas.parser.PdfCanvasProcessor.getFont(PdfCanvasProcessor.java:490)
at com.itextpdf.kernel.pdf.canvas.parser.PdfCanvasProcessor$SetTextFontOperator.invoke(PdfCanvasProcessor.java:811)
at com.itextpdf.kernel.pdf.canvas.parser.PdfCanvasProcessor.invokeOperator(PdfCanvasProcessor.java:454)
at com.itextpdf.kernel.pdf.canvas.parser.PdfCanvasProcessor.processContent(PdfCanvasProcessor.java:282)
at com.itextpdf.kernel.pdf.canvas.parser.PdfCanvasProcessor.processPageContent(PdfCanvasProcessor.java:303)
at com.example.pdf.util.Test.test(Test.java:138)
Whenever you add content to a PdfDocument like you do here
Document document = new Document(pdfDoc);
Paragraph title = new Paragraph(new Text("index"))
.setTextAlignment(TextAlignment.CENTER);
document.add(title);
you have to be aware that this content is not already stored in its final form; for example fonts used are not yet properly subset'ed. The final form is generated when you're closing the document.
Text extraction on the other hand requires the content to extract to be in its final form.
Thus, you should not apply text extraction to a document you're working on. In particular, don't apply text extraction to a page you've changed the content of.
If you need to extract text from the documents you create yourself, close your document first, open a new document from the output, and extract from that new document.

Solution to upload image file via WCF service?

Being surfing for last 3-4 days downloading, running and fixing issues with available demo projects online, none of them work so far.
I need to upload an image using WCF webservice. Where from client side end I like to upload it by means of form (multipart/form-data), including some file description.
Any solution working with proper answer? My mind is really stacked overflow trying different solution. One which I initially have I am able to upload a text file where file gets created with some extra content in it. I need to upload image file.
------------cH2ae0GI3KM7GI3Ij5ae0ei4Ij5Ij5
Content-Disposition: form-data; name=\"Filename\"
testing file gets upload...
When I upload image file, the image file is empty.
Initial Code (one implantation), method by means of which I get the .txt file as above, in case of image its blank (or say corrupt don't know)
private string uplaodFile(Stream stream)
{
StreamReader sr = new StreamReader(stream);
int length = sr.ReadToEnd().Length;
byte[] buffer = new byte[length];
stream.Read(buffer, 0, length);
FileStream f = new FileStream(Path.Combine(HostingEnvironment.MapPath("~/Upload"), "test.png"), FileMode.OpenOrCreate);
f.Write(buffer, 0, buffer.Length);
f.Close();
stream.Close();
return "Recieved the image on server";
}
another;
public Stream FileUpload(string fileName, Stream stream)
{
string FilePath = Path.Combine(HostingEnvironment.MapPath("~/Upload"), fileName);
int length = 0;
using (FileStream writer = new FileStream(FilePath, FileMode.Create))
{
int readCount;
var buffer = new byte[8192];
while ((readCount = stream.Read(buffer, 0, buffer.Length)) != 0)
{
writer.Write(buffer, 0, readCount);
length += readCount;
}
}
return returnJson(new { resp_code = 302, resp_message = "occurred." });
}

iText7: Error at file pointer when merging two pdfs

We are in the last steps of evaluating iText7. We use iText 7.1.0 and html2pdf 2.0.0.
What we do: we send a json_encoded collection with pdf-data (which includes html for header, body and footer) to our Java app. There we iterate over the collection, create a byteArrayOutputStream for each pdf-data element and merge them together. We then send the results to a script which echoes it to e.g. a browser. Although the pdf is displayed correctly, we encounter errors while creating it:
com.itextpdf.io.IOException: Error at file pointer 226,416.
...
Caused by: com.itextpdf.io.IOException: xref subsection not found.
... 73 common frames omitted
If we create only one part of the collection, no error is thrown.
Iterate over collection and merge:
#RequestMapping(value = "/pdf", method = RequestMethod.POST, produces = MediaType.APPLICATION_PDF_VALUE)
public byte[] index(#RequestBody PDFDataModelCollection elements, Model model) throws IOException {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
PdfWriter writer = new PdfWriter(byteArrayOutputStream);
try (PdfDocument resultDoc = new PdfDocument(writer)) {
for (PDFDataModel pdfDataModel : elements.getElements()) {
PdfReader reader = new PdfReader(new ByteArrayInputStream(creationService.createDatasheet(pdfDataModel)));
try (PdfDocument sourceDoc = new PdfDocument(reader)) {
int n = sourceDoc.getNumberOfPages(); //<-- IOException on second iteration
for (int i = 1; i <= n; i++) {
PdfPage page = sourceDoc.getPage(i).copyTo(resultDoc);
resultDoc.addPage(page);
}
}
}
}
return byteArrayOutputStream.toByteArray(); //outputs the final pdf
}
Creation of part:
public byte[] createDatasheet(PDFDataModel pdfDataModel) throws IOException {
PdfWriter writer = new PdfWriter(byteArrayOutputStream);
//Initialize PDF document
PdfDocument pdfDoc = new PdfDocument(writer);
try (
Document document = new Document(pdfDoc)
) {
//header, footer, etc
//body
for (IElement element : HtmlConverter.convertToElements(pdfDataModel.getBody(), this.props)) {
document.add((IBlockElement) element);
}
footer.writeTotalNumberOnPages(pdfDoc);
}
return byteArrayOutputStream.toByteArray();
}
We are grateful for any suggestion.
In createDatasheet you appear to re-use some byteArrayOutputStream without clearing it first.
In the first iteration, therefore, everything works as desired, at the end of createDatasheet you have a single PDF file in it.
In the second iteration, though, you have two PDF files in that byteArrayOutputStream, one after the other. This concatenation does not form a valid single PDF.
Thus, byteArrayOutputStream.toByteArray() returns something broken.
To fix this, either make the byteArrayOutputStream local to createDatasheet and create a new instance every time or alternatively reset byteArrayOutputStream at the start of createDatasheet:
public byte[] createDatasheet(PDFDataModel pdfDataModel) throws IOException {
byteArrayOutputStream.reset();
PdfWriter writer = new PdfWriter(byteArrayOutputStream);
[...]

Is it possible to merge several pdfs using iText7

I have several datasheets for products. Each is a separate file. What I want to do is to use iText to generate a summary / recommended set of actions, based on answers to a webform, and then append to that all the relevant datasheets. This way, I only need to open one new tab in the browser to print all information, rather than opening one for the summary, and one for each datasheet that is needed.
So, is it possible to do this using iText?
Yes, you can merge PDFs using iText 7. E.g. look at the iText 7 Jump-Start tutorial sample C06E04_88th_Oscar_Combine, the pivotal code is:
PdfDocument pdf = new PdfDocument(new PdfWriter(dest));
PdfMerger merger = new PdfMerger(pdf);
//Add pages from the first document
PdfDocument firstSourcePdf = new PdfDocument(new PdfReader(SRC1));
merger.merge(firstSourcePdf, 1, firstSourcePdf.getNumberOfPages());
//Add pages from the second pdf document
PdfDocument secondSourcePdf = new PdfDocument(new PdfReader(SRC2));
merger.merge(secondSourcePdf, 1, secondSourcePdf.getNumberOfPages());
firstSourcePdf.close();
secondSourcePdf.close();
pdf.close();
(C06E04_88th_Oscar_Combine method createPdf)
Depending on your use case, you might want to use the PdfDenseMerger with its helper class PageVerticalAnalyzer instead of the PdfMerger here. It attempts to put content from multiple source pages onto a single target page and corresponds to the iText 5 PdfVeryDenseMergeTool from this answer. Due to the nature of PDF files this only works for PDFs without headers, footers, and similar artifacts.
I found a solution that works quite well.
public byte[] Combine(IEnumerable<byte[]> pdfs)
{
using (var writerMemoryStream = new MemoryStream())
{
using (var writer = new PdfWriter(writerMemoryStream))
{
using (var mergedDocument = new PdfDocument(writer))
{
var merger = new PdfMerger(mergedDocument);
foreach (var pdfBytes in pdfs)
{
using (var copyFromMemoryStream = new MemoryStream(pdfBytes))
{
using (var reader = new PdfReader(copyFromMemoryStream))
{
using (var copyFromDocument = new PdfDocument(reader))
{
merger.Merge(copyFromDocument, 1, copyFromDocument.GetNumberOfPages());
}
}
}
}
}
}
return writerMemoryStream.ToArray();
}
}
Use
DirectoryInfo d = new DirectoryInfo(INPUT_FOLDER);
var pdfList = new List<byte[]> { };
foreach (var file in d.GetFiles("*.pdf"))
{
pdfList.Add(File.ReadAllBytes(file.FullName));
}
File.WriteAllBytes(OUTPUT_FOLDER + "\\merged.pdf", Combine(pdfList));
Autor:
https://www.nikouusitalo.com/blog/combining-pdf-documents-using-itext7-and-c/
If you want to add two array of bytes and return one array of bytes as PDF/A
public static byte[] mergePDF(byte [] first, byte [] second) throws IOException {
// Initialize PDF writer
ByteArrayOutputStream arrayOutputStream = new ByteArrayOutputStream();
PdfWriter writer = new PdfWriter(arrayOutputStream);
// Initialize PDF document
PdfADocument pdf = new PdfADocument(writer, PdfAConformanceLevel.PDF_A_1B, new PdfOutputIntent("Custom", "",
"https://www.color.org", "sRGB IEC61966-2.1", new FileInputStream("sRGB_CS_profile.icm")));
PdfMerger merger = new PdfMerger(pdf);
//Add pages from the first document
PdfDocument firstSourcePdf = new PdfDocument(new PdfReader(new ByteArrayInputStream(first)));
merger.merge(firstSourcePdf, 1, firstSourcePdf.getNumberOfPages());
//Add pages from the second pdf document
PdfDocument secondSourcePdf = new PdfDocument(new PdfReader(new ByteArrayInputStream(second)));
merger.merge(secondSourcePdf, 1, secondSourcePdf.getNumberOfPages());
firstSourcePdf.close();
secondSourcePdf.close();
writer.close();
pdf.close();
return arrayOutputStream.toByteArray();
}
The question doesn't specify the language, so I'm adding an answer using C#; this works for me. I'm creating three separate but related PDFs then combining them into one.
After creating the three separate PDF docs and adding data to them, I combine them this way:
PdfDocument pdfCombined = new PdfDocument(new PdfWriter(destCombined));
PdfMerger merger = new PdfMerger(pdfCombined);
PdfDocument pdfReaderExecSumm = new PdfDocument(new PdfReader(destExecSumm));
merger.Merge(pdfReaderExecSumm, 1, pdfReaderExecSumm.GetNumberOfPages());
PdfDocument pdfReaderPhrases = new PdfDocument(new PdfReader(destPhrases));
merger.Merge(pdfReaderPhrases, 1, pdfReaderPhrases.GetNumberOfPages());
PdfDocument pdfReaderUncommonWords = new PdfDocument(new PdfReader(destUncommonWords));
merger.Merge(pdfReaderUncommonWords, 1, pdfReaderUncommonWords.GetNumberOfPages());
pdfCombined.Close();
So the combined PDF is a PDFWriter type of PdfDocument, and the merged pieces parts are PdfReader types of PdfDocuments, and the PdfMerger is the glue that binds it all together.
Here is the minimum C# code needed to merge file1.pdf into file2.pdf creating new merged.pdf:
var path = #"C:\Temp\";
var src0 = System.IO.Path.Combine(path, "merged.pdf");
var wtr0 = new PdfWriter(src0);
var pdf0 = new PdfDocument(wtr0);
var src1 = System.IO.Path.Combine(path, "file1.pdf");
var fi1 = new FileInfo(src1);
var rdr1= new PdfReader(fi1);
var pdf1 = new PdfDocument(rdr1);
var src2 = System.IO.Path.Combine(path, "file2.pdf");
var fi2 = new FileInfo(src2);
var rdr2 = new PdfReader(fi2);
var pdf2 = new PdfDocument(rdr2);
var merger = new PdfMerger(pdf0);
merger.Merge(pdf1, 1, pdf1.GetNumberOfPages());
merger.Merge(pdf2, 1, pdf2.GetNumberOfPages());
merger.Close();
pdf0.Close();
Here is a VB.NET solution using open source iText7 that can merge multiple PDF files to an output file.
Imports iText.Kernel.Pdf
Imports iText.Kernel.Utils
Public Function Merge_PDF_Files(ByVal input_files As List(Of String), ByVal output_file As String) As Boolean
Dim Input_Document As PdfDocument = Nothing
Dim Output_Document As PdfDocument = Nothing
Dim Merger As PdfMerger
Try
Output_Document = New iText.Kernel.Pdf.PdfDocument(New iText.Kernel.Pdf.PdfWriter(output_file))
'Create the output file (Document) from a Merger stream'
Merger = New PdfMerger(Output_Document)
'Merge each input PDF file to the output document'
For Each file As String In input_files
Input_Document = New PdfDocument(New PdfReader(file))
Merger.Merge(Input_Document, 1, Input_Document.GetNumberOfPages())
Input_Document.Close()
Next
Output_Document.Close()
Return True
Catch ex As Exception
'catch Exception if needed'
If Input_Document IsNot Nothing Then Input_Document.Close()
If Output_Document IsNot Nothing Then Output_Document.Close()
File.Delete(output_file)
Return False
End Try
End Function
USAGE EXAMPLE:
Dim success as boolean = false
Dim input_files_list As New List(Of String)
input_files_list.Add("c:\input_PDF1.pdf")
input_files_list.Add("c:\input_PDF2.pdf")
input_files_list.Add("c:\input_PDF3.pdf")
success = Merge_PDF_Files(input_files_list, "c:\output_PDF.pdf")
'Optional: handling errors'
if success then
'Files merged'
else
'Error merging files'
end if

iText InvalidPdfException Rebuild failed: '>' not expected at file pointer

We're trying to concatenate many dynamically generated pdf together using PdfSmartCopy as follow
Document document = new Document();
PdfCopy copy = new PdfSmartCopy(document, outputStream);
document.open();
for (PdfReader reader : list) {
int n = reader.getNumberOfPages();
// page numbers start at 1
for (int page = 1; page <= n; page++) {
copy.addPage(copy.getImportedPage(reader, page));
}
copy.freeReader(reader);
reader.close();
}
document.close();
Once in a while, we keep getting this exception InvalidPdfException: Rebuild failed: '>' not expected at file pointer 65537; Original message: Invalid object number. at file pointer
Or InvalidPdfException: Rebuild failed: Dictionary key 0 is not a name. at file pointer 65536
Have anyone encountered this before?