Synchronize files from Box.com to AEM DAM - aem

I am trying to sync the files from my Box.com account to AEM(CQ5) DAM. I have written a service where I am able to authenticate to Box.com and get the files. But in order for me to upload those into AEM DAM, I need the files as InputStream. On Box.com documentation(https://github.com/box/box-java-sdk/blob/master/doc/files.md), I find the code snippet for Downloading a file.
BoxFile file = new BoxFile(api, "id");
BoxFile.Info info = file.getInfo();
FileOutputStream stream = new FileOutputStream(info.getName());
file.download(stream);
stream.close();
But I could not find anything where I can get the file in Inputstream so that I can use it to upload it into AEM DAM. When I tried to convert from OutputStream to Inputstream, its just not really working and creating ZERO bytes files in AEM.
Any pointers and help greatly appreciated !
Thanks in advance.

I had a similar problem where I tried to create a CSV within CQ and store it in JCR. The solutions are piped Streams:
final PipedInputStream pis = new PipedInputStream();
final PipedOutputStream pos = new PipedOutputStream(pis);
Though I then used an OutputStreamWriter to write into the output stream, but the FileOutputStream.download should work as well.
To actually write into JCR you need the ValueFactory, which you can get from a JCR Session (here the example for my CSV):
ValueFactory valueFactory = session.getValueFactory();
Node fileNode = logNode.addNode("log.csv", "nt:file");
Node resNode = fileNode.addNode("jcr:content", "nt:resource");
resNode.setProperty("jcr:mimeType", "text/plain");
resNode.setProperty("jcr:data", valueFactory.createBinary(pis));
session.save();
EDIT: untested example with BoxFile:
try {
AssetManager assetManager = resourceResolver.adaptTo(AssetManager.class);
BoxFile file = new BoxFile(api, "id");
BoxFile.Info info = file.getInfo();
final PipedInputStream pis = new PipedInputStream();
final PipedOutputStream pos = new PipedOutputStream(pis);
Executors.newSingleThreadExecutor().submit(new Runnable() {
#Override
public void run() {
file.download(pos);
}
});
Asset asset = assetManager.createAsset(info.getName(), pis, info.getMimeType(), true);
IOUtils.closeQuietly(pos);
IOUtils.closeQuietly(pis);
} catch (IOException e) {
LOGGER.error("could not download file: ", e);
}

If i understand the code correctly you are downloading the file to a file named info.getName(). Try using FileInputStream(info.getName()) to get the input stream from the downloaded file.
BoxFile file = new BoxFile(api, "id");
BoxFile.Info info = file.getInfo();
FileOutputStream stream = new FileOutputStream(info.getName());
file.download(stream);
stream.close();
InputStream inStream=new FileInputStream(info.getName());

Related

Not Reading bytes properly during FTP transfer in Spring Batch

I am doing a project where I have to efficiently transfer data(any file) from one endpoint(HTTP, FTP, SFTP) to other. I want to use springBatch concurrency and parallelism feature of Job. In my case, one file will be one job. So, I am Trying to read file(any extension) from ftp(running locally) and writing it to same ftp in different folder.
My Reader has:
FlatFileItemReader<byte[]> reader = new FlatFileItemReader<>();
reader.setResource(new UrlResource("ftp://localhost:2121/source/1.txt"));
reader.setLineMapper((line, lineNumber) -> {
return line.getBytes();
});
And Writer has:
URL url = new URL("ftp://localhost:2121/dest/tempOutput/TransferTest.txt");
URLConnection conn = url.openConnection();
DataOutputStream out = new DataOutputStream(conn.getOutputStream());
for (byte[] b : bytes) { //I am getting List<byte[]> in my writer
out.write(b);
}
out.close();
In case of text file, all content is showing in one line(omitting nextLine character) and in case of video file bytes are missing/corrupted as video is not getting played at destination.
What I am doing wrong or is there something better way to transfer file(irrespective of its extension).

BizTalk custom pipeline parsing POP3 PDF attachment error

I have a BizTalk custom pipeline component where I'm parsing a PDF attachment using itexsharp into a custom model. The pipeline is bound to a POP3 receiving port.
In the new created message if I return the attachment stream (outputMessage.GetPart("Body").Data = ms), then this is looking good in the BizTalk administration console. I have been able to save the message from here manually and this was parsed correctly using the same parsing method as in the pipeline.
When parsing the PDF directly in the pipeline, then I'm getting the following error: Rebuild failed: trailer not found.; Original message: xref subsection not found at file pointer 1620729
If I remove the default XMLDisassembler component from pipeline, then the parsing error disappeared, but in the console the message Body is empty, although the AttachmentSizeInBytes=1788
public IBaseMessage Execute(IPipelineContext pContext, IBaseMessage pInMsg)
{
return ExtractMessagePartToMessage(pContext, pInMsg);
}
private IBaseMessage ExtractMessagePartToMessage(IPipelineContext pContext, IBaseMessage pInMsg)
{
if (pInMsg.PartCount <= 1)
{
throw new InvalidOperationException("The email had no attachment, apparently.");
}
string partName;
IBaseMessagePart attachmentPart = pInMsg.GetPartByIndex(1, out partName);
Stream attachmentPartStream = attachmentPart.GetOriginalDataStream();
IBaseMessage outputMessage;
outputMessage = pContext.GetMessageFactory().CreateMessage();
outputMessage.AddPart("Body", pContext.GetMessageFactory().CreateMessagePart(), true);
outputMessage.Context = pInMsg.Context;
var ms = new MemoryStream();
attachmentPartStream.CopyTo(ms);
ms.Seek(0L, SeekOrigin.Begin);
Stream orderStream = PdfFormParser.Parse(ms);
outputMessage.GetPart("Body").Data = orderStream;
outputMessage.Context.Write("AttachmentName", "http://schemas.microsoft.com/BizTalk/2003/file-properties", partName);
outputMessage.Context.Write("AttachmentSizeInBytes", "http://schemas.microsoft.com/BizTalk/2003/file-properties", orderStream.Length.ToString());
pContext.ResourceTracker.AddResource(ms);
pContext.ResourceTracker.AddResource(orderStream);
return outputMessage;
}
public static Stream Parse(Stream pdfDocument)
{
using (var reader = new PdfReader(pdfDocument))
{
var outputStream = new MemoryStream();
var pdfForm = ParseInternal(reader);
var xmlDocument = new XmlDocument();
xmlDocument.LoadXml(pdfForm.Serialize());
xmlDocument.Save(outputStream);
return outputStream;
}
In pipelines when you read or write a Stream, you have to rewind the stream back to the beginning if something else is going to use it (especially the final message that you expect BizTalk to process).

Xtend Code Generator How to Copy Files

I am implementing my own DSL and using Xtend to generate codes. I need some static resources to be copied to my generate code. I was trying to use commons-io, but I couldn't get anywhere with that! What is the best way to do so? I am trying to avoid reading each file and writing to the corresponding file in output path...
This should do (taken from this web site, slightly modified, not tested)
def static void copyFileUsingChannel(File source, File dest) throws IOException {
FileChannel sourceChannel = null;
FileChannel destChannel = null;
try {
sourceChannel = new FileInputStream(source).getChannel();
destChannel = new FileOutputStream(dest).getChannel();
destChannel.transferFrom(sourceChannel, 0, sourceChannel.size());
}finally{
sourceChannel.close();
destChannel.close();
}
}

Populate database from internal project textfile wp7

I implemented a mango database in my windows phone project.
I want to add data from an internal project file.
I tried to use the StreamReader
StreamReader L= new StreamReader ("/Data/info.txt");
But the problem still exists.
Is there a solution to resolve this problem?
Cordially
The file 'File.txt' should be referenced as internal ressource.
In this case, 'test' is the project name.
var resource = App.GetResourceStream(new Uri(#"/test;component/File.txt", UriKind.Relative));
try
{
StreamReader streamReader = new StreamReader(resource.Stream);
string x = streamReader.ReadToEnd();
MessageBox.Show(x);
}
catch(Exception e)
{
MessageBox.Show(e.ToString());
}

JbossTextMessage Unicode convert failed in Linux

I'm trying to upload a xml (UTF-8) file and post it on a Jboss MQ. When reading the file from the listener UTF-8 characters are not correctly formatted ONLY in the Jboss (jboss-5.1.0.GA-3) instance running on Linux.
For an instance: BORÅS is converted to BOR¿S at Linux jboss instance.
When I copy and configure the same jboss instance to run at Windows (SP3) it works perfectly.
Also I have change the default setting in Linux by including JAVA_OPTS=-Dfile.encoding=UTF-8 in .bashrc and run.sh files.
inside the Listener JbossTextMessage.getText() is coming with incorrectly specified character.
Any suggestions or workarounds ?
Finally I was able to find a solution, BUT the solution is still a blackbox. If anyone have the answer to WHY it has failed/successful please update the thread.
Solution at a glance :
1. Captured the file contents as a byte arry and wrote it to a xml file in jboss tmp folder using FileOutputStream
When posting to the jboss Message queue, I used the explicitly wrote xml file (1st step) using a FileInputStream as a byte array and pass it as the Message body.
Code example:
View: JSP page with a FormFile
Controller Class :UploadAction.java
public ActionForward execute(ActionMapping mapping, ActionForm form, HttpServletRequest request, HttpServletResponse response){
...........
writeInitFile(theForm.getFile().getFileData()); // Obtain the uploaded file
Message msg = messageHelper.createMessage( readInitFile() ); // messageHelper is a customized factory method to create Message objects. Passing the newly
wrote file's byte array.
messageHelper.sendMsg(msg); // posting in the queue
...........
}
private void writeInitFile(byte[] fileData) throws Exception{
File someFile = new File("/jboss-5.1.0.GA-3/test/server/default/tmp/UploadTmp.xml"); // Write the uploaded file into a temporary file in jboss/tmp folder
FileOutputStream fos = new FileOutputStream(someFile);
fos.write( fileData );
fos.flush();
fos.close();
}
private byte[] readInitFile() throws Exception{
StringBuilder buyteArray=new StringBuilder();
File someFile = new File("/jboss-5.1.0.GA-3/test/server/default/tmp/UploadTmp.xml"); // Read the Newly created file in jboss/tmp folder
FileInputStream fstream = new FileInputStream(someFile);
int ch;
while( (ch = fstream.read()) != -1){
buyteArray.append((char)ch);
}
fstream.close();
return buyteArray.toString().getBytes(); // return the byte []
}
Foot Note: I think it is something to do with the Linux/Windows default file saving type. eg: Windows default : ANSI.