Mule flow adding weird characters to beginning of file - encoding

I'm simply moving files, but for some reason they are appending weird characters to the front of the file.
Before:
(Encoded with UTF-8)
testCol1,testCol2,testCol3
After: (Encoded with ANSI)
¬í ur [B¬óøTà xp žÌtestCol1,testCol2,testCol3
Things I've tried:
Explicitly setting the encoding to utf-8.
Explicitly setting the encoding to ANSI.
Using a Object to Byte Array with encoding set to utf-8.
Using a Object to String with encoding set to utf-8.
Here is my flow and relevant connectors:
<sftp:connector name="Sftp" maxConnectionPoolSize="3" doc:name="SFTP"/>
<file:connector name="FileConnector" autoDelete="true" streaming="false" validateConnections="true" doc:name="FileConnector"/>
<flow name="SftpOutBound" initialState="stopped">
<file:inbound-endpoint
path="${ftp.base}/${id}/export"
pollingFrequency="${polling.frequency.millis}"
responseTimeout="${standard.response.timeout.millis}"
fileAge="${standard.fileage.delay.millis}"
connector-ref="deletingFileConnector"
doc:name="OutBound">
<file:filename-wildcard-filter pattern="${out.filter}" />
</file:inbound-endpoint>
<object-to-byte-array-transformer doc:name="Object to Byte Array" />
<all doc:name="all">
<file:outbound-endpoint
path="${archive.out}/${id}"
outputPattern="#[header:INBOUND:originalFileName]_#[function:datestamp:${standard.date.format}]_#[function:systime]"
responseTimeout="${standard.response.timeout.millis}"
connector-ref="deletingFileConnector"
doc:name="Archive" />
<logger
level="WARN"
message="Uploaded file from ${ftp.base}/${id}/export/#[header:INBOUND:originalFileName]"
doc:name="Logger"/>
<sftp:outbound-endpoint
connector-ref="Sftp"
host="${host}"
port="${port}"
path="${in.path}"
user="${user}"
identityFile="${configFullPath}/${identity.file}"
passphrase="${passphrase}"
responseTimeout="${standard.response.timeout.millis}"
keepFileOnError="true"
duplicateHandling="overwrite"
exchange-pattern="one-way"
outputPattern="#[header:INBOUND:originalFileName]"
doc:name="SFTP"/>
</all>
</flow>
Thanks for any help!

So it seems to be an issue with Mule 3.5.0, when I upgraded to 3.6.1 the issue went away. Still not sure what was causing it, but the version change solved it.

The payload result from file:inbound-endpoint is of type InputStream. Why don't you leave as is (byte[]) to be saved by the file and sftp outbound end-points?
If you have to convert it into a string, then I suggest using <byte-array-to-string-transformer /> after you read it instead of <object-to-byte-array-transformer />

Related

NLog not using UTF-8 encoding

Since I upgraded NLog to the latest version 5.1.0, logging special characters (e.g. Ö, ä or ß) stopped working. Outout to file is like: ö ä ü for these chars.
I checked the log file with NotePad++ and it says that the encoding is ANSI. However, I have configured NLog to use utf-8. My config:
<target name="logfile" xsi:type="File" fileName="${basedir}/Logs/nLog.csv" archiveAboveSize="50000000" archiveNumbering="Sequence" maxArchiveFiles="3" encoding="utf-8" keepFileOpen="true">
<layout xsi:type="CsvLayout">
<column name="time" layout="${longdate}" />
<column name="level" layout="${level}"/>
<column name="category" layout="${event-context:item=category}" />
<column name="message" layout="${message}" />
</layout>
</target>
What am I missing?
Based on the comments, I was able to solve the problem. However, I do not know what the real cause of the problem was.
Solution: remove encoding="utf-8" from the nLog config AND read the log file as Encoding.UTF-8 (i.e. when opening the file with a StreamReader).
I don't know why this works since nLog uses UTF-8 as default...

How to ignore special characters in File Transformation Task of Azure DevOps Pipeline

I am having xyz.config file like below. I have created xyx.Release.config file to use in File Transformation task.
<formatters>
<add template="Timestamp: {timestamp}
Message: {message}
name="Reduced Text Formatter" />
</formatters>
After the File Transformation task in pipeline it transformed as below. The special characters are replaced after the execution of File Transformation task.
<formatters>
<add template="Timestamp: {timestamp}
Message: {message}
-
</formatters>
Anyone please let me know how to ignore special character transformation as part of File Transformation task.
Thanks in advance.
Mohan
As far as I know, the File transform task doesn't support to ignore special character transformation.
The file transfrom task will use the built-in method to do the file transform option. It doesn't support to use custom method to do the option.
Here is the doc about the built-in method: Web.config Transformation Syntax for Web Project Deployment Using Visual Studio
It supports overall replacement of strings, but currently does not support replacing only the part of value in the string and ignoring others.
For a workaround, you can use the replace method then just change the part of value in the string to keep the format of the string.
For example: xyx.Release.config
<?xml version="1.0"?>
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
<formatters>
<add template="Timestamp: test1
Message: test2
" name="Reduced Text Formatter" xdt:Transform="Replace" xdt:Locator="Match(template)" />
</formatters>
</configuration>

Calabash 1.0.23 throws on xsl-formatter step: ERROR: Failed to instantiate FO provider

I'm trying to use XML Calabash 1.0.23 to run XSLT transformation and FO formatting in a single pipeline. Although the XSLT step works fine, I cannot get the xsl-formatter step to work with FOP.
Every time I run the pipeline, Calabash throws:
ERROR: pipeline.xpl:13:68:Failed to instantiate FO provider
ERROR: Underlying exception: org/apache/fop/apps/FopFactory
My call to Calabash from the command line is:
java com.xmlcalabash.drivers.Main -c cfg.xml myPipeline.xpl
And the cfg.xml configuration file being referred to in the above line is:
<cc:xproc-config xmlns:cc="http://xmlcalabash.com/ns/configuration">
<cc:fo-processor class-name="com.xmlcalabash.util.FoFOP"/>
</cc:xproc-config>
For some reason, Calabash seems to ignore the config file setting, because regardless of the value of the class-name attribute on <cc:fo-processor>, it always throws the same error message. For example, if I use com.xmlcalabash.util.FoAH, the same happens; and if put a nonsensical value, the same occurs. It's always an exception on org/apache/fop/apps/FopFactory.
Just for the sake of completeness, this is my XPL:
<declare-step name="main" version="1.0" xmlns="http://www.w3.org/ns/xproc">
<input port="parameters" kind="parameter" />
<xslt name="transformation">
<input port="source">
<document href="myMarkup.xml" />
</input>
<input port="stylesheet">
<document href="myStylesheet.xsl" />
</input>
</xslt>
<xsl-formatter href="newDoc.pdf" >
<input port="source">
<pipe step="transformation" port="result" />
</input>
</xsl-formatter>
</declare-step>
Of course, if I manually pass the generated FO from the XSLT step to FOP 1.1, it converts it to PDF with no problems. The issue only arises when trying to do the conversion within the pipeline.
I could really use some help to solve this. I'm clueless at this point.
This may seem like a very pedantic reply, but do you have fop.jar (and fop-hyph.jar, which I think FOP requires) on your classpath? They're not bundled in the XML Calabash distribution, you have to get them from Apache.

How do I config xml namespace for a smooks input file using WSO2?

I couldn't find any advices on how to setup Smooks (on WSO2 Developer Studio 3.1) in order to it properly read a xml input which has a namespace declaration.
Without the namespace the transformation works just fine!
By informing the xmlns in the input, I get this exception:
Error on line 5, column 19 in free-marker-template
Expecting a string, date or number here, Expression .vars["order"]["order-items/order-item/#id"] is instead a freemarker.ext.dom.NodeListModel
The problematic instruction:
----------
==> ${.vars["order"]["order-items/order-item/#id"]} [on line 5, column 17 in free-marker-template]
These are both my input and output:
<order id='444' xmlns="http://example.com">
<header>
<customer number="555">Amila</customer>
</header>
<order-items>
<order-item id='1'>
<product>1</product>
<quantity>2</quantity>
<price>400</price>
</order-item>
</order-items>
</order>
<salesorder>
<details>
<orderid></orderid>
<customer>
<id></id>
<name></name>
</customer>
</details>
<itemList>
<item>
<id></id>
<productId></productId>
<quantity></quantity>
<price></price>
</item>
</itemList>
</salesorder>
I've also tried Smooks core´s namespace declarations as per Smooks site:
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd"
xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.4.xsd"
>
<core:namespaces>
<core:namespace prefix="ex" uri="http://example.com/"/>
</core:namespaces>
But it seems not to be supported in the IDE since the configuration editor raises this exception:
-Value 'org.eclipse.emf.ecore.xml.type.impl.AnyTypeImpl#53abd5a9 (mixed: null, anyAttribute: null)' is not legal. (platform:/resource/Corp/smooks-config.xml,6,20)
Well, any idea?
Instead of Smooks, can't you use a XSLT? It will avoid the namespace issue you have.

How do I optionally require a command line arguement for Ant?

I'm new to ant, and I want to require a file name if something other than the default target is used, so the calling syntax would be something like this:
ant specifictarget -Dfile=myfile
I'm using the ant contrib package to give me additional functionality, so I have this:
<if>
<equals arg1="${file}" arg2="" />
<then>
<!-- fail here -->
</then>
</if>
My thinking is that if file was not specified it might be equal to the empty string. Obviously, this didn't work, and I'm not finding any examples on google or the right syntax in the manual.
So what syntax should I use?
You don't really need the contrib package. This is more conveniently done using built-in ant capabilities like if/unless and depends. See below:
<target name="check" unless="file" description="check that the file property is set" >
<fail message="set file property in the command line (e.g. -Dfile=someval)"/>
</target>
<target name="specifictarget" if="file" depends="check" description=" " >
<echo message="do something ${file}"/>
</target>
You've got the right idea. The
ant specifictarget -Dfile=myfile
sets Ant Properties from the command line. All you really need is
<property name="file" value=""/>
for your default value. That way if file is not specified, it will be equal to the empty string.
Since properties are not mutable in Ant, you can add this:
<property name="file" value="" />
This will set the property file to an empty string if it hasn't already been set on the command line. Then your equality test will work as you intended.
Alternately, you can use escape the value since ant just spits out the actual text when it can't do a property substitution.
<if>
<equals arg1="${file}" arg2="$${file}" />
<then>
<echo>BARF!</echo>
</then>
</if>