How to feed data table from xml file received from Web API? - ado.net

My Web API returns a string looking like XML file below. I put all rows ending with \r\n to new line for readability.
{"DataTable.RemotingVersion":{"major":2,"minor":0,"build":-1,"revision":-1,"majorRevision":-1,"minorRevision":-1},"XmlSchema":" <?xml version=\"1.0\" encoding=\"utf-16\"?>\r\n
<xs:schema xmlns=\"\" xmlns:xs=\"http://www.w3.org/2001/XMLSchema\" xmlns:msdata=\"urn:schemas-microsoft-com:xml-msdata\">\r\n
<xs:element name=\"Test\">\r\n
<xs:complexType>\r\n
<xs:sequence>\r\n
<xs:element name=\"Col1\" type=\"xs:string\" msdata:targetNamespace=\"\" minOccurs=\"0\" />\r\n
<xs:element name=\"Col2\" type=\"xs:string\" msdata:targetNamespace=\"\" minOccurs=\"0\" />\r\n
</xs:sequence>\r\n
</xs:complexType>\r\n
</xs:element>\r\n
<xs:element name=\"tmpDataSet\" msdata:IsDataSet=\"true\" msdata:MainDataTable=\"Test\" msdata:UseCurrentLocale=\"true\">\r\n
<xs:complexType>\r\n
<xs:choice minOccurs=\"0\" maxOccurs=\"unbounded\" />\r\n
</xs:complexType>\r\n
</xs:element>\r\n
</xs:schema>","XmlDiffGram":"<diffgr:diffgram xmlns:msdata=\"urn:schemas-microsoft-com:xml-msdata\" xmlns:diffgr=\"urn:schemas-microsoft-com:xml-diffgram-v1\">\r\n
<tmpDataSet>\r\n
<Test diffgr:id=\"Test1\" msdata:rowOrder=\"0\">\r\n
<Col1>Column 1 value</Col1>\r\n
<Col2>Column 2 value</Col2>\r\n
</Test>\r\n
</tmpDataSet>\r\n
</diffgr:diffgram>"}
I wonder how I can feed this string to a datatable? I tried to feed the table like this
System.Data.DataTable dt = new System.Data.DataTable();
byte[] byteArray =Encoding.ASCII.GetBytes(<above string returned by Web API>);
MemoryStream stream = new MemoryStream( byteArray );
dt.ReadXml(stream);
However, the code failed with exception
System.Xml.XmlException: 'Data at the root level is invalid, position 1

Actually, your WebAPI didn't respond with a valid XML payload but return a JSON instead :
{
"DataTable.RemotingVersion": {...},
"XmlSchema":"... xml schema string ...",
"XmlDiffGram": " ... xml data string ..."
}
So you can't read the whole string by dt.ReadXml(stream);
Also note that the xml string begins with an empty string. It's better to invoke .Trim() before reading the XML string.
A working demo:
Let's create two helper methods that read the schema & data parts into DataTable:
// read the schema into DataTable
public static void ReadSchema(DataTable dt,string schema)
{
var stream = new MemoryStream();
ReadXMLToMemoryStream(schema, stream);
dt.ReadXmlSchema(stream);
}
// read the data into DataTable
public static void ReadData(DataTable dt,string xml)
{
var stream = new MemoryStream();
ReadXMLToMemoryStream(xml,stream);
dt.ReadXml(stream);
}
// read xml string into MemoryStream
private static void ReadXMLToMemoryStream(string xml, MemoryStream stream)
{
var doc = new XmlDocument();
doc.LoadXml(xml.Trim());
doc.Save(stream);
stream.Position = 0;
}
To fix the issue, we need parse the related json properties (I'm using Newtonsoft.Json) :
byte[] bytes = Encoding.ASCII.GetBytes(strReturnedFromWebApi);
MemoryStream ms = new MemoryStream(bytes);
using(var reader = new JsonTextReader(new StreamReader(ms))){
var json = JToken.Load(reader);
var version= json.SelectToken("$.['DataTable.RemotingVersion']");
var schema = (string)json.SelectToken("$.XmlSchema"); // the schema part
var data = (string)json.SelectToken("$.XmlDiffGram"); // the data part
System.Data.DataTable dt = new System.Data.DataTable();
ReadSchema(dt,schema); // read schema
ReadData(dt,data); // read data
}

Related

iText7: Error at file pointer when merging two pdfs

We are in the last steps of evaluating iText7. We use iText 7.1.0 and html2pdf 2.0.0.
What we do: we send a json_encoded collection with pdf-data (which includes html for header, body and footer) to our Java app. There we iterate over the collection, create a byteArrayOutputStream for each pdf-data element and merge them together. We then send the results to a script which echoes it to e.g. a browser. Although the pdf is displayed correctly, we encounter errors while creating it:
com.itextpdf.io.IOException: Error at file pointer 226,416.
...
Caused by: com.itextpdf.io.IOException: xref subsection not found.
... 73 common frames omitted
If we create only one part of the collection, no error is thrown.
Iterate over collection and merge:
#RequestMapping(value = "/pdf", method = RequestMethod.POST, produces = MediaType.APPLICATION_PDF_VALUE)
public byte[] index(#RequestBody PDFDataModelCollection elements, Model model) throws IOException {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
PdfWriter writer = new PdfWriter(byteArrayOutputStream);
try (PdfDocument resultDoc = new PdfDocument(writer)) {
for (PDFDataModel pdfDataModel : elements.getElements()) {
PdfReader reader = new PdfReader(new ByteArrayInputStream(creationService.createDatasheet(pdfDataModel)));
try (PdfDocument sourceDoc = new PdfDocument(reader)) {
int n = sourceDoc.getNumberOfPages(); //<-- IOException on second iteration
for (int i = 1; i <= n; i++) {
PdfPage page = sourceDoc.getPage(i).copyTo(resultDoc);
resultDoc.addPage(page);
}
}
}
}
return byteArrayOutputStream.toByteArray(); //outputs the final pdf
}
Creation of part:
public byte[] createDatasheet(PDFDataModel pdfDataModel) throws IOException {
PdfWriter writer = new PdfWriter(byteArrayOutputStream);
//Initialize PDF document
PdfDocument pdfDoc = new PdfDocument(writer);
try (
Document document = new Document(pdfDoc)
) {
//header, footer, etc
//body
for (IElement element : HtmlConverter.convertToElements(pdfDataModel.getBody(), this.props)) {
document.add((IBlockElement) element);
}
footer.writeTotalNumberOnPages(pdfDoc);
}
return byteArrayOutputStream.toByteArray();
}
We are grateful for any suggestion.
In createDatasheet you appear to re-use some byteArrayOutputStream without clearing it first.
In the first iteration, therefore, everything works as desired, at the end of createDatasheet you have a single PDF file in it.
In the second iteration, though, you have two PDF files in that byteArrayOutputStream, one after the other. This concatenation does not form a valid single PDF.
Thus, byteArrayOutputStream.toByteArray() returns something broken.
To fix this, either make the byteArrayOutputStream local to createDatasheet and create a new instance every time or alternatively reset byteArrayOutputStream at the start of createDatasheet:
public byte[] createDatasheet(PDFDataModel pdfDataModel) throws IOException {
byteArrayOutputStream.reset();
PdfWriter writer = new PdfWriter(byteArrayOutputStream);
[...]

How to acquire an XML Element Value using C#

The XML is being returned by a web API and it looks like this (but with more elements).
<?xml version="1.0" encoding="UTF-8"?>
<root response="True">
<movie title="TRON" />
</root>
I have C# that can query the Web API and then display the XML in the console. I need to be able to just display a specific element's value. For this example, I want to display the "title" element's value.
I have this C# code that just returns a blank console window.
// Process the XML HTTP response
static public void ProcessResponse(XmlDocument MovieResponse)
{
//This shows the contents of the returned XML (MovieResponse) in the console window//
//Console.WriteLine(MovieResponse.InnerXml);
//Console.WriteLine();
XmlNamespaceManager nsmgr = new XmlNamespaceManager(MovieResponse.NameTable);
XmlNode mTitle = MovieResponse.SelectSingleNode("/root/movie/title", nsmgr);
Console.WriteLine(mTitle);
Console.ReadLine();
}
something along these lines:
public List<string> GetMovieTitle(XDocument xdoc)
{
string xpath = #"//root/movie/title";
var query = xdoc.XPathSelectElements(xpath).Select(t => t.Value);
return query.ToList<string>();
}
More options can be found here : http://www.intertech.com/Blog/query-an-xml-document-using-linq-to-xml/
Edit using XmlDocument :
static public void ProcessResponse(XmlDocument MovieResponse)
{
string xpath = #"//root/movie/#title";
var query = MovieResponse.SelectSingleNode(xpath).Value;
Console.WriteLine(query) ;
Console.ReadLine();
}

How to fill XFA from using iText so it is Foxit Reader comptible

I used examples available on web to create an application that is able to get xml structure of XFA form and then set it back filled. Important code looks like this:
public void readData(String src, String dest)
throws IOException, ParserConfigurationException, SAXException,
TransformerFactoryConfigurationError, TransformerException {
FileOutputStream os = new FileOutputStream(dest);
PdfReader reader = new PdfReader(src);
XfaForm xfa = new XfaForm(reader);
Node node = xfa.getDatasetsNode();
NodeList list = node.getChildNodes();
for (int i = 0; i < list.getLength(); i++) {
if ("data".equals(list.item(i).getLocalName())) {
node = list.item(i);
break;
}
}
Transformer tf = TransformerFactory.newInstance().newTransformer();
tf.setOutputProperty(OutputKeys.ENCODING, "UTF-8");
tf.setOutputProperty(OutputKeys.INDENT, "yes");
tf.transform(new DOMSource(node), new StreamResult(os));
reader.close();
}
public void fillPdfWithXmlData(String src, String xml, String dest)
throws IOException, DocumentException {
PdfReader.unethicalreading = true;
PdfReader reader = new PdfReader(src);
PdfStamper stamper = new PdfStamper(reader, new FileOutputStream(dest), '\0', true);
AcroFields form = stamper.getAcroFields();
XfaForm xfa = form.getXfa();
xfa.fillXfaForm(new FileInputStream(xml));
stamper.close();
reader.close();
}
When I use it to fill this form: http://www.vzp.cz/uploads/document/tiskopisy-pro-zamestnavatele-hromadne-oznameni-zamestnavatele-verze-2-pdf-56-kb.pdf it works fine and I can see the filled form in Acrobat Reader. However, if I open the document in Foxit Reader I see a blank form (tested in latest version and 5.x version).
I tried to play a little bit with it and got these XfaForm(...).getDomDocument() data:
Filled by Acrobat Reader: http://pastebin.com/kXKyh9EM
Filled by Foxit Reader: http://pastebin.com/tiZ7EmfE
Filled by iText: http://pastebin.com/tTKLMERC
Filled by Foxit Reader after a fill by iText: http://pastebin.com/Uuq0jS4b
The field which was filled is . Is it possible to use iText in a way that it works even with Foxit Reader (and the XFA signature stays?)
In your Filled by iText example there's a superfluous <xfa:data> element:
<xfa:datasets xmlns:xfa="http://www.xfa.org/schema/xfa-data/1.0/">
<xfa:data>
<xfa:data>
<HOZ>
<!-- rest of your data here -->
</HOZ>
</xfa:data>
</xfa:data>
<!-- data description -->
</xfa:datasets>
This is because the fillXfaForm() method of XfaForm expects the XML data without <xfa:data> as the root element. So your XML data should just look like:
<HOZ>
<!-- rest of your data here -->
</HOZ>
I see that your readData() method that extract the existing form data including the <xfa:data> element:
<xfa:data>
<HOZ>
<!-- rest of your data here -->
</HOZ>
</xfa:data>
Stripping the outer element should fix your problem. For example:
XfaForm xfa = new XfaForm(reader);
Node node = xfa.getDatasetsNode();
NodeList list = node.getChildNodes();
for (int i = 0; i < list.getLength(); i++) {
if ("data".equals(list.item(i).getLocalName())) {
node = list.item(i);
break;
}
}
// strip <xfa:data>
node = node.getFirstChild();
// Transformer code here

How to calculate Amazon's MD5 hash for http request

Does anyone know how to calculate the MD5 hash that is needed to be used with Amazon's SubmitFeed API? I am using ColdFusion and every time I calculate the MD5 hash on my end it never matches what Amazon calculates.
Amazon responds with this error:
ContentMD5DoesNotMatch
the Content-MD5 HTTP header you passed for your feed (C7EF1CADB27497B46FCD6F69516F96E0) did not match the Content-MD5 we calculated for your feed (x+8crbJ0l7RvzW9pUW+W4A==)
I am using the built-in function that ColdFusion uses for hashing (example hash(myStr)). Is there a step I am missing?
public any function EncryptSignature(required string argValue,required string publicKey) hint="I create my own signature that I will matching later." {
local.filters=StructNew();
local.filters["F:publicKey"]=arguments.publicKey;
var jMsg=JavaCast("string",arguments.argValue).getBytes("iso-8859-1");
var thisSecretKey = getDAO().getSecretKey(local.filters).apiSecretKey;
var jKey=JavaCast("string",thisSecretKey).getBytes("iso-8859-1");
var key=createObject("java","javax.crypto.spec.SecretKeySpec");
var mac=createObject("java","javax.crypto.Mac");
key=key.init(jKey,"HmacSHA1");
mac=mac.getInstance(key.getAlgorithm());
mac.init(key);
mac.update(jMsg);
return lCase(binaryEncode(mac.doFinal(),'Hex'));
//return Encrypt(arguments.argValue,getapiUsersDAO().getSecretKey(arguments.publicKey),'HMAC-SHA1');
}
The problem is that Feed must have preceding
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
Even adding it as a Declaration is not enough
var memoryStream = new MemoryStream();
XDocument doc = new XDocument(
new XDeclaration("1.0", "utf-8", "yes"),
new XElement("AmazonEnvelope",
...
if you are using XmlWriter:
using (var xmlWriter = XmlWriter.Create(memoryStream))
{
doc.WriteTo(xmlWriter);
}
You have to save XDocument to file and then get stream from file. Only in this case XDocument preserves declaration (designed behaviour of Save() and WriteTo() methods):
var memoryStream = new MemoryStream();
doc.Save(memoryStream);
var file = Path.GetTempFileName();
using (var fileStream = File.OpenWrite(file))
{
var buffer = memoryStream.GetBuffer();
fileStream.Write(buffer, 0, (int)memoryStream.Length);
}
return File.Open(file, FileMode.Open, FileAccess.Read);
I checked this online tool and you just need to send that MD5 in base64 encoding. It is currently just hexadecimal encoded.
I'm afraid I don't know what the ColdFusion way to do that is, maybe this:
SHA or MD5 Digests in ColdFusion
Here is what I did to get this to work:
<cfset getMD5 = ToBase64(binaryDecode(hash(xmlRequest),'hex'))>
And bang it matched Amazons MD5 hash.
Here is an alternative java-way
found on http://www.kba.suche-spezialwerkzeug.de/pdf/MWSDeveloperGuide.pdf
public static String computeContentMD5HeaderValue(FileInputStream fis)
throws IOException, NoSuchAlgorithmException {
DigestInputStream dis = new DigestInputStream(fis,
MessageDigest.getInstance("MD5"));
byte[] buffer = new byte[8192];
while (dis.read(buffer) > 0)
;
String md5Content = new String(
org.apache.commons.codec.binary.Base64.encodeBase64(dis.getMessageDigest().digest())
);
// Effectively resets the stream to be beginning of the file via a
// FileChannel.
fis.getChannel().position(0);
return md5Content;
}
This will give your desire output.
<cfset binaryValue = binaryDecode( 'C7EF1CADB27497B46FCD6F69516F96E0', "hex" )>
<cfset base64Value = binaryEncode( binaryValue, "base64" )>
<cfdump var="#base64Value#">

How to best detect encoding in XML file?

To load XML files with arbitrary encoding I have the following code:
Encoding encoding;
using (var reader = new XmlTextReader(filepath))
{
reader.MoveToContent();
encoding = reader.Encoding;
}
var settings = new XmlReaderSettings { NameTable = new NameTable() };
var xmlns = new XmlNamespaceManager(settings.NameTable);
var context = new XmlParserContext(null, xmlns, "", XmlSpace.Default,
encoding);
using (var reader = XmlReader.Create(filepath, settings, context))
{
return XElement.Load(reader);
}
This works, but it seems a bit inefficient to open the file twice. Is there a better way to detect the encoding such that I can do:
Open file
Detect encoding
Read XML into an XElement
Close file
Ok, I should have thought of this earlier. Both XmlTextReader (which gives us the Encoding) and XmlReader.Create (which allows us to specify encoding) accepts a Stream. So how about first opening a FileStream and then use this with both XmlTextReader and XmlReader, like this:
using (var txtreader = new FileStream(filepath, FileMode.Open))
{
using (var xmlreader = new XmlTextReader(txtreader))
{
// Read in the encoding info
xmlreader.MoveToContent();
var encoding = xmlreader.Encoding;
// Rewind to the beginning
txtreader.Seek(0, SeekOrigin.Begin);
var settings = new XmlReaderSettings { NameTable = new NameTable() };
var xmlns = new XmlNamespaceManager(settings.NameTable);
var context = new XmlParserContext(null, xmlns, "", XmlSpace.Default,
encoding);
using (var reader = XmlReader.Create(txtreader, settings, context))
{
return XElement.Load(reader);
}
}
}
This works like a charm. Reading XML files in an encoding independent way should have been more elegant but at least I'm getting away with only one file open.
Another option, quite simple, is to use Linq to XML. The Load method automatically reads the encoding from the xml file. You can then get the encoder value by using the XDeclaration.Encoding property.
An example from MSDN:
// Create the document
XDocument encodedDoc16 = new XDocument(
new XDeclaration("1.0", "utf-16", "yes"),
new XElement("Root", "Content")
);
encodedDoc16.Save("EncodedUtf16.xml");
Console.WriteLine("Encoding is:{0}", encodedDoc16.Declaration.Encoding);
Console.WriteLine();
// Read the document
XDocument newDoc16 = XDocument.Load("EncodedUtf16.xml");
Console.WriteLine("Encoded document:");
Console.WriteLine(File.ReadAllText("EncodedUtf16.xml"));
Console.WriteLine();
Console.WriteLine("Encoding of loaded document is:{0}", newDoc16.Declaration.Encoding);
While this may not server the original poster, as he would have to refactor a lot of code, it is useful for someone who has to write new code for their project, or if they think that refactoring is worth it.