I need to preprocess a SOAP request (using HTTP request) populated using a CSV dataset in jMeter.
Some of the binded variables are null which is expected, but the service I am testing performs validation on these elements as well.
There was a similar question in the past (JMeter - Remove empty elements in SOAP request), but in my case I am not looking to only clear the blank element I want to clear the whole block.
Example:
I want this
<table>
<name>businessLicenceGroup</name>
<tableRow>
<sequence>1</sequence>
<field>
<sequence>0</sequence>
<name>authority</name>
<value>AD01</value>
</field>
<field>
<sequence>1</sequence>
<name>licenceNumber</name>
<value></value>
</field>
</tableRow>
</table>
to be
<table>
<name>businessLicenceGroup</name>
<tableRow>
<sequence>1</sequence>
<field>
<sequence>0</sequence>
<name>authority</name>
<value>AD01</value>
</field>
</tableRow>
</table>
The reference code which looks for an empty objects and removes their parent nodes would be something like:
def before = prev.getResponseDataAsString()
log.info('Before: ' + before)
def xml = new groovy.util.XmlParser().parseText(before)
def nodesWithoutText = xml.'**'.findAll { it.name() && !it.text() }
def removeNode = { node ->
def field = node.parent()
def tableRow = field.parent()
tableRow.remove(field)
}
nodesWithoutText.each{removeNode(it)}
log.info('After: ' + groovy.xml.XmlUtil.serialize(xml))
The above code assumes that your example XML comes as the response of a Sampler and needs to be put into JSR223 PostProcessor
Demo:
References:
Groovy: Processing XML
Apache Groovy - Why and How You Should Use It
Maybe you can use this groovy script for JSR223 PreProcessor. It's based on following articles. Regard. that the result has not the same format like the input xml.
Modifying / Removing nodes with Groovy
Groovy Goodness: Pretty Print XML
I found this after posting the answer. It might also help.
Groovy XmlSlurper replace node with given child node value
import groovy.xml.*
def xml = """
<table>
<name>businessLicenceGroup</name>
<tableRow>
<sequence>1</sequence>
<field>
<sequence>0</sequence>
<name>authority</name>
<value>AD01</value>
</field>
<field>
<sequence>1</sequence>
<name>licenceNumber</name>
<value></value>
</field>
</tableRow>
</table>
"""
log.info("XML before removing Node: " + xml);
def table = new XmlParser().parseText(xml);
log.info("Parsed XML before removing Node: " + table);
log.info("Node to remove: " + table.tableRow.field[1]);
// remove the node by replacement with empty element
table.tableRow.field[1].replaceNode {};
// Create Output
log.info("Parsed XML after removing Node: " + table);
def xmlOutput = new StringWriter()
def xmlNodePrinter = new XmlNodePrinter(new PrintWriter(xmlOutput))
xmlNodePrinter.print(table)
log.info("Parsed XML after removing Node: " + xmlOutput.toString());
Related
May be this is a basic question, but I have trouble binding the OData count in XML view.
In the following example, I want to bind the count of products from the OData model.
<List items="{/Categories}"} >
<ObjectListItem
title="{CategoryName}"
number="{path : 'Products/$count'}"
numberUnit="Products"/>
</List>
Each category needs to display count of products in the respective category as in
/Categories(1)/Products/$count
/Categories(2)/Products/$count
I had a similar issue. Although I am not thrilled with my solution, it uses expression binding and works without the need for a separate formatter:
<List items="{/Categories}"} >
<ObjectListItem
title="{CategoryName}"
number="{= ${Products}.length }"
numberUnit="Products" />
</List>
Like #Jasper_07, you still need to include Products in the expand, but you are ignoring most of the data coming back.
I dont think its currently possible
- $count is an OData query option, the equivalent in ODataListBinding is length, eg Products.length I cant think of a way to bind to it
you can achieve the count in a couple of ways using a formatter
option 1 - the simplest, create a list binding which reads the total number of products, it does a synchronous call and returns only the $count
function productCount(oValue) {
//return the number of products linked to Category // sync call only to get $count
if (oValue) {
var sPath = this.getBindingContext().getPath() + '/Products';
var oBindings = this.getModel().bindList(sPath);
return oBindings.getLength();
}
};
<List items="{/Categories}"} >
<ObjectListItem
title="{CategoryName}"
number="{path : 'CategoryName',formatter:'productCount'}"
numberUnit="Products"
</ObjectListItem>
</List>
option 2 - use an expand and return a very small set of data, in this case only CategoryName and ProductID, the caveat here is whether you have to by pass table paging to get full list
function productCount(oValue) {
//read the number of products returned
if (oValue) {
return oValue.length;
}
};
<List items="{/Categories,parameters:{expand:'Products', select:'CategoryName,Products/ProductID'}}">
<ObjectListItem
title="{CategoryName}"
number="{path : 'Products',formatter:'productCount'}"
numberUnit="Products"
</ObjectListItem>
</List>
Well.. I had exactly the same requirement and didn't want to perform the clever solution from #jasper as it will load all Products collection from the oData service.
This was the way I solve it:
View
Use a controller
Give your list an ID
Use a function on list's updateFinished event.
<mvc:View
controllerName="view.Root"
xmlns:mvc="sap.ui.core.mvc"
xmlns="sap.m"
>
<List id="list"
headerText="Categories"
items="{/Categories}"
growing="true"
growingThreshold="4"
growingScrollToLoad="true"
updateFinished=".countProducts"
>
<ObjectListItem
title="{description}"
numberUnit="Products"
/>
</List>
</mvc:View>
Controller
Implement countProducts function
Use jQuery to request the $count for each list item - Notice how the URL is generated concatenating model's service URL with the item's binding context
As jQuery uses asynchronous requests, by the time you get the first response, your for will be finished. So it can use IIFE to avoid filling just the last list item with your AJAX response
countProducts: function(e){
var m = sap.ui.getCore().getModel();
var items = this.byId("list").getItems();
for (var item_index = 0; item_index < items.length; item_index++) {
var item = items[item_index];
(function(_item) {
$.get(
m.sServiceUrl + _item.getBindingContextPath() + "/Categorias/$count",
function(count) {
_item.setNumber(count);
}
);
})(item);
}
}
I´d another solution using Manifest.json, Component.js and Controller.js for similar Issue.
First, I defined the Id in App.view.xml, for example:
<Title id="titleId" text="" level="H2"/>
After, I check Manifest.json, in especial:
{
"sap.app": {
"dataSources": {
"AXXX": {
"uri": "https://cors-anywhere.herokuapp.com/https://services.odata.org/Northwind/Northwind.svc/",
Next, in Componente.js at init:function() I put:
var oDataServiceUrl = this.getMetadata().getManifestEntry("sap.app").dataSources["AXXX"].uri;
console.log("oDataServiceUrl = ", oDataServiceUrl);
localStorage.setItem('oDataServiceUrl', oDataServiceUrl);
This code read Manifest.json and get Url to oDataService called AXXX.
Finnaly, I created one function in App Controller, such as:
countCustomersInAXXX : function (oEvent) {
var suffix = 'Customers/$count';
var oDataServiceUrl = localStorage.getItem('oDataServiceUrl');
var oDataServiceUri = oDataServiceUrl.concat(suffix);
console.log('App.controller.js: oDataServiceUri', oDataServiceUri);
var count = $.ajax({type: "GET", url: oDataServiceUri, async: false}).responseText;
console.log('App.controller.js: countCustomersInAXXX:' , count);
this.getView().byId("titleId").setText(count);
}
This code get the quantity of Customers and set the value in titleId.
To start this process you can user a button or one event, in my case I use this Table property:
updateFinished="countCustomersInAXXX"
I am gonna to use Google App Script to fetch the programme list from the website of radio station.
How can I select the specified elements in the webpage by specifying the id of the element?
Therefore, I can get the programs in the webpage.
Edit, Dec 2013: Google has deprecated the old Xml service, replacing it with XmlService. The script in this answer has been updated to use the new service. The new service requires standard-compliant XML & HTML, while the old one was forgiving of such problems as missing close-tags.
Have a look at the Tutorial: Parsing an XML Document. (As of Dec 2013, this tutorial is still on line, although the Xml service is deprecated.) Starting with that foundation, you can take advantage of the XML parsing in Script Services to navigate the page. Here's a small script operating on your example:
function getProgrammeList() {
txt = '<html> <body> <div> <div> <div id="here">hello world!!</div> </div> </div> </html>'
// Put the receieved xml response into XMLdocument format
var doc = Xml.parse(txt,true);
Logger.log(doc.html.body.div.div.div.id +" = "
+doc.html.body.div.div.div.Text ); /// here = hello world!!
debugger; // Pause in debugger - examine content of doc
}
To get the real page, start with this:
var url = 'http://blah.blah/whatever?querystring=foobar';
var txt = UrlFetchApp.fetch(url).getContentText();
....
If you look at the documentation for getElements you'll see that there is support for retrieving specific tags, for example "div". That finds direct children of a specific element, it doesn't explore the entire XML document. You should be able to write a function that traverses the document examining the id of each div element until it finds your programme list.
var programmeList = findDivById(doc,"here");
Edit - I couldn't help myself...
Here's a utility function that will do just that.
/**
* Find a <div> tag with the given id.
* <pre>
* Example: getDivById( html, 'tagVal' ) will find
*
* <div id="tagVal">
* </pre>
*
* #param {Element|Document}
* element XML document or element to start search at.
* #param {String} id HTML <div> id to find.
*
* #return {XmlElement} First matching element (in doc order) or null.
*/
function getDivById( element, id ) {
// Call utility function to do the work.
return getElementByVal( element, 'div', 'id', id );
}
/**
* !Now updated for XmlService!
*
* Traverse the given Xml Document or Element looking for a match.
* Note: 'class' is stripped during parsing and cannot be used for
* searching, I don't know why.
* <pre>
* Example: getElementByVal( body, 'input', 'value', 'Go' ); will find
*
* <input type="submit" name="btn" value="Go" id="btn" class="submit buttonGradient" />
* </pre>
*
* #param {Element|Document}
* element XML document or element to start search at.
* #param {String} elementType XML element type, e.g. 'div' for <div>
* #param {String} attr Attribute or Property to compare.
* #param {String} val Search value to locate
*
* #return {Element} First matching element (in doc order) or null.
*/
function getElementByVal( element, elementType, attr, val ) {
// Get all descendants, in document order
var descendants = element.getDescendants();
for (var i =0; i < descendants.length; i++) {
var elem = descendants[i];
var type = elem.getType();
// We'll only examine ELEMENTs
if (type == XmlService.ContentTypes.ELEMENT) {
var element = elem.asElement();
var htmlTag = element.getName();
if (htmlTag === elementType) {
if (val === element.getAttribute(attr).getValue()) {
return element;
}
}
}
}
// No matches in document
return null;
}
Applying this to your example, we get this:
function getProgrammeList() {
txt = '<html> <body> <div> <div> <div id="here">hello world!!</div> </div> </div> </html>'
// Get the receieved xml response into an XML document
var doc = XmlService.parse(txt);
var found = getDivById(doc.getElement(),'here');
Logger.log(found.getAttribute(attr).getValue()
+ " = "
+ found.getValue()); /// here = hello world!!
}
Note: See this answer for a practical example of the use of these utilities.
Someone has made an example here where the following custom functions are available for cut & paste use:
getElementById()
getElementsByClassName()
getElementsByTagName()
Then you can do something like this
function doGet() {
var html = UrlFetchApp.fetch('http://en.wikipedia.org/wiki/Document_Object_Model').getContentText();
var doc = XmlService.parse(html);
var html = doc.getRootElement();
var menu = getElementsByClassName(html, 'menu-classname')[0];
return menu;
}
I'm going to assume that you are referring to using UrlFetchApp's fetch() method. In which case, the answer is no, in the context of what you are thinking of.
If you look at the return type for fetch() in the documentation it returns HTTPResponse. There are a few methods for that, but most of them involve getting the returned data as a string. The good news is, you could still use any (well, most) of the traditional JS String methods documented here - so you could use search(), match(), etc. Depending on your project you could use those to find the data you are looking for in the response.
I'm looking for a way to declare form default values dynamically in a CFC, I'm calling via AJAX. The current CFC sends orders, which I need to break down into sub-orders.
I had been using this:
<!--- static defaults --->
<cffunction name="Defaults" access="public" returntype="struct" output="false"
hint="Assign default values to instance">
<cfscript>
var formDefaults = {
versenden=""
, speichern=""
...
}
</cfscript>
<cfreturn formDefaults />
</cffunction>
<cffunction name="Commit" access="public" returntype="struct" output="false" hint="database handler">
<!--- add dynamic form fields --->
<cfscript>
var LOCAL = {};
variables.defs = THIS.Defaults();
</cfscript>
<cfloop collection="#VARIABLES.Instance.FormData#" item="formField">
<cfscript>
if ( LEFT(formField, 5) EQ "MENGE"
OR LEFT(formField, 3) EQ "EAN"
OR LEFT(formField, 12) EQ "BESTELL_TEXT"
OR LEFT(formField, 10) EQ "BESTELLTYP"
...
) {
variables.defs[formField]="";
}
</cfscript>
</cfloop>
<cfscript>
structAppend(variables.defs, VARIABLES.Instance.FormData);
LOCAL.Basket = variables.defs;
</cfscript>
...
So I first declare static form fields (single instance only) and then try to dynamically append dynamic form fields to my array, which might be transferred multiple times (MENGE38, MENGE39, MENGE40 etc)
While this works ok, I need to add another counting element to my form-names, so I would have to change MENGE to something like counter.MENGE or MENGE.counter which will then send form values like this:
MENGE.1.38
MENGE.1.40
MENGE.1.41
MENGE.2.37
With the counter denoting the sub-order, this field is used for.
Problem is, this breaks my dynamic form field declaration and I don't understand why. I'm getting the following errors:
Diagnose: Element MENGE.1 is undefined in a CFML structure referenced as part of an expression.
Question:
Can anyone give me a hint on what the problem might be? Do I have to param the form fields on the HTML page as well (shouldn't have to)?
Thanks!
EDIT:
Problem was in my validate function, I also need to declare the modifications I did above. The new function looks like this:
<cffunction name="Validate" access="public" returntype="array" output="false" hint="validate form inputs and return an array of faulty field names.">
<cfscript>
var LOCAL = {};
var double = structNew();
double.form = VARIABLES.Instance.FormData;
double.criteria = VARIABLES.Instance.Validation;
</cfscript>
<!--- add dynamic form fields for validation... I FORGOT TO UPDATE THIS--->
<cfloop collection="#VARIABLES.Instance.FormData#" item="formField">
<cfscript>
if ( LEFT(formField, 5) EQ "MENGE"
OR LEFT(formField, 10) EQ 'BESTELLTYP'
OR LEFT(formField, 3) EQ "EAN"
OR LEFT(formField, 12) EQ "BESTELL_TEXT"
...
) {
VARIABLES.Instance.Validation[formField]="pass";
}
</cfscript>
</cfloop>
<!--- Get error names and type --->
<cfinvoke component="form_validate" method="validate_fields" double="#double#" returnvariable="validation_errors"></cfinvoke>
<cfset LOCAL.ErrorMessages = validation_errors />
<cfreturn LOCAL.ErrorMessages />
Because I did not add the new updated the if-clause in this function, I was getting the error.
To build on Dan Bracuk's answer, use underscores (though you'd need to change the name of "BESTELL_TEXT"). Use this with a combination of listFirst, listGetAt, and listLast to determine field name structure, using underscore as delimiter. Note how I cleaned up your big IF a bit using list function. This code as written probably doesn't do what you need, but wanted to illustrate the concepts without having to understand your business need.
<cfscript>
var orders=structNew();
item=listFirst(formField,'_');
orderNames = "MENGE,EAN,BESTELLTEXT,BESTELLTYPE";
if (listFindNoCase(orderNames,item,'_')){
if (!structKeyExists(orders,item)){
// initialize item
orders[item]=structNew();
}
orderID="";
subOrderId="";
if (listLen(formField,'_') gt 1) {
orderID=listGetAt(formField,2,'_');
}
if (listLen(formField,'_') eq 2) {
orders[item][orderId]=formData[formField];
}
if (listLen(formField,'_') eq 3) {
subOrderId=listLast(formField,'_');
orders[item][orderId][subOrderId]=formData[formField];
}
}
</cfscript>
I've searched everywhere and what I most found was doc.xpath('//element[#class="classname"]'), but this does not work no matter what I try.
code I'm using
import lxml.html
def check():
data = urlopen('url').read();
return str(data);
doc = lxml.html.document_fromstring(check())
el = doc.xpath("//div[#class='test']")
print(el)
It simply prints an empty list.
Edit:
How odd. I used google as a test page and it works fine there, but it doesn't work on the page I was using (youtube)
Here's the exact code I'm using.
import lxml.html
from urllib.request import urlopen
import sys
def check():
data = urlopen('http://www.youtube.com/user/TopGear').read(); #TopGear as a test
return data.decode('utf-8', 'ignore');
doc = lxml.html.document_fromstring(check())
el = doc.xpath("//div[#class='channel']")
print(el)
The TopGear page that you use for testing doesn't have any <div class="channel"> elements. But this works (for example):
el = doc.xpath("//div[#class='channel-title-container']")
Or this:
el = doc.xpath("//div[#class='a yb xr']")
To find <div> elements with a class attribute that contains the string channel, you could use
el = doc.xpath("//div[contains(#class, 'channel')]")
You can use lxml.cssselect to simplify class and id request: http://lxml.de/dev/cssselect.html
HTML uses classes (a lot), which makes them convenient to hook XPath queries. However XPath has no knowledge/support of CSS classes (or even space-separated lists) which makes classes a pain in the ass to check: the canonically correct way to look for elements having a specific class is:
//*[contains(concat(' ', normalize-space(#class), ' '), '$className')]
In your case this is
el = doc.xpath(
"//div[contains(concat(' ', normalize-space(#class), ' '), 'channel')]"
)
# print(el)
# [<Element div at 0x7fa44e31ccc8>, <Element div at 0x7fa44e31c278>, <Element div at 0x7fa44e31cdb8>]
or use own XPath function hasclass(*classes)
def _hasaclass(context, *cls):
return "your implementation ..."
xpath_utils = etree.FunctionNamespace(None)
xpath_utils['hasaclass'] = _hasaclass
el = doc.xpath("//div[hasaclass('channel')]")
I have code that is passed a string containing XML. This XML may contain one or more instances of (an entity reference for the blank space character). I have a requirement that these references should not be resolved (i.e. they should not be replaced with an actual space character).
Is there any way for me to achieve this?
Basically, given a string containing the XML:
<pattern value="[A-Z0-9 ]" />
I do not want it to be converted to:
<pattern value="[A-Z0-9 ]" />
(What I am actually trying to achieve is to simply take an XML string and write it to a "pretty-printed" file. This is having the side-effect of resolving occurrences of in the string to a single space character, which need to be preserved. The reason for this requirement is that the written XML document must conform to an externally-defined specification.)
I have tried creating a sub-class of XmlTextReader to read from the XML string and overriding the ResolveEntity() method, but this isn't called. I have also tried assigning a custom XmlResolver.
I have also tried, as suggested, to "double encode". Unfortunately, this has not had the desired effect, as the & is not decoded by the parser. Here is the code I used:
string schemaText = #"...<pattern value=""[A-Z0-9 ]"" />...";
XmlWriterSettings writerSettings = new XmlWriterSettings();
writerSettings.Indent = true;
writerSettings.NewLineChars = Environment.NewLine;
writerSettings.Encoding = Encoding.Unicode;
writerSettings.CloseOutput = true;
writerSettings.OmitXmlDeclaration = false;
writerSettings.IndentChars = "\t";
StringBuilder writtenSchema = new StringBuilder();
using ( StringReader sr = new StringReader( schemaText ) )
using ( XmlReader reader = XmlReader.Create( sr ) )
using ( TextWriter tr = new StringWriter( writtenSchema ) )
using ( XmlWriter writer = XmlWriter.Create( tr, writerSettings ) )
{
XPathDocument doc = new XPathDocument( reader );
XPathNavigator nav = doc.CreateNavigator();
nav.WriteSubtree( writer );
}
The written XML ends up with:
<pattern value="[A-Z0-9 ]" />
If you want it to be preserved, you need to double-encode it:  . The XML-reader will translate entities, that's more or less how XML works.
<pattern value="[A-Z0-9 ]" />
What I did above is replaced "&" with "&" thereby escaping the ampersand.