When making REST endpoint call to MarkLogic, is there an option to get only the data in the response without the additional metadata?
I am trying to use to make such connections using tools like Tableau, Qlik, Denodo etc
Options I use in the POST body on the URL: http://localhost:8000/v1/search
<search xmlns="http://marklogic.com/appservices/search">
<options>
<extract-document-data selected="all"></extract-document-data>
<transform-results apply="raw" />
</options>
</search>
Result I get:
<search:response snippet-format="raw" total="150" start="1" page-length="10" selected="all" xmlns:search="http://marklogic.com/appservices/search">
<search:result index="1" uri="/doc/21_doc.xml" path="fn:doc("/doc/21_doc.xml")" score="0" confidence="0" fitness="0" href="/v1/documents?uri=%2Fdoc%2F21_doc.xml" mimetype="application/xml" format="xml">
<root>
<col1>data1</col1>
<col2>data1</col2>
<col3>data1</col3>
</root>
<search:extracted kind="element">
<root>
<col1>data1</col1>
<col2>data1</col2>
<col3>data1</col3>
</root>
</search:extracted>
</search:result>
<search:metrics>
<search:query-resolution-time>PT0.0005236S</search:query-resolution-time>
<search:snippet-resolution-time>PT0.0001001S</search:snippet-resolution-time>
<search:extract-resolution-time>PT0.0003971S</search:extract-resolution-time>
<search:total-time>PT0.0021503S</search:total-time>
</search:metrics>
</search:response>
Expected:
<search:extracted kind="element">
<root>
<col1>data1</col1>
<col2>data1</col2>
<col3>data1</col3>
</root>
</search:extracted>
Also, why am I getting data in both extracted and result elements?
Both snippeting and data extraction provide access to content. Use either, or use them for different purposes. If you only like to get extracted data, then use:
<transform-results apply="empty-snippet" />
HTH!
If you want the original documents, specify an Accept header of multipart-mixed and specify only the content value for the a category URI parameter.
See: https://docs.marklogic.com/REST/POST/v1/search
and https://docs.marklogic.com/guide/rest-dev/bulk#id_65903
As an alternative, you may find it easier to provide rows to Business Intelligence Tools by creating TDE indexes and paging over an Optic query with the /v1/rows endpoint.
See: https://docs.marklogic.com/guide/rest-dev/search#id_34628
and https://docs.marklogic.com/REST/POST/v1/rows
Hoping that helps,
Related
I'm using Azure, I have some blobs in a container, what I'm looking for is to copy the urls of these blobs in a json file using azure data factory or data flow.
heres as example of a url :
the expected result is :
[[{'ur':'urltObLOB1'},{'ur':'urltObLOB2'},{'ur':'urltObLOB3'},.....]
is there a way to achieve that using azure data factory or data flow please ?
I could not find an activity or out of the box method to achieve the ask. However if you look at the REST api
The List Blobs operation returns a list of the blobs under the specified container.
In version 2013-08-15 and newer, the EnumerationResults element
contains a ServiceEndpoint attribute specifying the blob endpoint, and
a ContainerName field specifying the name of the container. In
previous versions these two attributes were combined together in the
ContainerName field. Also in version 2013-08-15 and newer, the Url
element under Blob has been removed.
Checkout MS DOC List blobs and snapshots
An example shows the result of a listing operation that returns blobs and snapshots in a container named mycontainer.
The request URI is as follows:
GET https://myaccount.blob.core.windows.net/mycontainer?restype=container&comp=list&include=snapshots&include=metadata
Sample XML response contains a block for each blob in the container.
<Blob>
<Name>blob3.txt</Name>
<Url>https://myaccount.blob.core.windows.net/mycontainer/blob3.txt</Url>
<Properties>
<Last-Modified>Wed, 09 Sep 2009 09:20:03 GMT</Last-Modified>
<Etag>0x8CBFF45D911FADF</Etag>
<Content-Length>16384</Content-Length>
<Content-Type>image/jpeg</Content-Type>
<Content-Encoding />
<Content-Language />
<Content-MD5 />
<Cache-Control />
<x-ms-blob-sequence-number>3</x-ms-blob-sequence-number>
<BlobType>PageBlob</BlobType>
<LeaseStatus>locked</LeaseStatus>
</Properties>
<Metadata>
<Color>yellow</Color>
<BlobNumber>03</BlobNumber>
<SomeMetadataName>SomeMetadataValue</SomeMetadataName>
</Metadata>
</Blob>
You can find the blob URL between <Url> </Url>
Next you can parse the XML output to store the url values in an array variable in the pipeline.
I am currently trying to work with the SuiteTalk 2017_2_0 API for a new integration with NetSuite. I have all the basics rolling of single record retrieval, etc.; however, I am having a problem trying to figure out how to list all of a given object/type in the system.
Example: I want to list ALL InventoryItem data.
Not sure what I am missing. Does anyone have a sample SOAP doc?
What I did so far
I have been working primarily with types: Sales Order, Customer and Inventory Item.
Tried using the getAll call defined in the WSDL - only supports limited fields:
budgetCategory
campaign*
currency
etc.
Tried using getList but a set of internal IDs are required in the baseRef/RecordRef (INVALID_KEY_OR_REF - The specified key is invalid.)
Fails:
<urn:getList>
<urn1:baseRef xsi:type="core:RecordRef" type="salesOrder" />
</urn:getList>
Succeeds:
<urn:getList>
<urn1:baseRef xsi:type="core:RecordRef" type="salesOrder" internalId="1" />
<urn1:baseRef xsi:type="core:RecordRef" type="salesOrder" internalId="2" />
</urn:getList>
Tried formulating a search that would return all the data but the types I need are unavailable (or I haven't been able to figure it out)
Define a saved search that pulls the appropriate data, then call the saved search.
[WCS 7, FixPack 7, Feature Pack 6]
I need to generate a feed with certain catalog entry (product) information such as name, image(s), category, price, seo-url, descriptive attributes and so on.
To generate this feed i will use the scheduler framework and created a controller command which will be invoked after a certain time has passed.
Now i need to receive the catalog entry data of each item in a specific store/catalog.
As far as i know there are several ways to achieve this:
Access Beans
SOA (new Access Profile with specific SQL)
I tried to use Access Beans to get all necessary information but I got stuck with seo-url, price etc.
Are there more ways and what is the best practice to get a specific set of catalog entry attributes?
I suggest you study/investigate in your WCS Development environment how site map generation schedule command is implemented (SitemapGenerateCmd) and how it is calling the JSP template file (sitemap.jsp)
you need to modify a bit in your command to create a jsp template for your feed and call that template from your scheduler command you've created .
calling template command for messaging system, make sure to use following properties in ActionForward tag to register the JSP for messaging from back-end:
example:
<forward className="com.ibm.commerce.struts.ECActionForward"
name="SitemapIndexView" path="/SitemapIndex.jsp">
<set-property property="direct" value="true"/>
<set-property property="resourceClassName" value="com.ibm.commerce.messaging.viewcommands.MessagingViewCommandImpl"/>
<set-property property="interfaceName" value="com.ibm.commerce.messaging.viewcommands.MessagingViewCommand"/>
<set-property property="properties" value="storeDir=no"/>
<set-property property="implClassName" value="com.ibm.commerce.messaging.viewcommands.MessagingViewCommandImpl"/>
</forward>
then the logic for extracting data from your product/catalog beans will be handled inside the JSP and you can easily form the output data as you want (XML, CSV, JSON .. etc)
the advantages and benefits of using this way are you can leverage Commerce Server OOTB JSTL tags , WCF tags for retrieving all information and using wcf:url for SEO URLS OOTB even you can call BOD/SOA commands using <wcf:getData tag and finally your will get more structured design that can easily maintain and reuse in future .
sitemap.jsp is a good resource for you how to iterate through catalog and sub-catalog to extract product info.
hope that those help you find your solution. need some search and understanding of existing sitemap generation utility .
Thanks.
Abed
Using the Restful Server Module I do this operation:
GET /api/v1/(ClassName)?(Field)=(Val)&(Field)=(Val) - searches for matching database records
The api should get the urls of all images related to a certain product like this:
http://domain.com/api/v1/Product?ArticleID=myID
unfortunately the returned XML does not show the full information of the ProductImages. just the attributes href and id.
here the xml:
<?xml version="1.0" encoding="UTF-8"?>
<ArrayList totalSize="1">
<Product href="http://domain.com/api/v1/Product/9.xml">
<ArticleID>myID</ArticleID>
<ID>9</ID>
<ProductImages linktype="has_many" href="http://domain.com/api/v1/Product/9/ProductImages.xml">
<ProductImage href="http://domain.com/api/v1/ProductImage/265.xml" id="265"/>
<ProductImage href="http://domain.com/api/v1/ProductImage/268.xml" id="268"/>
<ProductImage href="http://domain.com/api/v1/ProductImage/271.xml" id="271"/>
</ProductImages>
</Product>
</ArrayList>
ProductImage has much more attributes actually which are not displayed in the xml. How to make them visible? I can see all ProductImage Attributes when doing e.g. this:
http://domain.com/api/v1/Product/9/ProductImages
I use curl to load the urls.
In the worst case I think I have to take the ProductImage urls from the XML (e.g. http://domain.com/api/v1/ProductImage/271.xml) and do multiple calls.
many thx for any help,
florian
You need to increase the relationDepth inside RestfulServer. You can do that by passing in the relationdepth query string parameter.
As an aside, the reason it works like this is to keep the load low on the server by not having to instantiate all the image objects. It also means bandwidth is slightly lower too.
Quick problem here that I don't know how to solve and thought you guys could give me a heads-up on which way to go. I have successfully pulled all my connection data using the LinkedIn REST API both in XML and JSON and dumped them (the former using the cPickle plugin). The issue is that I need to refer to a single field within the data and so decided to use XML as it seemed to be the easiest one to use by far. When I refer to the specific field in my .pickle file, it gives me the following error:
AttributeError: 'str' object has no attribute 'location'
However, opening the pickle file with notepad, I can see that all my connections do have their location field stored in the XML format. It's very strange!
Here's my referral code:
import cPickle
connections_data = 'linkedin_connections.pickle'
response = cPickle.load(open(connections_data))
print response
locations = [ec.location for ec in response]
I have a print function set up to show what's in my file and all of the data appears as a normal XML output using the people call of the REST API. The XML data appears as follows:
<person>
<id>ID_number</id>
<first-name>blah</first-name>
<last-name>blah</last-name>
<headline>Business Development and Sales Executive at Computaris</headline>
<picture-url>picture_url</picture-url>
<api-standard-profile-request>
<url>profile_request</url>
<headers total="1">
<http-header>
<name>x-li-auth-token</name>
<value>name</value>
</http-header>
</headers>
</api-standard-profile-request>
<site-standard-profile-request>
<url>request_url</url>
</site-standard-profile-request>
<location>
<name>location</name>
<country>
<code>country_code</code>
</country>
</location>
<industry>industry</industry>
Any help will be much appreciated.