Mulesoft Insertion into Mongo DB database - mongodb

I am using a Mongo DB connector to insert data into Mongo Instance located in mLab.
I am sure I am using the correct credentials since these credentials are working when I am using them through a Java code.
But here using the Mongo DB connector constantly throws me an error as can be seen below.
org.mule.api.ConnectionException: Couldn't connect with the given
credentials org.mule.api.ConnectionException: Couldn't connect with
the given credentials at
org.mule.module.mongo.MongoCloudConnector.getDatabase(MongoCloudConnector.java:1304)
at
org.mule.module.mongo.MongoCloudConnector.connect(MongoCloudConnector.java:1173)
at
org.mule.module.mongo.connectivity.MongoCloudConnectorConnectionFactory.makeObject(MongoCloudConnectorConnectionFactory.java:56)
at
org.apache.commons.pool.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:1220)
at
org.mule.module.mongo.connectivity.MongoCloudConnectorConnectionManager.acquireConnection(MongoCloudConnectorConnectionManager.java:361)
at
org.mule.module.mongo.connectivity.MongoCloudConnectorConnectionManager.test(MongoCloudConnectorConnectionManager.java:444)
at
org.mule.tooling.metadata.api.utils.ConnectionTester.internalTestConnection(ConnectionTester.java:88)
at
org.mule.tooling.metadata.api.utils.ConnectionTester.testConnectionFor(ConnectionTester.java:113)
at
Is there something that I am missing here?
Below is the XML :
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:dw="http://www.mulesoft.org/schema/mule/ee/dw" xmlns:metadata="http://www.mulesoft.org/schema/mule/metadata" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:mongo="http://www.mulesoft.org/schema/mule/mongo" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/ee/dw http://www.mulesoft.org/schema/mule/ee/dw/current/dw.xsd
http://www.mulesoft.org/schema/mule/mongo http://www.mulesoft.org/schema/mule/mongo/current/mule-mongo.xsd">
<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration"/>
<http:request-config name="HTTP_Request_Configuration" host="developers.zomato.com" port="80" doc:name="HTTP Request Configuration"/>
<http:request-config name="HTTP_Request_Configuration1" host="api.mlab.com" port="80" doc:name="HTTP Request Configuration"/>
<mongo:config name="Mongo_DB" password="XXXXX" database="restaurant_data" host="ds241039.mlab.com" port="41039" doc:name="Mongo DB" username="XxXx"/>
<flow name="rest-webservice-applicationFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/rest" allowedMethods="GET" doc:name="HTTP"/>
<http:request config-ref="HTTP_Request_Configuration" path="api/v2.1/search" method="GET" doc:name="HTTP">
<http:request-builder>
<http:query-param paramName="entity_id" value="1"/>
<http:query-param paramName="entity_type" value="city"/>
<http:header headerName="user-key" value="XXXXXXXXX"/>
</http:request-builder>
</http:request>
<dw:transform-message doc:name="Transform Message" metadata:id="13f7b603-ac1e-45b4-9950-32c39a20ee36">
<dw:input-payload mimeType="application/json"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
{
results_found: payload.results_found,
results_start: payload.results_start,
results_shown: payload.results_shown,
restaurants: payload.restaurants map ((restaurant , indexOfRestaurant) -> {
restaurant: {
R: restaurant.restaurant.R,
id: restaurant.restaurant.id,
name: restaurant.restaurant.name,
url: restaurant.restaurant.url,
location: restaurant.restaurant.location,
switch_to_order_menu: restaurant.restaurant.switch_to_order_menu,
cuisines: restaurant.restaurant.cuisines,
average_cost_for_two: restaurant.restaurant.average_cost_for_two,
price_range: restaurant.restaurant.price_range,
currency: restaurant.restaurant.currency,
offers: restaurant.restaurant.offers map ((offer , indexOfOffer) -> offer),
thumb: restaurant.restaurant.thumb,
user_rating: restaurant.restaurant.user_rating,
photos_url: restaurant.restaurant.photos_url,
menu_url: restaurant.restaurant.menu_url,
featured_image: restaurant.restaurant.featured_image,
has_online_delivery: restaurant.restaurant.has_online_delivery,
is_delivering_now: restaurant.restaurant.is_delivering_now,
deeplink: restaurant.restaurant.deeplink,
has_table_booking: restaurant.restaurant.has_table_booking,
events_url: restaurant.restaurant.events_url
}
})
}]]></dw:set-payload>
</dw:transform-message>
<mongo:json-to-dbobject doc:name="Mongo DB"/>
<mongo:insert-object config-ref="Mongo_DB" doc:name="Mongo DB" collection="restaurant"/>
</flow>
</mule>

The MongoDB connector uses the deprecated MONGODB-CR authentication mechanism, MLab is expecting the SCRAM-SHA-1 mechanism.
The SCRAM-SHA-1 mechanism is available in the enterprise version of Mulesoft ESB.
The connection URI configuration is available on version 4.2.0 and above. This is how you configure it:
Add the connector to pom.xml:
<dependency>
<groupId>org.mule.connectors</groupId>
<artifactId>mule-mongo-connector</artifactId>
<version>4.2.0</version>
</dependency>
Configure a connection in your flow:
<mongo:config-connection-string name="MongoDB_Config"
connectionString="mongodb://jdoe:myPass#localhost:27017?authMechanism=SCRAM-SHA-1"/>

Related

Mule ESB Flow calling SOAP not working but worked from SOAP UI

Details : I have created on flow in MuleESB which is calling a web-service without any parameter just sending it username, password and token in a property and it is working fine.
But the second API I want to post some parameters while calling soap request but I don't know how to use it I tried to pass through set payload but no response.
<http:listener-config name="HTTP_Listener_Configuration" host="0.0.0.0" port="8081" doc:name="HTTP Listener Configuration"/>
<http:request-config name="HTTP_Request_Configuration" host="webservicehostadd" port="443" doc:name="HTTP Request Configuration">
<http:basic-authentication username="username" password="pass"/>
</http:request-config>
<ws:consumer-config name="Web_Service_Consumer" wsdlLocation="https://xxxx/1.0?wsdl" service="xxx" port="xxxx" serviceAddress="https://xxxxx/1.0" connectorConfig="HTTP_Request_Configuration" doc:name="Web Service Consumer"/>
<flow name="mycustomflow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/TEST" doc:name="HTTP"/>
<set-property propertyName="APIKey" value="xxx-xxx-xxx-xxx-xxx" doc:name="Property"/>
<dw:transform-message doc:name="Transform Message" metadata:id="xxx-xxx-xxx-xxx-xxxxxxxx">
<dw:input-payload mimeType="application/xml"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/xml
%namespace ns0 http://localhost/getDetails:getDetailsWSD
---
{
ns0#getDetails: {
getDetailsOrder: {
ID: payload.ns0#getDetails.getDetailsOrder.ID,
AllData: payload.ns0#getDetails.getDetailsOrder.AllData
}
}
}]]></dw:set-payload>
</dw:transform-message>
<ws:consumer config-ref="Web_Service_Consumer" operation="employeeDetails" doc:name="Web Service Consumer"/>
</flow>
It showed the below error:
Exception while executing:
[row,col]: [1,1]
Unexpected character '{' (code 123) in prolog; expected '<'
at [row,col {unknown-source}]: [1,1].
Updated answer:
<dw:transform-message metadata:id="XXXXX" doc:name="Transform Message">
<dw:set-payload><![CDATA[
%output application/xml skipNullOn="everywhere"
%namespace ns0 localhost/getDetails:getDetailsWSD
---
{
ns0#getDetails: {
getDetailsOrder: {
ID: payload.ns0#getDetails.getDetailsOrder.ID,
AllData: payload.ns0#getDetails.getDetailsOrder.AllData
}
}
}]]>
</dw:set-payload>
</dw:transform-message>
In your scenario: you are passing a body in XML format and sometimes an empty body in your Postman requests.
Passing an empty body results your payload to be {NullPayload}. To handle this, we have to remove explicitly defining the input mime type: <dw:input-payload mimeType="application/xml"/>.
In your transformation: ID: payload.ns0#getDetails.getDetailsOrder.ID,. You are retrieving a value from an empty payload and this will fail. To avoid failing, we have added: skipNullOn="everywhere". You can read more about it here.
I have tried the transformation myself which results to this:
<?xml version='1.0' encoding='UTF-8'?>
<ns0:getDetails xmlns:ns0="http://localhost/getDetails:getDetailsWSD">
<getDetailsOrder/>
</ns0:getDetails>
I think we are done with your initial issue regarding transformation of your empty payload. Your concern now is consuming the web service.
Thank you.

Can you update all Mongo documents in a collection at once in Mule?

I am trying to update the Effective date in my mongoDB collection with the current date. I want to update all documents in the collection where the Effective Date is null.
How do you configure the transformer to do this? The Query Reference field info message talks about an Id and the element field looks like it needs a payload containing a document that will replace the one being updated. I am just looking to update one field and currently getting an error with my query but it does work on the mongoDB command line.
This is my query that I want to implement:
db.stores.update({"EffectiveEndDateTime" : null}, {$set : {"EffectiveEndDateTime" : "2017-01-13T18:56:55.257Z"}})
My XML configure for Mongo in Mule so far:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:mongo="http://www.mulesoft.org/schema/mule/mongo" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:json="http://www.mulesoft.org/schema/mule/json"
xmlns:tracking="http://www.mulesoft.org/schema/mule/ee/tracking"
xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/ee/tracking http://www.mulesoft.org/schema/mule/ee/tracking/current/mule-tracking-ee.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/mongo http://www.mulesoft.org/schema/mule/mongo/current/mule-mongo.xsd">
<mongo:config name="Mongo_DB__Configuration" username="${mongodb.username}" password="${mongodb.password}" database="${mongodb.database}" host="${mongodb.host}" doc:name="Mongo DB: Configuration"/>
<flow name="deactivateCurrentDocumentsFlow">
<http:listener config-ref="httpListenerConfig" path="/mongo" allowedMethods="POST" doc:name="HTTP"/>
<mongo:update-documents-by-function config-ref="Mongo_DB__Configuration" collection="stores" function="$set : {"EffectiveEndDateTime": "2017-01-13T17:51:08.153Z" }" doc:name="Mongo DB"/>
<object-to-string-transformer doc:name="Object to String"/>
<logger level="INFO" doc:name="Logger"/>
</flow>
</mule>
Error I am getting:
Message : Failed to invoke updateDocumentsByFunction.
Payload : {NullPayload}
Payload Type : org.mule.transport.NullPayload
Element : /deactivateCurrentDocumentsFlow/processors/0 # test --------------------------------------------------------------------------------
Root Exception stack trace:
com.mongodb.MongoWriteException: Unknown modifier: $set : {"EffectiveEndDateTime": "2017-01-13T17:51:08.153Z" }
Thanks
You have to use update documents by Functions variant.
<mongo:update-documents-by-functions config-ref="Mongo_DB"
collection="myCollection" functions="$set,{"key":123}">
</mongo:update-documents-by-functions>

WSO2 ESB: Custom URL

I have created my proxy with custom url, based on:
http://wso2.com/library/knowledge-base/2011/01/custom-urls-wso2-esb-proxy-services/
Calling this custom URL with my SOAP message results in an error, I can still use the original url.
custom: /services/wss/PlanningOphaalServiceProxy_v1
original: /services/PlanningOphaalServiceProxy_v1
The error:
TID: [0] [ESB] [2015-08-19 15:47:05,039] ERROR {org.apache.axis2.engine.AxisEngine} - InvalidSecurity {org.apache.axis2.engine.AxisEngine}
org.apache.axis2.AxisFault: InvalidSecurity
at org.apache.rampart.handler.PostDispatchVerificationHandler.invoke(PostDispatchVerificationHandler.java:151)
at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:261)
at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:167)
at org.apache.synapse.transport.passthru.ServerWorker.processEntityEnclosingRequest(ServerWorker.java:411)
at org.apache.synapse.transport.passthru.ServerWorker.run(ServerWorker.java:183)
at org.apache.axis2.transport.base.threads.NativeWorkerPool$1.run(NativeWorkerPool.java:172)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
TID: [0] [ESB] [2015-08-19 15:47:05,041] ERROR {org.apache.synapse.transport.passthru.ServerWorker} - Error processing POST request for : /services/wss/PlanningOphaalServiceProxy_v1 {org.apache.synapse.transport.passthru.ServerWorker}
org.apache.axis2.AxisFault: InvalidSecurity
Solved: This is not possible. CustomURI does not work in combination with WS-Security, according to WSO2 SUpport
I tried Custom URI and working fine after adding custom dispatched in axis2.xml
below is URI for my proxy
http://localhost:8280/GenericProxy
and Proxy code as below:
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="CustomProxy"
transports="https,http"
statistics="disable"
trace="disable"
startOnLoad="true">
<target>
<inSequence>
<property name="messageType"
value="application/xml"
scope="default"
type="STRING"/>
<log level="full">
<property name="################## Body - In Seq###################"
expression="$body"/>
</log>
<send>
<endpoint>
<address uri="http://localhost:8090/services/OracleStoredProcedure.SOAP12Endpoint/"/>
</endpoint>
</send>
</inSequence>
<outSequence>
<log level="full">
<property name="############## Res Seq ############" value="Response"/>
</log>
</outSequence>
</target>
<parameter name="ServiceURI">/GenericProxy</parameter>
<description/>
</proxy>
And logs as below
[2015-08-21 16:35:25,173] INFO - ProxyService Successfully created the Axis2 se
rvice for Proxy service : CustomProxy
[2015-08-21 16:35:41,405] INFO - LogMediator To: /GenericProxy, MessageID: urn:
uuid:8e35439e-4d28-95be-4e25eb385843, Direction: request, #################
# Body - In Seq################### = <soapenv:Body xmlns:soapenv="http://schemas
.xmlsoap.org/soap/envelope/"><emp>
<ID>sample</ID>
</emp></soapenv:Body>, Envelope: <?xml version="1.0" encoding="utf-8"?><soapenv:
Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body
><emp>
<ID>sample</ID>
</emp></soapenv:Body></soapenv:Envelope>
[2015-08-21 16:35:42,200] INFO - LogMediator To: http://www.w3.org/2005/08/addr
essing/anonymous, WSAction: , SOAPAction: , MessageID: urn:uuid:0431fb5f-4b03-47
41-727bdb3637d8, Direction: response, ############## Res Seq ############ =
Response, Envelope: <?xml version="1.0" encoding="utf-8"?><soapenv:Envelope xml
ns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><soapenv:En
velope>

Apache camel cxfrs—Can't find the request for <URL> Observer

I tried to develop a rest service and expose the same via Apache Camel's CXFRS. I followed all the steps given in http://camel.apache.org/cxfrs.html and also referred to many samples given. I already referred to the question Can't find the the request for url Observer, but in my case it is a simple rest request. Below are the Service class, Route class, and cxf context used:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:camel="http://camel.apache.org/schema/spring"
xmlns:cxf="http://camel.apache.org/schema/cxf" xmlns:context="http://www.springframework.org/schema/context"
xmlns:jaxrs="http://cxf.apache.org/jaxrs"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://camel.apache.org/schema/spring
http://camel.apache.org/schema/spring/camel-spring.xsd
http://camel.apache.org/schema/cxf
http://camel.apache.org/schema/cxf/camel-cxf.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd
http://cxf.apache.org/jaxrs
http://cxf.apache.org/schemas/jaxrs.xsd">
<import resource="classpath:META-INF/cxf/cxf.xml" />
<import resource="classpath:META-INF/cxf/cxf-servlet.xml" />
<context:annotation-config />
<!-- enable Spring #Component scan -->
<context:component-scan base-package="org.camelsample.rest" />
<cxf:rsServer id="rsServer" address="/rest"
serviceClass="org.camelsample.rest.service.SampleRestService"
loggingFeatureEnabled="true" loggingSizeLimit="20">
<cxf:providers>
<bean class="org.codehaus.jackson.jaxrs.JacksonJsonProvider" />
</cxf:providers>
</cxf:rsServer>
<camel:camelContext id="samplerestservice"
xmlns="http://camel.apache.org/schema/spring">
<contextScan />
<jmxAgent id="agent" createConnector="true" />
</camel:camelContext>
</beans>
The Service Class:
package org.camelsample.rest.service;
import javax.ws.rs.GET;
import javax.ws.rs.Path;
public class SampleRestService {
#GET
#Path("/")
public String sampleService() {
return null;
}
}
The Route Class:
package org.camelsample.rest.route;
import org.apache.camel.spring.SpringRouteBuilder;
public class SampleRestRoute extends SpringRouteBuilder {
#Override
public void configure() throws Exception {
// TODO Auto-generated method stub
from("cxfrs:bean:rsServer").log("Into Sample Route").setBody(constant("Success"));
}
}
But when I try to hit and test using http://localhost:8080/rest, I always get the following error message:
2015-05-29 13:38:37.920 WARN 6744 --- [nio-8080-exec-2] o.a.c.t.servlet.ServletController : Can't find the the request for http://localhost:8080/favicon.ico's Observer
2015-05-29 13:38:40.295 WARN 6744 --- [nio-8080-exec-3] o.a.c.t.servlet.ServletController : Can't find the the request for http://localhost:8080/rest's Observer
Am using Spring boot to test the rest sample.
Does it work with this URL instead ?
http://localhost:8181/cxf/rest
If you just use address="/rest" as your address then you will probably get the default Jetty port 8181 and default CXF servlet path /cxf as the base URL.
If you specifically want to use the URL you have given then try this instead:
address="http://0.0.0.0:8080/rest"

Mule HTTP java.io.IOException: Attempted read on closed stream

When I used two HTTP endpoints in a flow, Mule throws this exception. I've found the way to deal with this problem: use the second HTTP endpoint in asynchronous mode, but it is not a good way.
ERROR DefaultSystemExceptionStrategy [[testdemo].connector.http.mule.default.receiver.03]: Caught exception in Exception Strategy: Attempted read on closed stream.
java.io.IOException: Attempted read on closed stream.
at org.apache.commons.httpclient.AutoCloseInputStream.isReadAllowed(AutoCloseInputStream.java:183)
at org.apache.commons.httpclient.AutoCloseInputStream.read(AutoCloseInputStream.java:126)
at org.mule.model.streaming.DelegatingInputStream.read(DelegatingInputStream.java:58)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1025)
at org.mule.transformer.simple.ObjectToOutputHandler$3.write(ObjectToOutputHandler.java:76)
at org.mule.transport.http.HttpServerConnection.writeResponse(HttpServerConnection.java:315)
at org.mule.transport.http.HttpMessageReceiver$HttpWorker.run(HttpMessageReceiver.java:164)
at org.mule.work.WorkerContext.run(WorkerContext.java:311)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
my code :
<flow doc:name="httpPost" name="httpPost">
<http:inbound-endpoint doc:name="HTTP" keep-alive="true"
exchange-pattern="request-response" host="localhost" path="service"
port="8082" />
<http:body-to-parameter-map-transformer />
<!-- This component is just set to show the message accecpted from the request -->
<scripting:component>
<scripting:script engine="groovy">
println payload['name']
return payload['name']+'123'
</scripting:script>
</scripting:component>
</flow>
<flow doc:name="httpPost" name="client">
<http:inbound-endpoint doc:name="HTTP" name="httpClient"
exchange-pattern="request-response" host="localhost" path="client"
port="8082" encoding="UTF-8" />
<http:body-to-parameter-map-transformer />
<scripting:component>
<scripting:script engine="groovy">
payload['name'] = 'opasso'
def paramstr = ""
for( param in payload){
paramstr = paramstr + "&" + param.key+ "=" + param.value
}
println "querystr:$paramstr"
return paramstr.substring(1)
</scripting:script>
</scripting:component>
<http:outbound-endpoint address="http://localhost:8082/service"
exchange-pattern="request-response" contentType="application/x-www-form-urlencoded"
method="POST" encoding="UTF-8" />
<!-- This component is just set to show the message accecpted from the request -->
<scripting:component>
<scripting:script engine="groovy">
def msg = "return payload:$payload;".toString()
println msg
return payload
</scripting:script>
</scripting:component>
</flow>
The problem is related to the funky business you're doing in the scripting:component with the streaming message payload generated by the http:outbound-endpoint. The script probably consumes the input stream, leaving it in a state where it can't used anymore.
Try adding a <object-to-string-transformer /> right after the http:outbound-endpoint to deserialize the stream into a string so the payload can be used both in the scripting:component and by Mule.