I am trying to create a REST API with wso2 API_Manager to gather data from a Postgres database (learning purpose). I struggle doing so and I would like to know whether:
I did not understand wso2 components' roles properly (new techno and subject for me)
or there is an error in the way I configured the manager.
System setup
I used this official docker image, added postgres jdbc jar in /repository/components/lib/ and added the following in /repository/conf/datasources/master-datasources.xml:
<datasource>
<name>s0m3dAtabas3</name>
<description>The db used for testing purposes</description>
<definition type="RDBMS">
<configuration>
<url>jdbc:postgresql://sandor_postgres:5432/s0m3dAtabas3</url>
<driverClassName>org.postgresql.Driver</driverClassName>
<username>s0m3us3rfr0mdAtAMaj0r</username>
<password>N0t5uchAs1mple1</password>
<maxActive>80</maxActive>
<minIdle>5</minIdle>
<maxWait>60000</maxWait>
<defaultAutoCommit>false</defaultAutoCommit>
<testOnBorrow>true</testOnBorrow>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
I made sure than the postgres' container named sandor_postgres is accessible from wso2's with these credentials. In this database, I have a table called something. The image comes with the following UIs:
admin
publisher
store
Graphical API creation
I first followed the WorldBank tutorial which seemed crystal clear (though I am not quite sure where the data came from). I then tried to adapt it.
Step 1: Design
I used the database name as context (s0m3dAtabas3) v.1.0.0. Since the table is called something, the url pattern I end up with is /s0m3dAtabas3/1.0.0/something
Step 2: Implement
This is where things start to be confusing. No matter the resource path I use in the Endpoint (end point type REST), I get a 404 and the logs are not very helpful
http://192.168.8.111:8280 -> 404
http://192.168.8.111:8280/something -> 404
http://192.168.8.111:9443/tried_several -> Invalid - Error connecting to backend
http://192.168.8.111:8243/tried_several -> Invalid - Error connecting to backend
INFO - InboundDBSyncRequestEvent Running DB sync task.
INFO - LogMediator STATUS = Message dispatched to the main sequence. Invalid URL., RESOURCE = /s0m3dAtabas3/1.0.0
INFO - CarbonAuthenticationUtil 'admin#carbon.super [-1234]' logged in at [2019-10-29 11:42:31,030+0000]
INFO - CarbonAuthenticationUtil 'admin#carbon.super [-1234]' logged in at [2019-10-29 11:42:31,197+0000]
INFO - LogMediator STATUS = Message dispatched to the main sequence. Invalid URL., RESOURCE = /s0m3dAtabas3/bullshit
INFO - CarbonAuthenticationUtil 'admin#carbon.super [-1234]' logged in at [2019-10-29 11:48:30,649+0000]
INFO - CarbonAuthenticationUtil 'admin#carbon.super [-1234]' logged in at [2019-10-29 11:48:30,790+0000]
INFO - LogMediator STATUS = Message dispatched to the main sequence. Invalid URL., RESOURCE = /
INFO - InboundDBSyncRequestEvent Running DB sync task.
INFO - LogMediator STATUS = Message dispatched to the main sequence. Invalid URL., RESOURCE = /
Did I miss some important configuration step or wso2 API Manager is not the standalone component I thought it was and requires another component to achieve what I am looking for?
It seems there is a misunderstanding in the concept-wise.
Here is the basic idea of a typical API Management solution.
You have a web service (REST, SOAP, etc...) which you need to expose as a managed API. Now, you can front your service with API Manager and expose it as a managed API with security, rate limiting, managed life cycle etc.
In your case, it seems you don't have such a service, but only have a database table. So, before using API Manager to front your service, you first need to expose your table as a service. For that purpose, I'd suggest you use the data service component of WSO2 EI 7.0.0. See [1] for how to do that. Once you have your service ready, you can use API Manager to expose it as a managed API.
[1] https://ei.docs.wso2.com/en/latest/micro-integrator/use-cases/tutorials/sending-a-simple-message-to-a-datasource/
Related
Spring Version: 5.0.8.RELEASE
Spring Boot Dependencies Version: 2.0.4.RELEASE
Java Version: 1.8.0_131
Spring Boot Admin reports that a client is down. However I can see that the client is running by navigating to it in the browser. On the details view for the client in Sprint Boot Admin the message under the health section is "Fetching health failed, Network Error". There are three URLs shown in the header of the details page:
http://localhost:8090/WorkOrderPrinting
http://localhost:8090/WorkOrderPrinting/manage
http://localhost:8090/WorkOrderPrinting/manage/health
Clicking them opens the respective views. Here is the output from the health view:
{"status":"UP","details":{"diskSpace":{"status":"UP","details":{"total":80482930688,"free":77726302208,"threshold":10485760}},"db":{"status":"UP","details":{"tenantRoutingDataSource":{"status":"UP","details":{"database":"Informix Dynamic Server","hello":104}},"userSetTenantRoutingDS":{"status":"UP","details":{"database":"Informix Dynamic Server","hello":104}},"dataSourceCbCommon":{"status":"UP","details":{"database":"Informix Dynamic Server","hello":104}},"dataSourceCbOrg":{"status":"UP","details":{"database":"Informix Dynamic Server","hello":104}}}}}}
It seems to be telling me that the client application is up and running. So I am not sure why Sprint Boot Admin UI is not reflecting that.
I have a second application running as a client and its working as expected. The result of the health URL is the same.
{"status":"UP","details":{"diskSpace":{"status":"UP","details":{"total":80482930688,"free":77726248960,"threshold":10485760}},"db":{"status":"UP","details":{"tenantRoutingDataSource":{"status":"UP","details":{"database":"Informix Dynamic Server","hello":104}},"userSetTenantRoutingDS":{"status":"UP","details":{"database":"Informix Dynamic Server","hello":104}},"dataSourceCbCommon":{"status":"UP","details":{"database":"Informix Dynamic Server","hello":104}},"dataSourceCbOrg":{"status":"UP","details":{"database":"Informix Dynamic Server","hello":104}}}}}}
The Spring Boot Admin Server log shows this error for Work Order Printing. Again, not sure why the URLs are working, but the log shows an error.
2018-09-13 09:19:19.071 DEBUG 5208 --- [ parallel-2] d.c.b.a.server.services.StatusUpdater : Update status for Instance(id=6212ad7c5ab4, version=1, registration=Registration(name=Work Order Printing Development, managementUrl=http://localhost:8090/WorkOrderPrinting/manage, healthUrl=http://localhost:8090/WorkOrderPrinting/manage/health, serviceUrl=http://localhost:8090/WorkOrderPrinting, source=http-api), registered=true, statusInfo=StatusInfo(status=DOWN, details={error=Found, status=302}), statusTimestamp=2018-09-13T13:15:08.169Z, info=Info(values={}), endpoints=Endpoints(endpoints={health=Endpoint(id=health, url=http://localhost:8090/WorkOrderPrinting/manage/health)}), buildVersion=null)
Spring Boot Admin Server Config
server.servlet.context-path=/adminserver
logging.file=/var/log/eti/webui/adminserver.log
logging.level.de.codecentric.boot.admin.server=INFO
Failing Client Config
#Admin Panel config
#
#This is the URL for the admin panel that this application will send its information to
spring.boot.admin.client.url=http://localhost:8080/adminserver
#This is required when deploying to Tomcat because the Admin panel cant seem to determine what the URL will be on its own
spring.boot.admin.client.instance.service-base-url=http://localhost:8090
#This is the name that will be displayed in the admin panel for this application
spring.boot.admin.client.instance.name=Work Order Printing
#
spring.boot.admin.auto-registration=true
#
#Actuator config needed to expose endpoints to admin panel
#
management.endpoints.web.base-path=/manage
management.endpoints.web.exposure.include:*
management.endpoint.health.show-details=always
Working Client Config
#Admin Panel config
#
#This is the URL for the admin panel that this application will send its information to
spring.boot.admin.client.url=http://localhost:8080/adminserver
#This is required when deploying to Tomcat because the Admin panel cant seem to determine what the URL will be on its own
spring.boot.admin.client.instance.service-base-url=http://localhost:8085
#This is the name that will be displayed in the admin panel for this application
spring.boot.admin.client.instance.name=LaunchPad
spring.boot.admin.auto-registration=true
#
#Actuator config needed to expose endpoints to admin panel
#
management.endpoints.web.base-path=/manage
management.endpoints.web.exposure.include:*
management.endpoint.health.show-details=always
So the problem turned out to be a configuration issue on my end. The Admin Server log shows the following:
2018-09-13 09:19:19.071 DEBUG 5208 --- [ parallel-2] d.c.b.a.server.services.StatusUpdater : Update status for Instance(id=6212ad7c5ab4, version=1, registration=Registration(name=Work Order Printing Development, managementUrl=http://localhost:8090/WorkOrderPrinting/manage, healthUrl=http://localhost:8090/WorkOrderPrinting/manage/health, serviceUrl=http://localhost:8090/WorkOrderPrinting, source=http-api), registered=true, statusInfo=StatusInfo(status=DOWN, details={error=Found, status=302}), statusTimestamp=2018-09-13T13:15:08.169Z, info=Info(values={}), endpoints=Endpoints(endpoints={health=Endpoint(id=health, url=http://localhost:8090/WorkOrderPrinting/manage/health)}), buildVersion=null)
Its basically saying that there is a 302 (redirect) happening so it cant reach the URL. The reason for this that I forgot to allow access to the URLs in Spring Security config. I could get to them with the browser because I was logged in. Spring Boot Admin could not, because it was not logged in.
I added a rule to allow access to the /manage/ urls
public void configure(WebSecurity web) throws Exception
{
web.ignoring().antMatchers("/css/**", "/fonts/**", "/img/**", "/js/**", "/close", "/webjars/**", "/manage/**");
}
Occasionally (?) the WSO2 IS user is unable to authenticate with following exception. When retrying, the user will be authenticated. Any ideas what could be reason / resolution? We set up the session caching.
Using WSO2 Identity Server 5.0.0.SP1 / SAML authentication with the authenticator set to advanced (single step, multiple options). I cannot find the correct source code commit to check out (to match the line number in the exception)
Thank you all in advance
Gabriel
TID: [0] [IS] [2016-02-15 13:07:22,914] ERROR
{org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultRequestCoordinator}
- Exception in Authentication Framework {org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultRequestCoordinator}
java.lang.NullPointerException at
org.wso2.carbon.identity.application.authentication.framework.handler.sequence.impl.DefaultStepBasedSequenceHandler.handle(DefaultStepBasedSequenceHandler.java:83)
at
org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultAuthenticationRequestHandler.handle(DefaultAuthenticationRequestHandler.java:121)
at
org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultRequestCoordinator.handle(DefaultRequestCoordinator.java:94)
at
org.wso2.carbon.identity.application.authentication.framework.servlet.CommonAuthenticationServlet.doPost(CommonAuthenticationServlet.java:54)
at
org.wso2.carbon.identity.application.authentication.framework.servlet.CommonAuthenticationServlet.doGet(CommonAuthenticationServlet.java:44)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:735) at
javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
Edit:
This exception occurs on the WSO2 IS 5.1.0 too
see the Source code line 105
StepConfig stepConfig = context.getSequenceConfig().getStepMap().get(currentStep);
// if the current step is completed
if (stepConfig.isCompleted()) {
stepConfig.setCompleted(false);
ERROR org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultRequestCoordinator} - Exception in Authentication Framework
java.lang.NullPointerException
at org.wso2.carbon.identity.application.authentication.framework.handler.sequence.impl.DefaultStepBasedSequenceHandler.handle(DefaultStepBasedSequenceHandler.java:105)
at org.wso2.carbon.identity.application.authentication.framework.handler.request.impl.DefaultAuthenticationRequestHandler.handle(DefaultAuthenticationRequestHandler.java:115)
it looks like the stepConfig 'dissapeared' from the authentication config. The setup is done on a single node with session persistence into a database.
Apparently it looks like a problem with concurrency.
When multiple concurrent requests are sent to the SSO endpoint while the user is already authenticated, all threads are attempting to process the request modifying the same authentication context object (currentStep counter) so the cached authentication context comes to an invalid state.
Valid use case is that the client should send only a single request to the SSO endpoint, so the team dealing with the UI have to fix it. But - that's only the a quick fix not preventing the issue in long term. We have to really pick it up with WSO2 (and fix the code ourselves maybe) :)
g.
I am getting the following error while doing baseline index of my Endeca application in ATG
15:26:47,891 ERROR [nucleusNamespace.atg.dynamo.security.opss.csf.CredentialStoreManager] (Thread-201) Unable to process any CSF calls as the Credential Store server i
s not enabled. Please check log for more details
15:26:47,913 INFO [nucleusNamespace.atg.commerce.search.StoreLocationOutputConfig] (Thread-201) Starting bulk load
15:26:47,915 INFO [nucleusNamespace.atg.commerce.endeca.index.CategoryToDimensionOutputConfig] (index-/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin) Fa
iled to cancel incremental load of /atg/commerce/endeca/index/CategoryToDimensionOutputConfig, probably because no bulk load was running.
15:26:47,916 INFO [nucleusNamespace.atg.endeca.index.ConfigImportDocumentSubmitter] (Thread-203) Opening configuration repository connection for application logistore
15:26:47,917 ERROR [nucleusNamespace.atg.dynamo.security.opss.csf.CredentialStoreManager] (Thread-203) Unable to process any CSF calls as the Credential Store server i
s not enabled. Please check log for more details
15:26:47,916 INFO [nucleusNamespace.atg.commerce.search.ProductCatalogOutputConfig] (index-/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin) Failed to can
cel incremental load of /atg/commerce/search/ProductCatalogOutputConfig, probably because no bulk load was running.
15:26:47,917 INFO [nucleusNamespace.atg.commerce.search.StoreLocationOutputConfig] (index-/atg/commerce/endeca/index/ProductCatalogSimpleIndexingAdmin) Failed to canc
el incremental load of /atg/commerce/search/StoreLocationOutputConfig, probably because no bulk load was running.
15:26:47,919 INFO [nucleusNamespace.atg.endeca.index.ConfigImportDocumentSubmitter] (Thread-199) Opening configuration repository connection for application logistore
15:26:47,919 ERROR [nucleusNamespace.atg.dynamo.security.opss.csf.CredentialStoreManager] (Thread-199) Unable to process any CSF calls as the Credential Store server i
s not enabled. Please check log for more details
15:26:47,919 INFO [nucleusNamespace.atg.commerce.endeca.index.ProductCatalogSimpleIndexingAdmin] (Thread-203) Indexing process cancelled, Endeca says: Could not retri
eve workbench credential properties from credential store.
15:26:47,919 INFO [nucleusNamespace.atg.endeca.index.ConfigImportDocumentSubmitter] (Thread-207) Opening configuration repository connection for application logistore
15:26:47,920 ERROR [nucleusNamespace.atg.dynamo.security.opss.csf.CredentialStoreManager] (Thread-207) Unable to process any CSF calls as the Credential Store server i
s not enabled. Please check log for more details
15:26:47,921 INFO [nucleusNamespace.atg.commerce.endeca.index.ProductCatalogSimpleIndexingAdmin] (Thread-207) Indexing process cancelled, Endeca says: Could not retri
eve workbench credential properties from credential store.
After doing extensive research I found that C:\ATG\ATG11.2\home\servers\atg_production_lockserver\localconfig\atg\dynamo\server\OPSSInitializer.properties has path for jps-config.xml ie
JPSConfigurationLocation=C:/ATG/ATG11.2/home/../home/security/jps-config.xml
This jps-config.xml has some CSF related configuration.
How can I get rid of this error for successful baseline indexing.
I am stuck on this part.
This happens if you change the default workbench password. Simple solution would be, change Endeca experience manager password back to admin and try.
Otherwise, password needs to be changed in multiple places.
Thanks,
Ajay Agrawal
Go to the OPSSInitializer component in dyn admin and check whether the path for jps-config.xml specified is correct there. If not, correct the path.
I have installed Oracle ATG v11 with the commerce reference store, when I startup the production server and go to the url domain/crs/storeus I see the blank white page, and have the following error in the console:
Oct 13, 2014 1:56:37 PM com.endeca.infront.site.SiteManager getSite
SEVERE: Unable to retrieve site definition for site id: /storeSiteUS
com.endeca.store.exceptions.PathNotFoundException: No node found at
path: [pages].
at com.endeca.store.configuration.InternalNode.getNode(InternalNode.java:153)
at com.endeca.store.configuration.InternalNode.getNodeInfo(InternalNode.java:221)
at com.endeca.store.configuration.InternalNode.getNode(InternalNode.java:150)
at com.endeca.store.configuration.InternalNode.getNode(InternalNode.java:61)
........................................
**** Error Mon Oct 13 13:00:47 +00:00 2014 1413205247448 /atg/endeca/assembler/droplet/InvokeAssembler A problem occurred
assembling the content for content item /content/Web/Home Pages. The
response received was {#type=ContentSlot,
atg:currentSiteProductionURL=/crs/storeus,
canonicalLink=com.endeca.infront.cartridge.model.NavigationAction#2b35e9c6,
ruleLimit=1, #error=com.endeca.infront.content.ContentException:
com.endeca.navigation.ENEConnectionException: Error establishing
connection to retrieve Navigation Engine request
http://localhost:15000/graph?node=0&profiles=sitegroup.siteGroupUS|NoPriceRange|site.storeSiteUS&offset=0&nbins=0&irversion=640'.
Tried all: '2' addresses, but could not connect over HTTP to server:
'localhost', port: '15000' Check MDEX Logs and specified query
parameters. , contentCollection=/content/Web/Home Pages}. Servicing
the error open parameter.
I am assuming this error is related to endeca? I have downloaded CAS, Tools And Frameworks with experience manager and MDX, and Platform Services. Do I need to start these or have I missed a part of the endeca install?
The value of the configurationPath attribute in the DefaultFileStoreFactory.properties located at \localconfig\atg\endeca\assembler\cartridge\manager may be incorrect.
In OOTB CRS, we normally provide the following value for configurationPath attribute :
/ToolsAndFrameworks/11.1.0/server/workspace/state/repository/CRS
Could you please verify the .zip is present at path provided in DefaultFileStoreFactory.properties.
Just check if you are able to connect the below url:
host:15000/admin?op=stats
If you are able to connect this URL, then MDEX is running. Also, you can login to the experience manager and check if the dgraphs and dgidx are running.
If you are not able to connect then check all the services are(tools and http) running and accessible. You can check the endeca logs to debug further.
Your DGraph is not (yet) started.
(Hit this URL in your browser and verify: http://localhost:15000/graph?node=0&profiles=sitegroup.siteGroupUS|NoPriceRange|site.storeSiteUS&offset=0&nbins=0&irversion=640&format=xml)
Possible reasons are:
You did not run baseline update from ATG (from
ProductCatalogSimpleIndexingAdmin dyn/admin component).
You did not run promote content (from your Endeca App's control folder).
Your Services are not working properly (or not started at all). Check that Platform Services and Tools And Frameworks are started.
The solution is to properly define the value for the property configurationPath=E:/Endeca/Apps/CRS/data/workbench/application_export_archive/CRS in "DefaultFileStoreFactory.properties"
If you are using the OS as Windows then define this path as Unix style as shown above.
Is it possible to let the server send messages to all connected clients without waiting for any action from them? Let me explain it :-) I've been reading the docs/examples and I have found nothing that satifies my needs: the flow is always the same; a client connects (e.g: a GET call to a Rest API), the connection is suspendend and until a new API call is received (e.g.: a POST call) the server simply waits (or at least this is what I have understood). My use case is pretty different: I want the server to send some "notifications" once new data become available. This would be my use case (pretty simplifed):
Client A connects to server
Connection is suspended since no new data is available at the moment
The server gets notified new data is available from an external
source and broadcasts it to client A
Go to step 2
What I have achieved so far is getting the connection successfully established. The next step is to solve this server issue. I must say this technology is completely new to me so it is possible I misunderstood how something works. If that's the case, let me know!
This is my stack:
Spring 3.2.0 RELEASE
Jersey 1.8
Atmosphere Jersey 1.0.13
Tomcat 7.0.40
Thank you all in advance!
UPDATE: After following this I get this warning, which I have no idea how to get rid of:
2013-06-04 09:40:36,284 WARN [org.atmosphere.cpr.AtmosphereFramework] - Failed using comet support: org.atmosphere.container.Tomcat7AsyncSupportWithWebSocket, error: Tomcat failed to detect this is a Comet application because context.xml is missing or the Http11NioProtocol Connector is not enabled.
If that's not the case, you can also remove META-INF/context.xml and WEB-INF/lib/atmosphere-compat-tomcat.jar Is the Nio or Apr Connector enabled?
2013-06-04 09:40:36,285 WARN [org.atmosphere.cpr.AtmosphereFramework] - Using org.atmosphere.container.Tomcat7BIOSupportWithWebSocket
I followed the app structure commented here, so this should not be a problem. I have noticed that by changing the transport to "websocket" instead of "long-polling" shows no errors. The server finally sends data tough :)
I followed your link and modified the code a little.
When you are in the step 3 "The server gets notified new data is available from an external source and broadcasts it to client A", you have to write a line like this:
BroadcasterFactory.getDefault().lookup("/*").broadcast(response);
At first I used the TextMessage received from my ActiveMQ Queue but I get this error, so I used a Jackson class as an object response and everything worked fine.
SEVERE: A message body writer for Java class org.apache.activemq.command.ActiveMQTextMessage, and Java type class org.apache.activemq.command.ActiveMQTextMessage, and MIME media type application/json was not found
jun 03, 2014 11:32:21 AM com.sun.jersey.spi.container.ContainerResponse write
SEVERE: The registered message body writers compatible with the MIME media type are:
application/json (JSONJAXBElementProvider, JSONArrayProvider, JSONObjectProvider, JSONRootElementProvider, JSONListElementProvider, ...)