Using custom connector i am able to load data from mysql to s3 in AWS Appflow but when creating a new table in mysql, it is not showing up in Appflow console.
New entities are discovered during the ListEnties event that is called recursively when building a flow amd selecting the source object. Since you are building a custom connector all the logic of how the connector discovers entities is in your custom code. Specifically the list_entities method of your MetadataHandler.
See the end-to-end flow of the readme of the Python custom connector SDK. I quote part here.
https://github.com/awslabs/aws-appflow-custom-connector-python
Step 5: AppFlow user then uses that ConnectorProfile to create Flow definitions. During the flow creation AppFlow invokes listEntity and describeEntity functions on the connector to retrieve the list of supported entities and the entity detail of the specific entity AppFlow user selected on the console / API.
With that said, if you can share im your question the code of your metadata handler as well as a description of the source database and why you think the new table should be discovered by your code.
I would also look at the CloudWatch logs or whereever you store logs for the Lambda hosting your custom connector. View the logs after trying to build a flow and not seeing the expected new entities listed. If there is an error trying to read a new entitiy you could see it here if you have the proper logging setup.
Related
I have a working app I made using Amplify (with AppSync API and Cognito). I'd like to make another app which is different, but shares some data with my first project (same product, but different targets, usages and security rules).
Is there a clean way to use Amplify for that new project, telling the GraphQL API to fetch some data in the DynamoDB from my first Amplify project ?
This data will change often and will be heavy - so I'm not too much into any synchronization solution.
I thought about those solutions, but I'm not experienced enough to tell if one of them is good:
Not using Amplify but SAM for this new project (but I will lose all the build pipeline provided by Amplify)
Using Amplify for the Hosting and the Auth, but configure AppSync with SAM and plug it into my existing DataSource
Maybe CloudFormation can be the answer, but I don't see how to interact directly with it within Amplify
There is an article from Amazon about microservice architectures that might be useful for you
https://aws.amazon.com/blogs/mobile/appsync-microservices/
I'm not sure if Amplify supports adding multiple APIs to it, if so, you could add it and then pull only that api on it.
IMO the easiest approach would be just creating a query endpoint to fetch the data from your other datasource, using lambda.
So, in that case you would edit your schema to something like this
query
{
externalData [ExternalData] #function(name: "getExternalData")
}
Then you need to add the lambda function getExternalData, which will be responsible to query the data as needed.
The article above has more in-depth details about this architecture
I have few custom modules created using Module Builder. I need to fetch data using external REST API (around 200-300k record per API) and without storing data in suite database I need to display data on custom modules (using UI provided by SuiteCRM only. NO CUSTOM UI). and also I have operations to be perform on data which will again call External API.
So, as per requirement, Data should not be saved at database. How to achieve this in SUITECRM (7.11.8 version)?
I have tried using Process record hook. But things aren't working.
I am running Mule Anypoint studio v6.2.4 under Windows. I have an application which gets data records from a SQL Server DB and populates them into Salesforce Org. The application was developed using examples and works fine in the current scenario of one DB and one SF parameters defined in a mule.properties file. This structure works but with serious limitations. Whenever a new DB instance is provided (DEV/QA/SAT/UAT...) and a different destination SF Org is chosen, all the mule applications need to be modified as the access parameters are defined in the property file.
I asked for help about how to dynamically define and use the access parameters in this question. (Defining various DB & Salesforce access information in Mule Anypoint Studio). While the proposed solution worked, it was still based on property file approach. Hence whenever a new DB or SF Org is to be worked on, all the mule applications will still need to be changed because of the property file approach.
A correct approach will be define all the access parameters in an external file. In the mule app, retrieve the appropriate access parameters based on DB name or Org name (provided as http query parameters). Use their values to initialise appropriate mule script variables. This way new DBs or Orgs can be added at will and mule app should work without any re-compilation or changes.
My mule app is batch based and config code fragment to access SQL DB is as follows::
<db:generic-config name="legacyDB" url="jdbc:jtds:sqlserver:${dbHost}:${dbPort};databaseName=${dbName};user=${dbUser};password=${dbPassword}" doc:name="Generic DB Config"/>
How would I accomplish it so that DB or SF access works as defined? Details will help me as I am not that proficient in Mule yet.
Thanks
Kishore
What about reading an external file using a JavaClass and returning a JavaObject to Mule, then you can easily manage those values.
Check this project and change the path in the LoadExternalFile.java to load your own json file. There is an example in the src/main/resources
https://github.com/angelalberici/external-property
I am installed and added data source (web data connector) inside tableau
Web data connector URL : http://localhost/datasouceexample/example.html
we data connector holds some data.
import data from web data connector and created graphs using the data's from the web data connector URL .Generated graph and saved in "tableau public" option.
Embed the code inside my web application . Graphs shows perfectly.
My question is about the automatic data updation .
1 ) I want to update graph automatically based on the value from web data connector URL
2 ) If any new updations are inside input datas (new inputs) , automatically sync with tableau and update the graph,so no changes in the embed code and graph updated automatically
Any settings available inside tableau for do this section? if it is possible ? Thanks
Only dashboards with a google sheets datasource can be automatically updated on Tableau Public. Maybe you could redesign your web service to write to a google sheets doc. If so, you could republish your dashboard to sync with that doc.
The scenario you are describing is not possible using Tableau Public (the free version of Tableau software) and the web data connector. Tableau Public does not support updating web data connector data once published on Public, you would need to refresh the data from your desktop application and republish.
The closest supported path to your use case would be, as Kara_F mentioned in the comments, to use Tableau Server. With that product, you can create web data connector datasources and publish them to your Server. You can then schedule a refresh job which will update your data at a certain interval. More info for keeping WDC data up to date can be found here: http://onlinehelp.tableau.com/v0.0/server/en-us/datasource_wdc.htm.
I followed very simple example from www.asp.net to create Web API project using Empty project template for Web API. This tutorial does not create database instead it just create model class and seed data inside api controller.
I successfully consumed API controller using angular service and able to show data on UI. Till here everything is fine.
However, moving one step further i would like create database but not sure about the steps.
I don't want to follow some other tutorial available on same site which does all of these things because i am not understand the overall concept even though i'll able to create database. Also, tutorial talks about using MVC approach which i dont want to use. I just looking for Web API, EF and Angular combination.
Can someone guide me for writing context class and overall database creation steps.?