Using a Web activity along with a linked service to call a rest api - azure-data-factory

I have to send data to a rest API via data factory.
I get batches of records from a database, send them in the body of the REST API call, which provides a response that informs the action performed on each record.
I created a linked service to the base API URL, and this linked service does the authentication to the API.
My question is how I use this linked service along with a web activity in a pipeline?
The web activity requires me to enter a full URL, which feels redundant as the base URL is already in the linked service.
The web activity does let me add multiple linked services but I'm unsure why it allows multiple linked services and how this is supposed to work.
I would appreciate expertise regarding how the web activity works with a linked service.
Thanks!

Related

REST API calls for setting namespace preferences and Program preferences

Can the namespace preferences and program preferences be set via REST API calls? If yes, what is the syntax for it?
Generally in Cloud Data Fusion, when we intend to perform the action on GCP side, like create/delete/restart etc. instance, it's feasible to use domestic Google Cloud API, giving the opportunity to interact with a service endpoint via JSON/HTTP calls interface as described in Google Cloud API design document.
Dedicated to Data Fusion you can follow the Cloud Data Fusion REST API reference document, nicely explaining the methods for composing REST API HTTP calls to manage Data Fusion instances, moreover every method description from the documentation contains Google API Explorer sub-panel, to get handy experience building JSON request on a live data.
Said above, I assume your initial question is related more to CDAP REST API, as it includes the methods for pure CDAP instance metadata/namespaces/application configuration.
From the user perspective your workflow might be the following:
Identify the CDAP API endpoint as explained in this guideline;
Compose an HTTP PUT/GET request relevant to Data Fusion
Namespace/Metadata/Preferences/Configuration
object via CDAP RESTful API.
Yes of course! You have two methods.
The first method is creating it from the platform. Follow the steps below:
Open your data fusion instance
Go to System Admin => Configuration => Make HTTP calls
To create a namespace, submit an HTTP PUT request:
PUT /v3/namespaces/<namespace-id>
Link of CDAP: CDAP
The second method is using terraform.

Transfer SCP Cloud User Information securely to ABAP

I have a UI5 Application hosted on SAP NEO which retrieves Data via an ABAP OData Service.
Users are maintained in SAP Cloud Identity and mapped with their Backenduser and the login is setup via Principal Propagation. This is all setup and works.
I got requested to change the Application to enable External Sales Representatives without SAP Backend Users to use the application.
The idea is to use one "technical user" with Basic Authentication instead of Principal Propagation.
My question is what would be the way to identify the original Cloud Username in ABAP(since there sy-uname would be a technical user).
Debugging in ABAP didn't reveal the original information and I am afraid the original User is not even passed to the Gateway
The SAP Cloud User API (https://help.sap.com/viewer/65de2977205c403bbc107264b8eccf4b/Cloud/en-US/1de599bf722446849d2b2e10132df42a.html) is not an option because the request could be manipulated in the browser
I heard of another option using a Java Servlet. But I am afraid that means we have to setup there the whole OData Service again and with every change in the Gateway we have to adjust the Java Servlet as well, or is there maybe a proxy.
If you are using Mobile Services of SAP Cloud Platform, you can activate a header with the username to be transferred to your ABAP system. It's called X-SMP-ENDUSERNAME.
Ref the documentation at https://help.sap.com/viewer/38dbd9fbb49240f3b4d954e92335e670/Cloud/en-US/defdadb71ee2476691d987689e3703a2.html
I assume you can get cloud user ID within your UI5 application and in case you access backend via Odata model you can use ODataModel.setHeaders function to provide your custom request headers which will be attached to every request sent to the backend. I would try to send cloud user id in some custom header value.
And on ABAP side you can use DP facade interface in service implementation to read custom headers:
lo_facade ?= /iwbep/if_mgw_conv_srv_runtime~get_dp_facade( ).
lt_client_headers = lo_facade->get_request_header( ).

How to create a dynamic API endpoint connection using HTTP or REST connectors in Azure Data Factory V2

I have an external REST based API that I need to create a connection to in order to retrieve data on a regularly scheduled basis (for BI purposes). This API is fairly robust, and supports around 60 distinct endpoints. Also, this same API is used to access information across multiple client sub-domains (e.g. client1.apisource.com, client2.apisource.com, client3.apisource.com, etc.). In other words, the API endpoints are the same for each client subdomain.
So what I'm trying to figure out is whether it's possible to create a single ADF that contains a complete set of pipeline actions for each endpoint THAT uses a "dynamic" URL based on the client subdomains? In other words...what I'm trying to see if its possible to create a single ADF that can manage a dynamic list of base URLs.
I tried to parameterize the HTTP and REST connections, but this doesn't appear to Is this possible yet. Any thoughts? Thanks!
Here is an example of a Web Activity to call a REST API using parameters and expressions. The URL can be an expression like:
#concat('https://management.azure.com/subscriptions/',pipeline().parameters.SubscriptionID,'/resourceGroups/',pipeline().parameters.ResourceGroup,'/providers/Microsoft.Sql/servers/',pipeline().parameters.Server,'/databases/',pipeline().parameters.DW,'?api-version=2014-04-01')

How to get URL from request within an action?

I'm writing a HATEOAS-driven REST API and it would be convenient to return responses from some actions with links to other resources.
As the REST API is currently implemented following a monolithic architecture, and in order to not only reduce redundancies but also simplify the job of migrating some parts to microservices, it would be nice if an action could send responses based on the URL that was routed to it.
So, does anyone know if ASP.NET core 2.1 offers any way to access the full URL that's routed to an action when a request is routed to said action?

503service unavailable in Salesforce

My goal is to create a REST API Integration from Salesforce to SAP application.
SUCCESS Through Chrome APP
1. All I need to do is retrieve values from sap application through the REST API. When I tried to use the Chrome APP 'Advanced Rest Client' and have passed the appropriate URL and Content with POST method I was able to retrieve the values from local server database.
For EG : If I pass request 92126 then I was able to get response  'SAN DIEGO' which is correct.
Here is the link (https://chrome.google.com/webstore/detail/advanced-rest-client/hgmloofddffdnphfgcellkdfbfbjeloo?hl=en-US) for Advanced REST Client.
PROBLEM from Salesforce :
I had created a remotesite setting 
When I created this REST class in SAlesforce and tried invoking the End Point then it's throwing this error.
System.HttpResponse[Status=Service Unavailable, StatusCode=503]
As the web api url which is provided to us is in local sql server i.e hosted in private, as we know in Salesforce for making callouts the URLs must be in public. But the URL is in private only for the security reasons not hosted in public. We should achieve it, any way is there to achieve it? What change should be done in Salesforce or server to communicate to each other, and allows to make the callout?
It is most likely that you endpoint does not allow access from outside some ip range which you indicated by saying it's not public. Salesforce is a SaaS application hosted outside the domain that your service is on. In order for Salesforce to access that endpoint resource you need to whitelist Salesforce IP ranges, which can be found here.
Whitelisting allows Salesforce to access the resource. The only caveat is that because Salesforce is multi-tenant it means that any instance of Salesforce on the range that you whitelist would have access to your endpoint. If this is not ok, you might want to add some sort of header or sign the request to the call to that identifies your Salesforce instance uniquely from any other instance to validate that the call originated from your Salesforce org.
(I am linking to the article instead of pasting the IP ranges here because these may change in the future).