Cadence Workflow: Pass host specific objects like database connections, service clients, etc to activity implementation - cadence-workflow

When a cadence worker comes up it initializes certain resources like database connections and client for other services. What is the correct pattern to make these resources accessible from activity implementations?

Go
You can pass anything your activity needs via the context to your activity.
Here is how it works:
1) Before you start your worker, set whatever you need as value to a context (below code happen at where you setup your worker):
myThriftClient := ... // create my thrift client
myContext := context.WithValue(context.Background(), "my_thrift_client", myThriftClient)
2) Use the context created in step1 as BackgroundActivityContext of the worker options that you use to start your worker:
workerOptions := cadence.WorkerOptions{
MetricsScope: myScope,
Logger: myLogger,
BackgroundActivityContext: myContext,
}
worker := cadence.NewWorker(service, domain, taskList, workerOptions)
3) In your activity code, retrieve your thrift client from the context:
func MyActivity(ctx context.Context) error {
myThriftClient := ctx.Value("my_thrift_client").(ThriftClient)
// now you can make thrift calls using myThriftClient
}
Java
Worker.registerActivityImplementations accepts an activity object instance. So any dependencies can be associated with this object before it is registered with the worker.
Worker.Factory factory = new Worker.Factory(DOMAIN);
Worker worker = factory.newWorker(TASK_LIST);
// Initialize activities instance with all its dependencies
MyActivities activities = new MyActivitiesImpl(dbPool, serviceAClient);
worker.registerActivitiesImplementations(activities);
// Start listening to the workflow and activity task lists.
factory.start();

In Java we use Suppliers to pass in stateful clients and services like gcs, database ...
so on each activity run based on how we have implemeneted the get method of the Supplier, the requested client/service is provided to the activity without the need to instantiating them beforehand upon activityImpl instantiation.

Related

Quarkus: "no tenant identifier specified" in callback

I try to add multi-tenancy support for my Quarkus app, following Quarkus hibernate-orm doc (see last section).
I have my CustomTenantResolver class and configure in application.properties, with multiple data sources, but no named persistent unit, see below:
# Default data source
quarkus.hibernate-orm.datasource=master
quarkus.hibernate-orm.database.generation=none
quarkus.hibernate-orm.multitenant=DATABASE
# ----- Tenant 'master' (default) ---------------
quarkus.datasource."master".db-kind=postgresql
quarkus.datasource."master".username=postgres
quarkus.datasource."master".password=password
quarkus.datasource."master".jdbc.url=jdbc:postgresql://localhost:5432/db_master
# ----- Tenant 'test' ---------------------------
quarkus.datasource.test.db-kind=postgresql
quarkus.datasource.test.username=postgres
quarkus.datasource.test.password=password
quarkus.datasource.test.jdbc.url=jdbc:postgresql://localhost:5432/db_test
Everything works fine for Web Services APIs functions - based on incoming web service calls, I can extract and supply tenant identifier for DB access.
Problem is, my app also needs to use callback method to listen on messages coming from Apache Pulsar queue. When a message comes in and triggers this callback, any DB access in this method will give this exception:
SessionFactory configured for multi-tenancy, but no tenant identifier specified: org.hibernate.HibernateException: SessionFactory configured for multi-tenancy, but no tenant identifier specified
at org.hibernate.internal.AbstractSharedSessionContract.<init>(AbstractSharedSessionContract.java:172)
at org.hibernate.internal.AbstractSessionImpl.<init>(AbstractSessionImpl.java:29)
at org.hibernate.internal.SessionImpl.<init>(SessionImpl.java:221)
at org.hibernate.internal.SessionFactoryImpl$SessionBuilderImpl.openSession(SessionFactoryImpl.java:1282)
at org.hibernate.internal.SessionFactoryImpl.openSession(SessionFactoryImpl.java:472)
at io.quarkus.hibernate.orm.runtime.session.TransactionScopedSession.acquireSession(TransactionScopedSession.java:86)
at io.quarkus.hibernate.orm.runtime.session.TransactionScopedSession.persist(TransactionScopedSession.java:138)
at io.quarkus.hibernate.orm.runtime.session.ForwardingSession.persist(ForwardingSession.java:53)
... (snipped)
Apparently my CustomTenantResolver class was not called during this listener callback as the callback is another fresh thread, hence no tenant id is supplied.
Do I miss anything? How about the scheduler in Quarkus - how does it support multi-tenancy in scheduled jobs?
Thanks for helps.
I had a similar issue when pulling messages from JMS. The cause of the issue is that io.quarkus.hibernate.orm.runtime.tenant.HibernateCurrentTenantIdentifierResolver ( which implements CurrentTenantIdentifierResolver and as the doc says Maps from the Quarkus {#link TenantResolver} to the Hibernate {#link CurrentTenantIdentifierResolver} model ) expects a request context to be active before calling our implementation of TenantResolver, as shown here:
// Make sure that we're in a request
if (!Arc.container().requestContext().isActive()) {
return null;
}
TenantResolver resolver = tenantResolver(persistenceUnitName);
String tenantId = resolver.resolveTenantId();
I solved it on my app by, first, enabling the request context on the JMS consumer:
Arc.container().requestContext().activate();
and, second, using a ThreadLocal to "pass" the current tenant id to the TenantResolver that will be called later by Hibernate ( through the HibernateCurrentTenantIdentifierResolver instance):
CurrentTenantLocal.setCurrentTenantId("public");
On my TenantResolver ( the class that implements TenantResolver ) I resolve the tenant from either an injected JsonWebToken jwt when it comes from a WebRequest, or using the ThreadLocal when consuming from JMS:
if ( CurrentTenantLocal.getCurrentTenantId() != null ) {
return CurrentTenantLocal.getCurrentTenantId();
}
Caveats:
Note that I haven't done an exhaustive search of the possible side effects of activating the request context... but I have no problems so far.

Terraform: add a message to queue

I create queue inside storage by the following way:
resource "azurerm_storage_queue" "myqueue" {
name = "myqueue"
storage_account_name = "${azurerm_storage_account.sto1.name}"
}
but for this queue I want to add init message (it's like a marker, when I sync with external resource last time and need to get data from the last execution). How can I configure it in terraform file?
This sounds like a use case for remote-exec.
Generally when using remote-exec, using a null provider is recommended. This lets you have closer control over what triggers the provider, rather than being dependent on the logic of the cloud service you're integrating it with.

What are the modifications needed to be done to get Wiremock running?

I have a .Net Core web API solution called ReportService, which calls another API endpoint (we can call this PayrollService) to get payroll reports. So my requirement is to mock the PayrollService using Wiremock.Net.
Also currently I have a automation test case written, which will directly call the ReportService controller and will execute all the service logic, and also classes which calls PayrollService and the DB layer logic and will get the HTTP result back from the ReportService.
Please note that the Automation test cases is a separate solution. So my requirement is to run the automation test cases like before on ReportService, and the payroll service will be mocked by Wiremock.
So, what are the changes that need to happen in the codebase? Do we have to change the url of the ReportService to be the Wiremock server base url in the ReportService solution? Please let us know, and please use the terms I have used in the question regarding the project names so I am clear.
Your assumption is indeed correct, you have make the base URL which is used by ReportService configurable.
So that for your unit / integration tests you can provide the URL on which the WireMock.Net server is running.
Example:
[Test]
public async Task ReportService_Should_Call_External_API_And_Get_Report()
{
// Arrange (start WireMock.Net server)
var server = WireMockServer.Start();
// Setup your mapping
server
.Given(Request.Create().WithPath("/foo").UsingGet())
.RespondWith(
Response.Create()
.WithStatusCode(200)
.WithBody(#"{ ""msg"": ""Hello world!"" }")
);
// Act (configure your ReportService to connect to the URL where WireMock.Net is running)
var reportService = new ReportService(server.Urls[0]});
var response = reportService.GetResport();
// Assert
Assert.Equal(response, ...); // Verify
}

Sharing objects with all verticles instances

My application, an API server, is thought to be organized as follows:
MainVerticle is called on startup and should create all necessary objects for the application to work. Mainly a mongoDB pool of connections (MongoClient.createShared(...)) and a global configuration object available instance-wide. It also starts the HTTP Listener, several instances of a HttpVerticle.
HttpVerticle is in charge of receiving requests and, based the command xxx in the payload, execute the XxxHandler.handle(...) method.
Most of the XxxHandler.handle(...) methods will need to access the database. In addition, some others will also deploy additional verticles with parameters from the global conf. For example LoginHandler.handle(...) will deploy a verticle to keep user state while he's connected and this verticle will be undeployed when the user logs out.
I can't figure out how to get the global configuration object while being in XxxHandler.handle(...) or in a "sub"-verticle. Same for the mongo client.
Q1: For configuration data, I tried to use SharedData. In `MainVerticle.start() I have:
LocalMap<String, String> lm = vertx.sharedData().getLocalMap("conf");
lm.put("var", "val");
and in `HttpVerticle.start() I have:
LocalMap<String, String> lm = vertx.sharedData().getLocalMap("conf");
log.debug("var={}", lm.get("var"));
but the log output is var=null.... What am I doing wrong ?
Q2: Besides this basic example with a <String, String> map type, what if the value is a mutable Object like JsonObject which actually is what I would need ?
Q3: Finally how to make the instance of the mongo client available to all verticles?
Instead of getLocalMap() you should be using getClusterWideMap(). Then you should be able to operate on shared data accross the whole cluster and not just in one verticle.
Be aware that the shared operations are async and the code might look like (code in Groovy):
vertx.sharedData().getClusterWideMap( 'your-name' ){ AsyncResult<AsyncMap<String,String>> res ->
if( res.succeeded() )
res.result().put( 'var', 'val', { log.info "put succeeded: ${it.succeeded()}" } )
}
You should be able to use any Serializable objects in your map.

How do I call a method on my ServiceWorker from within my page?

I have a ServiceWorker registered on my page and want to pass some data to it so it can be stored in an IndexedDB and used later for network requests (it's an access token).
Is the correct thing just to use network requests and catch them on the SW side using fetch, or is there something more clever?
Note for future readers wondering similar things to me:
Setting properties on the SW registration object, e.g. setting self.registration.foo to a function within the service worker and doing the following in the page:
navigator.serviceWorker.getRegistration().then(function(reg) { reg.foo; })
Results in TypeError: reg.foo is not a function. I presume this is something to do with the lifecycle of a ServiceWorker meaning you can't modify it and expect those modification to be accessible in the future, so any interface with a SW likely has to be postMessage style, so perhaps just using fetch is the best way to go...?
So it turns out that you can't actually call a method within a SW from your app (due to lifecycle issues), so you have to use a postMessage API to pass serialized JSON messages around (so no passing callbacks etc).
You can send a message to the controlling SW with the following app code:
navigator.serviceWorker.controller.postMessage({'hello': 'world'})
Combined with the following in the SW code:
self.addEventListener('message', function (evt) {
console.log('postMessage received', evt.data);
})
Which results in the following in my SW's console:
postMessage received Object {hello: "world"}
So by passing in a message (JS object) which indicates the function and arguments I want to call my event listener can receive it and call the right function in the SW. To return a result to the app code you will need to also pass a port of a MessageChannel in to the SW and then respond via postMessage, for example in the app you'd create and send over a MessageChannel with the data:
var messageChannel = new MessageChannel();
messageChannel.port1.onmessage = function(event) {
console.log(event.data);
};
// This sends the message data as well as transferring messageChannel.port2 to the service worker.
// The service worker can then use the transferred port to reply via postMessage(), which
// will in turn trigger the onmessage handler on messageChannel.port1.
// See https://html.spec.whatwg.org/multipage/workers.html#dom-worker-postmessage
navigator.serviceWorker.controller.postMessage(message, [messageChannel.port2]);
and then you can respond via it in your Service Worker within the message handler:
evt.ports[0].postMessage({'hello': 'world'});
To pass data to your service worker, the above mentioned is a good way. But in case, if someone is still having a hard time implementing that, there is an other hack around for that,
1 - append your data to get parameter while you load service-worker (for eg., from sw.js -> sw.js?a=x&b=y&c=z)
2- Now in service worker, fetch those data using self.self.location.search.
Note, this will be beneficial only if the data you pass do not change for a particular client very often, other wise it will keep changing the loading url of service worker for that particular client and every time the client reloads or revisits, new service worker is installed.