Several questions on Corda deployment:
Upon deployment, do all the JAR files in all Corda nodes have to be exactly the same - same build, same version, same hash, etc.. ? Does it mean all parties have to agree on the same code base beforehand?
How to implement your own node's logic variant? Is it possible to only share the JARs containing interface of the flow and have different JARs for implementation? Can we drop JAR into plugin directory as required without synchronizing with other nodes?
How do the first-time deployment and follow-up code (JAR) updates work for individual node? Please explain the typical deployment steps.
I have read the documentations but still not so clear about real-world production deployment scenario.
For 1. and 2.: See https://docs.corda.net/writing-a-cordapp.html#structure. The JARs containing the code for states and contracts must be identical across participants, since everyone must be in consensus about what is being agreed on the ledger and whether a given ledger update is valid. However, the flow logic can vary across nodes. As a result, CorDapps should be split into several JARs - some which will be shared and identical across nodes (contract and state definitions), and some which will be different (flow definitions, possibly other supporting classes)
For 3.: Here are instructions regarding the deployment of a node: https://docs.corda.net/deploying-a-node.html
Related
I have an issue with a service worker, I have two different projects that are in the same server but in different folders, and I want to precache the files on project number 2 using my service worker (My service worker is already working on project number 1). My question is, is it possible to do this? is there any other way I can attack this? Any help is very much appreciated.
In general, yes, as long as the service worker is hosted at a URL that is at the same level (or "higher") than the root of each of those projects. That would ensure that each project will be within scope of the service worker.
I'm assuming that one of the challenges you're asking about relates to creating a precache manifest within that service worker that contains build artifacts from both projects. There are a few different ways to tackle that, but I think the most straightforward would be to ensure that you always run the build process for each project at the same time, and then when you use Workbox's build tooling to create the precache manifest, you ensure that you grab all the assets that were output by each of the projects.
The specifics of configuring that build process depends on what you're currently using. You mention that there's a service worker (presumably using Workbox's precaching) already in place for the first project, so I think just using the same build setup, with tweaks to pick up the additional assets, would be easiest.
I’m currently using kubernetes and I came across of helm.
Let’s say I don’t like the idea of “infecting” my kubernetes cluster with a process that is not related to my applications but I would gladly accept it if it could be beneficial.
So I made some researches but I still can’t find anything I can’t easily do by using my yaml descriptor and kubectl so for now I can’t find an use except,maybe, for the environizing.
For example (taking it from guides I read:
you can easily install application, eg. helm install nginx —> I add an nginx image to my deployment descriptor, done
repositories -> I have docker ones (where I pull my images from)
you can easily helm rollback in case of release failure-> I just change the image version to the previous one in my kubernetes descriptor, easy
What bothers me is that, at level of commands, I do pretty much the same effort (helm update->kubectl apply).
In exchange for that I have a lot of boilerplate because of keeping the directory structure helm wants and I feel like missing the control I have with plain deployment descriptors ...what am I missing?
It is totally understandable your question. For small and simple deploys the benefits is not actually that great. But when the deploy of something is very complex Helm helps a lot.
Think that you have a couple squads that develop microservice for some company. If you can make a Chart that works for most of them, the deploy of each microservices would differ only by the image and the resources required. This way you get an standardized deployment and easier to all developers.
Another use case is deploying applications which requires a lot of moving parts. For example, if you want to deploy a Grafana server on Kubernetes you're probably going to need at least a Deployment and a Configmap, then you would need a service that matches this deployment. And if you want to expose it to the internet you need an ingress too.
One relatively simple application, would require 4 different YAMLs that you would to manually configure and make sure everything is correct instead you could do a simple helm install and reuse the configuration that someone has already made, sometimes even the company who created the Application.
There are a lot of other use cases, but these two are the ones that I would say are the most common.
Here's three suggestions of ways Helm can be useful:
Your continuous deployment system somewhat routinely produces new builds and wants to send them to the Kubernetes cluster. You can use templating to specify the image name and tag in a deployment, and so helm upgrade ... --set tag=201907211931 to request a specific tag.
You might have various service-specific controls like the log level or external database hostnames. The Helm values mechanism gives a uniform way to specify them, without having to know the details of the Kubernetes YAML files.
There is a repository of pre-packaged application charts, so if you want replicated PostgreSQL with in-cluster persistent storage, that's already built for you and you can just depend on it, rather than figuring out the right combination of StatefulSets and PersistentVolumeClaims yourself.
You can combine these in interesting (and potentially complex) ways: use an in-cluster database for developer testing but use a cloud-hosted and backed-up database for production, for example, and compute the database host name based on what combination of settings are provided.
There are, of course, alternative ways to do all of these things. Kustomize in particular can change the image value fairly straightforwardly, and is notable for having been included in the kubectl tool since Kubernetes 1.14 (see also Declarative Management of Kubernetes Objects Using Kustomize in the Kubernetes documentation). The "operator" pattern gives an alternate path to install software in your cluster, but even more so than Helm you're trusting an arbitrary program with API access.
I will be building an Android application (not a game) soon. I heard of containerized development and Docker/Kubernetes but I'm not well-versed in its functions and use cases.
Why should I build my Android application with Kubernetes?
Your question can be split up into two parts:
1. Why should I containerize my deployment?
I hope by "deployment", you are referring to the backend services that serve your Android application; not the application itself (not sure how one would do that...). Here is a good article.
Containerization is a powerful abstraction that can help you manage both your code and environment. Setting up a container with the correct dependencies, utilities etc., and securing them is a lot of work, as is the case with any server setup. However, once you have packaged everything into a container, you can deploy said container multiple times and build on-top of it. The value of the grunt work that you have done in the past is therefore carried forward in your future deployments; conversely, so are the bugs... Additionally, you can also leverage the Docker ecosystem and build on various community contributions greatly accelerating your workflows.
A possible unintended advantage is also protection against configuration drift. Whenever services fail or your application crashes, you can simply restart your container, and a fresh version of the service will be created again. However, to support these operations, you need to ensure that your containerized service behaves nicely across restarts and fails gracefully. There are many other caveats and advantages that are not listed here; you can find more discussion on Google.
2. Why should I use Kubernetes for my container orchestration?
If you have many containers (think in the order of 100s), then using a single-node solution like Docker/docker-compose to manage them becomes tedious.
If only there was a tool to manage across multiple nodes, implement service discovery between your nodes, have fault tolerance (ie. automatic restarts, backoff policies), do health-checking of your services, manage storage assets, and conveniently expose your containers to the public. That tool is Kubernetes.
Here is a more in-depth intro.
Hope this helps!
Our application environment in Websphere Application Server has 3 clusters
1. UI Cluster
2. Service Cluster
3. Integration Cluster
We have around 50 war files (Micro Services) deployed to Service cluster. All services are REST based and exposed through SPRING API. Restarting Service cluster takes close to 30 mins. This time is critical during live incidents in Production. For reasons, if Service cluster needs to be restarted, we need to have 30 mins downtime for all end users. We are looking to reduce to recycle time, please suggest us for any solution.
Is there a way to load all the Spring based jar files before the application starts?
i.e. for example there is a service war file called as xyz-1.0.war and there are Spring based jar files as maven dependencies. All 50 number of WAR files has the same set of dependencies, I am thinking in a way to see if we can load all the Spring based jars before the application is started by websphere server.
Please suggest.
I don't know that you can load them BEFORE the application starts (class loading is generally on-demand), but you might be able to speed things up through the use of shared libraries for your common files, so they'd be loaded by a single class loader rather than from each WAR's class loader. It won't eliminate the class loading activity, since each WAR would still need to load the necessary classes, but it'd speed up the mechanics of the class loads since the shared library loader would return the already-loaded class rather than searching its class path.
There are two different approaches you could take to this. Step one in both cases is to create a shared library with the classes that are shared among the applications. The options for step two:
1) Create a custom class loader on the server and associate the shared library with this new class loader. This will make the classes in the shared library visible to all applications running on the server.
2) In the shared library configuration, select "Use an isolated class loader for this shared library", then associate the shared library with any applications that require it. In the event that the shared classes are required only by some applications, this will provide them only to the applications that require them.
A couple points of caution:
If you require unique Class instances (for example, static values unique to each WAR), this approach won't work, because there will be only one instance of the Class loaded by the shared library loader. In that event, you'll have to stick with WAR-level packaging.
If you use the isolated class loader solution, note that those loaders use "parent last" class loading, in which they are searched before server class loaders. If you have anything in those libraries that conflicts with classes provided by the server, it could open you up to ClassCastExceptions or LinkageErrors.
Note that the shared library loaders operate as parents of the WAR loaders, and as such, classes in the libraries will not be able to "see" classes in the WARs. You'll need to make sure that the libraries are essentially self-contained in order for these approaches to be successful.
More specific details on the configuration steps can be found in this blog post: https://www.ibm.com/developerworks/community/blogs/aimsupport/entry/create_shared_library_and_associate_it_with_the_application_server_or_application_on_websphere_application_server?lang=en_us
If you are doing microservices then your services should be independently deployable, so they each should be in the separate cluster. Traditional WebSphere Application Server is a bit heavy weight for this solution (depending on how many resources you have on your nodes), so I'd suggest you to migrate your service cluster to WebSphere Liberty and in that case you could have each service in separate clusters. This would allow you to restart each service independently, at much shorter time.
If you are doing microservices, then your UI cluster should be prepared for any service unavailability - that is a primer when doing microservices, and display some message to end user, that this service is temporary unavailable.
Regarding your current setup - you could try the "Rollout update" option, which will restart your server sequentially, so services should be available on other nodes.
so-random-dude advice to use blue/green deployment is also very good. You could have 2 cells, and then switch plugin configuration after redeployment. If your services are written in a way that different versions can run in parallel during update, you would have no downtime.
If you want to reduce downtime further and improve performance, you should consider using Java EE/Rest services instead of Spring, as it will cut down significantly size of your app, amount of libs to be scanned, deployment and startup time. It is much better integrated and supported in WebSphere Liberty than tons of jars that you have to include with Spring.
I have a simple solution for you, just ditch the Websphere and deploy those 50 "wars" as independent jars with embedded netty/undertow/tomcat/jetty in it.
I am afraid, what you have currently is not at all a microservice architecture. Agreed, different teams/consultants/organizations has different interpretation of "micro"services. But this is an extreme, which you should avoid at any cost; because you are having all the pain points of microservice and ZERO benefits (benefits such as independent scalability/deployability etc).
Restarting Service cluster takes close to 30 mins. This time is
critical during live incidents in Production. For reasons, if Service
cluster needs to be restarted, we need to have 30 mins downtime for
all end users
Have you looked at different deployment strategies like Canary deployment / Blue-Green deployment? Do you have more than one instances behind a load balancer?
I have a Hybris deployment on a single Weblogic Managed Server. The problem is during performance testing it was found that it would be better to split the Hybris modules like Admin Cockpit and Product Catalogue to different Managed Servers.
EDIT
I Suppose I should also mention the fact that my Infra Team is asking me to separate out the EARs so that in case of code changes, only the affected module gets redeployed and not the whole bunch. That way even if we let the performance front out, still I need the splits
Now my problem is that for build Hybris produces a single EAR file.
Is there a way, in which I can break down the EAR file and have the modules optionally there...
So the structure would be:
Managed Server 1
Hybris Core
Admin Cockpit
Managed Server 2
Hybris Core
Product Catalogue
After this the links to the deployments be redirected via URL configuration
Any Suggestions??
I'm not sure if this will eliminate the problems you encounter as I don't think the admin cockpit by itself will be causing a performance bottleneck.
What is the performance issue? Quite often performance impact can come from admin/backend triggered functionality like e.g. cronjobs (e.g. updating product catalog with stock/product information), or solr indexing jobs etc.
One common approach I have seen in hybris cluster environments is to setup a cluster of multiple nodes and have one node reserved for backend activity (so that expensive cronjobs run on a dedicated node that is not served by the load balancer handling storefront requests).
But I think from a code deployment perspective the artifact would still be the same.
Hope this helps at least as food for thoughts :)
EDIT
In short: Multiple hybris servers accessing the same db need to be setup as cluster.
Multiple hybris servers with different sets of extensions can't share the same db (as the db layout will be different).
To be honest, this doesn't sound like a good approach to me.
In hybris you would use different localextension.xml files (which define which extensions (i.e. modules) are part of your code artifact). That being said, if you have two vastly different localextensions.xml files (one for your product catalog and one for admin) the resulting 'admin' deployment artifact would not contain the data model of the 'catalog' deployment, so the persistence layer wouldn't match up. In other words, in your admin server you wouldn't be able to even see the data model that is defined in your 'catalog' server because the 'catalog' specific extensions are not installed.
And if you go without a properly set up cluster environment, changes on one server (written down to the db) wouldn't be noticed on the other server unless you actively refresh/purge the hybris cache there either, so multiple hybris servers sharing the same db is only functioning if the servers are set up as a cluster.
I think if your admin server is supposed to work on the actual 'catalog' data, they both need to have the same set of extensions defined in their localextensions.xml in order for it to work at all.
Sharing the same database without being aware that there is a cluster (or basically other hybris servers accessing the same db) is not going to work IMO.
I still think your best shot would be to deploy the same code artifacts (in cluster environments you can still setup different behavior/configuration per node). You could still (if you are 100% sure of it) deploy a new release with code changes that affect only your 'catalog' node only on that catalog node if you want to reduce downtime etc, but its always a risk if you have a cluster with different deployments on each node.
Good luck :)