We are working on MTA project and using the SAP Cloud SDK pipeline (Piper) to integrate CI/CD for Cloud Foundry.
The project has multiple microservices based on SAP Cloud SDK for Java and each of these microservices has individual sets of integration tests (based on Java project template for SAP Cloud SDK).
We want to do MTA packaging of the microservices but problem arises when we run the mta build with the SAP Cloud SDK pipeline.
The pipeline errors out as it now (i.e. the new structure) expects integration-test module on the project root level but in our case integration tests are at the individual microservice level.
We understand integration tests for all the microservices need to be clubbed at the project root level for mta build but is there any other recommended approach to handle the integration tests at the individual microservice itself?
Appreciate if someone can guide us here to understand the new structure and the best practice to follow.
Just to add as an information - we run maven build using s4sdk pipeline for each of the aforementioned microservices. We want to club these microservices into MTA package for release deployment.
Let me say a few additional words:
The structure of MTA projects suitable for using SAP Cloud SDK Pipeline is documented here.
It is easier to start such a project based on the SAP Cloud Platform Business Application SAP Web IDE Template than to start with one of the SAP Cloud SDK archetypes.
I think that there is a slight contradiction between the idea of microservices and one mta project, because microservices should be deployable in isolation. That said, using mta to structure your application is perfectly fine, but in this case having the integration tests in the project root as documented and as you already discovered is the only way. You can use packages to separate logical entities of your integration tests.
The alternative if your code should be more strictly separated would be to have multiple repos (might use mta or not) and to setup a pipeline for each of those repos.
I hope this helps.
Related
We are using mta to structure our application consisting of multiple micro-services.
The mtar file is deployed to SAP Cloud Foundry in blue-green mode using the SAP Cloud SDK pipeline (Piper Project -> cloudFoundryDeploy step).
We are running into the below issue while trying partial deployment with the blue-green strategy.
Issue: If MTA with same ID is redeployed, pipeline creates new MTA color (Blue to Green and vice-versa) and all applications with the old color are deleted. This approach creates issue during partial deployments.
For instance - using blue-green deployment approach, assume that we have deployed 10 micro-services to SAP CF space.
After bug fixes, we want to do partial redeployment for only a couple of micro-services.
In this case, if we use the same MTA ID and include only the two micro-services in mta.yaml, pipeline deletes the other 8 micro-services which are already deployed to SAP CF.
We tried updating the MTA version but it doesn't make any difference.
As a result of this issue, we unnecessarily need to redeploy all the 10 micro-services again although fix was done for only a couple of micro-services.
On the flip side, if we use a different MTA ID for redeploying the two micro-services, pipeline treats those as initial deployment and triggers an action which is same as the standard deployment type. This results in downtime and defies the purpose of blue-green deployment approach.
Appreciate if someone can help us here to resolve this issue of partial deployment using blue-green strategy and guide us on the best practices to follow.
I don't think this is currently possible, and the right place to build this would probably be multiapps-cli-plugin. I think it would be best if you opened an issue in that repo.
On another note: With that requirement of "partial deployments" I'm not sure if MTA is the best choice. Is there a reason why you need MTA? If you had one maven project per microservice, with one pipeline each this would be closer to the general concept of microservices where each unit can be deployed independently.
Hope this helps
Florian
I have some SAPUI5 Fiori applications developed using SAP WEB IDE in the Neo platform. Now I need to migrate those applications to the cloud foundry environment. So far as I checked there are few differences in the application files of NEO & Cloud Foundry.
eg:
Neo has neo-app.json and cloud foundry has xs-app.json
Are there any guidance or information related on correctly migrating SAPUI5 apps from Neo to Cloud Platform.
There are indeed not that many differences but there is some different handling in terms of tooling files, etc ...
I would propose to create a new multi-target test project using the provided standard templates and then add a UI5 test template subproject.
That will give you an idea of how the CF UI5 project structure looks like and which files are being generated.
It also depends on whether you are going to use the new HTML5-repo application architecture or not. That newer format has more impact and changes under the hood.
I need a devOpts deployments tool that can do the following tasks:
Creating a project that can contain different queues or artifacts within a GUI application
Create Queues and artifacts via a GUI accessible to non WMQ experts within the DEV environment only (without relying on a script)
Once tested properly, having the tool deploy the project (more specifically the queues contained in the project) from DEV to INT via a GUI that is only accessible by the Deployment team
Does anyone have any suggestions of tools that can perform this other than using scripts
(I have heard mixed reviews about IBM uDeploy, that configurations can not be reused, etc)
There are lots of 3rd party MQ Administration/Configuration tools available for IBM MQ. You can find a list of them here.
What is the best way to achieve DevOps with XPages.
Multiple Developers working as a team, On Premises Servers [Dev, QA,
Prod] can we replicate to Bluemix? Source Control Automated Testing UI
/ Application, Unit testing business logic with testing framework, Automated Deployment
IDE/Tools
Domino Designer; are there other ways?
Note: Use of Views when the data is in a NSF, otherwise data is in the cloud, or SQL. No Forms or other classic Notes design elements.
What are your approaches to this?
This is a high level overview of the topics required to attempt what you're describing. I'm breezing past lots of details, so please search them out; I've tried to reference what I'm currently aware of as far as supporting documentation and blog posts, etc. of others. If anyone has anything good to add, I'm happy to add it in.
There are several components involved with what you're describing, generally amounting to:
scm workflow
building the app (NSF)
deploying the built app to a Domino server
Everything else, such as release workflow through a QA/QC environment, is secondary to the primary steps above. I'll outline what I'm currently doing with, attempting to highlight where I'm working on improving the process.
1. SCM Workflow
This can be incredibly opinionated and will depend a lot on how your team does/wants to use source control with your deployment / release process. Below I'll touch on performing tests, conceptually, during/around the build step.
I've switched from a fairly generic scm server implementation to a GitLab instance. Even running a CE instance is pretty fantastic with their CI runner capabilities. Previously, I had a Jenkins CI instance performing about the same tasks, but had to bake more "workflow" into the Jenkins task, whereas now most of that logic is in a unified script, referenced from a config file (.gitlab-ci.yml). This is similar to how a Travis CI or other similar CI config file works.
This config calls some additional helper work, but ultimately revolves around an adapted version of Egor Margineanu's PowerShell script which invokes the headless DDE build task.
2. Building an NSF from Source
I've blogged about my general build process, with my previous Jenkins CI implementation. I followed the blogging of Cameron Gregor and Martin Pradny for this. Ultimately, you need to:
configure a Windows environment with Domino Designer
set up Domino Designer to import from ODP (disable export), ensuring Build Automatically is enabled
the notes.ini will need a flag of DESIGNER_AUTO_ENABLED=true
the Jenkins CI or GitLab CI runner (or other) will need to run as the logged in user, not a Windows service; this allows it to invoke the "headless dde" command correctly, since it runs in the background as opposed to a true headless invocation
ensure that Domino Designer can start without prompting for a user's password
My blog post covers additional topics such as flagging the build as success or failure, by scanning the output logs for being marked as a failed build. It also touches on how to submit the code to a SonarQube instance.
Ref: IBM Notes/Domino App Dev Wiki page on headless designer
Testing
Any additional testing or other workflow considerations (e.g.- QA/QC approval) should go around the build phase, depending on how you set up your SCM workflow. A lot of the implementation will revolve around the specifics of your setup. A general idea is to allow/prevent deployment based on the outcome of the build + test phase.
Bluemix Concerns
IBM Bluemix, the only PaaS that runs IBM XPages applications, will require some additional consideration, such as:
their Git deploy process will only accept a pre-built NSF
the NSF must be signed by the account owner's Bluemix ID
Ref:
- IBM XPages on Bluemix
- Bluemix Docs: Building XPages apps for the Bluemix Runtime
3. Deploy
To Bluemix
If you're looking to deploy an XPages app to run on Bluemix, you would want to either ensure your headless build runs with the Bluemix ID, or is at least signed with it, and then deploy it for a production push either via a git connection or the cf/bluemix command line utility. Bluemix's receive hooks handle all the rest of the deployment concerns, such as starting/stopping the server instance, etc.
To On-Premise Server
A user ID with appropriate level credentials needs to perform the work of either performing a design replace/refresh or stopping a dev/test/staging server, performing the file copy of the .nsf, then starting it back up. I've heard rumors of Cameron Gregor making use of a plugin to Domino Designer to perform the operations needed for OSGi plugin development, which sounds pretty useful. As most of my Domino application development is almost purely NSF based, I'm focusing more on an approach of deploying to a staging/dev/test server, which I can then perform a design task on to do the needed refresh/replace; closer to the "normal" Domino way of doing things.
Summary
Again, there are a lot of moving pieces involved here, some of which gets rather opinionated rather quickly. For example, I'm currently virtualizing my build machine, so I can spool up a couple virtual machines of it, allowing for more than one build at a time. If there are major gaps in the process, let me know and I'll fill it what I can.
Status Quo
For our customer we are developing some libraries and applications that run as "modules" within a larger application that is delivered as a internal Java Web Start application. The customer maintains the infrastructure that this application is run on. The "server side" is made up of a few web services based on Axis2. Both parts are deployed to a single Tomcat instance as two separate web applications.
When we are releasing new versions of our artifacts, we just produce the necessary JAR files (e.g. ourapp-client.jar and ourapp-server.jar) and send them to our customer who in turn just drops them into the appropriate places and--if need be--restarts the Tomcat server.
Goals
We are currently Maven-izing all of our projects and in the future we also want to do our releases in a "Maven way". The main goal is to automate the release and deployment process on our side and make it less error prone and more reliable and comfortable on the client side.
Main problem
The tricky part is that our customer uses the same Tomcat web applications (Axis2 for the "server side" and the Web Start app) to include their self-developed modules into the application. So we cannot use the obvious solution and just deliver a fresh web app (WAR) that simply gets deployed into the server. That is why we are currently delivering single JAR files that are put "by hand" into the right location.
What strategies do you in general use for delivering your products to your customer? Does someone have had any experiences with a similar situation (i.e. shared runtime environment for 3rd party and self developed applications)?
The main goal is to automate the release and deployment process on our side and make it less error prone and more reliable and comfortable on the client side.
Not sure what you mean exactly by "release and deployment process" (in the maven lingua, this is about SCM tasks automation and deployment of artifacts to a remote repository).
If this is about deployment to production machines, I personally don't think this is really a job for Maven (and we don't use Maven for this). Maybe have a look at dedicated solutions like ControlTier, SmartFrog, Puppet, Chef, etc.
What strategies do you in general use for delivering your products to your customer? Does someone have had any experiences with a similar situation (i.e. shared runtime environment for 3rd party and self developed applications)?
We deliver the things we can control. If a customer wants to include his own bits in a given application, we would make the required subparts available but the packaging would probably be his responsibility.
Related questions
Automated Deployment solution for multiple Java web-apps