I see these both in Bluemix, but what is the difference between them?
Cloud Foundry and OpenWhisk are two Bluemix Compute models that a developer can used to power an application's workload.
I'll give a very high-level summary of both services and when I would use them...
Cloud Foundry
IBM Bluemix was originally based off Cloud Foundry's open technology. It is a cloud computing platform as a service that supports the full lifecycle, from initial development, through all testing stages, to deployment.
Cloud Foundry has a CLI program called cf which is the primary tool to interact with Bluemix (or Bluemix provides a web GUI for this).
Cloud Foundry introduces the concepts of Organizations that contain Spaces which you can think of as workspaces. Different spaces typically correspond to different lifecycle stages for an application.
Cloud Foundry introduces the concepts of Services and Applications. A Cloud Foundry service usually performs a particular function (like a database service), and an application usually has services and their keys bound to it.
OpenWhisk
OpenWhisk is a brand new IBM Cloud developed distributed event-driven compute model.
It has a distributed automatically scaling serverless architecture that executes application logic on events.
OpenWhisk also has a CLI program called wsk which can be used to run your code snippets, or actions, on OpenWhisk.
OpenWhisk introduces the concepts of Triggers, Actions, and Rules.
Triggers are a class of events emitted by event sources.
Actions encapsulate the actual code to be executed which support multiple language bindings including Node.js, Swift and arbitrary binary programs encapsulated in Docker Containers. Actions invoke any part of an open ecosystem including existing Bluemix services for analytics, data, cognitive, or any other 3rd party service.
Rules are an association between a trigger and an action.
Cloud Foundry vs. OpenWhisk
So the question remains: when should you use Cloud Foundry, or when should you use OpenWhisk?
In my limited experience using OpenWhisk, here are my thoughts. I like to think of OpenWhisk as an easily implementable automatically scaling architecture that application developers can use without needing much prior knowledge in backend development. I think of Cloud Foundry as a lower level in the software stack which might give you more customization, but will likely take more skill and knowledge for setting it up.
I would use Cloud Foundry if I...
Was a backend & application developer.
Had experience creating and connecting services together.
Needed functionality that just might not be possible using OpenWhisk.
I would use OpenWhisk if I...
Was an application developer.
Didn't want to worry about a server.
Didn't want to learn different programming languages, etc. to figure out how to set up my server.
Really wanted focus on developing my application and have the backend just work.
Hope that helped.
Edit:
Here's a cool image that I found that illustrates this:
CloudFoundry is a PaaS (Platform-as-a-service) platform, which means in a nutshell, that it hosts the platform for your application to run on. Examples of a platform include node.js or a JVM.
OpenWhisk is a serverless platform. The term FaaS (Function-as-a-service) seems to be emerging as well. You upload code, which is executed once an event happens. That event might be anything, ranging from a simple HTTP request to a change happening in your database.
The fundamental difference between the two is the mode of operation. PaaS means, you're still running a server-process. You'll have a long running process which listens to events and executes your logic, once an event happens. All the other time, the process is idle, still requiring CPU cycles and memory to actually listen for events.
In serverless, the platform takes the burden of "listening for events". Once an event happens, your code is instantiated and executed. That code is shutdown afterwards thus not requiring any resources anymore. That also explains why OpenWhisk actions have a time limitation of 5 minutes. It is not meant to have long running actions.
Disclaimer: Both platforms support a lot more than I described here, I tried to keep it down to the most substantial difference between the both.
Related
Can somebody bring some light in IBM Cloud deployments tools / platforms whatever?
I am new to it, so I am looking at their docs, watching videos and still i am confused.
What I want to achieve is typical scenario fetch code from repo, build it, test it, deploy it to cloud. I found strategies / platforms how to achieve that and i still can't see differences advantages / disadvantages between them.
So we have:
toolchain
cloud foundry
code engine
Continuous Delivery (service)
and maybe something more? :)
I am looking at Cloud Foundry explained video and the guy is saying if you want to do not care about the bottom part like networking, security, containers you can choose deploy using K8S service. Wtf? So from total automatic thing you can now handle something in the cloud foundry by yourself. So for me its total mix of everything together and i don't know now which tool / platform / strategy to use.
Any comment is appreciated.
It all depends on your requirements.
IBM DevOps/Continuous Delivery/Toolchanis is set of services that can build and deploy your application to given runtime. You can find various tutorials here - https://www.ibm.com/cloud/architecture/courses/toolchain-tutorials. These tutorials shows you various things that you can embed in your build pipelines (like code scanning with CRA, image scanning, signing etc)
These runtimes can be different depending on your requirements:
CloudFoundry, where you deploy app using a buildpak, but this is rather fading technology, so I wouldn't recommend that
as docker image in K8s/OpenShift cluster - use this if your organization is planning or already utilizing Docker/Kubernetes/OpenShift. You will need to create K8s/OpenShift cluster first.
as serverless app, using the IBM Code Engine
If you are just starting and just want to deploy simple, single app to the Cloud I'd consider using IBM Code Engine and not investigate Toolchains for now. Check basic demo here - How to deploy source code with IBM Cloud Code Engine
I have a single screen desktop application developed in Java. It is a tool to convert files, given a file in .abc format, the tool converts it to .xyz format. Basically the tool works offline and acts as a translator to convert file from one form to another.
So now, to improve the infrastructure, there are discussions to move the tool to Kubernetes or to provide REST services for the file conversion. I completely have no idea about the containers nor the REST APIs as I am a front-end developer.
More about the tool, as I told earlier, the tool is a single page application, very light doing very minimal job, totally used by 200 users approximately. So, this being the shape and size of the application, which one would be the best approach to go with and why? Basically, I am looking for a short evaluation report of Kubernetes vs REST service and architecture recommendation with reasons.
Currently your application is a standalone application which is quite an old concept.
I can mention high-level changes needs to be done when your file conversion logic would be exposed over Rest Api in Kubernetes world.
you can go through one by one following mentioned areas to get a better understanding design-wise:
java code would be a backend code and its public methods that take inputs from UI actions will be exposed over rest API.
There are multiple rest API's (jersey, rest easy, etc or spring/spring-boot framework also provides rest API support) that you can go through any of them to get an understanding.
once your backend is exposed over the rest API then it needs to be containerized means your backend will be running under the container. Can go through docker documentation and can build one sample containerized app. There is huge material present in this area.
once your backend is containerized then it will be installed in a Kubernetes cluster
Kubernetes is basically a container orchestration tool and it's quite a wide thing. you can through its official documentation for basic understanding.
SPA will be running on a client machine like today also you are able to launch from your desktop but it will communicate with the Kubernetes cluster where your application is presently packaged in a container.
References:
docker :
https://docs.docker.com/
Kubernetes :
https://kubernetes.io/
As I dive into the world of Cloud Composer, Airflow, Google Kubernetes Engine, and Kubernetes I've not yet found a good answer to what exactly makes Cloud Composer better than Helm and GKE.
Here are some things I've found that could be unique to Composer but mostly seem like they could be handled by GKE.
On their homepage:
End-to-end integration with Google Cloud products including BigQuery, Dataflow, Dataproc, Datastore, Cloud Storage, Pub/Sub, and AI Platform gives users the freedom to fully orchestrate their pipeline.
On the features page:
Identity-Aware Proxy protects the interface
Cloud Composer associates a Cloud Storage bucket with the environment. The associated bucket stores the DAGs, logs, custom plugins, and data for the environment.
The downsides of Composer I've seen include:
It takes many hours to spin up a new instance
It doesn't support Kubernetes Executor
It is risky to change the underlying GKE config because it could be changed back by a composer update
There are often errors that happen when auto-scaling often happen but are documented as known
Upgrading environments is still beta
To be clear, I'm not saying Cloud Composer is bad. I'm just having trouble seeing why people like it. When I've asked folks why it is better than Helm + GKE they haven't had any compelling answers despite that they can tell many stories of Composer being unpredictable and having lots of issues.
Are you comparing the same things?
On one side, GKE, you have a container orchestrator. Declare that you want, it will deploy and maintain the stability of the cluster according with declared configuration. This configuration can be packaged with helm to write it in an easier mode. Because you deploy container, you can use the language that you want in your services.
On the other side, you have a workflow manager, with scheduler, retry policies, parallel task, context forwarding. you write DAG in python (only!) and you have operators to interact with external product/services. It's mainly designed for data processing and used a lot by data scientist and data engineering team.
Note: Cloud Composer is deployed on top of GKE (scheduler and worker), redis, app engine and Cloud SQL.
You compare 2 different worlds: Ops world (GKE/Helm) and the App/Data world (Composer/Airflow). Have a look to this new video
Update 1:
My bad, I didn't understand!!! Anyway, personally I don't want to manage things by myself: a cluster, the update of K8S, VM patching, replicas, snapshot, backup/restore,...
If someone can do this for me, I prefer, and managed services are perfect for me!!
Do you ask yourselves this question about Cloud SQL and a database managed by yourselves on a Compute Engine instance? If not (because Cloud SQL solve a lot of boring issues), my opinion is the same for Composer.
But it's an opinion, I didn't test both and compare the performance, cost and easiness.
We have a kind of evaluation job which consists of several thousand invocations of a legacy binary with various inputs, each of which running like a minute. The individual runs are perfectly parallelizable (one instance per core).
What is the state of the art to do this in a hybrid cloud scenario?
Kubernetes itself does not seem to provide an interface for prioritizing or managing waiting jobs. Jenkins would be good at these points, but feels like a hack. Of course, we could hack something ourselves, but the problem should be sufficiently generic to already have an out-of-the box solution.
There are a lot of frameworks that helps managing jobs in Kubernetes cluster. The most popular are:
Argo for orchestrating parallel jobs on Kubernetes. Workflows is implemented as a Kubernetes CRD (Custom Resource Definition).
Airflow - has a modular architecture and uses a message queue to orchestrate an arbitrary number of workers. Also take a look for kubernetes-executor.
I recommend you to look for this video which describe each of framework and help you decide which is better for you.
You may be interested in following aricles about using Mesos for Hybrid Cloud
Xue, Noha & Haugerud, HĂ„rek & Yazidi, Anis. (2017). Towards a Hybrid Cloud Platform Using Apache Mesos. 143-148. 10.1007/978-3-319-60774-0_12.
Hybrid cloud technology is becoming increasingly popular as it merges private and public clouds to bring the best of two worlds together. However, due to the heterogeneous cloud installation, facilitating a hybrid cloud setup is not simple. Despite the availability of some commercial solutions to build a hybrid cloud, an open source implementation is still unavailable. In this paper, we try to bridge the gap by providing an open source implementation by leveraging the power of Apache Mesos. We build a hybrid cloud on the top of multiple cloud platforms, private and public.
Apache Mesos For All Your Hybrid Cloud Needs
Choosing the Best Approach to Hybrid Cloud
How exactly would someone define BlueMix to an engineering major with little to no knowledge in Comp. Sci?
Bluemix is a fully managed cloud service so all of the operations activities such as maintenance, availability, upgrades are part of the Bluemix service, so users don't have to worry about setting up their own infrastructure or installing software.
Bluemix makes it easy for application developers to write applications as it is a polygot environment that supports many different languages and runtimes.
Applications in Bluemix can easily be scaled as needed leveraging the elasticity of the cloud. There are over 100 services available in Bluemix (both IBM and 3rd party services) which can be tied together to make robust applications and also to implement a microservices architecture.
Bluemix cloud services are available as part of the public multi-tenant cloud offering which runs on softlayer. There are also dedicated and local Bluemix cloud offerings available for users who don't want to use the public cloud or want to combine the different offerings to create a hybrid cloud.
it is a platform that lets you build, run, deploy applications via cloud. moreover, it handles multiple languages
Check Bluemix Overview topic in the docs: https://www.ng.bluemix.net/docs/overview/index.html