The application stack I want to dockerize shall run within a CentOS container. The installation procedure verifies kernel version to ensure application requirements are met. Currently it is detected using "uname ...".
However the application now detects the host kernel version, which is "UBUNTU ..." and not "CentOS" ..."
Is it possible to detect the container's kernel version?
Thanks.
In fact, the kernel is the same in the host and in the container. That's the very principle of containerization: the kernel is shared (because actually, a container is a collection of processes running on top of the host kernel, with special isolation properties).
Does that pose a problem for your application?
Related
I am following this tutorial-
https://docs.confluent.io/platform/current/platform-quickstart.html
At step 3 when I click on "Connect" I see no option to add connector.
How do I add a connector?
For reference I am using M1 Mac book Air and Docker v4.12.0
You'll only be able to add a connector if you are running Kafka Connect server, and have properly configured Control Center to use it.
On Mac: Docker memory is allocated minimally at 6 GB (Mac). When using Docker Desktop for Mac, the default Docker memory allocation is 2 GB. Change the default allocation to 6 GB in the Docker Desktop app by navigating to Preferences > Resources > Advanced.
Assuming you already did that, then you need to look at the outputs from docker-compose ps and docker-compose logs connect to determine if the Connect containers are healthy and running.
Personally, I don't use Control Center since I prefer to manage connectors as config files, not copy/paste or click through UI fields. In other words, if Connect container is healthy, try using its HTTP endpoints directly with curl/postman, etc
I had exactly the same issue with there being no way to add a Connector.
Updating the container version from my old version 6.2.1 to 7.3.0 solved it.
I have a Yocto based OS on which I have everything installed to start the network.
Nevertheless, at each boot I need to do systemctl start networking to start it. Initially the service was even masked. I found out how to unmask it but I can't find a way to start it automatically.
I don't know much about systemd but the networking.service is located in generator.late folder. From what I understood, it's generated afterward.
How can I enable it?
It depends if you want to enable the service only on one particular device. If yes, it is simple:
systemctl enable networking
Append the parameter --now if you also want to start the service just now.
If you want to enable the service on all your devices (i.e. it will be automatically enabled in all your images coming from build), the best way is to extend the recipe, but please see below for other ways how to handle the network. The process is describe at NXP support for example.
Some notes about networking.service itself: I assume that your networking.service comes from init-ifupdown recipe. If yes, is there any reason to handle network configuration using old SysV init script in system with systemd? The service is generated from SysV init script by systemd-sysv-generator. So I would suggest to try other networking services like systemd's native "systemd-networkd", "NetworkManager" or "connman". The best choice depends on type of your embedded systemd. These services are integrated with systemd much better.
Some more information on activating or enabling the services: https://unix.stackexchange.com/questions/302261/systemd-unit-activate-vs-enable
I'm starting to play around with NixOS deployments. To that end, I have a repo with some packages defined, and a configuration.nix for the server.
It seems like I should then be able to test this configuration locally (I'm also running NixOS). I imagine it's a bad idea to change my global configuration.nix to point to the deployment server's configuration.nix (who knows what that will break); but is there a safe and convenient way to "try out" the server locally - i.e. build it and either boot into it or, better, start it as a separate process?
I can see docker being one way, of course; maybe there's nothing else. But I have this vague sense Nix could be capable of doing it alone.
There is a fairly standard way of doing this that is built into the default system.
Namely nixos-rebuild build-vm. This will take your current configuration file (by default /etc/nixos/configuration.nix, build it and create a script allowing you to boot the configuration into a virtualmachine.
once the script has finished, it will leave a symlink in the current directory. You can then boot by running ./result/bin/run-$HOSTNAME-vm which will start a boot of your virtualmachine for you to play around with.
TLDR;
nixos-rebuild build-vm
./result/bin/run-$HOSTNAME-vm
nixos-rebuild build-vm is the easiest way to do this, however; you could also import the configuration into a NixOS container (see Chapter 47. Container Management in the NixOS manual and the nixos-container command).
This would be done with something like:
containers.mydeploy = {
privateNetwork = true;
config = import ../mydeploy-configuration.nix;
};
Note that you would not want to specify the network configuration in mydeploy-configuration.nix if it's static as that could cause conflicts with the network subnet created for the container.
As you may already know, system configurations can coexist without any problems in the Nix store. The problem here is running more than one system at once. For this, you need an isolation or virtualization tools like Docker, VirtualBox, etc.
NixOS Containers
NixOS provides an efficient implementation of the container concept, backed by systemd-nspawn instead of an image-based container runtime.
These can be specified declaratively in configuration.nix or imperatively with the nixos-container command if you need more flexibility.
Docker
Docker was not designed to run an entire operating system inside a container, so it may not be the best fit for testing NixOS-based deployments, which expect and provide systemd and some services inside their units of deployment. While you won't get a good NixOS experience with Docker, Nix and Docker are a good fit.
UPDATE: Both 'raw' Nix packages and NixOS run in Docker. For example, Arion supports images from plain Nix, NixOS modules and 'normal' Docker images.
NixOps
To deploy NixOS inside NixOS it is best to use a technology that is designed to run a full Linux system inside.
It helps to have a program that manages the integration for you. In the Nix ecosystem, NixOps is the first candidate for this. You can use NixOps with its multiple backends, such as QEMU/KVM, VirtualBox, the (currently experimental) NixOS container backend, or you can use the none backend to deploy to machines that you have created using another tool.
Here's a complete example of using NixOps with QEMU/KVM.
Tests
If the your goal is to run automated integration tests, you can make use of the NixOS VM testing framework. This uses Linux KVM virtualization (expose /dev/kvm in sandbox) to run integrations test on networks of virtual machines, and it runs them as a derivation. It is quite efficient because it does not have to create virtual machine images because it mounts the Nix store in the VM. These tests are "built" like any other derivation, making them easy to run.
Nix store optimization
A unique feature of Nix is that you can often reuse the host Nix store, so being able to mount a host filesystem in the container/vm is a nice feature to have in your solution. If you are creating your own solutions, depending on you needs, you may want to postpone this optimization, because it becomes a bit more involved if you want the container/vm to be able to modify the store. NixOS tests solve this with an overlay file system in the VM. Another approach may be to bind mount the Nix store forward the Nix daemon socket.
I am new to kuberenetes.
is it possible to turn every end user machine (PCs, Macs, RPi ... etc) who with full consent downloaded my electron research app that should turn their machines into nodes that ultimately comprise a k8s cluster which then i can run kubeflow.org on to do ML research?
Thanks
Kubernetes relies on some container engine. Usually that's docker, there are efforts to create a common container interface for kubernetes and that's where CRI-O comes in, an abstraction that would allow any container engine to run underneath it.
That being said, containers "don't exist" they are a native abstraction in the linux kernel comprised of cgroups and namespaces and what that means is that the abstraction and isolation doesn't live in the hypervisor (which usually talks to the kernel) as is the case with regular virtual machines, but rather in the actual linux kernel.
MacOS uses its own kernel which, to the extent of my knowledge, doesn't support any sort of containers.
Windows does support containers via Hyper-V and i believe that windows server has a more native built-in support for them. See this link for a better explanation https://learn.microsoft.com/en-us/virtualization/windowscontainers/about/ and also for kubernetes https://kubernetes.io/docs/getting-started-guides/windows/.
As far as Raspberry PI goes there is an ongoing effort that brought k8s to ARM see this link (https://github.com/luxas/kubernetes-on-arm). That being said, you need an entire cluster of raspberry pis to actually make that work, as it would require a lot of resources. One raspberry pi won't get you very far.
How to go about this?
You need linux to run kubernetes. Everywhere.
If you want to create a "giant" kubernetes cluster your best bet is to use a virtualization technology for the PC that is running windows or for the Mac and create virtual machines that you can use as kubernetes nodes.
In short, you create virtual machines where there's no Linux and install kubernetes natively where there is.
Parallels, Veertu or plain Xhyve is a good way of running virtualization on MacOs.
VmWare or VirtualBox are good virtualizations for both windows and mac.
Libvirt and virtualbox are good solutions for linux virtualisation.
I just completed my vagrant box for a product that made by my company.
I needed that because we're running same product on different
operating systems. I want to serve sites inside virtual machines, I
have questions:
Am I on correct way? Can a virtual machine used as production
server?
If you say yes:
How should I keep virtualbox running? Are there any script or sth
to restart if something crashes?
What happens if somebody accidentally gives "vagrant destroy"
command? What should I do if I don't want to lose my database and user
uploaded files?
We have some import scripts that running every beginning of the
month. sometimes they're using 7gb ram (running 1500 lines of mysql
code with lots of asynchronised instances). Can it be dangerous to run
inside VirtualBox?
Are there any case study blog post about this?
Vagrant is mainly for Development environment. I personally recommend using Type 1 hypervisor (Bare metal), VirtualBox is a desktop virtulization tool (Type 2, running on top of a traditional OS), not recommended for production.
AWS is ok, the VMs are running as Xen guest, Xen is on bare metal;-)
I wouldn't.
The w/ Vagrant + Virtualbox is that these are development instances. I would look at Amazon Web Services for actually deploying your project into the wild.