Is it possible to build a standalone Service Fabric cluster where Microsoft Nano is used as nodes?
I presume you mean for on-premise deployment. If it is then the answer is no as the minimum requirement wont be met.
https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-cluster-standalone-deployment-preparation
Prerequisites for each machine that you want to add to the cluster:+
A minimum of 16 GB of RAM is recommended.
A minimum of 40 of GB available disk space is recommended.
A 4 core or greater CPU is recommended.
Related
RAM- 4GB,
PROCESSOR-i3 5010ucpu #2.10 GHz
64 bit OS
can Cassandra and MongoDB be installed in such a laptop? Will it run successfully?
The hardware configuration proposed does not meet the minimum requirements. For Cassandra, the documentation requests a minimum of 8GB of RAM and at least 2 cores.
MongoDB's documentation also states that it will need at least 2 real cores or one multi-core physical CPU. With 4GB in RAM, the WiredTiger will allocate 1.5GB for the cache. Please also note that MongoDB will require changes in BIOS to allow memory interleaving to enable Non-Uniform Access Memory, a.k.a. NUMA, such changes will impact the performance of the laptop for other processes.
Will it run successfully?
This will depend on the workload expected to be executed; there are documented examples where Cassandra was installed on a Raspberry Pi array, which since the design it was expected to have slow performance and have a limited amount of data that can be held in the cluster.
If you are looking to have a small sandbox to start using these databases there are other options, MongoDB has a service named Atlas, with a model of a database as a service, it offers a free tier for a 3-node replica and up to 512Mb of storage. For Cassandra there are similar options, AWS offers in the free tier a small cluster of their Managed Cassandra Service (MCS), Datastax is also planning to offer similar services with Constellation
Is the Service Fabric service running on the primary node itself a stateful service?
The reason for asking is capacity planning. Recommended is d3v2, but is doing with d2v2 more than enough for Primary with Windows OS?
We are using Standard_B2ms for our system nodes. You could probably go lower on the system nodes, they are not under heavy load anyway
The Service Fabric system services that run on the primary node type are indeed stateful services. If you are doing capacity planning, I recommend you visit the following document which has our production readiness checklist. https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-production-readiness-checklist
For production workloads:
It's recommended to dedicate your clusters primary NodeType to system services, and use placement constraints to deploy your application to secondary NodeTypes.
The recommended VM SKU is Standard D3 or Standard D3_V2 or equivalent with a minimum of 14 GB of local SSD.
The minimum supported use VM SKU is Standard D1 or Standard D1_V2 or equivalent with a minimum of 14 GB of local SSD.
The 14 GB local SSD is a minimum requirement. Our recommendation is a minimum of 50 GB. For your workloads, especially when running Windows containers, larger disks are required.
Partial core VM SKUs like Standard A0 are not supported for production workloads.
Standard A1 SKU is not supported for production workloads for performance reasons.
What is the cheapest possible vm sku you can run a service fabric cluster on in Azure?
I just tried to create a 1 node cluster using DS1_V2 and got a warning "Not enough disk space". Then I tried DS2_V2 Promo and the warning goes away.
It cost 142.85 USD and you need 5 of them. So that will be a total cost of 714.25 $ month plus usage.
Is the minimum cost for a service fabric cluster really around 1.000 USD a month.
What are the minimum requirement for running it on premise?
Is it possible to deploy 1 virtual machine in azure, install service fabric on that and deploy to that. (I know that wont scale, be fault tolerant etc)
For a production environment, you are correct you will need at least 5x D class machines for a Service fabric cluster.
For a QA environment you can set up a 3 node cluster with a Bronze durability level which should bring down the costs a bit.
For a development environment, you could use the Service Fabric Local Cluster manager which allows you to emulate a 1 Node or a 5 Node environment on your local machine and recently there is a new option in Azure to create and run a 1 Node cluster - see below.
As for capacity planning, you can find some good guidelines in the official docs.
For production workloads
The recommended VM SKU is Standard D3 or Standard D3_V2 or equivalent with a minimum of 14 GB of local SSD.
The minimum supported use VM SKU is Standard D1 or Standard D1_V2 or equivalent with a minimum of 14 GB of local SSD.
Partial core VM SKUs like Standard A0 are not supported for production workloads
Standard A1 SKU is specifically not supported for production workloads for performance reasons.
I am running an instance group of 20 Preemptible GCE instance to read ORC files on Google storage, The data partitioned by hour, each hour about 2GB.
What type of instances should i use ?
How many of the Ram should be used by the JVM ?
I am using autoscale configuration of 80% CPU and 10 minute cooldown, Is there more subtitle config for Presto ?
Is there a solution for servers shutdowns, due to lack of resources ?
Partial responses will be appreciated as well.
As 0.199 version of PrestoDB there's no google cloud storage connector for Presto, which makes impossible to query GCS data.
Regarding hardware requirements, I'll cite Terada doc here.
Memory
You should allocate a minimum of 16GB of RAM per node for Presto. But
recommend 64GB for most production workloads.
Network Bandwidth
It is recommended to have 10 Gigabit Ethernet between all the nodes in
the cluster.
Other Recommendations
Presto can be installed on any normally configured Hadoop cluster.
YARN should be configured to account for resources dedicated to
Presto. For example, if a node has 64GB of RAM, perhaps you would
normally allocate 60GB to YARN. If you install Presto on that node and
give Presto 32GB of RAM, then you should subtract 32GB from the 60GB
and let YARN only allocate 28GB per node. An optimized configuration
might choose to have separate Presto and Hadoop nodes. The optimized
configuration allows you to give more memory to Presto, and thus
perform larger join queries, for example.
I am working on a setup where I am running a Ubuntu VM on a Fedora 18 host using QEMU/KVM and libvirt. Now I have pinned 2 vCPUs to my VM. I can see the pinned vCPUs using virsh. But, is there any other way to find that out? Ideally, I want to write a function which will return the number/id of pinned vCPUs.
You can use element in domain XML to determine which pinned CPUs for which vcpu. see section CPU Tuning. By default, the vcpu is pinned to all physical CPUs.
For example:
<cputune>
<vcpupin vcpu="0" cpuset="1-4,^2"/>
<vcpupin vcpu="1" cpuset="0,1"/>
</cputune>