I wanna use WarmUpRoute to prepare two cold memcached data node, I use this config:
{
"pools": {
"cold": {
"servers": ["xxxxx:11212", "xxxx:11213"]
},
"warm": {
"servers": ["xxxxx:11211"]
}
},
"route": {
"type": "WarmUpRoute",
"cold": "PoolRoute|cold",
"warm": "PoolRoute|warm"
}
}
But when I do some test, I connet mcrouter and set some data, found that only 1 cold server and warm server are success to save data, another cold node cannot set data successfuly. I am confuse, my config has some problem or is this a bug?
Related
We are having issues deploying our Service Fabric cluster to Azure and have it handle both IPv4
and IPv6 traffic.
We are developing an application that have mobile clients on iOS and Android which communicate with
our Service Fabric cluster. The communication consist of both HTTP traffic as well as TCP Socket communication.
We need to support IPv6 in order to have Apple accept the app in their App Store.
We are using ARM template for deploying to Azure as it seems the portal does not support configuring
load balancer with IPv6 configuration for Virtual Machine Scale Sets (ref: url). The linked page also states other limitations
to the IPv6 support, such as private IPv6 addresses cannot be deployed to VM Scale Sets. However according
to this page the possibility to assign private IPv6 to VM Scale Sets is available in preview
(although this was last updated 07/14/2017).
For this question I have tried to keep this as general as possible, and based the ARM template on a template found
in this tutorial. The template is called "template_original.json" and can be downloaded from
here. This is a basic template for a service fabric cluster with no security for simplicity.
I will be linking the entire modified ARM template in the bottom of this post, but will highlight the
main modified parts first.
Public IPv4 and IPv6 addresses that are associated with the load balancer. These are associated with their respective backend pools:
"frontendIPConfigurations": [
{
"name": "LoadBalancerIPv4Config",
"properties": {
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPv4Name'),'-','0'))]"
}
}
},
{
"name": "LoadBalancerIPv6Config",
"properties": {
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPv6Name'),'-','0'))]"
}
}
}
],
"backendAddressPools": [
{
"name": "LoadBalancerIPv4BEAddressPool",
"properties": {}
},
{
"name": "LoadBalancerIPv6BEAddressPool",
"properties": {}
}
],
Load balancing rules for frontend ports on respective public IP addresses, both IPv4 and IPv6.
This amounts to four rules in total, two per front end port. I have added port 80 for HTTP here and port 5607 for Socket connection.
Note that I have updated the backend port for IPv6 port 80 to be 8081 and IPv6 port 8507 to be 8517.
{
"name": "AppPortLBRule1Ipv4",
"properties": {
"backendAddressPool": {
"id": "[variables('lbIPv4PoolID0')]"
},
"backendPort": "[parameters('loadBalancedAppPort1')]",
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPv4Config0')]"
},
"frontendPort": "[parameters('loadBalancedAppPort1')]",
"idleTimeoutInMinutes": "5",
"probe": {
"id": "[concat(variables('lbID0'),'/probes/AppPortProbe1')]"
},
"protocol": "tcp"
}
},
{
"name": "AppPortLBRule1Ipv6",
"properties": {
"backendAddressPool": {
"id": "[variables('lbIPv6PoolID0')]"
},
/*"backendPort": "[parameters('loadBalancedAppPort1')]",*/
"backendPort": 8081,
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPv6Config0')]"
},
"frontendPort": "[parameters('loadBalancedAppPort1')]",
/*"idleTimeoutInMinutes": "5",*/
"probe": {
"id": "[concat(variables('lbID0'),'/probes/AppPortProbe1')]"
},
"protocol": "tcp"
}
},
{
"name": "AppPortLBRule2Ipv4",
"properties": {
"backendAddressPool": {
"id": "[variables('lbIPv4PoolID0')]"
},
"backendPort": "[parameters('loadBalancedAppPort2')]",
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPv4Config0')]"
},
"frontendPort": "[parameters('loadBalancedAppPort2')]",
"idleTimeoutInMinutes": "5",
"probe": {
"id": "[concat(variables('lbID0'),'/probes/AppPortProbe2')]"
},
"protocol": "tcp"
}
},
{
"name": "AppPortLBRule2Ipv6",
"properties": {
"backendAddressPool": {
"id": "[variables('lbIPv6PoolID0')]"
},
"backendPort": 8517,
"enableFloatingIP": "false",
"frontendIPConfiguration": {
"id": "[variables('lbIPv6Config0')]"
},
"frontendPort": "[parameters('loadBalancedAppPort2')]",
/*"idleTimeoutInMinutes": "5",*/
"probe": {
"id": "[concat(variables('lbID0'),'/probes/AppPortProbe2')]"
},
"protocol": "tcp"
}
}
Also added one probe per load balancing rule, but omitted here for clarity.
The apiVerison for VM Scale set is set to "2017-03-30" per recommendation from aforementioned preview solution.
The network interface configurations are configured according to recommendations as well.
"networkInterfaceConfigurations": [
{
"name": "[concat(parameters('nicName'), '-0')]",
"properties": {
"ipConfigurations": [
{
"name": "[concat(parameters('nicName'),'-IPv4Config-',0)]",
"properties": {
"privateIPAddressVersion": "IPv4",
"loadBalancerBackendAddressPools": [
{
"id": "[variables('lbIPv4PoolID0')]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[variables('lbNatPoolID0')]"
}
],
"subnet": {
"id": "[variables('subnet0Ref')]"
}
}
},
{
"name": "[concat(parameters('nicName'),'-IPv6Config-',0)]",
"properties": {
"privateIPAddressVersion": "IPv6",
"loadBalancerBackendAddressPools": [
{
"id": "[variables('lbIPv6PoolID0')]"
}
]
}
}
],
"primary": true
}
}
]
With this template I am able to successfully deploy it to Azure. Communication using IPv4 with the
cluster works as expected, however I am unable to get any IPv6 traffic through at all. This is the
same for both ports 80 (HTTP) and 5607 (socket).
When viewing the list of backend pools for the load balancer in the Azure portal it displays the
following information message which I have been unable to find any information about. I am unsure
if this affects anything in any way?
Backend pool 'loadbalanceripv6beaddresspool' was removed from Virtual machine scale set 'Node1'. Upgrade all the instances of 'Node1' for this change to apply Node1
load balancer error message
I am not sure why I cannot get the traffic through on IPv6. It might be that there is something I
have missed in the template, or some other error on my part? If any additional information is required
dont hesitate to ask.
Here is the entire ARM template. Due to the length and post length limitations I have not embedded it, but here is a Pastebin link to the full ARM Template (Updated).
Update
Some information regarding debugging the IPv6 connectivity. I have tried slightly altering the ARM template to forward the IPv6 traffic on port 80 to backend port 8081 instead. So IPv4 is 80=>80 and IPv6 80=>8081. The ARM template has been updated (see link in previous section).
On port 80 I am running Kestrel as a stateless web server. I have the following entries in the ServiceManifest.xml:
<Endpoint Protocol="http" Name="ServiceEndpoint1" Type="Input" Port="80" />
<Endpoint Protocol="http" Name="ServiceEndpoint3" Type="Input" Port="8081" />
I have been a bit unsure specifically which addresses to listen for in Kestrel. Using FabricRuntime.GetNodeContext().IPAddressOrFQDN always returns the IPv4 address. This is currently how we start it. For debugging this I currently get ahold of all the IPv6 addresses, and hardcoded hack for port 8081 we use that address. Fort port 80 use IPAddress.IPv6Any, however this always defaults to the IPv4 address returned by FabricRuntime.GetNodeContext().IPAddressOrFQDN.
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
var endpoints = Context.CodePackageActivationContext.GetEndpoints()
.Where(endpoint => endpoint.Protocol == EndpointProtocol.Http ||
endpoint.Protocol == EndpointProtocol.Https);
var strHostName = Dns.GetHostName();
var ipHostEntry = Dns.GetHostEntry(strHostName);
var ipv6Addresses = new List<IPAddress>();
ipv6Addresses.AddRange(ipHostEntry.AddressList.Where(
ipAddress => ipAddress.AddressFamily == AddressFamily.InterNetworkV6));
var listeners = new List<ServiceInstanceListener>();
foreach (var endpoint in endpoints)
{
var instanceListener = new ServiceInstanceListener(serviceContext =>
new KestrelCommunicationListener(
serviceContext,
(url, listener) => new WebHostBuilder().
UseKestrel(options =>
{
if (endpoint.Port == 8081 && ipv6Addresses.Count > 0)
{
// change idx to test different IPv6 addresses found
options.Listen(ipv6Addresses[0], endpoint.Port);
}
else
{
// always defaults to ipv4 address
options.Listen(IPAddress.IPv6Any, endpoint.Port);
}
}).
ConfigureServices(
services => services
.AddSingleton<StatelessServiceContext>(serviceContext))
.UseContentRoot(Directory.GetCurrentDirectory())
.UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
.UseStartup<Startup>()
.UseUrls(url)
.Build()), endpoint.Name);
listeners.Add(instanceListener);
}
return listeners;
}
Here is the endpoints shown in the Service Fabric Explorer for one of the nodes: Endpoint addresses
Regarding the socket listener I have also altered so that IPv6 is forwarded to backend port 8517 instead of 8507. Similarily as with Kestrel web server the socket listener will open two listening instances on respective addresses with appropriate port.
I hope this information is of any help.
It turns out I made a very stupid mistake that is completely my fault, I forgot to actually verify that my ISP fully supports IPv6. Turns out they don't!
Testing from a provider with full IPv6 support works as it should and I am able to get full connectivity to the nodes in the Service Fabric cluster.
Here is the working ARM template for anyone that needs a fully working example of Service Fabric cluster with IPv4 and IPv6 support:
Not allowed to post pastebin links without a accompanied code snippet...
Update:
Due to length constraints the template could not be pasted in this thread in its entirety, however over on the GitHub Issues page for Service Fabric I crossposted this. The ARM template is posted as a comment in that thread, it will hopefully be available longer than the pastebin link. View it here.
We are trying to use Drool as our rule engine service. What we done till now is listed below
Deployed workbench 7.2.Final
Deployed KIE server 7.2.0.Final
Configured some data objects, rules, deployed the changes to KIE server and we are able to execute the rule using rest API
Most of our requirements satisfied by stateless session (Give a set of data, execute the rule and return the data, that's it) . But using stateless we have to compromise many of the important features provided by Drools stateful session.
So we are trying to use stateful session per request. Which means the session should get disposed as soon as the request end. Also, parallel request should not interfere each other even if the session name is same
We found about container runtime strategy configuration (Workbench > Deploy > {any container} > Process Configuration > Runtime strategy)
But even after configure the container strategy to Per Request, it still behave same as Singleton (the session is not getting disposed after each request)
Few place we read it as, run time strategy only implemented in jBPM
The way we make request to KIE server is shown below
Request: POST {HOST}/kie-server/services/rest/server/containers/instances/TestRequest_1.0.4
{
"lookup": "ab-session", //stateful session
"commands": [
{
"insert": {
"out-identifier": "125",
"object": {
"com.myteam.testrequest.Product": {
"id": "123",
"name": "Hoo Hoo",
"count": 0
}
},
"return-object": "true"
}
},
{
"insert": {
"out-identifier": "126",
"object": {
"com.myteam.testrequest.Product": {
"id": "123",
"name": "Hoo Hoo",
"count": 0
}
},
"return-object": "true"
}
},
{"fire-all-rules": "hf2"}
]
}
We need help in achieving this requirement. Also, please help understand if we done something wrong
In kmodule.xml you may try to add "prototype" scope, because default is "singleton":
<ksession name="SessionName" type="stateful" default="false" clockType="realtime" scope="prototype"/>
I'm experimenting on creating a new service fabric cluster using ARM template and modify the template to add certificates, etc. The cluster and all resources are successfully created, but I noticed that initially the number of node instances are 2x, plus 1 than what I set to. For example, if I set "vmInstanceCount" to 3, I see 7 instances are currently creating.
But if I just wait and let them finish, then 4 instances were deleted and it will keep the three instances. One problem here is that it randomly select what to keep, thus, the names to keep can be node_1, node_4, node_6 which is messy.
Here's my snippet of nodeType:
"nodeTypes": [
{
"name": "[variables('vmNodeType0Name')]",
"applicationPorts": {
"endPort": 30000,
"startPort": 20000
},
"clientConnectionEndpointPort": "[variables('fabricTcpGatewayPort')]",
"ephemeralPorts": {
"endPort": 65534,
"startPort": 49152
},
"httpGatewayEndpointPort": "[variables('fabricHttpGatewayPort')]",
"isPrimary": true,
"vmInstanceCount": "[variables('vmInstanceCount')]",
"reverseProxyEndpointPort": "[variables('reverseProxyEndpointPort')]",
"durabilityLevel": "Bronze"
}
]
...
"sku": {
"name": "[variables('vmssSkuName')]",
"capacity": "[variables('vmssSkuCapacity')]",
"tier": "Standard"
}
I was talking to a Microsoft support earlier and this issue is actually a new feature as we can see here https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-design-overview#overprovisioning
I will close this issue as I found the answer. However, I still have some concern on the naming part but I will throw that question to MS.
I have tried to fetch my RedShift cluster details. I'm able to see many details about the cluster but few details got missed.
For Ex:- Details like Storageand Memory
The below is the code:-
redshiftClient = boto3.client('redshift', aws_access_key_id = role.credentials.access_key,
aws_secret_access_key = role.credentials.secret_key, aws_session_token = role.credentials.session_token, region_name='us-west-2')
#Getting all the clusters
clusters = redshiftClient.describe_clusters()
can you please check provide the way to get it.
Thanks.
The describe-clusters command does not return that type of information. The output of that command is:
{
"Clusters": [
{
"NodeType": "dw.hs1.xlarge",
"Endpoint": {
"Port": 5439,
"Address": "mycluster.coqoarplqhsn.us-east-1.redshift.amazonaws.com"
},
"ClusterVersion": "1.0",
"PubliclyAccessible": "true",
"MasterUsername": "adminuser",
"ClusterParameterGroups": [
{
"ParameterApplyStatus": "in-sync",
"ParameterGroupName": "default.redshift-1.0"
} ],
"ClusterSecurityGroups": [
{
"Status": "active",
"ClusterSecurityGroupName": "default"
} ],
"AllowVersionUpgrade": true,
"VpcSecurityGroups": \[],
"AvailabilityZone": "us-east-1a",
"ClusterCreateTime": "2013-01-22T21:59:29.559Z",
"PreferredMaintenanceWindow": "sat:03:30-sat:04:00",
"AutomatedSnapshotRetentionPeriod": 1,
"ClusterStatus": "available",
"ClusterIdentifier": "mycluster",
"DBName": "dev",
"NumberOfNodes": 2,
"PendingModifiedValues": {}
} ],
"ResponseMetadata": {
"RequestId": "65b71cac-64df-11e2-8f5b-e90bd6c77476"
}
}
You will need to retrieve Memory and Storage statistics from Amazon CloudWatch.
See your other question: Amazon CloudWatch is not returning Redshift metrics
If you actually want to retrieve information about a standard cluster (that is, the amount of storage and memory assigned to each node, rather than current memory and storage usage), that is not available from an API call. Instead see: Amazon Redshift Clusters
I have already created a service fabric cluster with azure diagnostics and it is functional currently with my services deployed into that cluster. I have an ETW EventSource in my service that I would like to start collecting events from because my service code already uses this event source to write my service related events. Since the cluster is already enabled for azure diagnostics and my services are already deployed into that cluster, I think it is a simple matter of updating the ETW provider with my event source in this service fabric cluster. Here is the exported template (only a partial is shown that is relevant for azure diagnostics):
{
"properties": {
"publisher": "Microsoft.Azure.Diagnostics",
"type": "IaaSDiagnostics",
"typeHandlerVersion": "1.5",
"autoUpgradeMinorVersion": true,
"settings": {
"WadCfg": {
"DiagnosticMonitorConfiguration": {
"overallQuotaInMB": "50000",
"EtwProviders": {
"EtwEventSourceProviderConfiguration": [
{
"provider": "Microsoft-ServiceFabric-Actors",
"scheduledTransferKeywordFilter": "1",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableActorEventTable"
}
},
{
"provider": "Microsoft-ServiceFabric-Services",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableServiceEventTable"
}
},
{
"provider": "Bb.ServiceFabric.Infrastructure.Container",
"scheduledTransferPeriod": "PT1M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableServiceEventTable"
}
}
],
"EtwManifestProviderConfiguration": [
{
"provider": "cbd93bc2-71e5-4566-b3a7-595d8eeca6e8",
"scheduledTransferLogLevelFilter": "Information",
"scheduledTransferKeywordFilter": "4611686018427387904",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricSystemEventTable"
}
}
]
}
}
},
"StorageAccount": "sfdgsmsraghuplaygrou6827"
}
},
"name": "VMDiagnosticsVmExt_vmNodeType0Name"
}
I would like to update following EtwProviders/EtwEventSourceProviderConfiguration to contain following section (as MyCompany.MyServices.MyStatelessService is the name of my service's EventSource):
{
"provider": "MyCompany.MyServices.MyStatelessService",
"scheduledTransferPeriod": "PT5M",
"DefaultEvents": {
"eventDestination": "ServiceFabricReliableServiceEventTable"
}
}
Here are my questions:
Is this the correct way of inserting an ETW provider/EventSource (from my service) into an existing cluster (that is already enabled with azure diagnostics)?
Can I add this event source (as a ETW event source provider) using a powershell command(s)?
If so, what is the exact powershell command (using all the information from the above code fragment)?
Note: I am using .net framework 4.5.2.
All seems good with the added configuration above. Just be aware that for ETWProviders the EventDestination cannot contain hyphens (-), yours don't so you are ok.
To update the Windows Azure Diagnostics (WAD) agent configuration, you can use either PowerShell or Cloud Explorer in Visual Studio.
For the former, simply update the ARM template and use the New-AzureRmResourceGroupDeployment cmdlet. See here for further information: https://azure.microsoft.com/en-us/documentation/articles/service-fabric-diagnostics-how-to-setup-wad/#update-diagnostics-to-collect-and-upload-logs-from-new-eventsource-channels
For using Cloud Explorer in Visual Studio. Browse to your Virtual Machine Scale Set (as this is the Azure resource that holds the WAD configuration). Right-click and choose Update Diagnostics. In the dialog shown, you have the option to upload a private and public configuration file. Simple take a .json document containing the {"WadCfg": {}} element, and upload that as a public configuration.
If you need to update the private configuration specifies the storage account name and AccessKey:
{
"storageAccountName": "",
"storageAccountKey": "",
"storageAccountEndPoint": "https://core.windows.net",
}
Hope this helps.
Mikkel