How to distribute cache using memchached approach between multiple servers? - memcached

I'm trying to distribute cache across multiple servers. I have installed Memcached on 2 Linux servers. I'm using .Net Client Libraries to manipulate cache on those servers.
It's always store value on the first server from configuration. I tried to change sequence of the servers and it stores but only on the first in the list.
I use Putty to check if object exist on the server.
The questions are:
How to store object on both servers at the same time?
if I want to verify that this object stored on all servers from configuration file
How I can do it from client libraries?
Here is the example of my code:
static void Main(string[] args)
{
var mc = new MemcachedClient();
mc.FlushAll();
mc.Store(StoreMode.Add, "key1", "some information");
Console.WriteLine(mc.Get("key1"));
}
and configuration
<enyim.com>
<memcached>
<servers>
<add address="20.23.24.105" port="11211"/>
<add address="20.23.24.106" port="11211"/>
</servers>
<socketPool minPoolSize="10" maxPoolSize="100" connectionTimeout="00:10:00" deadTimeout="00:02:00"/>
</memcached>
</enyim.com>

I don't think that it is possible to do this with the .NET client because that's not how Memcached is supposed to work. Internally the client will hash each key to only one server. If that server goes down the client will take that into account when hashing future keys. One way around this though would be to do a separate set for each server. This would require you to create two clients, one connected to each server, and send the request twice. If the .NET client provides async sets then you can do this with the latency only being as long as the longest of the two set operations.

Related

Multiple server names in a single connection string

In the PostgreSQL documentation https://www.postgresql.org/docs/10/libpq-connect.html, it has been said that multiple hosts can be specified in a single connection string such that all the hosts will be tried in order one after the other until one of the server gets succeeds.
But when i tried to implement the same setting in the tag present in my ASP.net web.config file, it is throwing error as no such host name. I am using NpgSQL provider in order to connect to PostgreSQL database.
I need to add multiple server names in the connection string such that if the server#1 fails then it should try for the next server server#2 immediately provided in the order until it succeeds
Can you please suggest on how multiple hosts can be provided in the connection string?
The Npgsql driver does not currently support this functionality. The issue tracking this is https://github.com/npgsql/npgsql/issues/732, I'm still hoping we can get this into the next release but there's a lot going on.
Load balancing and failover is avaialble in Npgsql version 6. At the time of writing v.6 is in preview.
Simple failover example (server2 is only used if a connection could not be established to server1):
Host=server1,server2;Username=test;Password=test
Example with load balancing (round robin I guess):
Host=server1,server2,server3,server4,server5;Username=test;Password=test;Load
Balance Hosts=true;Target Session Attributes=prefer-standby
https://www.npgsql.org/doc/failover-and-load-balancing.html

Keys created from one HSM client are not available for use in another client

I am building a solution to store keys and encrypt\decrypt data using an HSM. I am using a network HSM manufactured by Thales. The thing I have noticed is that a key generated in client machine 1 is inaccessible in client machine 2. The key can only be used to encrypt\decrypt data in client machine 1. Is there any thing that needs to be changed in my implementation or is there something to be changed in net-HSM configuration to enable this. I am using PKCS11Iterop library for all the key management operations.
I am using token based OCS protection.
I suppose your client machine 1 has a new file in kmdata/local directory associated to the new key generated.
But your client machine 2 has not this file in his kmdata/local directory.
You have to find a way to share the kmdata/local directory, for instance, using NFS.

Implementation of Proxy on Liberty for Java

I use "Liberty for Java" app and Statica service(Proxy) on Bluemix.
We set http.proxyHost/http.proxyPort/https.proxyHost/https.proxyPort as system properties in Java code every transactions.
for example:
URL url = new URL(xxx);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
........
System.setProperty("http.proxyHost", host);
System.setProperty("http.proxyPort", port);
System.setProperty("https.proxyHost", host);
System.setProperty("https.proxyPort", port);
........
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
I have an issue that one transaction go from the app to a target server directly in spite of tens of thousands of transactions passed the proxy.
Question 1:
Do "Liberty for Java" app on Bluemix clear or update system properties, http.proxyHost/http.proxyPort/https.proxyHost/https.proxyPort?
I wonder "Liberty for Java" app updated with null to access outer servers in multi-thread environment.
Question 2:
Do "Liberty for Java" app on Bluemix communicate with outer servers?
I found the following log in Statica.
https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.agents.na.apm.ibmserviceengage.com
https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.gateway.prd.na.ca.ibmserviceengage.com
( I masked a part of URL.)
P.S. We will change java code with ProxySelector class or Proxy class.
Re #1: No.
Re #2: Potentially yes. In your case, it seems your app is bound with a Monitoring & Analytics service? If so, a data collector will be installed and will send collected data to remote servers.
What's the reason that you need to set the proxy system properties in your code? Is it because you want some connections to go through the proxy and others not?
If so, then the way you do this is not right because the system proxy setting is a global setting, not a thread-scoped setting. This means if one thread sets the proxy setting, all threads will then use that proxy; if one thread unsets it, all threads will then do direct connections. That may explain why you are intermittently seeing some direct connections. The right way is to use a http client lib that supports proxy as parameters, like https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/config/RequestConfig.Builder.html#setProxy%28org.apache.http.HttpHost%29
If you want all connections to go through the http proxy, then you should simply set the JAVA_OPTS environment variable to pass in those system properties, e.g., "-Dhttp.proxyHost=x.x.x.x -Dhttp.proxyPort=xx".

Windows ServiceBus 1.1 for Windows Server

I did move the databases from our ServiceBus test enviroment.
I started by leaving the farm with the single node, then I moved the databases.
After rejoining the farm I see that GatewayDBConnectionString is till pointing to the old one.
I can't find any valid PowerShell command to reconfigure the value in question.
Anyone know how to fix this?
Thank you in advance.
To answer this I will need you to understand this a bit more - and hence giving a high-level overview of Service Bus 1.1 Server farm configuration:
Service Bus Server 1.1 is a platform where users can create highly-durable distributed Pub-Sub (messaging Queues/Topics) entities. In simple words - the main job of this is to translate the Compute (your VMs) and Data (your MsgContainer databases) into messaging functionality Durable Queues and Topics. So, in short - the configuration wizard or the Powershell cmdlets used to configure ServiceBus 1.1 Server will try to take the VMs and Databases from you.
The Db SBManagementDB is considered to be the authoritative source of truth for any Farm level configuration -> like Nodes that are part of the Farm (Store.Nodes), Ports opened on each of the nodes, Gateway database connection string (Cluster Config) etc. Also pl. note that - as per the Windows Server product guidelines - any information that has to be securely persisted will be encrypted - so as the Gateway DB connection String.
a) when you did New-SBFarm (with a Gateway DB connection string) - you have essentially communicated to SBMgmtDB - the Gateway DB Server, database name etc.
b) when you do Add-SBHost - again you have communicated to SBMgmtDb that you want to add one Node to this Farm
Gateway db connection string is the one place for Truth for all Gateway Services to find any run-time info -> like Container Databases, entity to container mapping etc.
again, when you do New-SBMessageContainer PSCmdlet --> you communicated to SBGatewayDB that you are adding one db
Now, with this background - lets see how the action you did above will take into effect:
- When you moved all the Databases to a different Server - you changed the Gateway Database connection string - But the Gateway connection string you had communicated to the SBManagementDB (using the New-SBFarm cmdlet) was pointing to the Old Server.
- When you removed the Node from the Farm and again Joined back - you removed one node from the configuration and re-added it - no affect :)
The ANSWER
Use Restore-SBFarm PS Cmdlet to communicate to the SBManagementDB that you changed the GW db
and then Use Restore-SBMessageContainer PS Cmdlet to communicate to Gateway DB that you changed the Container databases.
Now, add the Nodes back to this restored farm.
HTH!
Sree

Configure AppFabric Cache without listing servers in web.config

I am trying to understand how to properly configure AppFabric Caching on a web site. We are planning to use SQL Server as the cache manager and as far as I can understand the SQL will contain a list of the cache hosts in the cluster.
However, when running
DataCacheFactory factory = new DataCacheFactory();
I get
Server collection cannot be empty.
which, I guess, is to be expected since I have not added any servers in the web.config.
However, I do not want to maintain a server list on each web server, I want that to be done centrally on the SQL Server. I assume there is a way to point to the SQL Server, but I cannot find information on how to do this.
(I have also tried with the XML configration option, but it cannot even find that file. I have checked the health of the service in power shell.)
How do I centralize the server cache host list?
We are planning to use SQL Server as the cache manager and as far as I
can understand the SQL will contain a list of the cache hosts in the
cluster.
It's false. SQL Server can perform cluster management but it's only for managing the cache hosts, and ultimately, the cache cluster. It's just for internal management and your clients can use this configuration and they don't need to have acces to Sql Server.
DataCacheFactory factory = new DataCacheFactory();
This code will try to load default datacacheclient in config. In your case, it should be empty that's why you get this error.
You can still use code to configure cache host in this way.
// Declare array for cache host(s).
DataCacheServerEndpoint[] servers = new DataCacheServerEndpoint[1];
servers[0] = new DataCacheServerEndpoint("CacheServer1", 22233);
DataCacheFactoryConfiguration factoryConfig = new DataCacheFactoryConfiguration();
factoryConfig.Servers = servers;
DataCacheFactory mycacheFactory = new DataCacheFactory(factoryConfig);
DataCache myDefaultCache = mycacheFactory.GetCache("NamedCache1");
You don't need to specify all host names here, because AppFabric Caching will route request to the correct cache host, event if it is not in your list.