When using the Memcached protocol to connect to Apache Ignite, any value greater than 16384 characters in length will save incorrectly. When you attempt to retrieve this data it will appear 'garbled'. But any string 16384 or less will save and retrieve correctly.
We have tried a number of Ignite configurations, including the one specified here: http://apacheignite.gridgain.org/v1.1/docs/php
This same issue happens when using both the PHP and Java Memcached libraries. And both work perfectly fine with values less than 16385 characters in length.
Ignite JIRA ticket for this issue is created: https://issues.apache.org/jira/browse/IGNITE-3772
Related
Sorry i'm new to MongoDB so I'm so confused between
mongodb+srv://username:<password>#cluster0.accdl.mongodb.net/website?retryWrites=true&w=majority
and
mongodb://[port]:27017/[database_name]
what's difference and how is it impact our code?
well, as mongodb.com in https://www.mongodb.com/developer/article/srv-connection-strings/ said :
What is this mongodb+srv syntax?
Well, in MongoDB 3.6 we introduced the concept of a seed list that is specified using DNS records, specifically SRV and TXT records. You will recall from using replica sets with MongoDB that the client must specify at least one replica set member (and may specify several of them) when connecting. This allows a client to connect to a replica set even if one of the nodes that the client specifies is unavailable.
and :
Note that without the SRV record configuration we must list several nodes (in the case of Atlas we always include all the cluster members, though this is not required). We also have to specify the ssl and replicaSet options.
then in short words , mongodb +srv syntax , is way to connect to mongodb database , released starting from mongodb 3.6 , and allows you connect to the whole replicaset including all nodes , instead of mention a specific node in the traditional connection way .
I think mongodb+srv is used when you are using cluster and one instance of db
both of them will work for one instance but I think mongodb is work only for one instance
Q1. I have a crate cluster of version 1.0.2 and I am using older version of crate JDBC driver to connect to it from java program. I have specified all nodes of crate in the JDBC driver URL by separating them with comma. When I fire queries from my java program to crate, I can see memory and CPU usage of only 1 crate node increases and this node is 1st in the comma separated list given in the connection URL. After some time, that node runs out of memory . Could someone please explain why this happens ? I remember reading documentation of crate driver which indicated that crate driver load balances the queries across all specified client nodes. All my nodes are client enabled.
Q2. I tried same experiment with Crate 2.1.6 and JDBC driver 2.1.7 and I can see same behavior. I have verified that all the queries are getting fired on the data which is spread across multiple nodes. In latest documentation, I can see a new property got added viz loadBalanceHosts https://crate.io/docs/clients/jdbc/en/latest/connecting.html#jdbc-url-format
Right now I do not have this property added. Was this property present and required on JDBC driver version 2.1.7 ? Why do developer have to do worry about load balancing when crate cluster and JDBC drivers are supposed to provide that ?
FYI, most of my queries have group by clause and I have few billions of records to experiment with. Memory configured is 30GB per node.
This has been fixed in the lastest driver: https://github.com/crate/crate-jdbc/blob/master/docs/connecting.txt#L62
I have a distributed application and I use zookeeper to manage configuration data in all distributed servers.My service in each server needs some dlls to run . I am trying to build a centralized system from where I can copy my dlls to all the server.
Can I achieve that using zookeeper ?
I am aware that "ZooKeeper is generally not designed for large size storage" . My dll files are of size less the 3mb.
There is a 1mb soft limit on how large node data can get. According to the docs you can increase the max data size:
jute.maxbuffer:
(Java system property: jute.maxbuffer)
This option can only be set as a Java system property. There is no zookeeper prefix on it. It specifies the maximum size of the data that can be stored in a znode. The default is 0xfffff, or just under 1M. If this option is changed, the system property must be set on all servers and clients otherwise problems will arise. This is really a sanity check. ZooKeeper is designed to store data on the order of kilobytes in size.
I would not recommend using Zookeeper for this purpose, (you could much more easily host the binaries on a web server instead,) but it does seem possible in theory.
Zookeeper is designed to transfer messages inside the cluster.
Best thing you can do is create a Znode_A that will contain Znodes,
watch znode a for changes. Each Znode in Znode_A will represent a dll and will contain a dll path. Each node on the cluster watch for Znode_A data changes, so when a new dll (znode) will be created the nodes will know to copy the dll from a main repository.
In order to transfer files you can use SCP.
As data you can pass file path of your dlls. Using SCP you can pull files from base repository.
Yesterday, I found this cool script 'memcache-top' which nicely prints out stats of memcached live. It looks like,
memcache-top v0.6 (default port: 11211, color: on, refresh: 3 seconds)
INSTANCE USAGE HIT % CONN TIME EVICT/s READ/s WRITE/s
127.0.0.1:11211 88.8% 94.8% 20 0.8ms 9.0 311.3K 162.8K
AVERAGE: 88.8% 94.8% 20 0.8ms 9.0 311.3K 162.8K
TOTAL: 1.8GB/ 2.0GB 20 0.8ms 9.0 311.3K 162.8K
(ctrl-c to quit.)
it even makes certain text red when you should pay attention to something!
Q. Broadly, what are some useful tools/techniques you've used to check that memcached is set up well?
Good interface to accessing Memcached server instances is phpMemCacheAdmin.
I prefer access from the command line using telnet.
To make a connection to Memcached using Telnet, use the following telnet localhost 11211 command from the command line.
If at any time you wish to terminate the Telnet session, simply type quit and hit return.
You can get an overview of the important statistics of your Memcached server by running the stats command once connected.
Memory is allocated in chunks internally and constantly reused. Since memory is broken into different size slabs, you do waste memory if your items do not fit perfectly into the slab the server chooses to put it in.
So Memcached allocates your data into different "slabs" (think of these as partitions) of memory automatically, based on the size of your data, which in turn makes memory allocation more optimal.
To list the slabs in the instance you are connected to, use the stats slab command.
A more useful command is the stats items, which will give you a list of slabs which includes a count of the items store within each slab.
Now that you know how to list slabs, you can browse inside each slab to list the items contained within by using the stats cachedump [slab ID] [number of items, 0 for all items] command.
If you want to get the actual value of that item, you can use the get [key] command.
To delete an item from the cache you can use the delete [key] command.
For a production systems, you should really set up active monitoring (with downtime alerts, automated restarts etc.) of Memcache using something like Monit. Here is an example config: Monitoring Memcache with Monit
It is good to monitor overall memory usage of memcached for resource planning.
Track the eviction statistics counter to know how often cached items are getting evicted due to lack of memory.
Track cache hit/misses, reclaims(The number of expired items removed to allow space for new writes), current connections, flush cmd which is available in stats.
Memcached stats (can be read from telnet, libmemcached, language specific library)
stats
stats slabs
stats items
stats sizes
stats detail
stats settings
run the above commands using telnet
or simply run using netcat
echo "stats settings" | nc 127.0.0.1 11211
Other scripts/tools
https://github.com/memcached/memcached/tree/master/scripts
memcached top
memcached metrics per slab
This is what memcached metrics per slab looks like
desc for some fields can be found here.
Memcached Prometheus exporter - Exports metrics from memcached servers for consumption by Prometheus.
Assume I have N servers, each operating as a web server and a mongodb member of a replica set.
I'd like the slaveOk reads to be satisfied first by the local mongodb instance, rather than a remote machine across the network.
The documentation says slaveOk reads are satisfied by an arbitrary member. Is it possible to override that?
Mongodb 1.8, C-sharp driver 1.2.
The documentation says slaveOk reads are satisfied by an arbitrary member. Is it possible to override that?
Not without changing the C# driver. You'd probably have to look somewhere in this file to make those changes.
Assume I have N servers, each operating as a web server and a mongodb member of a replica set.
As a note, this is generally not the expected usage for MongoDB. Implemented in this way, your web server will be competing for RAM with MongoDB. If a server gets overloaded the web server will starve the mongod process which will cause connections to back up and exacerbate the issue.
It sounds like you're trying to use MongoDB as a local cache and there are far better tools for this job.
The closest you could come to what you are describing is for each web application to open a separate direct connection (not in replica set mode) to the local mongodb and use that separate connection for reads.