Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
I have a problem. I run an open source game software on one of my servers (eAthena) and they use MySQL as their preferred database. However, every time I run mysqldump
mysqldump -u backup -p*** eathena > test-backup.sql
it ends up hanging the server. The eAthena software hangs, and ends up shutting down once it's complete. Is this a size problem? Seeing as how the database is 4.7gb, I'm not sure what to blame. Is there some type of alternative my mysqldump?
I'm running CentOS 6 64bit, MySQL version 5.1.52.
it seems that your mysql installation is running out of memory. Check /etc/my.cnf or /etc/mysql/my.cnf (or wherever it is on CentOS).
increase the table_cache and key_buffer_size . See the examples of confs that comes with the mysql installation: my-huge.cnf’,my-large.cnf’, my-medium.cnf’, andmy-small.cnf’. Compare your confs with this:
[mysqld]
# Reduced to 200 as memory will not be enough for 500 connections.
# memory=key_buffer+sort_buffer_size+read_buffer_size)*max_connections
# which is now: 64 + (1 + 1) * 200 = 464 MB
# max_connections = approx. MaxClients setting in httpd.conf file
# Default set to 100.
max_connections = 100
max_allowed_packet = 1M
max_connect_errors = 10
key_buffer = 512M # 128M for 1GB, 256M for 2GB, 512 for 4GB
join_buffer_size = 4M # 1M for 1GB, 2M for 2GB, 4M for 4GB
read_buffer_size = 4M # 1M for 1GB, 2M for 2GB, 4M for 4GB
sort_buffer_size = 2M # 1M for 1GB, 2M for 2GB, 4M for 4GB
# myisam_sort_buffer_size used for ALTER, OPTIMIZE, REPAIR TABLE commands.
myisam_sort_buffer_size = 32M
# Checked opened tables and adjusted accordingly after running for a while.
table_cache = 1024
# thread_concurrency = 2 * (no. of CPU)
thread_concurrency=4
thread_cache_size = 286
# log slow queries is a must. Many queries that take more than 2 seconds.
# If so, then your tables need enhancement.
log_slow_queries=/var/log/mysqld.slow.log
long_query_time=2
# Reduced wait_timeout to prevent idle clients holding connections.
wait_timeout = 10
connect_timeout = 10
interactive_timeout = 10
# Enable it for vast improvement and it may be all you need to tweak.
query_cache_limit = 1M
query_cache_size = 128M
query_cache_type = 1
[mysqld_safe]
open_files_limit = 8192
[mysqldump]
quick
max_allowed_packet = 16M
[myisamchk]
key_buffer = 256M # 64M for 1GB, 128M for 2GB, 256 for 4GB
sort_buffer = 256M # 64M for 1GB, 128M for 2GB, 256 for 4GB
read_buffer = 64M # 16M for 1GB, 32M for 2GB, 64M for 4GB
write_buffer = 64M # 16M for 1GB, 32M for 2GB, 64M for 4GB
[mysqlhotcopy]
interactive-timeout
don't forget to restart mysql server before try again
Related
I'm struggling with resizing a CentOs Partition on a Server. I found some steps, but I'm not sure which circumstances I face and whats the correct approach and i definitely cannot mess that up.
The space should already be available, but the partition is not resized as far as I can tell.
The goal is to extend the partition /dev/sdb1 from 197GB to 1TB
Below are the "lsblk", "df -h" and "fdisk -l" results which should show my current situation.
[ ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 50G 0 disk
├─sda1 8:1 0 1G 0 part /boot
├─sda2 8:2 0 3.7G 0 part [SWAP]
└─sda3 8:3 0 45.3G 0 part /
sdb 8:16 0 1T 0 disk
└─sdb1 8:17 0 1024G 0 part /var/www/vhosts
sdc 8:32 0 50G 0 disk
└─sdc1 8:33 0 50G 0 part /var/lib/psa
sr0 11:0 1 680M 0 rom
[ ~]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.8G 0 7.8G 0% /dev
tmpfs 7.8G 0 7.8G 0% /dev/shm
tmpfs 7.8G 12M 7.8G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/sda3 45G 7.0G 36G 17% /
/dev/sda1 976M 135M 775M 15% /boot
/dev/sdc1 50G 53M 47G 1% /var/lib/psa
/dev/sdb1 197G 126G 62G 68% /var/www/vhosts
tmpfs 1.6G 0 1.6G 0% /run/user/0
[ ~]# fdisk -l
Disk /dev/sda: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0009c4b4
Device Boot Start End Blocks Id System
/dev/sda1 * 2048 2099199 1048576 83 Linux
/dev/sda2 2099200 9910271 3905536 82 Linux swap / Solaris
/dev/sda3 9910272 104855551 47472640 83 Linux
Disk /dev/sdb: 1099.5 GB, 1099511627776 bytes, 2147483648 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x8e948ef1
Device Boot Start End Blocks Id System
/dev/sdb1 2048 2147483647 1073740800 83 Linux
Disk /dev/sdc: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x7677284e
Device Boot Start End Blocks Id System
/dev/sdc1 2048 104857599 52427776 83 Linux
I found this answer here on an external page, but I'm not familiar with the commands and cannot tell, if thats the right way to go (if allowed I can paste the url). Partition Paths have not beed update to mine.
There are three steps to make:
alter your partition table so sda2 ends at end of disk
reread the partition table (will require a reboot)
resize your LVM pv using pvresize
Step 1 - Partition table Run fdisk
/dev/sda. Issue p to print your current partition table and copy that
output to some safe place. Now issue d followed by 2 to remove the
second partition. Issue n to create a new second partition. Make sure
the start equals the start of the partition table you printed earlier.
Make sure the end is at the end of the disk (usually the default).
Issue t followed by 2 followed by 8e to toggle the partition type of
your new second partition to 8e (Linux LVM).
Issue p to review your new partition layout and make sure the start of
the new second partition is exactly where the old second partition
was.
If everything looks right, issue w to write the partition table to
disk. You will get an error message from partprobe that the partition
table couldn't be reread (because the disk is in use).
Step 2 Reboot your system This step is neccessary so the partition table gets
re-read.
Step 3 Resize the LVM PV After your system rebooted invoke pvresize
/dev/sda2. Your Physical LVM volume will now span the rest of the
drive and you can create or extend logical volumes into that space.
The question is, is that the right way to increase the partition size without loosing any data on it for a CentOs System?
Thank you
As you can see the partition
sdb 8:16 0 1T 0 disk
└─sdb1 8:17 0 1024G 0 part /var/www/vhosts
is already 1TB. So you need to extend the filesystem. If your filesystem is ext4 you can use command:
resize2fs /var/www/vhosts
if your filesystem is xfs you can use command:
xfs_growfs /var/www/vhosts
I set up a cephfs cluster on my virtual machine, and then want to use this cluster to store a batch of image data (total 1.4G, each image is about 8KB). The cluster stores two copies, with a total of 12G of available space. But when I store data inside, the system prompts that the available space is insufficient. How to solve this?The details of the cluster are as follows:
Cluster Information:
cluster:
id: 891fb1a7-df35-48a1-9b5c-c21d768d129b
health: HEALTH_ERR
1 MDSs report slow metadata IOs
1 MDSs report slow requests
1 full osd(s)
1 nearfull osd(s)
2 pool(s) full
Degraded data redundancy: 46744/127654 objects degraded (36.618%), 204 pgs degraded
Degraded data redundancy (low space): 204 pgs recovery_toofull
too many PGs per OSD (256 > max 250)
clock skew detected on mon.node2, mon.node3
services:
mon: 3 daemons, quorum node1,node2,node3
mgr: node2(active), standbys: node1, node3
mds: cephfs-1/1/1 up {0=node1=up:active}, 2 up:standby
osd: 3 osds: 2 up, 2 in
data:
pools: 2 pools, 256 pgs
objects: 63.83k objects, 543MiB
usage: 10.6GiB used, 1.40GiB / 12GiB avail
pgs: 46744/127654 objects degraded (36.618%)
204 active+recovery_toofull+degraded
52 active+clean
Cephfs Space Usage:
[root#node1 0]# df -hT
文件系统 类型 容量 已用 可用 已用% 挂载点
/dev/mapper/nlas-root xfs 36G 22G 14G 62% /
devtmpfs devtmpfs 2.3G 0 2.3G 0% /dev
tmpfs tmpfs 2.3G 0 2.3G 0%
/dev/shm
tmpfs tmpfs 2.3G 8.7M 2.3G 1% /run
tmpfs tmpfs 2.3G 0 2.3G 0%
/sys/fs/cgroup
/dev/sda1 xfs 1014M 178M 837M 18% /boot
tmpfs tmpfs 2.3G 28K 2.3G 1%
/var/lib/ceph/osd/ceph-0
tmpfs tmpfs 471M 0 471M 0%
/run/user/0
192.168.152.3:6789,192.168.152.4:6789,192.168.152.5:6789:/ ceph 12G 11G 1.5G 89% /mnt/test
Ceph OSD:
[root#node1 mnt]# ceph osd pool ls
cephfs_data
cephfs_metadata
[root#node1 mnt]# ceph osd pool get cephfs_data size
size: 2
[root#node1 mnt]# ceph osd pool get cephfs_metadata size
size: 2
ceph.dir.layout:
[root#node1 mnt]# getfattr -n ceph.dir.layout /mnt/test
getfattr: Removing leading '/' from absolute path names
# file: mnt/test
ceph.dir.layout="stripe_unit=65536 stripe_count=1 object_size=4194304 pool=cephfs_data"
Storing small files, you need to watch the minimum allocation size. Until the Nautilus release, this defaulted to 16k for SSD and 64k for HDD, but with the new Ceph Pacific the default minimum allocation has been tuned to 4k for both.
I suggest you use Pacific, or manually tune Octopus to the same numbers if that's the version you installed.
You also want to use replication (as opposed to Erasure Coding) if your files are under a multiple of the minimum allocation size, as the chunks of EC would use the same minimum allocation and will waste slack space otherwise. You already made the right choice here by using replication, I am just mentioning it here because you may be tempted by EC's touted space-saving properties -- which unfortunately do not apply to small files.
you need to set bluestore_min_alloc_size to 4096 by default its value is 64kb
[osd]
bluestore_min_alloc_size = 4096
bluestore_min_alloc_size_hdd = 4096
bluestore_min_alloc_size_ssd = 4096
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 2 years ago.
Improve this question
I'm using buildroot to build a custom linux system for my raspi A+.
Using genimage, I've created two partitions on a 1 GB sdcard. The first partion is the boot partition. It's vfat and it is 32 MB. The second partition is ext4, it is the rootfs and it is 512 MB.
Once I boot my raspi with the newly burned sdcard and that I type df -h I get this in the output:
Filesystem Size Used Available Use% Mounted on
/dev/root 17.1M 14.0M 1.8M 89% /
devtmpfs 200.6M 0 200.6M 0% /dev
tmpfs 200.7M 0 200.7M 0% /dev/shm
tmpfs 200.7M 0 200.7M 0% /tmp
tmpfs 200.7M 4.0K 200.7M 0% /run
as you can see, /dev/root is 17.1 MB instead of 512 MB.
Then, I issue cat /proc/partitions:
major minor #blocks name
1 0 4096 ram0
1 1 4096 ram1
1 2 4096 ram2
1 3 4096 ram3
1 4 4096 ram4
1 5 4096 ram5
1 6 4096 ram6
1 7 4096 ram7
1 8 4096 ram8
1 9 4096 ram9
1 10 4096 ram10
1 11 4096 ram11
1 12 4096 ram12
1 13 4096 ram13
1 14 4096 ram14
1 15 4096 ram15
179 0 969728 mmcblk0
179 1 32768 mmcblk0p1
179 2 524288 mmcblk0p2
We clearly see that the sdcard (mmcblk0) is 1 GB, the boot partition (mmcblk0p1) is 32 MB and the rootfs partition (mmcblk0p2) is 512 MB.
So, to convince myself that the mmcblk0p2 partition may have been imporperly mounted, I mount it again with mount -t ext4 -o rw /dev/mmcblk0p2 /mnt and then I issue df -h again:
Filesystem Size Used Available Use% Mounted on
/dev/root 17.1M 14.0M 1.8M 89% /
devtmpfs 200.6M 0 200.6M 0% /dev
tmpfs 200.7M 0 200.7M 0% /dev/shm
tmpfs 200.7M 0 200.7M 0% /tmp
tmpfs 200.7M 4.0K 200.7M 0% /run
/dev/mmcblk0p2 17.1M 14.0M 1.8M 89% /mnt
Again, I see that mmcblk0p2 size is 17.1 MB.
So, my question is Why is cat /proc/partitions returning the expected size for my rootfs partition while df -h returns a totally different and bogus size ?
TL;DR: set BR2_TARGET_ROOTFS_EXT2_BLOCKS to 524288.
You have to distinguish the partition from the filesystem on the partition.
The partition sizes and offsets are specified in the partition table, and you can view them with cat /proc/partitions. Paritions are created with a tool like fdisk (or when you're using Buildroot, it's often created by genimage).
The filesystem size is specified in the filesystem superblock, a piece of metadata that specifies the size of the filesystem, any options (e.g. if journalling is used), cluster sizes, etc. This is created by a tool like mke2fs. When you use mke2fs directly on a partition, it will use the full space of the partition for the filesystem, which is typically what you want. However, when you create the filesystem before partitioning the SD card (as is often the case when you generate an image with e.g. Buildroot), you have to specify the size to mke2fs (cfr. the man page: the second argument is blocks-count).
In Buildroot, you typically create an image as a file and don't write directly to the SD card. That is because the size of the SD card is not known a priori, and because you have to be root to be able to write the SD card. Therefore, there is no way for Buildroot to know how large the ext4 filesystem should be when you create the filesystem. Before the 2017.05 release of Buildroot, it would try to estimate how large the filesystem should be to fit everything, and create a filesystem of exactly that size. You are probably in that situation.
To fix this, you should set the configuration variable BR2_TARGET_ROOTFS_EXT2_BLOCKS to 524288 (= 512MB in 1024-byte blocks). Or if you use Buildroot more recent than the 2017.05 release, set BR2_TARGET_ROOTFS_EXT2_SIZE to 512M (the new option is in bytes but allows suffixes K, M, G).
I got two identical servers, in both is installed postgresql server version 9.0.4 with the same configuration. If I launch a .sql file that performs about 5k inserts, on the first one it takes a couple of seconds, on the second one it takes 1 minute and 30 seconds.
If I set synchronous_commit, speed dramatically reduces (as expected), and the performances of the two servers are comparable. But if I set synchronous_commit to on, on one server the insert script execution time increases of less than one second, on the other one it increases too much, as I said in the first period.
Any idea about this difference in performances? Am I missing some configuration?
Update: tried a simple disk test: time sh -c "dd if=/dev/zero of=ddfile bs=8k count=200000 && sync"
fast server output:
1638400000 bytes (1.6 GB) copied, 1.73537 seconds, 944 MB/s
real 0m32.009s
user 0m0.018s
sys 0m2.298s
slow server output:
1638400000 bytes (1.6 GB) copied, 4.85727 s, 337 MB/s
real 0m35.045s
user 0m0.019s
sys 0m2.221s
Common features (both servers):
SATA, RAID1, controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller, distribution: linux centOS. mount -v output:
/dev/md2 on / type ext3 (rw)
proc on /proc type proc (rw)
none on /dev/pts type devpts (rw,gid=5,mode=620)
/dev/md1 on /boot type ext3 (rw)
fast server: kernel 2.6.18-238.9.1.el5 #1 SMP
Disk /dev/sda: 750.1 GB, 750156374016 bytes
255 heads, 63 sectors/track, 91201 cylinders, total 1465149168 sectors
Units = sectors of 1 * 512 = 512 bytes
Device Boot Start End Blocks Id System
/dev/sda1 3906 4209029 2102562 fd Linux raid autodetect
/dev/sda2 4209030 4739174 265072+ fd Linux raid autodetect
/dev/sda3 4739175 1465144064 730202445 fd Linux raid autodetect
slow server: kernel 2.6.32-71.29.1.el6.x86_64 #1 SMP
Disk /dev/sda: 750.2 GB, 750156374016 bytes
64 heads, 32 sectors/track, 715404 cylinders, total 1465149168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0006ffc4
Device Boot Start End Blocks Id System
/dev/sda1 2048 4194303 2096128 fd Linux raid autodetect
/dev/sda2 4194304 5242879 524288 fd Linux raid autodetect
/dev/sda3 5242880 1465147391 729952256 fd Linux raid autodetect
Could it be useful to address the performance issue?
I suppose your slow server with newer kernel has working barriers. This is good, as otherwise you can loose data in case of a power failure. But it is of course slower than running with write cache enabled and without barriers, aka running with scissors.
You can check if barriers are enabled using mount -v — search for barrier=1 in output. You can disable barriers for your filesystem (mount -o remount,barrier=0 /) to speed up, but then you risk data corruption.
Try to do your 5k inserts in one transaction — Postgres won't have to write to disk on every row inserted. The theoretical limit for number of transactions per second wound be comparable to disk rotational speed (7200rpm disk ≈ 7200/60 tps = 120 tps) as a disk can only write to a sector once per rotation.
To me this sounds like in the "fast" server there is a write cache enbled for the harddisk(s), whereas in the slow server the harddisk(s) are really writing the data when PG writes it (by calling fsync)
On CentOS has recently started to happen a strange thing - the user can not create files in its directory:
[Deployer # server ~] $ echo test> test.file
-Bash: echo: write error: Disk quota exceeded
Although quotas had not been established:
[Deployer # server ~] $ quota
Disk quotas for user deployer (uid 500): none
Disk space is sufficient:
[Deployer # server ~] $ df-h
Filesystem Size Used Avail Use% Mounted on
/ Dev / vzfs 9.6G 6.9G 2.8G 72% /
none 256M 4.0K 256M 1% / dev
Problems with inodes should not be:
[Deployer # server ~] $ df-i
Filesystem Inodes IUsed IFree IUse% Mounted on
/ Dev / vzfs 10000000 130959 9869041 2% /
none 65536 95 65441 1% / dev
Can you please tell what could be the problem?
You have group quota also enabled for that user:
repquota -avug|grep username
repquota -avug|grep groupname
Edit quota for group:
edquota -g groupname