Our company has network issue and had some test since last couple of months.
It was super slow last Jan. but it got better this month. Network team didn't take any action and nothing changed. But when we compare tracert results, it seems that something must have been changed.
** tracert 1 month ago
1 * 44 ms 66 ms 172.21.64.1
2 * 65 ms 66 ms 172.21.64.254
3 286 ms 272 ms 293 ms 172.30.247.121
4 277 ms 267 ms 299 ms 192.168.254.233
5 * 262 ms 287 ms 192.168.253.113
6 321 ms 310 ms 271 ms 10.245.124.33
7 308 ms 311 ms 294 ms 10.245.124.9
8 268 ms 270 ms 303 ms 10.245.64.19
**tracert today
1 3 ms <1 ms <1 ms 172.21.64.1
2 1 ms 1 ms 1 ms 172.21.64.254
3 262 ms 262 ms 262 ms 172.30.247.121
4 262 ms 273 ms * 192.168.254.233
5 265 ms 266 ms 263 ms 192.168.253.113
6 266 ms 275 ms 302 ms 10.245.124.33
7 289 ms 287 ms 280 ms 10.245.124.9
8 269 ms 267 ms 263 ms 10.245.64.19
I'm not good at network nor couldn't find related topics from Google.
Could this be an evidence that some optimization or configuration could have been made?
Any kinds of view will be appreciated!
Thanks in advance!
It looks like your first and second hop routers response time is much better, but overall, the other routers area only slightly faster. These numbers are way too high. Anything over a 100 ms is too high for my liking. I would look at the 3rd hop, 172.30.247.121. That is where the slow down starts. The rest of the trip doesn't add much to the average, so they are ok.
I would look into that router and see if it is just too small for the traffic that it routes. Maybe it needs an update, maybe it needs a reboot, or maybe there is just too much traffic using it. It could have some odd Quality of Service setup that is slowing ICMP. A tracert is just a 32-byte ICMP ping that sets its "time-to-live" to "1" and increments each time.
Related
How much storage does Rundeck need in S3 for log storage for say 100 jobs, 100 executions a day?
There is no information in the docs about minimum storage requirements, hence the question.
You can do that with a simple experiment. If one execution (on my example it's a simple "hello world!" print) takes 2.4 kB then 2.4 kB x 100 = 240 kB (100 executions), and 240 kB x 100 = 24.000 kB = 24 MB per day (100 executions of 100 jobs). Of course, that depends on your job outputs, my example is just a "hello world!" string printed.
Anyway, this is not a big deal for the Amazon S3 service (check "How much data can I store in Amazon S3?" section).
I am verifying some of the configuration in my production Postgres instance. Our DB server has 32 GB RAM. From pg_settings, I see that effective_cache_size is set to:
postgres=> select name, setting, unit from pg_settings where name like 'effective_cache_size';
name | setting | unit
----------------------+---------+------
effective_cache_size | 7851762 | 8kB
(1 row)
As per my understanding, this value accounts to 7851762 X 8 KB = 62.8 GB. If my calculation is right, we are basically telling the optimizer that we have 62 GB for this parameter whereas we have only 32 GB of physical RAM.
Please correct me if I am calculating this parameter wrong. I always get confused with calculating parameter allocations for units with 8 KB.
7851762 times 8 kB is approximately 60 GB.
I would configure the setting to 30 GB if the machine is dedicated to the PostgreSQL database.
This parameter tells PostgreSQL how much memory there is available for caching its files. If the value is high, PostgreSQL will estimate nested loop joins with an index scan on the inner side cheaper, because it assumes that the index will probably be cached.
I have a C# backend running on AWS Lambda. It connects to a PostgreSQL DB in the same region.
I observed extremely slow cold-startup execution time after adding the DB connection code. After allocating more memory to my Lambda function, the execution time has significantly reduced.
The execution time as measured by Lambda console:
128 MB (~15.5 sec)
256 MB (~9 sec)
384 MB (~6 sec)
512 MB (~4.5 sec)
640 MB (~3.5 sec)
768 MB (~3 sec)
In contrast, after commenting out the DB connection code:
128 MB (~5 sec)
256 MB (~2.5 sec)
So opening a DB connection has contributed a lot to the execution time.
According to AWS documentation:
In the AWS Lambda resource model, you choose the amount of memory you
want for your function, and are allocated proportional CPU power and
other resources.
Since the peak memory usage has consistently stayed at ~45 MB, this phenomenon seems to suggest that database connection establishment is a computationally intensive activity. If so, why?
Code in question (I'm using Npgsql):
var conn = new NpgsqlConnection(Db.connStr);
conn.Open();
using(conn)
{ // print something }
Update 1
I set up a MariaDB instance with the same configuration and did some testing.
Using MySql.Data for synchronous operation:
128 MB (~12.5 sec)
256 MB (~6.5 sec)
Using MySqlConnector for asynchronous operation:
128 MB (~11 sec)
256 MB (~5 sec)
Interestingly, the execution time on my laptop has increased from ~4 sec (for Npgsql) to 12~15 sec (for MySql.Data and MySqlConnector).
At this point, I'll just allocate more memory to the Lambda function to solve this issue. But if anyone knows why the connection establishment took so long, I'd appreciate an answer. I might post some profiling results later.
When I run any SQL in PostgreSQL manager I'm coming across execution time: 328 ms; total time: 391 ms. I'm wondering what is this two times that is execution time and
total time.
Not sure what PostgreSQL manager is, but it is most likely combination of those:
Planning time: 0.430 ms
Execution time: 150.225 ms
Planning is how long it takes for Postgres to decide how to get your data. You send query and server might try to optimize it, that takes time.
Execution is how long it took to actually run that plan.
You can verify it yourself if you send your query like that:
EXPLAIN (ANALYZE)
SELECT something FROM table WHERE whatever = 5;
I have a very update heavy table.
The problem is that the table grows a lot, as the autovacuum can not catch up.
The autovacuum kick in every 2 minutes, so it is running file.
I have an application which is making like 50k updates(and a few inserts) every 60-70 seconds.
I am forced to do CLUSTER every week, which is not the best solution IMO.
My autovacuum settings are as follows (plus related):
maintenance_work_mem = 2GB
autovacuum_vacuum_cost_limit = 10000
autovacuum_naptime = 10
autovacuum_max_workers = 5
autovacuum_vacuum_cost_delay = 10ms
vacuum_cost_limit = 1000
autovacuum_vacuum_scale_factor=0.1 # this is set only for this table
Here is the autovacuum log:
2018-01-02 12:47:40.212 UTC [2289] LOG: automatic vacuum of table
"mydb.public.sla": index scans: 1 pages: 0 removed, 21853 remain, 1
skipped due to pins, 0 skipped frozen tuples: 28592 removed, 884848
remain, 38501 are dead but not yet removable buffer usage: 240395
hits, 15086 misses, 22314 dirtied avg read rate: 29.918 MB/s, avg
write rate: 44.252 MB/s system usage: CPU 0.47s/2.60u sec elapsed 3.93
sec
It seems to me that 29 MB/s is way to low, considering my hardware is pretty OK
I have Intel NVMe devices, and my I/O doesn't seem to be saturated.
Postgresql version: 9.6
CPU : Dual Intel(R) Xeon(R) CPU E5-2620 v4 # 2.10GHz
Any ideas how to improve this?