I am working on an adding a new extension on citus and am in the process of collecting and analyzing the stack traces for the extension, postgres, citus and other libs by running ycsb workloads against a cluster. I am using perfcollect to collect and perfview to view the stack traces. In the traces for the postmaster processes I am finding a lot of "unknown" modules which I believe are unresolved postgres symbols-
unknown_symbols_png
I have tried several things to resolve these symbols -
Build postgres from source with debug flags for the configure file (--enable-debug, CFLAGS = -ggdb -O0 -g)
Build postgres using yum package manager - this builds the debug symbols and executables in different folders, so have to then use eu-unstrip command to combine the symbols and executables into a single file
Build the extension with debug symbols
None of this has helped me resolve the unknown symbols when using perfcollect. However, when trying to collect the traces using the linux perf tool with the specific PID for postgres and then using the Firefox profiler tool to view the traces, I see much less unknown modules - some are still there though.
Any ideas what else can be done to resolve these unknown symbols?
Related
I have blindly followed this tutorial for installing PostgreSQL and Apache AGE and would like to understand more about the process better. In the video, he has used a lot of options and flags for the ./configure command which I have copied below:
./configure --prefix=$(pwd) --enable-cassert --enable-debug CFLAGS="-glldb -ggdb -0g -g3 -fno-omit-frame-pointer"
He has mentioned --prefix=$(pwd) is required for setting a custom location as he has multiple of Postgres instances intalled. Does this mean without doing this, Postgres will be installed to the same location every time? If so, where will it be installed? I am unable to find it in any of my system files, unlike when I have installed Postgres using the packages and installers, which showed up in my /libraries directory.
--enable-cassert and --enable-debug are used to enable dev tools for debugging but it is not clear to me the differences between them.
As for the CFLAGS, I have no idea what is going on.
Thank you in advance.
Most of the answers to your questions can be found on the official documentation for Postgres. I've linked it below, but I'll also focus on the points you mentioned.
https://www.postgresql.org/docs/current/install-procedure.html#CONFIGURE-OPTIONS
--prefix=$(pwd) --enable-cassert --enable-debug
All the above are configuration parameters that postgres itself uses. Taken directly from the documentation, here is what they mean:
--prefix=PREFIX
Install all files under the directory PREFIX instead of /usr/local/pgsql. The actual files will be installed into various subdirectories; no files will ever be installed directly into the PREFIX directory.
--enable-debug
Compiles all programs and libraries with debugging symbols. This means that you can run the programs in a debugger to analyze problems. This enlarges the size of the installed executables considerably, and on non-GCC compilers it usually also disables compiler optimization, causing slowdowns. However, having the symbols available is extremely helpful for dealing with any problems that might arise. Currently, this option is recommended for production installations only if you use GCC. But you should always have it on if you are doing development work or running a beta version.
--enable-cassert
Enables assertion checks in the server, which test for many “cannot happen” conditions. This is invaluable for code development purposes, but the tests can slow down the server significantly. Also, having the tests turned on won't necessarily enhance the stability of your server! The assertion checks are not categorized for severity, and so what might be a relatively harmless bug will still lead to server restarts if it triggers an assertion failure. This option is not recommended for production use, but you should have it on for development work or when running a beta version.
CFLAGS="-glldb -ggdb -0g -g3 -fno-omit-frame-pointer"
As for the above, these are options you can set for the C compiler itself, not postgres. You can look them up individually for more info, but to give you an idea, -0g is a compiler flag that decides the level of code optimization performed by the compiler. Here's a link containing more info:
https://wiki.gentoo.org/wiki/GCC_optimization/en#-O
I have a data ware house running on PostgreSQL and I would like to check what all are the missing indices are in my database.
I tried to install the extension pg_qualstats but it is giving the below error.
root#Ubuntu-1604-xenial-64-minimal ~ # sudo pip install pg_qualstats
Collecting pg_qualstats
Could not find a version that satisfies the requirement pg_qualstats (from versions: )
No matching distribution found for pg_qualstats
You are using pip version 18.1, however version 19.2.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
root#Ubuntu-1604-xenial-64-minimal ~ #
So how can i install this extension? I tried the CREATE EXTENSION pg_qualstats but it was giving error.
Also is there any other database maintenance need to be done for the database regularly?
What all parameter I have to check?
Can we automate the maintenance activity?
I was a SQL server DB admin and it was much easier to find out the missing index, understand the execution plan, DB maintenance but I find it hard, when it comes to PostgreSQL.
So any guidance will be of great help.
I am only answering the immediate question regarding the installation of pg_qualstats - the rest of the questions are way too broad for a platform like stackoverflow (or dba.stackexchange).
Many interesting extensions are provided as source code in Postgres (that's one of the reasons why it's highly recommended to run Postgres on Linux, because compiling the extensions is way easier in Linux than it is on Windows, and may extensions are only developed for Linux).
pg_qualstats is no different.
It is provided together with PoWA and the installation of the extension is documented as part of their installation guide
In a nutshell:
Download the source:
wget https://github.com/powa-team/pg_qualstats/archive/1.0.7.tar.gz -O pg_qualstats-1.0.7.tar.gz
tar zxvf pg_qualstats-1.0.7.tar.gz
cd pg_qualstats-1.0.7
the compile it:
make
make install
Register the shared libraries by editing postgresql.conf and after restarting Postgres the extension can be created using CREATE EXTENSION pg_qualstats;
I'm posting here with the hopes that someone will be kind enough to help me. Just to clarify, I'm sort of a newb, so I'd be really grateful if you give more details to your explanations.
I'm on Windows 10 64 bit, QT 5.9.1, PostgreSQL is 9.6.3 version.
I'm working on a project, where I need to get a postgresql database running and connect it to the code I'm doing right now. Unfortunately, when I try to install the driver, I run into all kinds of problems. I've been reading on threads on sites, but they are outdated and some folders do not match, which my first problem arises. For some reason I cannot get qmake to work, even though I edit the path in the variables.
I tried running the commands from here.
cd %QTDIR%\qtbase\src\plugins\sqldrivers\ps
ql
qmake "INCLUDEPATH+=C:/psql/include" "LIBS+=C:/psql/lib/ms/libpq.lib" psql.pro
nmake
Basically my folders don't aren't 100% as the guide, also I don't have a libpq.lib file
On my final attempt I tried the following:
qmake "INCLUDEPATH+=C:/PostgreSQL/pg96/include" "LIBS+=C:/PostgreSQL/pg96/lib/libpq.lib"
Project ERROR: Cannot run compiler 'cl'. Maybe you forgot to setup the environment?
You'll need to have PostgreSQL installed so you have libpq.lib and libpq.dll. You cannot compile the Qt PostgreSQL driver without these. You'll also need to find the include directory for your PostgreSQL install, the one that contains libpq-fe.h.
Then determine the paths to those. Set the PATH to include the directory with libpq.dll. Set LIBS to include the directory with libpq.lib. And set the INCLUDEPATH to include the directory with libpq-fe.h.
You'll probably need to do this within a "Visual Studio Command Prompt" or run vcvarsall.bat fist, too, so qmake can find Visual Studio.
I'm having trouble cross compiling PostgreSQL for my TI Sitara AM335x EVM SK. My host system is an i386 machine running Ubuntu 12.04.
My application is written in C++ using Qt. When I try and compile, I get the error that libpq.so is incompatible. I believe this is because the cross compiler is trying to use the host libpq.so instead of one for the target system (which as I have found out, doesn't exist).
I've downloaded the source for PostgreSQL with the intention of cross compiling that in order to give me the libpq.so library that will be compatible with my target system, however there is virtually no information on how to do this.
I have tried using the CC argument with the configure file to change my compiler to the following: CC=/home/tim/ti-sdk-am335x-evm-06.00.00.00/linux-devkit/sysroots/i686-arago-linux/usr/bin/arm-linux-gnueabihf-gcc but the configure script gives me this error: configure: error: cannot run C compiled programs. If you meant to cross compile, use --host.
The configure file makes a small reference to the --host option, but the only information in the file that I could find is in reference to mingw and windows, which isn't what I want.
I've done some quick searching through the configure file, and it references the --host option, but with no explanation of what is a valid host. I'm assuming that with --host option there will be an associated --target.
What arguments can I give the configure script so that it will cross compile with the correct compiler to generate a library that my target device can use? Are there any resources out there that I haven't found in regards to how the --host/--target works or how to use them?
OK, so after fiddling around for a little while, I think I was actually able to cross compile PostgreSQL and answer my own question.
Before I went any further, I had realized I had forgotten to add the path to my cross compiler to the PATH environment variable. I used the command export PATH=/path/to/cross/compiler:$PATH to insert the compiler path to the PATH environment variable.
Next, I did some experimenting with the --host option. To start off with I tried using ./configure --host=arm-linux-gnueabihf and running the configure script. The configure script seemed to accept this as the host argument. I then went to the next step of running the makefile. Running this makefile resulted in errors being generated. The errors were selected processor does not support Thumb mode. I did a quick search to see what information I could find about this error and came to this webpage: http://www.postgresql.org/message-id/E1Ra1sk-0000Pq-EL#wrigleys.postgresql.org.
This webpage gave me a bit more information since it seemed like the person was trying to do something very similar to me. One of the responders to the post mentioned that --disable-spinlocks is intended for processors that aren't supported by default by PostgreSQL. I emulated the arguments that were used in the website listed above and used the command: ./configure --host=arm-linux CC=arm-linux-gnueabihf-gcc AR=arm-linux-gnueabihf-ar CPP=arm-linux-gnueabihf-cpp --without-readline --without-zlib --disable-spinlocks to generate my makefile. This makefile actually generated all of the files, including the libpq.so library file I was needing.
Hope this helps somebody else in the future!
I have Installed and worked with MemSQL on Ubuntu 12.04 and CentOS 6. But I am doing a fresh install on CentOs 6.4 (FInal). When running ./check_system script I am getting the following error.
Warning: The number of allowed open file descriptors should be at least 64000. This OS is currently configured with the limit set to 1024. Please visit http://developers.memsql.com/docs/2.5/admin/admin_concepts.html#configuring-the-linux-file-descriptor-limit
./check_system: line 313: 6173 Illegal instruction (core dumped) $SCRIPT_DIR/lib/check_crc32 > /dev/null 2> /dev/null
Warning: SSE4.2 is not supported. Resorting to software CRC32C. MemSQL recovery and log writing performance will be negatively impacted.
I am familiar with the warning and I know how to fix it. its the Illegal instructions error I am trying to understand.
Any help in this regard will be useful.
For the file descriptor warning, you can either read that link, or simply use the rpm or deb package to install - both of these will configure that limit for you.
The warning about SSE4.2 is not a fatal warning - the server will run fine, generally with single-digit percentage slowdown to insert and update queries.
Read, Installation Best Practices HOWTO:
http://docs.memsql.com/latest/setup/best_practices/#increase-file-descriptor-limit
Also try restart svr