Is there any way to create RT without reloading the searchd?
For example, I have script generating sphinx.conf file, where the number of indexes
depends on how many instances of some class I have.
If I create another instance and reload searchd, then for some period of time other
instances can not use search.
Can I create RT indexes dynamically?
Related
I have a postgresql database and i want to use sphinx search heavily to get a lot of data from many tables (more than 30 table) , do i have to define index for each table or i can just define listen socket and it will work fine?
I tried the normal way which is define index for each table and it's working fine but i have to define the index for all tables!
I'm trying to define listen on the searchd section on sphinx.conf but it's not working.
No. Sphinx doesn't have 'auto-indexes'. They have to be created explicitly.
Frankly the variations are too many. What fields to include, what rows to include (eg exclude 'deleted' rows) etc. What attributes should be included . Too much to be deduced universally.
Having said that, the config file can be created by code. So the code knows how how you want each index to work, so just generates the config file automatically. But its probably only worth the trouble if your tables change regually.
I have implemented postgres trigger procedure, and python worker to feed Sphinx RT index. Take a look
https://github.com/narg/sphinx-search-feeder
I have a table of around 3TB on a redshift cluster. As part of some pre-processing step I need to create a few derived variable. The logic for them is very simple, e.g. a variable with difference of two variables etc.
Currently I use a update command to create such variable. The problem with update command is that it bloats the table size and requires a vacuum command to free up space. I am trying to find some way in which I can create such derived variable without the need of vacuum command. I tried creating a different table with the derived variable and joining them on my primary key. But this is equally time consuming as creating the new table and update on it requires almost the same amount of time.
Any other way I can achieve this which is more efficient?
Holding you new data in a separate table and joining it should be relatively quick as long as you use DISTSTYLE KEY on both tables with the same key and you include the DISTKEY in the join between them.
I'm doing a lot of stuff with big databases in MongoDB and sometimes there's a need in making a copy of database where copydb command fits perfectly, however it does add global lock for whole mongod instance at the end of process for creating index.
I know that there's an argument to create index in background "background:true", but I didn't find such argument for copydb command (or copydatabase).
Is there way to avoid global lock and force background index creation for Copy Database?
In my current environment I have a HUGE list of tables to scroll down through and finding that specific table I need to double-click is tedious (almost like trying to find a needle in a haystack).
Is there a way to open a specific table upon connecting to a database?
Alternatively, is there a way to create "shortcuts" (something like "favorites") to certain tables, so that they are easily accessible/findable upon SQL Developer startup?
I don't believe it is possible to set up a set of "Favorite" tables. However, if you right-click on Tables in your connection, there is an Apply Filter option. That lets you specify criteria to filter the set of tables that are displayed based on the name of the table or on other attributes like the last DDL time, etc. That's generally the easiest way to reduce the list to a reasonable number of tables.
I'm copying several tables (~1.5M records) from one data source to another, but it is taking a long time. I'm looking to speed up my use of DBD::Pg.
I'm currently using pg_getcopydata/pg_putcopydata, but I suppose that the indexes on the destination tables are slowing the process down.
I found that I can find some information on table's indexes using $dbh->statistics_info, but I'm curious if anyone has a programmatic way to dynamically drop/recreate indexes based on this information.
The programmatic way, I guess, is to submit the appropriate CREATE INDEX SQL statements via DBI that you would enter into psql.
Sometimes when copying a large table it's better to do it in this order:
create table with out indexes
copy data
add indexes