SphinxSearch: 3.3.1
Ubuntu 22.04
PHP 8.1.12
I am trying to use SphinxQL with PHP to find rows where two different categories match, such as:
$db = new PDO('mysql:host=127.0.0.1;port=9306', '', '');
$sql = "SELECT id FROM index,delta,rtindex WHERE category2 > 0 AND category2 = category1 LIMIT 1000 OPTION max_matches=1000";
$stmt = $db->query($sql);
However, I am receiving a syntax error whenever I do this. I thought this query had worked in the past, but I'm just now noticing it in the logs. Is this the correct way to do it, or is there some other way dictated by sphinx's rules?
Related
I have this python3 code :
conn = psycopg2.connect( ... )
curr = conn.cursor()
curr.execute(code)
rows = curr.fetchall()
where 'code' has the select query statement
After executing this, 'rows' list will have lists of only the selected row values. How do I run 'curr.execute' in such a way that I also get the respective col headers too?
Meaning if I have say
Select col1, col2 from table Where some_condition;
I want my 'rows' list to have something like [['col1', 'col2'], [some_val_for_col1, some_val_for_col2] ....]. Any other ways of getting these col headers are also fine, but the select query in 'code' shouldn't change.
you have to execute 2 commands
curr.execute("Select * FROM people LIMIT 0")
colnames = [desc[0] for desc in curs.description]
curr.execute(code)
you can follow steps mentioned in https://kb.objectrocket.com/postgresql/get-the-column-names-from-a-postgresql-table-with-the-psycopg2-python-adapter-756
I have a native query in Doctrine, which generate the right SQL, which runs in PostgreSQL. However the Doctrine returns null/empty results.
public count unReadedNotifications(User $user) {
$rsm = new ResultSetMapping();
$sql =
"SELECT count(ele->'status') FROM notification CROSS JOIN LATERAL jsonb_array_elements(items) status(ele) WHERE notification.id = ? and (ele->'status')::jsonb #> '1'";
$query = $this->getEntityManager()->createNativeQuery($sql,$rsm)->setParameter('1', $user->getNotification()->getId());
return $query->getSingleScalarResult();
}
This Doctrine native query returns null.
If I check the profiler I see the generated query is:
SELECT count(ele->'status') FROM notification CROSS JOIN LATERAL jsonb_array_elements(items) status(ele) WHERE notification.id = 21 and (ele->'status')::jsonb #> '1';
If I copy-paste this into pgAdmin it runs and it retrieve the desired "2".
So pgAdmin can gives me the right result, but Doctrine not.
Can somebody see, where and what went wrong?
Doctrine version: 2.6.3, Postgres is 10.
I need make search on billion records in MySQL and it's very long process (it's works now). May be Sphinx help me? How correctly to configure Sphinx for search numbers? Should I use integer attribute for searching (not string field)?
I need to get only row where the timestamp 'nearest or equal' to query:
CREATE TABLE test ( date TIMESTAMP(6) UNIQUE, num INT(32) );
| 2018-07-02 05:50:33.084011 | 282 |
| 2018-07-02 05:50:33.084028 | 475 |
...
(40 M such rows... all timestamps is unique, so this column are unique index so I no need in create additional index I suppose.)
sphinx.conf:
source src1
{
type = mysql
...
sql_query = SELECT * FROM test
}
indexer...
Sphinx 3.0.3
...
indexing index 'test'...
collected 40000000 docs, 0.0 MB
In my test I find nearest timestamp to query:
$start = microtime(true);
$query = '2018-07-02 05:50:33.084011';
$connMySQL = new PDO('mysql:host=localhost;dbname=test','','');
$sql = "SELECT * FROM test WHERE date <= '$search' ORDER BY date DESC LIMIT 1";
$que = $connMySQL->query($sql);
$result = $que->fetchAll(PDO::FETCH_ASSOC);
$query = $connMySQL->query('reset query cache');
$connMySQL = null;
print_r ($result);
echo 'Time MySQL:'.(microtime(true) - $start).' sec.';
$start = microtime(true);
$query = '2018-07-02 05:50:33.084029';
$connSphinxQL = new PDO('mysql:host=localhost;port=9306;dbname=test','root','');
$sql = "SELECT * FROM test WHERE date <= '$search' ORDER BY date DESC LIMIT 1";
$que = $connSphinxQL->query($sql);
$result = $que->fetchAll(PDO::FETCH_ASSOC);
$query = $connSphinxQL->query('reset query cache');
$connSphinxQL = null;
print_r ($result);
echo 'Time Sphinx:'.(microtime(true) - $start).' sec.';
Output:
[date] => 2018-07-02 05:50:33.084011 [num] => 282 Time MySQL: 0.00193 sec.
[date] => 2018-07-02 05:50:33.084028 [num] => 475 Time Sphinx: 0.00184 sec.
I suggested to see some different resuts, but noticed that before indexing I have got the same result, so I think Sphinx searches directy in MySQL by the reason of my wrong configuration.
Only ask here I found: no text search
Should I use integer attribute for searching (not string field)?
Yes. But an added complication, is a index NEEDS at least one field (sphinx isnt really designed as a general database, its intended for text queries!)
Can synthesize a fake one.
sql_query = SELECT unix_timestamp(`date`) AS id, 'a' AS field, num FROM test
sql_attr_uint = num
Also shows that need a unique integer as the first column, to be a document_id, seems as your timestamp is unique, can use that. a UNIX_TIMESTAMP is a nice easy way to represent a timestamp as a plain integer.
Can use id in queries too, for filtering, so would need to convert to a timestamp at the same time.
$query = '2018-07-02 05:50:33.084011';
$id = strtotime($query)
$sql = "SELECT * FROM test WHERE id <= '$id' ORDER BY id DESC LIMIT 1";
Is it possible to do something like an AND condition with OR?
This is my simple sphinx query with AND only...
Job.search('', {with: {canonical_type: Zlib::crc32('SubCategory')}, conditions: { sub_category_ids: [4]}})
Sphinx Query (22.8ms) SELECT * FROM `job_core`, `job_delta` WHERE MATCH('#sub_category_ids [4]') AND `canonical_type` = 1916160457 AND `sphinx_deleted` = 0 LIMIT 0, 20 OPTION max_matches=50000
Sphinx Found 1 results
I'd like to add an OR... something like...
WHERE (MATCH('#sub_category_ids [4]') AND `canonical_type` = 1916160457 OR MATCH('#sub_category_ids [4]') AND `canonical_type` = 4282022807)
so that I can search for IDs with different canonical_type.
Not sure how to do it in Thinking Sphinx.
Thanks!
This is actully a limitation of Sphinx itself. It doesnt support OR in WHERE (nor nesting) - which is why it not in thinking-sphinx.
But
WHERE (MATCH('#sub_category_ids [4]') AND `canonical_type` = 1916160457
OR MATCH('#sub_category_ids [4]') AND `canonical_type` = 4282022807)
could be written as SphinxQL:
WHERE MATCH('#sub_category_ids [4]') AND `canonical_type` IN (1916160457,4282022807)
... ie IN() operator is kinda like 'OR'. Wouldn't be surprised if thinkinx-sphinx does it automatically with a array
with: {canonical_type: [Zlib::crc32('SubCategory1'), Zlib::crc32('SubCategory2')] }
based on http://freelancing-gods.com/thinking-sphinx/searching.html#filters
I am new to zend framework
I want to calculate number of rows in my query
this is my code:
$nm = new Zend_Db_Table('emp');
$row = $nm->fetchRow($nm->select()->where('id= ?', $a));
yes you can try in this way to get total number of rows return by your sql query.
$select = $this->select();
$select->where('id= ?', $a);
$result=$this->fetchAll($select);
if(empty($result)){
return false;
}else{
echo "Total number of users : -> ".count($result->toArray());
}
let me know if i can help you more.
If you are testing for the existance of a record by primary key then you can use $nm->find($a) and check for any results.
if you expect the result set to be small then you can do
$nm->fetchAll($nm->select()->where("id = ?", $a);
If you expect the result set to get big and all you are really after is the count then authoring a query that asks for the count of a field would probably make the most sense to keep the query from eating up a lot of server memory:
$row = $nm->getAdapter()->fetchRow("select count(*) as num_rows".
" from ".$nm->info(Zend_Db_Table_Row_Abstract::NAME).
" where ".$nm->getAdapter()->quoteInto("id = ?", $a));
echo "Users ".$row["num_rows"];
In Zend Framework 1, you can use count() function at the last of query for getting total number of rows in the table.
For example:
$row = $nm->fetchAll($nm->select()->where('id = ?', $a ))->count();
NOTE: This will only return total number of rows in the table. Output is Integer.