SetSortMode in sphinx not working with delta index - sphinx

So my sphinx.conf file contains something similar. Basically I am using delta index to make things quick.
source main
{
#...
sql_query_pre = SET NAMES utf8
sql_query_pre = REPLACE INTO sph_counter SELECT 1, MAX(id) FROM photo
sql_query = \
SELECT p.id AS id, p.search AS search, COUNT(li.id) AS total_likes \
FROM `photo` p \
LEFT JOIN `like` li \
ON p.id = li.photo_id \
WHERE p.id <= ( SELECT max_doc_id FROM sph_counter WHERE counter_id=1 ) \
GROUP BY \
p.id
#...
sql_query_info = SELECT * FROM photo WHERE id=$id
}
source delta : main
{
sql_query_pre = SET NAMES utf8
sql_query = \
SELECT p.id AS id, p.search AS search, COUNT(li.id) AS total_likes \
FROM `photo` p \
LEFT JOIN `like` li \
ON p.id = li.photo_id \
WHERE p.id > ( SELECT max_doc_id FROM sph_counter WHERE counter_id=1 ) \
GROUP BY \
p.id
}
And in the php when I retrieve data I also want to have some sort of sorting methods.
$s->SetSortMode(SPH_SORT_EXTENDED, '#relevance DESC, total_likes DESC, #id DESC');
$result = $s->Query($data['query'], "delta main");
Sorting was working fine when I had only main index. But now when I search with both indexes, results from the delta index is appended at the front. What I actually want is results from both indexes are fetched and then sorted according to preferences i.e. #relevance DESC, total_likes DESC, #id DESC in my case. That is total_likes should be given preference over id

Thanks #barryhunter for the solution. The solution was that in the delta index second sql_query_pre had to be overwritten.
sql_query_pre = SET NAMES utf8
sql_query_pre =
sql_query = \

Related

How to use "with table as" in pyspark sql function?

The code below has an error
spark.sql("WITH q AS (SELECT b.*
FROM df \
WHERE parent_id = 59 \
UNION ALL \
SELECT d.* \
FROM df d \
JOIN q \
ON d.parent_id = q.id) \
SELECT * \
FROM q").show()
the error is:
AnalysisException: Table or view not found: q; line 1 pos 221;
So I need to create a view of 'q', but how?

How to Finding Array Differences using postgres SQL

I have a table in Postgres DB as below:
SOURCE
Entity
DiFF
PRD
E1,E2,E3,E4,E5
MC
E1,E2
GT1
E1,E2,E3
I Need to insert the differences Between PRD and MC,GT1 into the DIFF column using postgres SQL
Expected result
SOURCE
Entity
DiFF
PRD
E1,E2,E3,E4,E5
MC
E1,E2
E3,E4,E5
GT1
E1,E2,E3
E4,E5
This is a terrible data model, but anyways:
The following query will get the "difference" between the column separated lists (not arrays) for MC and GT1:
select source,
(select string_agg(xp.item, ',')
from data p
cross join unnest(string_to_array(p.entity, ',')) as xp(item)
where p.source = 'PRD'
and not exists (select *
from unnest(string_to_array(d.entity, ',')) as x(item)
where xp.item = x.item)) as diff
from data d
where source in ('MC', 'GT1')
Given your sample data this returns:
source | diff
-------+---------
MC | E3,E4,E5
GT1 | E4,E5
This can be used to UPDATE the table (not "insert"!)
update data
set diff = t.diff
from (
select source,
(select string_agg(xp.item, ',')
from data p
cross join unnest(string_to_array(p.entity, ',')) as xp(item)
where p.source = 'PRD'
and not exists (select *
from unnest(string_to_array(d.entity, ',')) as x(item)
where xp.item = x.item)) as diff
from data d
where source in ('MC', 'GT1')
) t
where data.source = t.source;

SQLAchemy ORM: LEFT JOIN LATERAL() ON TRUE

I am trying to replicate the following raw query:
SELECT r.id, r.name, e.id, e.title, e.start, e.end
FROM room r
LEFT JOIN LATERAL (
SELECT evt.id, evt.title, evt.start, evt.end
FROM event evt, calendar cal
WHERE
r.calendar_id=cal.id AND evt.calendar_id=cal.id AND evt.end>%(start)s
ORDER BY abs(extract(epoch from (evt.start - %(start)s)))
LIMIT 1
) e ON TRUE
WHERE r.company_id=%(company_id)s;
with the SQLAlchemy ORM:
start = datetime.datetime.now()
company_id = 6
event_include = session.query(
Event.id,
Event.title,
Event.start,
Event.end) \
.filter(
Room.calendar_id == Calendar.id,
Event.calendar_id == Calendar.id,
Event.end > start,
) \
.order_by(func.abs(func.extract('epoch', Event.start - start))) \
.limit(1) \
.subquery() \
.lateral()
query = session.query(Room.id, Room.name, event_include) \
.filter(Room.company_id == company_id)
Which produces the following SQL:
SELECT room.id AS room_id, room.name AS room_name, anon_1.id AS anon_1_id, anon_1.title AS anon_1_title, anon_1.start AS anon_1_start, anon_1."end" AS anon_1_end
FROM room, LATERAL (
SELECT event.id AS id, event.title AS title, event.start AS start, event."end" AS "end"
FROM event, calendar
WHERE room.calendar_id = calendar.id AND event.calendar_id = calendar.id AND event."end" > %(end_1)s ORDER BY abs(EXTRACT(epoch FROM event.start - %(start_1)s)
)
LIMIT %(param_1)s) AS anon_1
WHERE room.company_id = %(company_id_1)s
This returns all the rooms and their next calendar event, but only if there is a next calendar event available. It needs to be a LEFT JOIN LATERAL() ON TRUE, but I'm not sure how to do that.
Any help here would be great.
Use outerjoin with true expression
from sqlalchemy import true
query = session.query(Room.id, Room.name, event_include) \
.outerjoin(event_include, true()) \
.filter(Room.company_id == company_id)

Postgres get rows which hasnt match in other table

I need your help. I need an advanced Query to my database. Im showing part of my database following:
Place (id, name, address)
Local (id, place_id, name)
PlaceReservation(id, local_id, date)
Media_Place (id, place_id, type)
Now I need a query, which gets all places with logo, which have AT LEAST ONE local which hasn't been reserved on a specific day e.g: 2015-07-01.
Help me please, because I haven't an idea how to do it. I thought about an outer join but I don't know how use it.
I was trying by:
$query = 'SELECT DISTINC *,
(SELECT sum(po.rating)/count(po.id)
FROM "Place_Opinion" po
WHERE po.place_id = p.id AND po.deleted = false) AS rating,
mp.path as logo_path
FROM "Place" p
INNER JOIN "Media_Place" mp ON mp.place_id = p.id
JOIN Local ON Local.place_id = Place.id
LEFT JOIN (
SELECT id AS rr, local_id
FROM PlaceReservation
WHERE date_start = \'2015-07-01\') Reserved ON Reserved.local_id = Local.id
WHERE mp.type = ' . Model_Row_MediaPlace::LOGO_TYPE . '
AND mp.deleted = false
AND p.deleted = false
AND rr IS NULL';
Looking for things that do not exist in a database is usually very inefficient. But you can change the logic around by finding places that do have a booking for the specified date, then LEFT JOIN that to all places with a logo and filter out the records with a reservation:
SELECT DISTINCT p.*, po.rating, mp.path as logo_path
FROM "Place" p
JOIN "Media_Place" mp ON mp.place_id = p.id AND mp.deleted = false AND mp.type = ?
JOIN Local ON Local.place_id = p.id
LEFT JOIN (
SELECT id AS rr, local_id
FROM PlaceReservation
WHERE date_start = '2015-07-01') reserved ON reserved.local_id = Local.id
LEFT JOIN (
SELECT place_id, avg(rating) AS rating
FROM "Place_Opinion"
WHERE deleted = false
GROUP BY place_id) po ON po.place_id = p.id
WHERE p.deleted = false
AND reserved.rr IS NULL;
The average rating per places is calculated in a separate sub-query. The error you had was because you referenced the "Place" table (p.id) before it was defined. For simple columns you can do that, but for sub-queries you can't.

Sphinxql - filtering with having

I'm trying to filter query with HAVING but all I get this error:
mysql> SELECT id FROM related_tags GROUP BY application_id HAVING COUNT(*)=10;
ERROR 1064 (42000): sphinxql: syntax error, unexpected IDENT, expecting $end near 'HAVING COUNT(*)=10'
I'm using Sphinx 2.2.6-id64-release, it supports HAVING
This is my index, if it does matter (application_id attribute is for grouping by id).
sql_query = \
SELECT `id`, `id` as `application_id`, `clear_title`\
FROM `applications`\
WHERE `id`>=$start AND `id`<=$end
sql_query_range = SELECT MIN(id),MAX(id) FROM applications
sql_attr_uint = application_id
sql_attr_multi = uint tag_id from query; \
select application_id, tag_id \
from application_tag_stemmed2;
I think you have to make it a virtual attribute, arbitary expressions not allowed in the HAVING clause itself.
SELECT id,COUNT(*) AS cnt FROM related_tags GROUP BY application_id HAVING cnt=10;