I have a table called user, and I need to get another users in an expecific distance range and the distance between them. My query is this:
select ( 6371 * acos(
cos( radians(users.latitude) )
* cos( radians(-22.9035) )
* cos( radians(-43.2096) - radians(users.longitude) )
+ sin( radians(users.latitude) )
* sin( radians(-22.9035) )
)
) as distance, users from users where users.id != 41 and distance > 50
I need to recover the user list and the distance, but the "as distance" is not working:
ERROR: column "distance" does not exist.
I tried with:
with distance as (
select ( 6371 * acos(
cos( radians(users.latitude) )
* cos( radians(-22.9035) )
* cos( radians(-43.2096) - radians(users.longitude) )
+ sin( radians(users.latitude) )
* sin( radians(-22.9035) )
)
) from users
)
select * from users where users.id != 41 and distance > 50
However, the error remains the same.
In Postgres, you cannot use column aliases in a where clause. That's why your first query does not work.
Your second query using a CTE can be made to work, but distance is a virtual table, not a column. Join with it.
with users_distance as (
select
id,
( 6371 * acos(
cos( radians(users.latitude) )
* cos( radians(-22.9035) )
* cos( radians(-43.2096) - radians(users.longitude) )
+ sin( radians(users.latitude) )
* sin( radians(-22.9035) )
)
) as distance
from users
)
select *
from users
join users_distance ud on users.id = ud.id
where users.id != 41 and ud.distance > 50
If you do this a lot, consider adding a generated column to the table so you don't have to recalculated it all the time.
alter table users add column distance numeric
generated always as (
6371 * acos(
cos( radians(users.latitude) )
* cos( radians(-22.9035) )
* cos( radians(-43.2096) - radians(users.longitude) )
+ sin( radians(users.latitude) )
* sin( radians(-22.9035) )
)
) stored
Try it
Finally, rather than doing these calculations yourself, consider using the very powerful PostGIS extension.
Related
i was stuck for days in this schema now.
I am trying to populate distance column in a different table from other 2 tables. Inside the table there are lat, long, city id, distance, and location id.
This is the current table that i wanted to populate
This is the two tables that i can get to calculate the distance from
LocationID are the same as ID in the first table
To calculate the distance to the nearest city i calculate it using lat long, this is what my code look like for the nearest distance
select location_id, distance
from (SELECT t.table1.location_id as location_id,
( 6371 * acos( cos( radians(6.414478) ) *
cos( radians(t.table1.latitude::float8) ) *
cos( radians(t.table1.longitude::float8) - radians(12.466646) )
+ sin( radians(6.414478) ) * sin( radians(t.table1.latitude::float8) ) ) ) AS distance
FROM t.table1
INNER JOIN t.table2
on t.table1.location_id = t.table2.id
) km
group by location_id, distance
Having distance < 2000
order by distance limit 20;
but the table only returns null value
I'm using PostgreSQL for this code and the application used for visualising is metabase.
I would recommend you to use ST_Distance function from PostGIS extension for distance calculation instead of doing it yourself. It will be easier and definitely much faster.
Edited: Probably misunderstood the initial intentions.
This should work:
select d.city_id, d.distance, latitude,location_id, longitude
from t.table1
left join lateral (
select city_id, distance from (
select location_id city_id, ( 6371 * acos( cos( radians(table3.latitude) ) *
cos( radians(t.table1.latitude::float8) ) *
cos( radians(t.table1.longitude::float8) - radians(table3.longitude) )
+ sin( radians(table3.latitude) ) * sin( radians(t.table1.latitude::float8) ) ) ) AS distance
from t.table3
) d
where distance < 2000
order by distance
limit 1
) cities on true
Try it out.
Best regards,
Bjarni
I have multiple tables in a json_agg for psql:
SELECT json_agg(t) FROM (SELECT *,
( SELECT row_to_json(b) FROM ( SELECT * from (SELECT *, ( (3959 * acos( cos( radians(6.414478) ) * cos( radians( lat ) ) * cos( radians( lng ) - radians(12.466646) ) + sin( radians(6.414478) ) * sin( radians( lat ) ) ) ) * 1.609344 ) AS distance
from farm_location ) al WHERE farm_location_id=supply_forecast.farm_location_id and distance < 100 ) b) as farm_location,
( SELECT json_agg(c) FROM ( SELECT *
FROM supply_forecast_status_history WHERE supply_forecast_id=supply_forecast.supply_forecast_id) c) as supply_forecast_status
FROM supply_forecast WHERE delete = B'0' ORDER BY farm_location.distance desc) t;
I'm trying to calculate the distance from a farm location. The issue is I'm trying to sort the data by "distance" but it's throwing an error:
ERROR: missing FROM-clause entry for table "farm_location"
LINE 6: ...FROM supply_forecast WHERE delete = B'0' ORDER BY farm_locat...
^
********** Error **********
ERROR: missing FROM-clause entry for table "farm_location"
SQL state: 42P01
Character: 642
If I remove the ORDER BY farm_location.distance desc the query works but the data is not sorted by distance. Any idea on how to go about this?
json_agg function can support order by parameter.
SELECT json_agg(t.* ORDER BY distance desc) FROM
(SELECT *,
(SELECT row_to_json(b) FROM
(SELECT * from
(SELECT *
, ((3959 * acos( cos( radians(6.414478)) * cos( radians( lat )) * cos(radians( lng )
- radians(12.466646) ) + sin( radians(6.414478) ) * sin( radians( lat ) ) ) ) * 1.609344 ) as distance
from farm_location) al
where farm_location_id=supply_forecast.farm_location_id and distance < 100
) b
) as farm_location
, (select json_agg(c)
from (SELECT * FROM supply_forecast_status_history
WHERE supply_forecast_id=supply_forecast.supply_forecast_id) c
)as supply_forecast_status
FROM supply_forecast WHERE delete = B'0'
) t;
I apply ORDER BY inside function json_agg() and itÅ› work OK
select
jsonb_agg(jsonb_build_object('toma_parada', app51.app51_tomas__toma_parada)|| jsonb_build_object('lado', app51.app51_tomas__lado)|| jsonb_build_object('existen_tomas_paradas', app51.app51_tomas__existen_tomas_paradas)|| jsonb_build_object('b', app51.app51_tomas__b)|| jsonb_build_object('a', app51.app51_tomas__a)|| jsonb_build_object('m', app51.app51_tomas__m)|| jsonb_build_object('nombre', app51.app51_tomas__nombre, 'id_publico', app51.app51_tomas__id_publico , 'id', app51.app51_tomas__id )
order by app51.app51_tomas__id desc) as app51_tomas,
app51.fecha_modificacion,
app51.fecha_creacion,
app51.id_cliente,
app51.id_rumas,
app51.id_usuario,
app51.id,
app51.id_publico,
app51.habilitado,
app51.borrado,
app51.produccion_fin,
app51.produccion_inicio
from
(
select
distinct app51.id_cliente,
app51.id_rumas,
app51.id_usuario,
app51.id,
app51.id_publico,
app51.habilitado,
app51.borrado,
app51.atributos ->> 'produccion_fin' as produccion_fin,
app51.atributos ->> 'produccion_inicio' as produccion_inicio,
app51_tomas.atributos ->> 'toma_parada' as app51_tomas__toma_parada,
app51_tomas.atributos ->> 'lado' as app51_tomas__lado,
app51_tomas.atributos ->> 'existen_tomas_paradas' as app51_tomas__existen_tomas_paradas,
app51_tomas.atributos ->> 'b' as app51_tomas__b,
app51_tomas.atributos ->> 'a' as app51_tomas__a,
app51_tomas.atributos ->> 'm' as app51_tomas__m,
app51_tomas.atributos ->> 'nombre' as app51_tomas__nombre,
app51_tomas.id as app51_tomas__id ,
app51_tomas.id_publico as app51_tomas__id_publico ,
jsonb_build_object() as id_usuario_creacion,
jsonb_build_object() as id_usuario_modificacion,
(
select
TO_CHAR(app51.fecha_creacion::timestamptz at time zone 'GMT' , 'YYYY-MM-DD"T"HH24:MI:SS.ss0"Z"')) as fecha_creacion ,
(
select
TO_CHAR(app51.fecha_modificacion::timestamptz at time zone 'GMT' , 'YYYY-MM-DD"T"HH24:MI:SS.ss0"Z"')) as fecha_modificacion
from
app51
left join app51_app51_tomas on
app51_app51_tomas.id_app51 = app51.id
left join app51_tomas on
app51_app51_tomas.id_app51_tomas = app51_tomas.id
and ((app51_tomas.habilitado = true
and app51_tomas.borrado = false)
or app51_tomas is null )
where
(app51.habilitado = true
and app51.borrado = false)
order by
app51_tomas.id desc) as app51
group by
app51.fecha_modificacion,
app51.fecha_creacion,
app51.id_cliente,
app51.id_rumas,
app51.id_usuario,
app51.id,
app51.id_publico,
app51.habilitado,
app51.borrado,
app51.produccion_fin,
app51.produccion_inicio
I have a PostgreSQL query that looks like this:
SELECT *,
2 * 3961 * asin(sqrt((sin(radians((latitude - 40.2817993164062) / 2))) ^ 2 + cos(radians(40.2817993164062)) * cos(radians(latitude)) * (sin(radians((longitude - -111.720901489258) / 2))) ^ 2)) as distance,
(SELECT json_agg(deals.*) FROM deals WHERE vendors.id = deals.vendorid) as deals FROM vendors
WHERE ( category = 'Food' )
AND (distance < 80)
AND (nationwide IS FALSE OR nationwide is NULL)
ORDER BY featured ASC, created DESC, distance ASC
I'm getting the distance in miles using the second select part.
The problem is the part that says AND (distance < 80) I get the following error: column "distance" does not exist the weird thing is that if I remove the AND (distance < 80) it works and it also sorts correctly by distance, also the outputted data includes distance, so it's grabbing the distance correctly but for some reason wont let me use the distance as a filter in the WHERE clauses and I can't figure out why.
distance is just an alias. You could try something like:
WITH vendors_distance as (
SELECT *,
2 * 3961 * asin(sqrt((sin(radians((latitude - 40.2817993164062) / 2))) ^ 2 + cos(radians(40.2817993164062)) * cos(radians(latitude)) * (sin(radians((longitude - -111.720901489258) / 2))) ^ 2)) as distance
FROM vendors
WHERE ( category = 'Food' )
AND (nationwide IS FALSE OR nationwide is NULL)
)
SELECT vendors_distance.*,
(SELECT json_agg(deals.*) FROM deals WHERE vendors_distance.id = deals.vendorid) as deals
FROM vendors_distance
WHERE (distance < 80)
ORDER BY featured ASC, created DESC, distance ASC
SELECT DISTINCT "Users"."id" , "Users".name,
"Users"."surname", "Users"."gender",
"Users"."dob", "Searches"."start_date"
FROM "Users"
LEFT JOIN "Searches" ON "Users"."id" = "Searches"."user_id"
WHERE (SQRT( POW(69.1 * ("Users"."latitude" - 45.465454), 2) + POW(69.1 * (9.186515999999983 - "Users"."longitude") * COS("Users"."latitude" / 57.3), 2))) < 20
AND "Users"."status" = true
AND "Users"."id" != 18
AND "Searches"."activity" = \'clubbing\'
AND "Users"."gender" = \'m\'
AND "Users"."age" BETWEEN 18 AND 30
ORDER BY ABS( "Searches"."start_date" - date \'2016-07-07\' )
For some reasons the above query returns the following error:
for SELECT DISTINCT, ORDER BY expressions must appear in select list
I only want to return unique users but I really don't know what's wrong with it.
Thanks for your help
Just doing what the error message says I would include the expression ABS( "Searches"."start_date" - date '2016-07-07' ) in the SELECT list. No need to change your query logic.
absdiffdate can be discarded later when processing the result.
SELECT DISTINCT "Users"."id" , "Users".name,
"Users"."surname", "Users"."gender",
"Users"."dob", "Searches"."start_date",
ABS( "Searches"."start_date" - date '2016-07-07' ) absdiffdate
FROM "Users"
LEFT JOIN "Searches" ON "Users"."id" = "Searches"."user_id"
WHERE (SQRT( POW(69.1 * ("Users"."latitude" - 45.465454), 2) + POW(69.1 * (9.186515999999983 - "Users"."longitude") * COS("Users"."latitude" / 57.3), 2))) < 20
AND "Users"."status" = true
AND "Users"."id" != 18
AND "Searches"."activity" = 'clubbing'
AND "Users"."gender" = 'm'
AND "Users"."age" BETWEEN 18 AND 30
ORDER BY ABS( "Searches"."start_date" - date '2016-07-07' )
Will this new column results in possibly more records when DISTINCT is applied?
I don't think so because you are subtracting a constant from start_date and for similar start_date corresponds similar outcome.
In Postgres, you can use DISTINCT ON to get one row per user id:
SELECT DISTINCT ON (u."id") u."id", u.name, u."surname", u."gender", u."dob", s."start_date"
FROM "Users" u LEFT JOIN
"Searches" s
ON u."id" = s."user_id"
WHERE (SQRT( POW(69.1 * (u."latitude" - 45.465454), 2) + POW(69.1 * (9.186515999999983 - u."longitude") * COS(u."latitude" / 57.3), 2))) < 20 AND
u."status" = true AND
u."id" != 18 AND "Searches"."activity" = \'clubbing\' AND
u."gender" = \'m\' AND
u."age" BETWEEN 18 AND 30
ORDER BY users.id, ABS(s."start_date" - date \'2016-07-07\' );
Notice how table aliases make the query easier to write and to read.
In t-sql, it's possible to run multiple select statements without a ;. Example:
select 1 select 2 is valid, and returns two datasets of 1 and 2 respectively.
In postgres, it is not possible to run multiple select statements... you need a ; delimiter otherwise you get a syntax error.
Referencing the docs: http://www.postgresql.org/docs/current/interactive/libpq-exec.html
Multiple queries sent in a single PQexec call are processed in a single transaction, unless there are explicit BEGIN/COMMIT commands included in the query string to divide it into multiple transactions.
How can I do this?
Let's say I want to run these two queries on the server: select 1 select 2: should it look like this:
begin
select 1
commit;
begin
select 2
commit
I'm ok with it only returning the last query as the result set, but I need to know that the first query was executed on the server, even if it's not returning with that result set.
Why I want to do this: I have a complex sql script that has ~6 temp tables to build that the main query will use. By delimiting the temp tables with the ; syntax, I can't schedule this script in cron to run on a schedule. If I can get the temp tables to run and the main query to access them in the same PGexec call, I'd be very very happy.
You don't need libpq directly, you can just use he psql front end (in cron, you might need to specify the absolute pathname for the binary)
#!/bin/sh
psql -U my_user mydb <<OMG
begin;
select tralal 1;
commit;
begin;
select domtidom 2;
commit;
OMG
I was able to accomplish what I was looking for with CTEs rather than temp tables... one long chain of CTEs (acting as temp tables) waterfalling into the main query.
A simple example:
with first as (
select 1 as col
),
second as (
select 2 as col
)
select * from first union all select * from second
A more complex example:
with COGS as (
select 'Product1' Vertical, 3.0 Credit, 1.00 Debit, 2.75 Blend, 4.30 Amex, 0.25 ACH union
select 'Product2', 3.1, 2.2, 2.8, 4.5, 0.25 union
),
Allocable_Card_Volume as (
select MPR.Date, sum(MPR.Card_Volume_Net_USD) Allocable_Card_Volume
from mpr_base MPR
where MPR.Gateway in ('YapProcessing') and MPR.Vertical not in ('HA-Intl','HA')
group by MPR.Date
),
COGS_Financials_Base as (
select '2013-01-31'::DATE Date , 1000 Total_COGS , 200 Homeaway , (select Allocable_Card_Volume from Allocable_Card_Volume where Date in ('2013-01-31') ) Allocable_Card_Volume union
),
Initial_COGS as (
select
MPR.Date,
sum(
case when MPR.PaymentTypeGroup in ('ACH_Scan','AmEx') then (Txn_Count * COGS.ACH) else 0 end +
case when MPR.Vertical not in ('HA') and MPR.PaymentTypeGroup in ('Card','AmEx-Processing') then
coalesce( ((Credit_Card_Net_USD - Amex_Processing_Net_USD) * COGS.Credit * 0.01),0) + coalesce((Debit_Card_Net_USD * COGS.Debit * 0.01),0) + coalesce((Amex_Processing_Net_USD * COGS.Amex * 0.01),0) + coalesce((case when TPV is null and PaymentTypeGroup in ('Card') then TPV_Billing else 0 end * COGS.Blend * 0.01),0)
when MPR.Vertical in ('HA') and MPR.PaymentTypeGroup in ('Card','AmEx-Processing') and FeePaymentType in ('PropertyPaid') then
coalesce(COGS_Financials.Homeaway,0)
else 0 end
) Initial_COGS
from
mpr_base MPR
left join COGS on COGS.Vertical = MPR.Vertical and MPR.Gateway in ('YapProcessing') and MPR.PaymentTypeGroup not in ('Cash')
left join COGS_Financials_Base COGS_Financials on MPR.Date = COGS_Financials.Date and MPR.Gateway in ('YapProcessing') and MPR.PaymentTypeGroup in ('Card')
where MPR.Gateway in ('YapProcessing') and MPR.Vertical not in ('HA-Intl') and MPR.PaymentTypeGroup not in ('Cash')
group by
MPR.Date
),
COGS_Financials as (
select
COGS_Financials_Base.*, (COGS_Financials_Base.Total_COGS - Initial_COGS.Initial_COGS) Allocation
from
COGS_Financials_Base
join Initial_COGS on COGS_Financials_Base.Date = Initial_COGS.Date
),
MPR as (
select
MPR.Date,MPR.Gateway,MPR.Vertical, MPR.ParentAccountId, MPR.ParentName ,
MPR.PaymentTypeGroup ,
sum(TPV_USD) TPV_USD,
sum(TPV_Net_USD) TPV_Net_USD,
sum(Revenue_Net_USD) Revenue_Net_USD ,
sum(coalesce(
case when MPR.PaymentTypeGroup in ('ACH_Scan','AmEx') then (Txn_Count * COGS.ACH) else 0 end +
case when MPR.Vertical not in ('HA') and MPR.PaymentTypeGroup in ('Card','AmEx-Processing') then
coalesce( ((Credit_Card_Net_USD - Amex_Processing_Net_USD) * COGS.Credit * 0.01),0) + coalesce((Debit_Card_Net_USD * COGS.Debit * 0.01),0) + coalesce((Amex_Processing_Net_USD * COGS.Amex * 0.01),0) + coalesce((case when TPV is null and PaymentTypeGroup in ('Card') then TPV_Billing else 0 end * COGS.Blend * 0.01),0)
+(coalesce( ( ( cast(Card_Volume_Net_USD as decimal(18,2) ) / cast(COGS_Financials.Allocable_Card_Volume as decimal(18,2)) ) * COGS_Financials.Allocation ), 0) ) -- Excess
when MPR.Vertical in ('HA') and MPR.PaymentTypeGroup in ('Card','AmEx-Processing') and MPR.FeePaymentType in ('PropertyPaid') then coalesce(COGS_Financials.Homeaway,0)
else 0
end,0)
) COGS_USD,
sum(Txn_Count) Txn_Count
from
mpr_Base MPR
left join COGS on COGS.Vertical = MPR.Vertical and MPR.Gateway in ('YapProcessing') and MPR.PaymentTypeGroup not in ('Cash')
left join COGS_Financials on MPR.Date = COGS_Financials.Date and MPR.Gateway in ('YapProcessing') and MPR.PaymentTypeGroup in ('Card','AmEx-Processing')
where
MPR.Date in ('2016-02-29')
group by
MPR.Date,MPR.Gateway,MPR.Vertical , MPR.ParentAccountId ,MPR.ParentName,
MPR.PaymentTypeGroup
)
select
Vertical,
sum(TPV_USD)::money as TPV_USD,
sum(Revenue_Net_USD)::money as Revenue_Net_USD,
sum(COGS_USD)::money COGS_USD,
round((sum(Revenue_Net_USD)-sum(COGS_USD))/sum(Revenue_Net_USD)*100,2) Accounting_Margin
from
MPR
where Date in ('2016-02-29')
group by
Vertical
union all
select
'Total' ,
sum(TPV_USD)::money as TPV_USD,
sum(Revenue_Net_USD)::money as Revenue_Net_USD,
sum(COGS_USD)::money COGS_USD,
round((sum(Revenue_Net_USD)-sum(COGS_USD))/sum(Revenue_Net_USD)*100,2) Accounting_Margin
from
MPR
where Date in ('2016-02-29')
I said it would be complex :-)
From your answer, you could also do this
SELECT * FROM a
UNION ALL
SELECT * FROM b
UNION ALL
SELECT * FROM c
...