There is geometry type column in database like Postgis or h2gis(I am using it).
In the console provided by database, I can create a geometry value with select ST_GeomFromText('POINT(12.3 12)', 4326).
Or select a column with geometry type simply by select * from geom.
However I don't know how to insert a geometry value (a string actually) into a table or the opposite direction conversion.
There are also several miscellaneous question below.
Here is the table definition in slick:
class TableSimple(tag:Tag) extends Table[ (Double,String,String) ](tag,"tb_simple"){
def col_double = column[Double]("col_double",O.NotNull)
def col_str = column[String]("col_str",O.NotNull)
def geom = column[String]("geom",O.DBType("Geometry"))
def * = (col_double,col_str,geom)
}
1. About select
The most simple one:
sql" select col_double,col_str, geom from tb_simple ".as[(Double,String,String)]
won't work unless casting geom to string explicitly like:
sql" select col_double,col_str, cast( geom as varchar) from tb_simple ".as[(Double,String,String)]
The first sql throws the error java.lang.ClassNotFoundException: com.vividsolutions.jts.io.ParseException
Q1: How does slick know com.vividsolutions.jts.io.ParseException (it is lib used by h2gis)? Is it an error on the server side or client side(slick side)?
Q2: How to convert/treat column geom as string without writing too much code(e.g. create a new column type in slick)?
2. About insert
First of all the following sql works
StaticQuery.updateNA(""" insert into tb_simple values(11,'abcd',ST_GeomFromText('POINT(5.300000 1.100000)', 4326)) """).execute
I hope code like TableQuery[TableSimple] += (10.3,"hello","ST_GeomFromText('POINT(0.300000 1.100000)'") would work but it doesn't.
It shouldn't because slick translate it to
insert into tb_simple values(11,'abcd','ST_GeomFromText(''POINT(5.300000 1.100000)'', 4326)')
Notice the function ST_GeomFromText become a part of string, that's why it doesn't work.
Q3: Can I implant a string directly for a column instead of wrapped with '' in slick?
I hope I can insert a row as easy as TableQuery[TableSimple] += (10.3,"hello","ST_GeomFromText('POINT(0.300000 1.100000)'") or similar code.
Q4 What's the most convenient way in Slick to implement bidirectional conversion to and from String for a geometry or other self-defined column in the database?
Answering you main question: Slick-pg offers mapping of geometry types in the db to actual geometry types in your model.
It works for Postgis, but maybe it can also work with H2Gis.
You can find slick-pg at https://github.com/tminglei/slick-pg
Related
I'm trying to convert each row in a jsonb column to a type that I've defined, and I can't quite seem to get there.
I have an app that scrapes articles from The Guardian Open Platform and dumps the responses (as jsonb) in an ingestion table, into a column called 'body'. Other columns are a sequential ID, and a timestamp extracted from the response payload that helps my app only scrape new data.
I'd like to move the response dump data into a properly-defined table, and as I know the schema of the response, I've defined a type (my_type).
I've been referring to the 9.16. JSON Functions and Operators in the Postgres docs. I can get a single record as my type:
select * from jsonb_populate_record(null::my_type, (select body from data_ingestion limit 1));
produces
id
type
sectionId
...
example_id
example_type
example_section_id
...
(abbreviated for concision)
If I remove the limit, I get an error, which makes sense: the subquery would be providing multiple rows to jsonb_populate_record which only expects one.
I can get it to do multiple rows, but the result isn't broken into columns:
select jsonb_populate_record(null::my_type, body) from reviews_ingestion limit 3;
produces:
jsonb_populate_record
(example_id_1,example_type_1,example_section_id_1,...)
(example_id_2,example_type_2,example_section_id_2,...)
(example_id_3,example_type_3,example_section_id_3,...)
This is a bit odd, I would have expected to see column names; this after all is the point of providing the type.
I'm aware I can do this by using Postgres JSON querying functionality, e.g.
select
body -> 'id' as id,
body -> 'type' as type,
body -> 'sectionId' as section_id,
...
from reviews_ingestion;
This works but it seems quite inelegant. Plus I lose datatypes.
I've also considered aggregating all rows in the body column into a JSON array, so as to be able to supply this to jsonb_populate_recordset but this seems a bit of a silly approach, and unlikely to be performant.
Is there a way to achieve what I want, using Postgres functions?
Maybe you need this - to break my_type record into columns:
select (jsonb_populate_record(null::my_type, body)).*
from reviews_ingestion
limit 3;
-- or whatever other query clauses here
i.e. select all from these my_type records. All column names and types are in place.
Here is an illustration. My custom type is delmet and CTO t remotely mimics data_ingestion.
create type delmet as (x integer, y text, z boolean);
with t(i, j, k) as
(
values
(1, '{"x":10, "y":"Nope", "z":true}'::jsonb, 'cats'),
(2, '{"x":11, "y":"Yep", "z":false}', 'dogs'),
(3, '{"x":12, "y":null, "z":true}', 'parrots')
)
select i, (jsonb_populate_record(null::delmet, j)).*, k
from t;
Result:
i
x
y
z
k
1
10
Nope
true
cats
2
11
Yep
false
dogs
3
12
true
parrots
I'm transforming some old PostgreSQL code to jOOQ, and I'm currently struggling with SQL that has multiple WITH clauses, where each one depends on previous. It would be best to keep the SQL logic the way it was written and not to change it (e.g. multiple queries to DB).
As it seems, there is no way to do SELECT on something that is UPDATE ... RETURNING, for example
dsl.select(DSL.asterisk())
.from(dsl.update(...)
.returning(DSL.asterisk())
)
I've created some test tables, trying to create some sort of MVCE:
create table dashboard.test (id int primary key not null, data text); --test table
with updated_test AS (
UPDATE dashboard.test SET data = 'new data'
WHERE id = 1
returning data
),
test_user AS (
select du.* from dashboard.dashboard_user du, updated_test -- from previous WITH
where du.is_active AND du.data = updated_test.data
)
SELECT jsonb_build_object('test_user', to_jsonb(tu.*), 'updated_test', to_jsonb(ut.*))
FROM test_user tu, updated_test ut; -- from both WITH clauses
So far this is my jOOQ code (written in Kotlin):
dsl.with("updated_test").`as`(
dsl.update(Tables.TEST)
.set(Tables.TEST.DATA, DSL.value("new data"))
.returning(Tables.TEST.DATA) //ERROR is here: Required Select<*>, found UpdateResultStep<TestRecord>
).with("test_user").`as`(
dsl
.select(DSL.asterisk())
.from(
Tables.DASHBOARD_USER,
DSL.table(DSL.name("updated_test")) //or what to use here?
)
.where(Tables.DASHBOARD_USER.IS_ACTIVE.isTrue
.and(Tables.DASHBOARD_USER.DATA.eq(DSL.field(DSL.name("updated_test.data"), String::class.java)))
)
)
.select() //here goes my own logic for jsonBBuildObject (which is tested and works for other queries)
.from(
DSL.table(DSL.name("updated_test")), //what to use here
DSL.table(DSL.name("test_user")) //or here
)
Are there any workarounds for this? I'd like to avoid changing SQL if possible.
Also, in this project this trick is used very often to get JSON(B) from UPDATE clause (table has JSON(B) columns too):
with _updated AS (update dashboard.test SET data = 'something' WHERE id = 1 returning *)
select to_jsonb(_updated.*) from _updated;
and it will be a real step back for us if there is no workaround for this.
I'm using JOOQ version 3.13.3, and Postgres 12.0.
This is currently not supported in jOOQ, see:
https://github.com/jOOQ/jOOQ/issues/3185
https://github.com/jOOQ/jOOQ/issues/4474
The workaround is, as always, when some vendor specific syntax is unsupported, to resort to plain SQL templating
E.g.
// If you don't need to map data types
dsl.fetch("with t as ({0}) {1}", update, select);
// If you need to map data types
dsl.resultQuery("with t as ({0}) {1}", update, select).coerce(select.getSelect()).fetch();
I have user define type like
CREATE TYPE point ( pointId int, floor text);
And I have table like:
CREATE TABLE path (
id timeuuid,
val timeuuid,
PointList list<frozen <point>>,
PRIMARY KEY(id,val)
);
And have create index like
create index on path(PointList);
But the problem is I am not able to execute select query where PointList = [floor : "abc"].
I google for 2 hours but not able to find the hint.
I am using this query to execute select query
Select * from path where val = sdsdsdsdsds-dsdsdsd-dssds-sdsdsd and PointList contains {floor: 'eemiG8NbzdRCQ'};
I can see this data in my cassandra table but not able to get that data using above query.
I want select query where we can only use floor and val. Because we only have data for floor and val
I tried many different ways but nothing is working.
I would appreciate any kind of hint or help.
Thank you,
Frozen point means point type is frozen, you can't partially provide point value, you have to provide the full value of point
Example Query :
select * from path where pointlist CONTAINS {pointId : 1, floor : 'abc'};
I'm trying to select and order stores by their distance to a point with GeoAlchemy2 / PostGIS but for some reason I keep getting an error.
It seems GeoAlchemy2 wraps things with ST_AsBinary, but when I try to select the distance it tries to wrap the result of the distance calculation. I have no idea how to fix this.
I use this ORM query.
distance = (
Store.coordinates.cast(Geometry)
.distance_centroid(query_centroid)
.label('distance')
)
stores = stores.order_by(distance).add_columns(distance)
The model.
class Store(db.Model):
__tablename__ = 'stores'
store_id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)
address_details = db.Column(db.String)
coordinates = db.Column(Geography('POINT'))
The error I get.
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) function st_asbinary(double precision) does not exist
LINE 1: ...Binary(stores.coordinates) AS stores_coordinates, ST_AsBinar...
^
HINT: No function matches the given name and argument types. You might need to add explicit type casts.
[SQL: 'SELECT stores.store_id AS stores_store_id,
stores.name AS stores_name,
stores.address_details AS stores_address_details,
ST_AsBinary(stores.coordinates) AS stores_coordinates,
ST_AsBinary(CAST(stores.coordinates AS geometry(GEOMETRY,-1)) <-> ST_GeomFromEWKT(%(param_1)s)) AS distance
FROM stores ORDER BY distance']13 -46.730347)'}]
[parameters: {'param_1': 'POINT(-23.3569
The problem is precisely in this part...
ST_AsBinary(
CAST(stores.coordinates AS geometry(GEOMETRY,-1))
<->
ST_GeomFromEWKT(%(param_1)s)
) AS distance
Notice how ST_AsBinary wraps the distance between the two points instead of wrapping just the geom, for example? (I'm not sure it should wrap the geom in this case, either)
Can anyone help? I just want to know how far things are.
An average user at freenode answered it for me.
GeoAlchemy2 will convert columns of type Geometry if they are in the select statement. Even though the result of the distance expression is a double, and not a Geometry, GeoAlchemy2 isn't smart enough to figure that out.
The column needs to be explicit cast in the ORM.
The fixed query:
distance = (
Store.coordinates.cast(Geometry)
.distance_centroid(query_centroid)
.cast(db.Float)
.label('distance')
)
stores = stores.order_by(distance).add_columns(distance)
Using Open JPA 2.0, Database is DB2 9.7. For query like SELECT COUNT(1) FROM USER WHERE FNAME := fname, JPA is converting the query to SELECT COUNT(CAST(? AS BIGINT)) FROM TABLENAME.
How to avoid the CAST to BIGINT?
Code sample below:
query = entityManager.createNamedQuery("qry.checkuser");
query.setParameter("fname", fname);
Long count = (Long)query.getSingleResult();
Which one is the problem in the CAST?
I think you can't avoid it since is SQL generated by the JPA provider.
BTW, I allways use Number super class instead of specific subclass:
query = entityManager.createNamedQuery("qry.checkuser", Number.class);
query.setParameter("fname", fname);
Number count = query.getSingleResult();
// Do whatever is needed
if (count.longValue()...
This way there's no problem if the JPA provider returns a integer, long or BigXXXX.
What worked for me is:
SELECT COUNT(USER_ID) FROM USER WHERE FNAME := fname
Basically we need to use a non nullable column like Primary key column and with this change the CAST can be avoided which consumes additional CPU, a minor gain.