Using unnest as a field rather than a table in jOOQ - postgresql

This is the query I am trying to run in PostgreSQL:
SELECT * FROM message WHERE id IN (
SELECT unnest(message_ids) "mid"
FROM session_messages WHERE session_id = '?' ORDER BY "mid" ASC
);
However, I am not able do something:
create.selectFrom(Tables.MESSAGE).where(Tables.MESSAGE.ID.in(
create.select(DSL.unnest(..))
Because DSL.unnest is a Table<?>, which makes sense since it is trying to take a List-like object (mostly a literal) and convert it to table.
I have a feeling that I need to find a way to wrap the function around my field name, but I have no clue as to how to proceed.
NOTE. The field message_ids is of type bigint[].
EDIT
So, this is how I am doing it now, and it works exactly as expected, but I am not sure if this is the best way to do it:
Field<Long> unnestMessageIdField = DSL.field(
"unnest(" + SESSION_MESSAGES.MESSAGE_IDS.getName() + ")",
Long.class)
.as("mid");
Field<Long> messageIdField = DSL.field("mid", Long.class);
MESSAGE.ID.in(
ctx.select(messageIdField).from(
ctx.select(unnestMessageIdField)
.from(Tables.CHAT_SESSION_MESSAGES)
.where(Tables.CHAT_SESSION_MESSAGES.SESSION_ID.eq(sessionId))
)
.where(condition)
)
EDIT2
After going through the code on https://github.com/jOOQ/jOOQ/blob/master/jOOQ/src/main/java/org/jooq/impl/DSL.java I guess the right way to do this would be:
DSL.function("unnest", SQLDataTypes.BIGINT.getArrayType(), SESSION_MESSAGES.MESSAGE_IDS)
EDIT3
Since as always lukas is here for my jOOQ woes, I am going to capitalize on this :)
Trying to generalize this function, in a signature of sort
public <T> Field<T> unnest(Field<T[]> arrayField) {
return DSL.function("unnest", <??>, arrayField);
}
I don't know how I can fetch the datatype. There seems to be a way to get DataType<T[]> from DataType<T> using DataType::getArrayDataType(), but the reverse is not possible. There is this class I found ArrayDataType, but it seems to be package-private, so I cannot use it (and even if I could, it does not expose the field elementType).

Old PostgreSQL versions had this funky idea that it is OK to produce a table from within the SELECT clause, and expand it into the "outer" table, as if it were declared in the FROM clause. That is a very obscure PostgreSQL legacy, and this example is a good chance to get rid of it, and use LATERAL instead. Your query is equivalent to this one:
SELECT *
FROM message
WHERE id IN (
SELECT "mid"
FROM session_messages
CROSS JOIN LATERAL unnest(message_ids) AS t("mid")
WHERE session_id = '?'
);
This can be translated to jOOQ much more easily as:
DSL.using(configuration)
.select()
.from(MESSAGE)
.where(MESSAGE.ID).in(
select(field(name("mid"), MESSAGE.ID.getDataType()))
.from(SESSION_MESSAGES)
.crossJoin(lateral(unnest(SESSION_MESSAGES.MESSAGE_IDS)).as("t", "mid"))
.where(SESSION_MESSAGES.SESSION_ID.eq("'?'"))
)

The Edit3 in the question is quite close to a decent solution for this problem.
We can create a custom generic unnest method for jOOQ which accepts Field and use it in jOOQ query normally.
Helper method:
public static <T> Field<T> unnest(Field<T[]> field) {
var type = (Class<T>) field.getType().getComponentType();
return DSL.function("unnest", type, field);
}
Usage:
public void query(SessionId sessionId) {
var field = unnest(SESSION_MESSAGES.MESSAGE_IDS, UUID.class);
dsl.select().from(MESSAGE).where(
MESSAGE.ID.in(
dsl.select(field).from(SESSION_MESSAGES)
.where(SESSION_MESSAGES.SESSION_ID.eq(sessionId.id))
.orderBy(field)
)
);
}

Related

Stop jOOQ stripping characters from SQL field

I am implementing a function to get an estimate of the count as described in the PostgreSQL documentation here https://wiki.postgresql.org/wiki/Count_estimate
I'm using the function:
public static Field<Integer> countEstimate(final QueryPart query) {
final String sql = String.format("count_estimate(%s)", escape(query.toString()));
return field(sql(sql), PostgresDataType.INT);
}
Which looks fine until I pass it an IN clause array field in the query. When this happens jOOQ strips the array curly braces from within my SQL. e.g. Calling it with this java code:
final UUID[] ids = new UUID[]{UUID.randomUUID()};
return db.select(countEstimate(db.select(TABLE.ID)
.from(TABLE)
.where(overlaps(ids, TABLE.FILTER_IDS))));
Results in both the variable sql and DSL.sql(sql) in the above function rendering:
count_estimate(E'select "schema"."table"."id"
from "schema"."table"
where (
((\'{"75910f3b-83e6-41ed-bf57-085c225e0131"}\') && ("schema"."table"."filter_ids"))
)')
But field(sql(sql), PostgresDataType.INT) renders this:
count_estimate(E'select "schema"."table"."id"
from "schema"."table"
where (
((\'"75910f3b-83e6-41ed-bf57-085c225e0131"\') && ("schema"."table"."filter_ids"))
)')
Is there any way to work around this and to tell jOOQ to leave my query alone?
(jOOQ 3.8.3, PG 9.5.5, PG driver 9.4-1203-jdbc4)
It turns out it only strips '{}' style arrays. Replacing the code that turns the UUID[] into sql from
DSL.val(ids)
with
DSL.array(Arrays.stream(ids)
.map(UUID::toString)
.collect(Collectors.toList())
.toArray(new String[0]))
.cast(PostgresDataType.UUID.getArrayDataType()
results in it rendering cast(array[\'75910f3b-83e6-41ed-bf57-085c225e0131\'] as uuid[]) prevents it being stripped

Groovy sql.rows returns org.postgresql.util.PSQLException: No hstore extension installed

I am using Groovy Sql in Grails with named parameters to get results from a Postgres DB. My statement is generated dynamically, i.e. concatenated to become the final statement, with the params being added to a map as I go along.
sqlWhere += " AND bar = :namedParam1"
paramsMap.namedParam1 = "blah"
For readability, I am using the groovy string syntax which allows me to write my sql statement over multiple lines, like this:
sql = """
SELECT *
FROM foo
WHERE 1=1
${sqlWhere}
"""
The expression is evaluated as a string containing the linebreaks as \n:
SELECT *\n ...
This is not a problem when I pass params like this
results = sql.rows(sqlString, paramsMap)
but it does become one if paramsMap is empty (which happens since AND bar = :namedParam1 is not always concatenated into the query). I then get an error
org.postgresql.util.PSQLException: No hstore extension installed
which does not really seem to relate to the true nature of the problem. I have for now fixed this with an if...else
if (sqlQuery.params.size() > 0) {
results = sql.rows(sqlString, paramsMap)
} else {
results = sql.rows(sqlString.replace('\n',' '))
}
But this seems a bit weird (especially since it does not work if I use the replace in the if-branch as well).
My question is: why do I really get this error message and is there a better way to prevent it from occuring?
It's certainly a bug in groovy.sql.SQL implementation. The method rows() can't deal with an empty map passed as params. As a workaround, you can test for it and pass an empty list instead.
def paramsMap = [:]
...
if (paramsMap.isEmpty())
paramsMap= []
Issue created at https://issues.apache.org/jira/browse/GROOVY-8082

hooks causing issues with insert using subqueries

I'm not sure why this is causing me an issue, but I'm using Orient 2.1.19, found this in 2.1.12 as well. We are building some hooks to implement a method of encryption. I know 2.2 implements some encryption, but we had some further requirements.
Anyway, we have hooks for onRecordAfterRead, onRecordBeforeCreate and onRecordBeforeUpdate. It works for most statements fine, but with the hook in place, running a query that sets a link property using a subquery in an insert fails. Here's an example query:
create EDGE eThisEdge from (select from vVertex where thisproperty = 'this') to (select from vVertex where thatProperty = 'that' ) set current = (select from lookupCurrent where displayCurrentPast = 'Current');
Runnning this query gives me the error:
com.orientechnologies.orient.core.exception.OValidationException: The field 'eThisEdge.current' has been declared as LINK but the value is not a record or a record-id.
It's some issue with the way a subquery is ran during just an insert though, because if I run the insert without setting any properties, then run an update to set the properties, that works. I'd hate to have to rewrite all of our inserts for our base data and our coding just as a work around for this, and it seems like I'm just missing something here.
Has anyone seen this kind of issue with hooks as well?
The biggest issue seems to be surrounding the onRecordBeforeCreate code. We are trying to have a generic hook that encrypts strings in our database. Here's the basics of the onRecordBeforeCreate method:
public RESULT onRecordBeforeCreate( ODocument oDocument) {
RESULT changed = RESULT.RECORD_NOT_CHANGED;
try {
if(classIsCipherable(oDocument)) {
for (String field : oDocument.fieldNames()) {
if (oDocument.fieldType(field) != null && oDocument.fieldType(field) == OType.STRING && oDocument.field(field) != null) {
oDocument.field(field, crypto.encrypt(oDocument.field(field).toString()));
changed = RESULT.RECORD_CHANGED;
}
}
}
return changed;
} catch (Exception e) {
throw new RuntimeException( e );
}
Is there anything there that looks obvious that I'd have issues with running a create edge statement that sets properties with a property that is a link?
The query select from lookupCurrent where displayCurrentPast = "Current" return more than one element, you must use a LinkList or a LinkSet

Using linq select to provide a grid datasource, properties are read-only

I am using the return from the following call as the datasource for a grid.
public object GetPropertyDataSourceWithCheckBox( )
{
return (
from p in LocalProperties
join c in GetCities( ) on p.CityID equals c.CityID
orderby p.StreetNumber
select new { Selected = false, p.PropertyID, p.StreetNumber, p.StreetName, c.CityName } ).ToList( );
}
I get a checkbox in the grid, but it is READ-ONLY. [For the record, the grid is DevExpress.] Is there a way around this, short of creating a non-anonymous class?
Continuing research tells me that, in fact, anonymous classes returned by linq are always read-only, so, apparently, creating an actual class is best (only?) solution.

Linq to Entities and Xml Fields

I have this scenario:
A SQL Server table myTable with field1, xmlField (nvarchar(50) and xml sql server data type)
Linq to entities
Now I'd like to get a query like this:
SELECT Field1, XmlField
FROM MyTable
WHERE CAST(XmlField AS nvarchar(4000)) = '<myXml />'
Obviously this is a correct query in SQL Server but I can't find a solution to write this in L2E.
Please notify that this code doesn't work:
var query = from row in context.MyTables
where (string)row.XmlField == "<myXml />"
select row
and other cast methods too.
This just because in L2E the "ToString" does't work correctly.
Now my idea is this one: an extension method:
var query = from row in context.MyTables
select row
query = query.CompareXml("XmlField", "<myXml />")
and this is the extended method:
public static IQueryable<TSource> CompareXml<TSource>(this IQueryable<TSource> source, string xmlFieldName, string xmlToCompare)
{
ConstantExpression xmlValue = Expression.Constant(xmlToCompare);
ParameterExpression parameter = Expression.Parameter(typeof(TSource), source.ElementType.Name);
PropertyInfo propertyInfo = typeof(TSource).GetProperty(xmlFieldName);
MemberExpression memberAccess = Expression.MakeMemberAccess(parameter, propertyInfo);
var stringMember = Expression.Convert(memberAccess, typeof(string));
BinaryExpression clauseExpression = Expression.Equal(xmlValue, stringMember);
return source.Where(Expression.Lambda<Func<TSource, bool>>(clauseExpression, parameter));
}
and again this doesn't work too.
Now I'd like to understand how I can force a "Convert" using Cast so I can compare Xml and nvarchar.
Thanks in advance
Massimiliano
Unfortunately EF still doesn't properly support XML columns. I'm afraid pretty much the only choice I know of is to create a view that does the cast and map that to a different entity. This will probably make the code awkward but also offers additional possible scenarios; for example, with a lot of SQL code, you could map elements in the XML columns to actual columns in the view, allowing you to make queries on specific parts of the XML.
On the bright side, at least inserting values in an XML column works pretty much as expected.