dynamically choose columns in select with R2DB2 - spring-data-r2dbc

I have been trying to pass the columns I want to select in but out of the box it appears it is not possible. I have tried things like
#Query("SELECT :columns FROM USERS u WHERE u.LOCALE = :locale AND u.id IN (:ids)")
Flux<Users> retrieveExportData(#Param("columns") String columns,
#Param("locale") String locale,
#Param("ids") String[] ids);
and with
private final R2dbcEntityTemplate template;
I tried to create my own query and but that was not working because it has to be of type Criteria and that was just creating a complexity that just was not worth it.
It would be nice if I could add the columns like
criteriaList.add(
Criteria.where("LOCALE").is(locale)
);
Criteria criteria = Criteria.from(criteriaList);
and execute it like
Flux<Users> users = this.template.select(User.class)
.matching(Query.query(criteria))
.all();
or just calling the repository like in my first example.
Has anyone been able to do this successfully?
----- update 1 -----
I tried doing like so:
import org.springframework.r2dbc.core.DatabaseClient;
DatabaseClient databaseClient = DatabaseClient.create(connectionFactory);
String sql = "SELECT " + columns + " FROM USERS u WHERE u.LOCALE=" + locale + " AND u.id IN (" + ids + ")";
return databaseClient.sql(sql)
.fetch()
.all().cast(User.class);
but Since Spring 4.3.6.RELEASE, LinkedCaseInsensitiveMap doesn't extend LinkedHashMap and HashMap, but only implements Map interface.
This results in a
Cannot cast org.springframework.util.LinkedCaseInsensitiveMap to User at java.base/java.lang.Class.cast
error.
I then tried the jooq approach suggested in the answers but it just produces syntax errors. Example
private final DSLContext ctx = DSL.using(connectionFactory);
private final Users users = ctx.newRecord(Users.USERS); <-- USERS not found
#Query("SELECT :columns FROM USERS u WHERE u.LOCALE = :locale AND u.id IN (:ids)")
public Flux<Users> retrieveExportData(
List<Field<?>> columns,
String locale,
String[] ids
) {
return Flux.from(ctx
.select(columns)
.from("USERS")
.where(users.LOCALE.eq(locale)) <--- LOCALE not found
.and(users.ID.in(ids)) <--- ID not found
).map(r -> r.into(Users.class)); <---- into not found
}
the library look promising. I will try to get it working.

You cannot replace a bind parameter (:columns) by syntactic elements like this, other than actual bind values. For this type of dynamic SQL, you'll have to resort to some sort of query building mechanism.
Perhaps look at jOOQ, which has R2DBC support? Your implementation would then look like this:
#Query("SELECT :columns FROM USERS u WHERE u.LOCALE = :locale AND u.id IN (:ids)")
public Flux<Users> retrieveExportData(
List<Field<?>> columns,
String locale,
String[] ids
) {
return Flux.from(ctx
.select(columns)
.from(USERS)
.where(USERS.LOCALE.eq(locale))
.and(USERS.ID.in(ids))
).map(r -> r.into(Users.class));
}
Disclaimer: I work for the company behind jOOQ.

you can write dynamic SQL like so.
import org.springframework.r2dbc.core.DatabaseClient;
DatabaseClient databaseClient = DatabaseClient.create(connectionFactory);
String sql = "SELECT " + columns + " FROM USERS u WHERE u.LOCALE=" + locale + " AND u.id IN (" + ids + ")";
return databaseClient.sql(sql)
.fetch()
.all().cast(User.class);
and you will need this dependency
implementation "org.springframework.boot:spring-boot-starter-data-r2dbc:2.6.2"

Related

#Query annotation not working in JPA Repository

#Query(value = "SELECT * FROM MEMBER m " +
"LEFT JOIN COMPANY_TYPE c ON c.TYPE_CODE = m.TYPE_CODE " +
"LEFT JOIN COMPANY_LOCATION l on l.LOCATION_CODE = m.LOCATION_CODE " +
"WHERE l.LOCATION_CODE = :location AND c.TYPE_CODE = :type", nativeQuery = true)
List<Member> findByOption(#Param("type")Long companyType, #Param("location")Long companyLocation);
This is the method I'm trying to use in MemberRepository interface that extends JPARepository<Member, Long>.
#ManyToOne
#JoinColumn(name = "TYPE_CODE")
private CompanyType company_Type;
#ManyToOne
#JoinColumn(name = "LOCATION_CODE")
private CompanyLocation company_Location;
And this is part of entity class Member.
I am trying to get filtered list of members according to the company type and location code(id). However, the slice test results always return all of members.
I want to know if there are other settings that needs to be set in order for the #Query annotation to work, and if the native query I put in the annotation is wrong.
Any other ideas of how to get filtered results in Spring Data JPA is also welcomed.
This is a very subtle mistake, but rather than using SELECT * you should be using SELECT m.*:
Query(value = "SELECT m.* FROM MEMBER m " +
"LEFT JOIN COMPANY_TYPE c ON c.TYPE_CODE = m.TYPE_CODE " +
"LEFT JOIN COMPANY_LOCATION l on l.LOCATION_CODE = m.LOCATION_CODE " +
"WHERE l.LOCATION_CODE = :location AND c.TYPE_CODE = :type", nativeQuery = true)
List<Member> findByOption(#Param("type")Long companyType, #Param("location")Long companyLocation);
The reason for this is that SELECT m.* returns only the columns from the Member entity/table. SELECT * instructs JPA to return columns from all tables in the query. You would need a method return signature of List<Object[]> in that case.

Entity Framework Core 2.0 FromSql and SQL Injection

I need to know the correct way to handle SQL Injection when using the FromSQL command.
At runtime, I need to dynamically create a where statement. So I am using FromSql to create the SQL command. Now I know that using use string interpolation is the way to go. However, I need to step through a list of "Where Parameters" to generate the command. Simple enough to do;
foreach (var x in wp)
{
if (!string.IsNullOrEmpty(results))
results = $"{results} and {x.Field} = {x.Value}";
if (string.IsNullOrEmpty(results))
results = $"where {x.Field} = {x.Value}";
}
Problem is that this return a simple string and would not be string interpolation. How can I do this correctly?
Entityframework will parameterize your queries if you put it in the following format:
db.something.FromSql("SELECT * FROM yourTable WHERE AuthorId = {0}", id)
Is x.Field a form field that has a fixed number of possibilities? i.e. title, firstname etc. If so then something like the following:
var sqlstring = new StringBuilder();
var sqlp = new List<SqlParameter>();
var i = 0;
foreach (var x in wp)
{
var param = "#param" + i.ToString();
if (i!=0)
{
sqlstring.Append($" AND {x.Field} = " + param);
sqlp.Add(new SqlParameter(param, x.Value));
}
if (i==0)
{
sqlstring.Append($"WHERE {x.Field} = " + " #param" + i.ToString());
sqlp.Add(new SqlParameter(param, x.Value));
}
i++;
}
You'd then need to do something like this:
db.something.FromSql(sqlstring.ToString(), sqlp.ToArray())
Might be a better/cleaner way but that should work.
My solution to this problem is a VS extension, QueryFirst. QueryFirst generates a C# wrapper for sql that lives in a .sql file. As such, parameters are the only way to get data into your query and SQL injection is near impossible. There are numerous other advantages: you edit your sql in a real environment, it's constantly validated against your db, and using your query in your code is very simple.

JPQL & EclipseLink Concat

When I am concatenating fields in em.CreateQuery concat returns NULL, cause one of my fields can have NULL. I want check for NULL as in Native MSSQL ISNULL(field,'somthing') function.
MyQuery is.
entManager.createQuery("SELECT NEW " + ThumbNail.class.getName() + "(p.id,p.thumbnail,p.thumbNailModifiedDate, CONCAT(p.firstName,' ',p.lastName,' ',p.middleName)) FROM Person p").setHint("eclipselink.refresh", "true").getResultList();
Tried this one too
entManager.createQuery("SELECT NEW " + ThumbNail.class.getName() + "(p.id,p.thumbnail,p.thumbNailModifiedDate, CONCAT(FUNC('ISNULL',p.firstName,''),' ',FUNC('ISNULL',p.lastName,''),' ',FUNC('ISNULL',p.middleName,'')) FROM Person p").setHint("eclipselink.refresh", "true").getResultList();
And it didn't work
JPQL supports the coalesce function, which returns the first of its non-null arguments. So you could use
CONCAT(coalesce(p.firstName, ''),
' ',
coalesce(p.lastName, ''),
' ',
coalesce(p.middleName, ''))
I would put the middle name in the middle, rather than putting it at the end, though.
SOLVED!
But not with JPQL functions.
I just added new constructor to my pojo object and filled it with JPQL then concated there with checkig for null. Thanx guys for answers.
modifying the jpa query is a mess...
just add a notNull method:
private String notNull(String input){ if (input != null) { return input.trim(); } return ""; }
and put this on every (String) setter on your Entity-Class
setFirstName(String name) { this.name = notNull(name) }
so all your "nulls" get "" and the jpa works
BUT: then for db-nulls you get String="", so be aware..

can i use custom lambda method in entity framework?

i have some methods like:
public static string ToOtherFormat (this string inp)
{
// some code to change inp
return inp;
}
and in my select i want to have code like this:
var DetailMembers = db.TB_Members
.Where(x=> x.FName == obj.ToOtherFormat())
.Select( x=> new { name = (x.FName.ToOtherFormat() + " " + x.LName) , x.ActCode });
i try and just have error. is it possible?
thanks!
i receive this error in simple convert to integer
LINQ to Entities does not recognize the method 'Int32 ToInt32(System.String)' method, and this method cannot be translated into a store expression.
with this code
.Where(x => x.MemberID == Convert.ToInt32(Hmemid.Hash_two_Decrypt())
Looks like you are querying against the database. Your current query will get translated into SQL query and since SQL doesn't recognize your function that is why you get error.
You may get the data from the tables using a query without that function and then later do the formatting on the result set.
i found it on use .AsEnumerable() method like:
var DetailMembers = db.TB_Members.AsEnumerable()
.Where(x=> x.FName == obj.ToOtherFormat())
.Select( x=> new { name = (x.FName.ToOtherFormat() + " " + x.LName) , x.ActCode });

using the TSqlParser

I'm attempting to parse SQL using the TSql100Parser provided by microsoft. Right now I'm having a little trouble using it the way it seems to be intended to be used. Also, the lack of documentation doesn't help. (example: http://msdn.microsoft.com/en-us/library/microsoft.data.schema.scriptdom.sql.tsql100parser.aspx )
When I run a simple SELECT statement through the parser it returns a collection of TSqlStatements which contains a SELECT statement.
Trouble is, the TSqlSelect statement doesn't contain attributes such as a WHERE clause, even though the clause is implemented as a class. http://msdn.microsoft.com/en-us/library/microsoft.data.schema.scriptdom.sql.whereclause.aspx
The parser does recognise the WHERE clause as such, looking at the token stream.
So, my question is, am I using the parser correctly? Right now the token stream seems to be the most useful feature of the parser...
My Test project:
public static void Main(string[] args)
{
var parser = new TSql100Parser(false);
IList<ParseError> Errors;
IScriptFragment result = parser.Parse(
new StringReader("Select col from T1 where 1 = 1 group by 1;" +
"select col2 from T2;" +
"select col1 from tbl1 where id in (select id from tbl);"),
out Errors);
var Script = result as TSqlScript;
foreach (var ts in Script.Batches)
{
Console.WriteLine("new batch");
foreach (var st in ts.Statements)
{
IterateStatement(st);
}
}
}
static void IterateStatement(TSqlStatement statement)
{
Console.WriteLine("New Statement");
if (statement is SelectStatement)
{
PrintStatement(sstmnt);
}
}
Yes, you are using the parser correctly.
As Damien_The_Unbeliever points out, within the SelectStatement there is a QueryExpression property which will be a QuerySpecification object for your third select statement (with the WHERE clause).
This represents the 'real' SELECT bit of the query (whereas the outer SelectStatement object you are looking at has just got the 'WITH' clause (for CTEs), 'FOR' clause (for XML), 'ORDER BY' and other bits)
The QuerySpecification object is the object with the FromClauses, WhereClause, GroupByClause etc.
So you can get to your WHERE Clause by using:
((QuerySpecification)((SelectStatement)statement).QueryExpression).WhereClause
which has a SearchCondition property etc. etc.
Quick glance around would indicate that it contains a QueryExpression, which could be a QuerySpecification, which does have the Where clause attached to it.
if someone lands here and wants to know how to get the whole elements of a select statement the following code explain that:
QuerySpecification spec = (QuerySpecification)(((SelectStatement)st).QueryExpression);
StringBuilder sb = new StringBuilder();
sb.AppendLine("Select Elements");
foreach (var elm in spec.SelectElements)
sb.Append(((Identifier)((Column)((SelectColumn)elm).Expression).Identifiers[0]).Value);
sb.AppendLine();
sb.AppendLine("From Elements");
foreach (var elm in spec.FromClauses)
sb.Append(((SchemaObjectTableSource)elm).SchemaObject.BaseIdentifier.Value);
sb.AppendLine();
sb.AppendLine("Where Elements");
BinaryExpression binaryexp = (BinaryExpression)spec.WhereClause.SearchCondition;
sb.Append("operator is " + binaryexp.BinaryExpressionType);
if (binaryexp.FirstExpression is Column)
sb.Append(" First exp is " + ((Identifier)((Column)binaryexp.FirstExpression).Identifiers[0]).Value);
if (binaryexp.SecondExpression is Literal)
sb.Append(" Second exp is " + ((Literal)binaryexp.SecondExpression).Value);
I had to split a SELECT statement into pieces. My goal was to COUNT how many record a query will return. My first solution was to build a sub query such as
SELECT COUNT(*) FROM (select id, name from T where cat='A' order by id) as QUERY
The problem was that in this case the order clause raises the error "The ORDER BY clause is not valid in views, inline functions, derived tables, sub-queries, and common table expressions, unless TOP or FOR XML is also specified"
So I built a parser that split a SELECT statment into fragments using the TSql100Parser class.
using Microsoft.Data.Schema.ScriptDom.Sql;
using Microsoft.Data.Schema.ScriptDom;
using System.IO;
...
public class SelectParser
{
public string Parse(string sqlSelect, out string fields, out string from, out string groupby, out string where, out string having, out string orderby)
{
TSql100Parser parser = new TSql100Parser(false);
TextReader rd = new StringReader(sqlSelect);
IList<ParseError> errors;
var fragments = parser.Parse(rd, out errors);
fields = string.Empty;
from = string.Empty;
groupby = string.Empty;
where = string.Empty;
orderby = string.Empty;
having = string.Empty;
if (errors.Count > 0)
{
var retMessage = string.Empty;
foreach (var error in errors)
{
retMessage += error.Identifier + " - " + error.Message + " - position: " + error.Offset + "; ";
}
return retMessage;
}
try
{
// Extract the query assuming it is a SelectStatement
var query = ((fragments as TSqlScript).Batches[0].Statements[0] as SelectStatement).QueryExpression;
// Constructs the From clause with the optional joins
from = (query as QuerySpecification).FromClauses[0].GetString();
// Extract the where clause
where = (query as QuerySpecification).WhereClause.GetString();
// Get the field list
var fieldList = new List<string>();
foreach (var f in (query as QuerySpecification).SelectElements)
fieldList.Add((f as SelectColumn).GetString());
fields = string.Join(", ", fieldList.ToArray());
// Get The group by clause
groupby = (query as QuerySpecification).GroupByClause.GetString();
// Get the having clause of the query
having = (query as QuerySpecification).HavingClause.GetString();
// Get the order by clause
orderby = ((fragments as TSqlScript).Batches[0].Statements[0] as SelectStatement).OrderByClause.GetString();
}
catch (Exception ex)
{
return ex.ToString();
}
return string.Empty;
}
}
public static class Extension
{
/// <summary>
/// Get a string representing the SQL source fragment
/// </summary>
/// <param name="statement">The SQL Statement to get the string from, can be any derived class</param>
/// <returns>The SQL that represents the object</returns>
public static string GetString(this TSqlFragment statement)
{
string s = string.Empty;
if (statement == null) return string.Empty;
for (int i = statement.FirstTokenIndex; i <= statement.LastTokenIndex; i++)
{
s += statement.ScriptTokenStream[i].Text;
}
return s;
}
}
And to use this class simply:
string fields, from, groupby, where, having, orderby;
SelectParser selectParser = new SelectParser();
var retMessage = selectParser.Parse("SELECT * FROM T where cat='A' Order by Id desc",
out fields, out from, out groupby, out where, out having, out orderby);