I started using 'ANY()' function in query instead of 'IN' due to some parameter bound error. Currently it's something like that.
Select *
FROM geo_closure_leaf
WHERE geoId = ANY(:geoIds)
But it has a huge impact on performance. Using the query with IN is very much faster than with ANY.
Any suggestion how can we bound array of string parameters can be passed in 'IN' expression.
I have tried temporary fix using
Select *
FROM geo_closure_leaf
WHERE geoId IN (''('' || array_to_string(:geoIds::text[] ,''),('') || '')'')
Select *
FROM geo_closure_leaf
WHERE geoId IN (select unnest(:geoIds::text[]))
geoIds = array of strings
It's working this way.
**public override T Query<T>(string query, IDictionary<string, object> parameters, Func<IDataReader, T> mapper)**
{
T Do(NpgsqlCommand command)
{
IDataReader reader = null;
try
{
** command.CommandText = query;
reader = command.AddParameters(parameters).ExecuteReader();**
return mapper(reader);
}
finally
{
CloseDataReader(reader);
}
}
return Execute(Do);
}
Object is array of string.
Expected is: I should be able to do this without having to put extra logic in sql.
Select *
FROM geo_closure_leaf
WHERE geoId IN (:geoIds)
The performance difference cannot be IN versus = ANY, because PostgreSQL will translate IN into = ANY during query optimization.
The difference must be the subselect. If you are using unnest, PostgreSQL will always estimate that the subquery returns 100 rows, because that is how unnest is defined.
It must be that the estimate of 100 somehow produces a different execution plan that happens to work better.
We'd need the complete execution plans to say anything less uncertain.
https://dba.stackexchange.com/questions/125413/index-not-used-with-any-but-used-with-in
Found this post explaining how indeexs are getting used in different constructors of 'ANY' & 'IN'.
Related
I'm trying to parameterize my postgresql query in order to prevent SQL injection in my ruby on rails application. The SQL query will sum a different value in my table depending on the input.
Here is a simplified version of my function:
def self.calculate_value(value)
calculated_value = ""
if value == "quantity"
calculated_value = "COALESCE(sum(amount), 0)"
elsif value == "retail"
calculated_value = "COALESCE(sum(amount * price), 0)"
elsif value == "wholesale"
calculated_value = "COALESCE(sum(amount * cost), 0)"
end
query = <<-SQL
select CAST(? AS DOUBLE PRECISION) as ? from table1
SQL
return Table1.find_by_sql([query, calculated_value, value])
end
If I call calculate_value("retail"), it will execute the query like this:
select location, CAST('COALESCE(sum(amount * price), 0)' AS DOUBLE PRECISION) as 'retail' from table1 group by location
This results in an error. I want it to execute without the quotes like this:
select location, CAST(COALESCE(sum(amount * price), 0) AS DOUBLE PRECISION) as retail from table1 group by location
I understand that the addition of quotations is what prevents the sql injection but how would I prevent it in this case? What is the best way to handle this scenario?
NOTE: This is a simplified version of the queries I'll be writing and I'll want to use find_by_sql.
Prepared statement can not change query structure: table or column names, order by clause, function names and so on. Only literals can be changed this way.
Where is SQL injection? You are not going to put a user-defined value in the query text. Instead, you check the given value against the allowed list and use only your own written parts of SQL. In this case, there is no danger of SQL injection.
I also want to link to this article. It is safe to create a query text dynamically if you control all parts of that query. And it's much better for RDBMS than some smart logic in query.
I would like to get the result of a query using rowMode="array" (as this is a potentially very large table and I don't want it formatted to object format) but I couldn't figure out how to pass in a array/list parameter for use in an IN operator.
const events = await t.manyOrNone({text: `select * from svc.events where user_id in ($1:list);`, rowMode: "array"}, [[1,2]]);
However, the above gives an error: syntax error at or near ":"
Removing the :list did not work either:
const events = await t.manyOrNone({text: `select * from svc.events where user_id in ($1);`, rowMode: "array"}, [[1,2]]);
Error: invalid input syntax for integer: "{"1","2"}"
I understand that this might be because I'm forced to use ParameterizedQuery format for rowMode="array" which does not allow those snazzy modifiers like :list, but this then leads to the question, if I were to use ParameterizedQuery format, then how do I natively pass in a Javascript array so that it is acceptable to the driver?
I guess an alternative formulation to this question is: how do I use arrays as parameters for ParameterizedQuery or PreparedStatements...
Answering my own question as I eventually found an answer to this issue: how to pass in arrays as params for use in the IN operator when using rowMode="array" | ParameterizedQuery | PreparedStatements.
Because this query is being parameterized in the server, we cannot use the IN operator, because the IN operator parameterize items using IN ($1, $2, $3...). Instead we need to use the ANY operator, where ANY($1) where for $1 an array is expected.
So the query that will work is:
const events = await t.manyOrNone({text: `select * from svc.events where user_id=ANY($1);`, rowMode: "array"}, [[1,2]]);
In a SQL Server database I have a column of decimal datatype defined something like this:
CREATE TABLE MyTable
(
Id INT,
Number DECIMAL(9, 4)
)
I use Entity Framework and I would like to return column Number converted to a string with only the digits right of the decimal separator that are actually needed. A strict constraint is that a result must be an IQueryable.
So my query is:
IQueryable queryable = (
from myTable in MyDatabase.NyTable
select new
{
Id = myTable.Id,
Number = SqlFunctions.StringConvert(myTable.Number,9,4)
}
);
The problem with is that it always convert number to string with 4 decimals, even if they are 0.
Examples:
3 is converted to "3.0000"
1.2 is converted to "1.2000"
If I use other parameters for StringConvert i.e.
SqlFunctions.StringConvert(myTable.Number, 9, 2)
the results are also not OK:
0.375 gets rounded to 0.38.
StringConvert() function is translated into SQL Server function STR.
https://learn.microsoft.com/en-us/sql/t-sql/functions/str-transact-sql?view=sql-server-2017
This explains the weird results.
In the realm of Entity Framework and LINQ I was not able to find a working solution.
What I look for is something like C# function
String.Format("0.####", number)
but this cannot be used in a LINQ query.
In plain simple SQL I could write my query like this
SELECT
Id,
Number = CAST(CAST(Number AS REAL) AS VARCHAR(15))
FROM
MyTable
I have not managed to massage LINQ to produce query like that.
A workaround would be to forget doing this in LINQ, which is quite inflexible and messy thing, borderline on useless and just return type DECIMAL from database and do my formatting on a client side before displaying. But this is additional, unnecessary code and I would hate to di it that way if there perhaps is a simpler way via LINQ.
Is it possible to format numbers in LINQ queries?
I would absolutely return a decimal from he database and format it when needed. Possible directly after the query. But usually this is done at display time to take into account culture specific formatting from the the client.
var q =
(from myTable in MyDatabase.NyTable
select new
{
Id = myTable.Id,
Number = myTable.Number
})
.AsEnumerable()
.Select(x => new { Id = x.Id, Number = x.Number.ToString("G29") });
I am trying to run a search on one particular field of a table with a list of values. Not able to find a solution so far. Any help is really appreciated.
Here is the scenario
var records = new PagedList<Required>();
var result = db.Required.Where(x => filter == null || (x.Title.Contains(filter)) || x.CID.Contains(filter));
foreach (string str in SelectedNetwork)
{
string tempStr = str;
result = result.Where(x => x.Network == tempStr);
records.TotalRecords = result.Count();
}
records.Content = result
.Where(x => filter == null ||
(x.Title.Contains(filter))
|| x.CID.Contains(filter)
)
.OrderBy(sort + " " + sortdir)
.Skip((page - 1) * Convert.ToInt32(records.PageSize))
.Take(Convert.ToInt32(records.PageSize))
.ToList();
highlighted code in the foreach loop fails to run as per expectation. Is there any way, I can fix it?
Thanks
Tutumon
You must take into account that LINQ expressions are queries, until you materialize them. To materialize them you need to either enumerate them, or convert them to a list, array, or whatever, i.e. enumerate their members in a foreach, or call a method like ToList(), or ToArray().
In your code the original query stored in result is not materialized, so everytime a foreach loop is executed, a new Where contidion is added to the original query. To vaoid this behavior you need to recreate the whole results query in each iteration, so that you get a fresh copy of the unfilterd expression.
There would be another solution which would be to materialize the result query and then run the foreach loop as is. The problem of this solution would be that you would get all the data from the database, keep it in memory and run the Where and the Count on the in-memory copy. Unless there is a very small number of rows in Required that would be a very bad idea.
I don't know how to write when it returns max ID. Please Give me answer.
I have written this in sessionbean but it returns -1.
#Override
public int searchMaxAdvID() {
int adid=em.createNativeQuery("select max(AdvID) from Advertisement").getMaxResults();
return adid;
}
Problem is that you misinterpreted getMaxResults-method. It does not have any connection to the MAX function. As said in documentation, it should behave following way:
The maximum number of results the query object was set to retrieve.
Returns Integer.MAX_VALUE if setMaxResults was not applied to the
query object.
In your case it seems to return -1 instead of Integer.MAX_VALUE. This is clearly incorrect. For example Hibernate 3.6.8.Final returns Integer.MAX_VALUE (2147483647) as specified. Method setMaxResults limit maximum number of rows returned by query.
In your case getSingleResult is right method to use. If there is not any rows in table, result will be null.
Integer x = (Integer) em.createNativeQuery(
"select max(AdvID) from Advertisement").getSingleResult();
If there is not specific reason to use native query, use JPQL query as suggested in Nayan Wadekar's answer.
You can try the below regular JPQL query rather than using native query.
Integer maxAdvId = (Integer) em.createQuery("SELECT MAX(a.advID) from Advertisement a").getSingleResult();