Column must appear in the GROUP BY clause or be used in an aggregate function - postgresql

I'm updating a Qt software, to make it compatible with both SQLite and PostgreSQL.
I have a C++ method that is used to count elements of a given table with given clauses.
In SQLite, the following worked and gave me a number N (the count).
SELECT COUNT(*) FROM table_a
INNER JOIN table_b AS
ON table_b.fk_table_a = table_a.id
WHERE table_a.start_date_time <> 0
ORDER BY table_a.creation_date_time DESC
With PostgreSQL (I'm using 9.3), I have the following error :
ERROR: column "table_a.creation_date_time" must appear in the
GROUP BY clause or be used in an aggregate function
LINE 5: ORDER BY
table_a.creation_date_time DESC
If I add, GROUP BY table_a.creation_date_time, it gives me a table with N rows.
I've read a lot of stuff about how different DBMS allow you to omit columns in the GROUP BY clause. Now, I'm just confused.
For those who are curious, the C++ method is:
static int count(const QString &table, const QString &clauses = QString(""))
{
int success = -1;
if (!table.isEmpty())
{
QString statement = QString("SELECT COUNT(*) FROM ");
statement.append(table);
if (!clauses.isEmpty())
{
statement.append(" ").append(clauses) ;
}
QSqlQuery query;
if(!query.exec(statement))
{
qWarning() << query.lastError();
qWarning() << statement;
}
else
{
if (query.isActive() && query.isSelect() && query.first())
{
bool ok = false;
success = query.value(0).toInt(&ok);
if (ok == false)
{
success = -1;
return success;
}
}
}
}
return success;
}

If you're just doing a count(*) on the table in order to get a single scalar-value result, then surely having the order by present is obsolete ?
solution
Remove the obsolete order by to get "standard" query behavior across multiple dbms

Related

How to get Function arguments in Postgresql

I am changing pg-pool to cater cypher queries for Apache AGE and I have the following code, which takes the select statement. From that it gets the FROM clause, then from the FROM clause it gets the function names. So now I want that function arguments also. How would I do that. The code is as follows:
if (IsA(node, SelectStmt))
{
SelectStmt *stmt = (SelectStmt *) node;
List* fromClause = stmt->fromClause;
ListCell *fl;
//Match the first cypher function call in the FROM clause. Could be multiple tables
// e.g. FROM table1, table2, cypher(),table3....
foreach(fl, fromClause)
{
Node *n = lfirst(fl);
if (IsA(n, RangeFunction))
{
RangeFunction *rf = (RangeFunction*) n;
List* functions = rf->functions;
if (functions->length == 1)
{
List *sublist = (List *) lfirst(list_head(functions));
FuncCall *fntree = (FuncCall *) lfirst(list_head(sublist));
StringInfoData str;
initStringInfo(&str);
_outNode(&str,fntree->funcname);
if (!strcmp("\"cypher\"",str.data)){
return true;
}
}
}
}
}
The arguments are stored in FuncCall. Since you already have that, you can get them with
fntree->args
Note that your code only covers function calls in the top FROM list. What if there are subqueries?

Force Entity Framework to read each line with all details

I am having trouble with en EF method returning duplicate rows of data. When I am running this, in my example, it returns four rows from a database view. The fourth row includes details from the third row.
The same query in SSMS returns four individual rows with the correct details. I have read somewhere about EK and problems with optimization when there are no identity column. But - is there anyway to alter the below code to force EK to read all records with all details?
public List<vs_transactions> GetTransactionList(int cID)
{
using (StagingDataEntities db = new StagingDataEntities())
{
var res = from trans in db.vs_transactions
where trans.CreditID == cID
orderby trans.ActionDate descending
select trans;
return res.ToList();
}
}
Found the solution :) MergeOption.NoTracking
public List<vs_transactions> GetTransactionList(int cID)
{
db.vs_transactions.MergeOption = MergeOption.NoTracking;
using (StagingDataEntities db = new StagingDataEntities())
{
var res = from trans in db.vs_transactions
where trans.CreditID == cID
orderby trans.ActionDate descending
select trans;
return res.ToList();
}
}

What is the relevant rules of Flink Window TVF and CEP SQL?

I am trying to parse Flink windowing TVF sql column level lineage, I initial a custom FlinkChainedProgram and set some Opt rules.
Mostly works fine except Window TVF SQL and CEP SQL.
for example, I get a logical plan as
insert into sink_table(f1, f2, f3, f4)
SELECT cast(window_start as String),
cast(window_start as String),
user_id,
cast(SUM(price) as Bigint)
FROM TABLE(TUMBLE(TABLE source_table, DESCRIPTOR(event_time), INTERVAL '10' MINUTES))
GROUP BY window_start, window_end, GROUPING SETS ((user_id), ());
rel#1032:FlinkLogicalCalc.LOGICAL.any.None: 0.[NONE].[NONE](input=FlinkLogicalAggregate#1030,select=CAST(window_start) AS EXPR$0, CAST(window_start) AS EXPR$1, null:BIGINT AS EXPR$2, user_id, null:VARCHAR(2147483647) CHARACTER SET "UTF-16LE" AS EXPR$4, CAST($f4) AS EXPR$5)
As we seen, the Optimized RelNode Tree contains null column so that MetadataQuery can't get origin column info.
What rules should I set in Logical Optimized phase to parse Window TVF SQL and CEP SQL? Thanks
I solved the field blood relationship method of Flink CEP SQL and added the getColumnOrigins(Match rel, RelMetadataQuery mq, int iOutputColumn) method in org.apache.calcite.rel.metadata.org.apache.calcite.rel.metadata. RelMdColumnOrigins:
/**
* Support field blood relationship of CEP.
* The first column is the field after PARTITION BY, and the other columns come from the measures in Match
*/
public Set<RelColumnOrigin> getColumnOrigins(Match rel, RelMetadataQuery mq, int iOutputColumn) {
if (iOutputColumn == 0) {
return mq.getColumnOrigins(rel.getInput(), iOutputColumn);
}
final RelNode input = rel.getInput();
RexNode rexNode = rel.getMeasures().values().asList().get(iOutputColumn - 1);
RexPatternFieldRef rexPatternFieldRef = searchRexPatternFieldRef(rexNode);
if (rexPatternFieldRef != null) {
return mq.getColumnOrigins(input, rexPatternFieldRef.getIndex());
}
return null;
}
private RexPatternFieldRef searchRexPatternFieldRef(RexNode rexNode) {
if (rexNode instanceof RexCall) {
RexNode operand = ((RexCall) rexNode).getOperands().get(0);
if (operand instanceof RexPatternFieldRef) {
return (RexPatternFieldRef) operand;
} else {
// recursive search
return searchRexPatternFieldRef(operand);
}
}
return null;
}
Source address: https://github.com/HamaWhiteGG/flink-sql-lineage/blob/main/src/main/java/org/apache/calcite/rel/metadata/RelMdColumnOrigins.java
I have given detailed test cases, you can refer to: https://github.com/HamaWhiteGG/flink-sql-lineage/blob/main/src/test/java/com/dtwave/flink/lineage/cep/CepTest.java
Flink CEP SQL test case:

How to assure the return StringList will be ordered : Scala

I am using Scala 2.11.8
I am trying to read queries from my Property File. Each Query Set has multiple parts (explained below)
And i have certain sequence in which these queries must execute.
Code:
import com.typesafe.config.ConfigFactory
object ReadProperty {
def main(args : Array[String]): Unit = {
val queryRead = ConfigFactory.load("testqueries.properties").getConfig("select").getStringList("caseInc").toArray()
val localRead = ConfigFactory.load("testqueries.properties").getConfig("select").getStringList("caseLocal").toArray.toSet
queryRead.foreach(println)
localRead.foreach(println)
}
}
PropertyFile Content :
select.caseInc.2 = Select emp_salary, emp_dept_id from employees
select.caseLocal.1 = select one
select.caseLocal.3 = select three
select.caseRemote.2 = Select e1.emp_name, d1.dept_name, e1.salary from emp_1 e1 join dept_1 d1 on(e1.emp_dept_id = d1.dept_id)
select.caseRemote.1 = Select * from departments
select.caseInc.1 = Select emp_id, emp_name from employees
select.caseLocal.2 = select two
select.caseLocal.4 = select four
Output:
Select emp_id, emp_name from employees
Select emp_salary, emp_dept_id from employees
select one
select two
select three
select four
As we can see in output, The result is Sorted . In the property if you see i have tried numbering the queries in the sequence it should run.(passing the caseInc, caseLocal as arguments).
With getStringList() i am always getting the Sorted List on the basis of the sequence number i am providing.
Even when i tried using toArray() & toArray().toSet i am getting sorted output.
So far its Good
But how to be sure that it will always return in Sorted Order which i have provided in the property file. I am confused because somehow i am not able to find the API which says that the returned List will be Sorted.
I think you can rely on this fact. Looking into the code of DefaultTransformer you can see following piece of logic:
} else if (requested == ConfigValueType.LIST && value.valueType() == ConfigValueType.OBJECT) {
// attempt to convert an array-like (numeric indices) object to a
// list. This would be used with .properties syntax for example:
// -Dfoo.0=bar -Dfoo.1=baz
// To ensure we still throw type errors for objects treated
// as lists in most cases, we'll refuse to convert if the object
// does not contain any numeric keys. This means we don't allow
// empty objects here though :-/
AbstractConfigObject o = (AbstractConfigObject) value;
Map<Integer, AbstractConfigValue> values = new HashMap<Integer, AbstractConfigValue>();
for (String key : o.keySet()) {
int i;
try {
i = Integer.parseInt(key, 10);
if (i < 0)
continue;
values.put(i, o.get(key));
} catch (NumberFormatException e) {
continue;
}
}
if (!values.isEmpty()) {
ArrayList<Map.Entry<Integer, AbstractConfigValue>> entryList = new ArrayList<Map.Entry<Integer, AbstractConfigValue>>(
values.entrySet());
// sort by numeric index
Collections.sort(entryList,
new Comparator<Map.Entry<Integer, AbstractConfigValue>>() {
#Override
public int compare(Map.Entry<Integer, AbstractConfigValue> a,
Map.Entry<Integer, AbstractConfigValue> b) {
return Integer.compare(a.getKey(), b.getKey());
}
});
// drop the indices (we allow gaps in the indices, for better or
// worse)
ArrayList<AbstractConfigValue> list = new ArrayList<AbstractConfigValue>();
for (Map.Entry<Integer, AbstractConfigValue> entry : entryList) {
list.add(entry.getValue());
}
return new SimpleConfigList(value.origin(), list);
}
}
Note how keys are parsed as integer values and then sorted using Integer.compare

Extending Zend_Db

I apologize if my title is a bit misleading and it turns out that it's some other class under Zend_Db.
I use the following method of extracting data from a MSSQL:
// $_config contains information about how to connect to my MSSQL server
$config = new Zend_Config($_config);
$db = Zend_Db::factory($config->database);
$sql = "SELECT * FROM table1";
print_r($db->fetchAll($sql));
So far no problem and everything runs smooth :).
Now I need to run some more complex queries with multiple rowsets:
$sql2 = <<<EOD
DECLARE #test INT
SET #test = 42
SELECT * FROM table1 WHERE col1 = #test
SELECT col2 FROM table2 UNION
SELECT col3 FROM table2
SELECT * FROM table3
EOD;
print_r($db->fetchAll($sql2));
I hope you get the idea.
Using $db->fetchAll($sql2); will result in
Fatal error: Uncaught exception
'Zend_Db_Statement_Exception' with
message 'SQLSTATE[HY000]: General
error: 10038 Attempt to initiate a new
SQL Server operation with results
pending. [10038] (severity 7)
[(null)]' in
\Sacp026a\sebamweb$\prod\includes\Zend\Db\Statement\Pdo.php:234
The following function will return all the correct rowsets:
function sqlquery_multiple($zdb, $sql) {
$stmt = $zdb->query($sql);
$rowsets = array();
do {
if ($stmt->columnCount() > 0) {
$rowsets[] = $stmt->fetchAll(PDO::FETCH_ASSOC);
}
} while ($stmt->nextRowset());
return $rowsets;
}
print_r(sqlquery_multiple($db, $sql2));
Now my big question is:
How do I extend Zend_Db, so I can implement and use the function above as $db->fetchMulti($sql2); instead of sqlquery_multiple($db, $sql2)?
Thanks in advance :)
NB: It's worth mentioning that I'm using the ODBC-patch in order to be able to fetch multiple rowsets in the first place.