Hi everyone especially Talend users.
I need to filter data load in tMongoDBInput_1 via the query with no success.
Fyi, the code runs well on the mongodb environment / shell.
db.getCollection("tes_date").find({
created_at : { $gte: new Date(ISODate().getTime() - 1000 * 60 * 60 * 24 * 2 ) }
})
but then I tried to run on talend tMongoDBInput_1 with this query
"{
created_at : { $gte: new Date(ISODate().getTime() - 1000 * 60 * 60 * 24 * 2 ) }
}"
with the following error results
> tMongoDBInput_1 - The query does not contain any reference an index. [ _id ]
> Exception in component tMongoDBInput_1 (Mongodb_query)
> org.bson.json.JsonParseException: JSON reader expected an integer or a string but found 'ISODate'.
> at org.bson.json.JsonReader.visitDateTimeConstructor(JsonReader.java:913)
> at org.bson.json.JsonReader.visitNew(JsonReader.java:566)
> at org.bson.json.JsonReader.readBsonType(JsonReader.java:253)
> at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:167)
> at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:46)
> at org.bson.codecs.DocumentCodec.readValue(DocumentCodec.java:258)
> at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:169)
> at org.bson.codecs.DocumentCodec.decode(DocumentCodec.java:46)
> at org.bson.Document.parse(Document.java:110)
> at org.bson.Document.parse(Document.java:95)
> at local_project.mongodb_query_0_1.Mongodb_query.tMongoDBInput_1Process(Mongodb_query.java:810)
> at local_project.mongodb_query_0_1.Mongodb_query.runJobInTOS(Mongodb_query.java:1335)
> at local_project.mongodb_query_0_1.Mongodb_query.main(Mongodb_query.java:1173)
any help is appreciated, or any alternative script for dynamic query H-1 date range in talend are welcome, thanks guys.
Related
I am trying to select all records where the date (a5) is greater than 2021. I keep getting false results back. I have tried the following.
SELECT * WHERE a5 > "3/02/2021"
SELECT * WHERE a5 > '3/02/2021'
SELECT * WHERE parseInt(a5) > 3/02/2021
I went through the documentation but don't see anything about Date format. Here are my negative results.
You can use Date.parse() function from Javascript e.g.
SELECT * WHERE Date.parse(a5) > Date.parse('3/02/2021') and NR > 1
or (if you click that "Input table has header" checkbox):
SELECT * WHERE Date.parse(a.hireDate) > Date.parse('3/02/2021')
i am trying to fetch users list who older than and between 20 to 60 minute randomly. here is my query
SELECT *
FROM t_users
WHERE create_time <= NOW() - INTERVAL CONCAT(floor(random()* (60-20 + 1) + 20),' minutes');
it's giving me error ERROR: syntax error at or near "CONCAT"
You can't use concat() like that to create an interval. The easiest solution is to use make_interval:
WHERE create_time <= NOW() - make_interval(mins => (floor(random()* (60-20 + 1) + 20))::int )
SELECT_QUERY = `SELECT * FROM events WHERE c_id = ? AND start_time > ? and
end_time < ?`
query := sr.db.Raw(SELECT_QUERY, request.GetCId(), startTime, endTime)
var v = request.GetVIds()
if len(v) > 0 {
query = query.Where(` v_id IN (?) `, v)
} //Only this block introduces first ) after end_time
var c = request.GetStatus().String()
if len(c) > 0 {
query = query.Where( " status = ? ", c) // this introduces the other opening brace //after AND
}
Following is the query generated and found in logs
SELECT * FROM events WHERE c_id = 1 AND start_time > '2020-04-16 18:42:00' and
end_time < '2020-04-16 18:45:50' ) AND ( v_id IN (1,2)) AND ( status = 'STATUS_MIDDLE_CLASS' ORDER BY start_time DESC LIMIT 5 OFFSET 1
The other solution in stackoverflow and internet article doesn't help.
PS: Is it because I mix db.Raw( ) and query.Where() ?
Changing ? to $1 doesn't fix the issue.
Basically a few things fixed the issue.
1) Mixing Raw and query.Where was one issue.
After making the Raw query to sr.db.Where
2)
SELECT_QUERY = `SELECT * FROM events WHERE c_id = ? AND start_time > ? and
end_time < ?`
already has select * from. And then using query := sr.db.Raw(SELECT_QUERY, request.GetCId(), startTime, endTime) introduces nested select *.
So, changed SELECT_QUERY as follows
SELECT_QUERY = `events WHERE c_id = ? AND start_time > ? and
end_time < ?`
solved the issue.
I found a workaround to the error which I received when I tried to add a timestamp filed in Go/Gorm solution with PostgreSQL equivalent to default: now() which is default: time.Now().Format(time.RFC3339)
I received the error because I use AutoMigrate() to create the tables in PostgreSQL. The one problem I found is when trying to use the default value of a function instead of a string (which one can use for a fixed timestamp).
So I had to go into DataGrid (since I use JetBrains, but you can use any PostgreSQL admin tool like pgAdmin) and manually add the timestamp field with default of now() or merely update the existing field to add the default of now(). Then the error goes away when doing your next build in Go.
I am using OrientDB 2.2.17 (enterprise edition) and I have a problem searching on the data:
I have a vertex called: EN70 with a property: EP70_4 from type 'embedded list = LONG'.
I added a record to this vertex with the value of [2,3,4] in this 'EP70_4' property.
The index on this property: Not_Unique (sbtree). [must support range queries]
SELECT * FROM EN70 WHERE (EP70_4 >= 1) is working and returns my record.
SELECT * FROM EN70 WHERE (EP70_4 <= 3) is working and returns my record.
The operator "Between" is not woking and throw exception:
SELECT * FROM EN70 WHERE (EP70_4 BETWEEN 1 AND 3 )
I tried to simulate a "between" operator:
SELECT * FROM EN70 WHERE ((EP70_4 >= 1) AND (EP70_4 <= 3)) IS NOT WORKING
but it is not working.
both sections 6 and 7 throws an exception:
Error on using index EN70.EP70_4 in query 'SELECT * FROM EN70 WHERE (EP70_4 BETWEEN 1 AND 3 )'. Probably you need to rebuild indexes. Now executing query using cluster scan
java.lang.ClassCastException: java.util.ArrayList cannot be cast to java.lang.Long
at com.orientechnologies.common.serialization.types.OLongSerializer.preprocess(OLongSerializer.java:36)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.iterateEntriesBetweenAscOrder(OSBTree.java:1474)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.iterateEntriesBetween(OSBTree.java:771)
at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine.iterateEntriesBetween(OSBTreeIndexEngine.java:185)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doIterateIndexEntriesBetween(OAbstractPaginatedStorage.java:2065)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.iterateIndexEntriesBetween(OAbstractPaginatedStorage.java:2053)
at com.orientechnologies.orient.core.index.OIndexMultiValues.iterateEntriesBetween(OIndexMultiValues.java:275)
at com.orientechnologies.orient.core.index.OIndexAbstractDelegate.iterateEntriesBetween(OIndexAbstractDelegate.java:104)
at com.orientechnologies.orient.core.index.OIndexTxAwareMultiValue.iterateEntriesBetween(OIndexTxAwareMultiValue.java:339)
at com.orientechnologies.orient.core.sql.operator.OQueryOperatorBetween.executeIndexQuery(OQueryOperatorBetween.java:131)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:2184)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:1001)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:209)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:530)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:512)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:488)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:74)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:2624)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:2570)
at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract.execute(OCommandRequestTextAbstract.java:69)
at com.orientechnologies.orient.server.network.protocol.http.command.post.OServerCommandPostCommand.execute(OServerCommandPostCommand.java:106)
at com.orientechnologies.orient.graph.server.command.OServerCommandPostCommandGraph.execute(OServerCommandPostCommandGraph.java:37)
at com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpAbstract.service(ONetworkProtocolHttpAbstract.java:169)
at com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpAbstract.execute(ONetworkProtocolHttpAbstract.java:622)
at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:77)
What can it be??? Please help me...
I'd say it's a bug, I strongly suggest you to report it here: https://github.com/orientechnologies/orientdb/issues
I'm trying to use Npgsql and/or Dapper to query a table and I keep running into Npgsql.PostgresException 42601: syntax error at or near "$1".
Here is what I've got trying it with NpgsqlCommand:
using (var conn = new NpgsqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["postgres"].ConnectionString))
{
conn.Open();
using (NpgsqlCommand command = new NpgsqlCommand("select * from Logs.Logs where Log_Date > current_date - interval #days day;", conn))
{
command.Parameters.AddWithValue("#days", days);
var reader = command.ExecuteReader();
I've also tried it with Dapper(my preferred method) with:
using (var conn = new NpgsqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["postgres"].ConnectionString))
{
conn.Open();
var logs = conn.Query<Log>("select * from Logs.Logs where Log_Date > current_date - interval #days day;", new {days = days});
Either way I get the same Npgsql.PostgresException 42601: syntax error at or near "$1" error. The Statement in the Exception shows: select * from Logs.Logs where Log_Date > current_date - interval $1 day
Note, if I do the following it works fine, but it's not properly parameterized:
var logs = conn.Query<Log>("select * from Logs.Logs where Log_Date > current_date - interval '" + days + "' day;");
What am I doing wrong? I very much appreciate any feedback. Thank you.
PostgreSQL doesn't allow you to stick a parameter anywhere in a query. What you want can be achieved with the following:
var command = new NpgsqlCommand("select * from Logs.Logs where Log_Date > current_date - #days", conn))
command.Parameters.AddWithValue("#days", TimeSpan.FromDays(days));
This way you're passing the interval directly from Npgsql to PostgreSQL, rather than a part of the expression designed to create that interval.
i got this error using DapperExtensions
adding
DapperExtensions.DapperExtensions.SqlDialect = new PostgreSqlDialect();
DapperAsyncExtensions.SqlDialect = new PostgreSqlDialect();
before creating the connection fixed the issue
To subtract days from a date (assuming log_date is data type date), you can simplify:
"SELECT * FROM logs.logs WHERE log_date > CURRENT_DATE - #days;"
And provide #days as unquoted numeric literal (digits only) - which is taken to be an integer. This is even more efficient, since date - integer returns date, while date - interval returns timestamp.
The manual about interval input.