I am doing same query in spark shell and same in dbeaver application on same server and port then why i m getting different results do you have any idea ?
I wanted my result to be which is in second snapshot , sparkshell is showing incorrect result for this query . not able to figure out why
Related
Hi guys unable to copy one field value to another field in mongo db update many options, I am using mongo db 3.6, i attached my query below pls help me to overcome this issue.
db.am_task.updateMany(
{"reselleraccountid":{$exists:true}},
{$set:{"reselleraccountid":"$parantid"}}
)
Actual Results:
Expected Result
its should replace its value, not print its key name.
I have created a mongodb database using mongolite and I create index on the _row key on the database using following command:
collection$index(add = '{"_row" : 1}')
when I query a document via Robo3T program with the db.getCollection('collection').find({"_row": "ENSG00000197616"}) command, my index works and it takes less than a second to query the data.
Robo3T screen shot >>> pay attention to the query time
This is also the case when I query the data using pymongo package in python.
python screenshot >>> pay attention to query time
Surprisingly, when I perform the same query with mongolite, it takes more than 10 seconds to query data:
system.time(collection$find(query = '{"_row": "ENSG00000197616"}'))
user system elapsed
12.221 0.005 12.269
I think this can only come from mongolite package, otherwise, it wouldn't work on the other programs as well.
Any input is highly appreciated!
I found the solution here:
https://github.com/jeroen/mongolite/issues/37
The time consuming part is not data query but simplifying it in a dataframe.
I have a mongodb catalog under etc/catalog named mongodb.properties
When I run the presto shell and execute command :
Show tables;
I shows the collections of mongodb but when I run select query it gives me errors.
presto> select * from <catalog>.<schema>.<collection/table>;
Gives error :
java.sql.SQLException: Query failed (#20190429_125534_00001_qxggq): line 1:8: SELECT * not allowed in queries without FROM clause
at io.prestosql.jdbc.PrestoResultSet.resultsException(PrestoResultSet.java:1839)
at io.prestosql.jdbc.PrestoResultSet.getColumns(PrestoResultSet.java:1749)
at io.prestosql.jdbc.PrestoResultSet.<init>(PrestoResultSet.java:118)
at io.prestosql.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:251)
at io.prestosql.jdbc.PrestoStatement.execute(PrestoStatement.java:229)
at io.prestosql.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:78)
at spark_mongo_poc.SparkMongo.process_query(SparkMongo.java:32)
at spark_mongo_poc.SparkMongo.main(SparkMongo.java:76)
Presto> select name from <mongodb>.<schema>.<collection>;
Gives error :
java.sql.SQLException: Query failed (#20190429_125718_00002_qxggq): line 1:8: Column 'name' cannot be resolved
at io.prestosql.jdbc.PrestoResultSet.resultsException(PrestoResultSet.java:1839)
at io.prestosql.jdbc.PrestoResultSet.getColumns(PrestoResultSet.java:1749)
at io.prestosql.jdbc.PrestoResultSet.<init>(PrestoResultSet.java:118)
at io.prestosql.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:251)
at io.prestosql.jdbc.PrestoStatement.execute(PrestoStatement.java:229)
at io.prestosql.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:78)
at spark_mongo_poc.SparkMongo.process_query(SparkMongo.java:32)
at spark_mongo_poc.SparkMongo.main(SparkMongo.java:76)
I want same data as we get when we do db.collection.find({}); which gives me proper result in the form of documents
Please help
I Fixed the above issue by just changing the mongodb collection name
to all Lower Case. There is an issue with presto mongodb catalog that
it doesnot recognise Upper case Lettered collection name and that was
the reason it was not able to identify the collection.
So I changed my collection case to lower and it started working. :)
I had similar issue. I fixed it adding this to catalog configuration.
mongodb.case-insensitive-name-matching=true
Please help with error below when running in mongodDb shell, thanks,
> use mvp_demo
switched to db mvp_demo
> show collections;
1476851599865_IND_TRX
1476851599865_NIN_TRX
1476851599865_SWF_TRX
configuration
> db.1476851599865_IND_TRX.remove({})
2017-02-16T22:34:29.377-0500 E QUERY [thread1] SyntaxError: identifier starts immediately after numeric literal #(shell):1:2
The problem is that mongodb wasn't designed to have a number as the first character in a collection name, similar to how JavaScript doesn't let you begin a variable name with a number. But to get around this limitation please try running the following:
db["1476851599865_IND_TRX"].remove({})
I'm trying to use the following line through flask:
collection = 'mongo.db.vehicules.distinct("date_debut", {"modele":"Q5"})'
When I don't insert the second arguments, the query works fine.
With the second one, I have the following error:
TypeError: distinct() takes 2 positional arguments but 3 were given
The same query in Mongo shell works fine.
Anyone has faced the same issue?
Thanks