I am trying to convert a huge set of dates to ISODate objects and storing it in the same collection and field. I am using the following query:
db.collection.find().forEach(function(element){
element.StartTime = ISODate(element.StartTime);
element.StopTime = ISODate(element.StopTime);
db.collection.save(element);
});
The query ran for about 10 minutes, and then gave an error:
2017-06-15T15:48:10.419+0200 E QUERY [thread1] Error: invalid ISO date :
ISODate#src/mongo/shell/types.js:65:1
#(shell):2:23
DBQuery.prototype.forEach#src/mongo/shell/query.js:501:1
#(shell):1:1
I had a look into the entries in the DB and it looks like it did convert a lot of the data set, but I am having troubles now fixing the error and finding the location where it stopped. What I tried is using Studio3T to find where either "StartTime" or "StopTime" has a value of 2017-06-15T15:48:10.419+0200 or at least starts with 2017-06, but as I expected (since there shouldn't be any data from June 2017 in there), it can't find anything.
Now when I run the query again, it gives the error immediately.
My question is if it's possible to find the document in the collection responsible for this error or what the cause might be. Are there recommendations for bulk operations like this preventing these errors?
Related
I'm currently taking a course in university.mongodb but suddenly the queries stopped working. I was practicing with the UpdateOne and noticed that it didn't worked and didn't modified the document I was trying to modify. I'm now trying to do a simple find but it returns a 0 like there's no document in the database, and I already checked if the database was set in the one I had to use.
I reconnected and used the db I needed but the query didn't worked neither.
db.movieDetails.find({title: "The Martian"})
It's not a syntax problem. I expected the output to be 1 because there is only 1 document with that name but it returned a 0.
Screenshot of the console
db.movieDetails.find({title: "The Martian"})
What you are doing is:
db.MovieDetails.find({title: "The Martian"})
Collection name is case sensitive
I have a mongodb catalog under etc/catalog named mongodb.properties
When I run the presto shell and execute command :
Show tables;
I shows the collections of mongodb but when I run select query it gives me errors.
presto> select * from <catalog>.<schema>.<collection/table>;
Gives error :
java.sql.SQLException: Query failed (#20190429_125534_00001_qxggq): line 1:8: SELECT * not allowed in queries without FROM clause
at io.prestosql.jdbc.PrestoResultSet.resultsException(PrestoResultSet.java:1839)
at io.prestosql.jdbc.PrestoResultSet.getColumns(PrestoResultSet.java:1749)
at io.prestosql.jdbc.PrestoResultSet.<init>(PrestoResultSet.java:118)
at io.prestosql.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:251)
at io.prestosql.jdbc.PrestoStatement.execute(PrestoStatement.java:229)
at io.prestosql.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:78)
at spark_mongo_poc.SparkMongo.process_query(SparkMongo.java:32)
at spark_mongo_poc.SparkMongo.main(SparkMongo.java:76)
Presto> select name from <mongodb>.<schema>.<collection>;
Gives error :
java.sql.SQLException: Query failed (#20190429_125718_00002_qxggq): line 1:8: Column 'name' cannot be resolved
at io.prestosql.jdbc.PrestoResultSet.resultsException(PrestoResultSet.java:1839)
at io.prestosql.jdbc.PrestoResultSet.getColumns(PrestoResultSet.java:1749)
at io.prestosql.jdbc.PrestoResultSet.<init>(PrestoResultSet.java:118)
at io.prestosql.jdbc.PrestoStatement.internalExecute(PrestoStatement.java:251)
at io.prestosql.jdbc.PrestoStatement.execute(PrestoStatement.java:229)
at io.prestosql.jdbc.PrestoStatement.executeQuery(PrestoStatement.java:78)
at spark_mongo_poc.SparkMongo.process_query(SparkMongo.java:32)
at spark_mongo_poc.SparkMongo.main(SparkMongo.java:76)
I want same data as we get when we do db.collection.find({}); which gives me proper result in the form of documents
Please help
I Fixed the above issue by just changing the mongodb collection name
to all Lower Case. There is an issue with presto mongodb catalog that
it doesnot recognise Upper case Lettered collection name and that was
the reason it was not able to identify the collection.
So I changed my collection case to lower and it started working. :)
I had similar issue. I fixed it adding this to catalog configuration.
mongodb.case-insensitive-name-matching=true
I'm trying to filter oplog.rs to view operations that were logged after a certain date. But comparison operators don't seem to work with dates in MongoDB:
db['oplog.rs'].find({ts: {$gte: ISODate("2014-12-15T00:00:00Z")}})
What's confusing is a lot of online sources are saying to do it this way but it is obviously not working. I want to return results where the ts field is at least December 15th or more recent, but I am getting results that are clearly before this period. If I switch $gte with $lte no results at all are displayed, even though there are definitely entries both before and after this specified date.
Also, I can't seem to sort the results either. If I try this:
db['oplog.rs'].find().sort({ts: -1})
I get this:
error: {
"$err" : "Runner error: Overflow sort stage buffered data usage of 33566005 bytes exceeds internal limit of 33554432 bytes",
"code" : 17144
}
If I could filter the results to make them more recent, I am hoping it would overcome this sorting error, but I can't even do that with MongoDB's basic operators. How can I filter the results of a find operation by date?
Comparion operators work just fine with date, but ts is a Timestamp object - not a date.
Your query needs to look something like this:
db['oplog.rs'].find({ts: {$gte: Timestamp(ISODate("2014-12-15T00:00:00Z").getTime(),0)}})
This creates a Timestamp object based on an ISODate and uses it in the query.
I am trying to post a query to Sails.js via HTTP POST in order to find records created after a certain point in time.
Using the following syntax works perfectly and gives me back one record with the correct time:
localhost:1337/object?where={"createdAt":"2014-08-19T14:36:16.047Z"}
When trying to add an operator the query does not work any longer giving me no records in the object collection :
localhost:1337/object?where={"createdAt":{">":"2014-08-19T14:36:16.047Z"}}
Due to the fact that the date conversion from ISODate to dateTime works for the exact query I suppose it should also work for the ">" query.
Highly appreciate some help on this one.
Attempting to setup / populate a basic mongo db using the shell, I get the following error message.
I have tried different databases but the error is the same. Can anyone kindly explain what I'm doing wrong?
{
"user":"bob",
"email":"bob#home.com"
}
Mon May 26 00:54:28.228
SyntaxError: Unexpected token :
Attempting to setup / populate a basic mongo db using the shell, I get the following error message.
I have tried different databases but the error is the same. Can anyone kindly explain what I'm doing wrong?
The code you mentioned is just the document itself. Check out the docs for how insert works. To insert, you should use:
db.collection.insert()
An example of an insert with the document in your example:
db.collection.insert({ "user":"bob", "email":"bob#home.com" })
Edit---
To clarify, you must replace "collection", in the above example with the name of the collection you're trying to insert into.
Also, ensure that you're using the correct database. When you connect to the shell, select your db with:
use testdb