Here I am in a position to write a Unix Shell script(.sh) to remove MongoDB collection documents.
I know how to remove based on a condition like below and it works for me.
eval 'db.Collection.remove({TimeStamp:{$lte: "'$var'"}})
But I need to change the remove statement based on a new parameter ,lets say PID which will receive bunch of inputs(Many PID's).
I don't need to remove the collection based on the Timestamp field ,Instead my condition has to be changed as per my previous statement.
I went through many forums ,But I am not able to get the solution.
Please help me to resolve this in my UNIX shell script.
Related
I am going to use SQL terminology because I am new to Parse, apologies if that is confusing. I have a table in an app and in order to introduce new functionality I need to add a new column and set all the records to a default value. In SQL I would just run
update <table> set <column> = <value>;
Parse Server has MongoDB as the back end and I am not clear whether the correct approach is to directly access the MongoDB and run statements through the command line or if this would cause an issue with Parse. I found this helpful link for translating SQL syntax to MongoDB for that, https://docs.mongodb.com/manual/reference/sql-comparison/.
I also noticed that there were some tools, such as studio 3t, but the ones I saw all required expensive licenses. If direct MongoDB access is OK, any help understanding how to get to that would be helpful, I installed parse-server from the Bitnami stack on the AWS marketplace and to date I have only been interacting with it through the provided dashboard which doesn't have an "Update all records" option.
Right now my work around is to write a Swift script that runs the update in a loop, but I have to think that if I had millions or records instead of thousands this would be the incorrect approach. What is the proper environment and code to update my existing Parse server so that I can run something like the SQL above?
I want to know is there any handler or trigger kind of functionality exist in MongoDB which can help me to run my mongo command on find query?
Basically, I want to run specific set of Mongo queries on find query. You can say a listener on find query to run other mongo commands.
Other than that is there anything like create a view from a remote database, I know MongoDB provides creating a view from the same database but I want from the remote database?
I do not want any application level solution like any third party module or watcher script, I want something which is within the MongoDB itself like inbuilt function or any event handling.
I have tried auditing but, it only records the operation and all, does not provide any handler through which I can run my mongo commands like Oracle providing in Fine-grained auditing.
Any help related to this is appreciable, Thanks in advance.
I have a mongo database db.ns, db.0, db.1, ... db.7
Accidentally I remove all the data from a collection, but in the database files (explorer with vim) it's all (or part of) the data.
After trying to recover the data moving to another mongodb instance, or mongod --restore, also, I try with the mongodump, but the collection appears empty.
I try to recover from scratch, directly from the files. I try with bsondump for each one, and for a single file (cat db.ns db.1 ... > bigDB) but nothing.
I don't know what other ways are from recover the data from a mongo database file.
Any suggestion?? Thx!!!
[SOLVED]
I will try to explain what I do to "solved" the problem.
First. Theory.
In this SlideShare, can see a little of how files MongoDB database work.
http://www.slideshare.net/mdirolf/inside-mongodb-the-internals-of-an-opensource-database
Options:
When you remove accidentally a collection:
the first thing that you have to do is quickly copy all the database (normally in /data/db or /var/lib/mongodb) and stop the service.
Remove the journal directory try to recover from this copy and pray ;D
You can see more about that, here:
mongodb recovery removed records
In my case, this did not work for me.
In Journaly case, mongo no update its database files directly only their indexes.
So that, you can access to the files (appointed as database.ns, database.0, database.1 ...) and try to recover this.
These files are as cleaved BSONs and binary. So, you can open, and see all the information
In my case, I create a easy function in PHP that first read the file, and explode the file in smallers files.
Before, takes one to one and apply some regular expresions to remove Hexadecimal values, explode the info into the registers (you can see the "_id" key to do that) and do some others task to clean the info.
And finally, I have to process manually all the preprocessed info to obtain all the information.
I think, I have lost, at least, the 15-25% of the information. But I prefer to think that I have recovered the 75% of the lost info.
Caution:
This is not a easy and secure way to solve this problem. In my case, the db only recive information, and not modify or update this.
With this method, a lot of information will be lost, Mongo IDs, integers, dates, can't be recovered.
The proccess is 100% manually, you can spend your time on automating certain tasks, but will depend on your database structure.
I have a time field in mysql table. based on this time field I need to import data from mongodb collection.
So in Pentaho transformations first I have a Table Input step which gets the required date.
Next I have a mongodb input step.Here how do i filter records based on the output from previous step?
I saw that in mongodb input query it accepts parameters only if its an environment variable or defined in another transformation, but does not recognize variable from previous step.
How do I load from previous step, please help me I am a fresher in Pentaho and trying for this solution since a week.
Thank you,
Deepthi
You've already answered your own question:
I saw that in mongodb input query it accepts parameters only if its an environment variable or defined in another transformation, but does not recognize variable from previous step. How do I load from previous step, please help me I am a fresher in Pentaho and trying for this solution since a week.
If there is no way for a step to accept an input stream, you'll have to do exactly what you describe. In one transformation, access the MySQL table to get the time and store it in a variable. Then in another transformation access that variable in your MongoDB step.
Note that you will have to do this in two transformations to ensure that the variable is set by the time the MongoDB step runs.
Take a look at optiq. This is bleeding edge, but allows sql access to mongodb, so in theory you could use it in a table input step rather than a mongo input step:
http://julianhyde.blogspot.co.uk/2013/06/efficient-sql-queries-on-mongodb.html
It can be achieved vai passing query as parameter.
In Transformation setting Add a parameter (eg : MONGO_QRY )
Example
In MongoDB Query expression (json)
${MONGO_QRY}
It works fine for us try that . If not lets know .
I'm at an impasse that probably has a simple solution, but I can't see it. I've done everything in the Sphinx documentation up to the point of the Quick Tour, but when I test the search using test.php in PuTTy, it returns zero results.
I've put in all my correct database info in sphinx.conf and I've assembled the SQL query. I'm not getting any errors at all, just that it says it's returning 0 results every time I search.
Is it looking at my databases? Let me know if you need to see any code. searchd is running (as far as I can tell).
Sphinx has 2 different phases:
1) Indexing
2) Searching
I belive from your question that you skipped by mistake part where you need to index data (run indexer) so searching would have data to search through. In indexing part sphinx will take all of data from your db and search will actually be searching that and not your DB.
Make sure that indexer --all showing that it found and indexed actual documents.
Besides the API there is another convenient method to test sphinx using SphinxQL
Add line "listen = 9306:mysql41" line in searchd section in sphinx.conf as described in http://astellar.com/2011/12/replacing-mysql-full-text-search-with-sphinx/ and start the daemon.
Then run
mysql -h0 -P 9306
and then fire the query against sphinx
SELECT * FROM <your_sphinx_index>;
Hope that helps!