i have a collection of financial data stored in mongodb. each company symbol has its data. the question is how to iterate over the collection and change the value of the key which is the symbol company to print out the whole collection and this is my list of companies ['TSLA','TYO','C','LULU','DTV','SHS',' ZNGA'] and this is my cod which return the data of one company:
from pymongo import MongoClient
import csv
host = "localhost"
port = 27017
databaseName = "finance000"
collection_name = "income"
client = MongoClient(host, port)
database = client[databaseName]
collection = database[collection_name]
def finance_data(symbol):
Earnings_BeforeInterestAndTaxes = symbol['statement'[0]EarningsBeforeInterestAndTaxes']['content']
Total_Revenue = symbol['statement'][0]['TotalRevenue']['content']
return Earnings_BeforeInterestAndTaxes,Total_Revenue
i =collection.find({'symbol':'C'})
with open('D:/INCOMEdata.csv', "w") as output:
writer = csv.writer(output, lineterminator='\n')
for key in i :
print finance_data(key)
writer.writerow(finance_data(key))
If I understood you correctly. You want to retrieve the document/data for a given company. The company is denoted by a symbol example 'TYSLA'.
If that is what you need. Here is how you do it (assuming each symbol is unique)
company_data = collection.find({'symbol':'TYSLA'})
The above will return a dictionary. To access an element within the document you just use:
company_data['profit'] #profit is an example of an attribute name. You can use any attribute you like.
Assuming you have multiple companies with the same symbol. If you used the above command. you will get a cursor. to get each company just do a for loop example:
company_data = collection.find({'symbol':'TYSLA'})
Now to loop:
for one in company_date:
print one['profit'] #python 2.7
Now to edit the say for example the profit attribute use $set
collection.update_one({'symbol':'TYSLA'},{'profit':100})
the above will change TYSLA's company profit to 100
Update
Assuming you have a collection with the symbol being any value of ['TSLA','TYO','C','LULU','DTV','SHS',' ZNGA']. to get the data for any of the symbols you use (assuming symbol contains any of the names (only one name)):
you use the $or as:
collection.find({"$or":[ {'symbol':'TSLA},{'symbol':'TYO},... ]},{}) #the ... are the rest of the names you need
The above returns the whole data for a given symbol. To return the specifics Total_Revenue and Earnings_BeforeInterestAndTaxes you use the following:
collection.find({"$or":[ {'symbol':'TSLA},{'symbol':'TYO},... ]},{'Total_Revenue ':1,'Earnings_BeforeInterestAndTaxes ':1, _id:0 }) #if you remove _id it will always be returned
I hope that helps.
Related
I have a collection in mongodb. In my python program, I have a variable named coll point at it. I want to create an index on a specified field, digestedOn, which will cause expiration of the record after 7776000 seconds.
I know how to create a simple index in python: coll.create_index([( "digestedOn", pymongo.ASCENDING)]). Where do I stick the {"expireAfterSeconds": 7776000} part?
Here's my whole program, I need the last line fixed so that the index is created with expireAfterSeconds.
import pymongo
import ssl
def connect_to_mongo(host, port, ssls, user, password, auth_source):
return pymongo.MongoClient(host, port, ssl=ssls, username=user, ssl_cert_reqs=ssl.CERT_NONE,
password=password, authSource=auth_source,
authMechanism='SCRAM-SHA-1', maxPoolSize=None)
client = connect_to_mongo(host="10.10.10.10", port=27017, ssls=True, user="user",
password="password",auth_source="admin")
db = client['logs']
colnames = db.list_collection_names()
coll = db[colnames[0]]
coll.create_index([( "digestedOn", pymongo.ASCENDING )])
Just pass it as a named parameter:
coll.create_index([( "digestedOn", pymongo.ASCENDING )], expireAfterSeconds=7776000)
I need to search for users by name, their first name and last names are stored in separate columns in a postgresql database. The columns need to be concatenated for search to work properly. Typing the full first and last name of a user should match a result.
What could I pass as a query to the find method of a Feathers service that would allow me to do this?
As in the answer linked you can pass the where clause to the Feathers service by modifying params.query in a before hook:
app.service('users').before({
find(hook) {
const where = Sequelize.where(Sequelize.fn("concat",
Sequelize.col("firstname"),
Sequelize.col("lastname")), {
like: '%John Do%'
}
);
hook.params.query.where = where;
}
})
When I use mongo-ruby-driver and I insert new document it returns generated '_id':
db = MongoClient.new('127.0.0.1', '27017').db('ruby-mongo-examples')
id = db['test'].insert({name: 'example'})
# BSON::ObjectId('54f88b01ab8bae12b2000001')
I'm trying to get the '_id' of a document after doing an insertion using Moped:
db = Moped::Session.new(['127.0.0.1:27017'])
db.use('ruby-mongo-examples')
id = db['coll'].insert({name: 'example'})
# {"connectionId"=>15, "n"=>0, "syncMillis"=>0, "writtenTo"=>nil, "err"=>nil, "ok"=>1.0}
How I get the id using Moped?
Update:
I also try use safe mode but it doesn't work:
db = Moped::Session.new(['127.0.0.1:27017'])
db.use('ruby-mongo-examples')
db.with(safe: true) do |safe|
id = safe['coll'].insert({name: 'example'})
# {"connectionId"=>5, "n"=>0, "syncMillis"=>0, "writtenTo"=>nil, "err"=>nil, "ok"=>1.0}
end
After inserting/saving, the returned object will have a property inserted_id which is a BSON::ObjectId:
# I'm using insert_one
result = safe['coll'].insert_one({name: 'example'})
result.methods.sort # see list of methods/properties
result.inserted_id
result.inserted_id.to_s # convert to string
From this issue:
It would be nice, but unfortunately Mongo doesn't give us anything
back when inserting (since it's fire and forget), and when in safe
mode it still doesn't give the id back if it generated it on the
server. So there really isn't any possible way for us to do this
unless it was a core feature in MongoDB.
Your best bet would be to generate the id before inserting the document:
document = { _id: Moped::BSON::ObjectId.new, name: "example" }
id = document[:_id]
I want to access a document in collection by 'name' attribute for getting its ObjectId so that i can insert that unique objectid to other document for reference.
cursorObject = db.collectionIngredient.find({'name': 'sugar'})
I want _id field of cursorObject.
cursorObject.'_id' or cursorObject._id not working.
I have tried __getitem__, __getattribute__ and so much internet surfing but couldn't able to find a way.
Please help
First, as #jjmartinez pointed out, find returns a cursor, which you need to iterate over, in order to get hold of the documents returned by your query. The _id field belongs to the documents, not the cursor.
I'm guessing that in your case the name is unique, so you can avoid cursor/iterating if you use find_one instead of find. Then you get the document directly.
Then, to access the _id, you just need a standard dict-item access:
id = doc['_id']
So we get:
ingredient = db.collectionIngredient.find_one({'name': 'sugar'})
if ingredient is not None:
id = ingredient['_id']
else:
id = None
When you do cursorObject = db.collectionIngredient.find({'name': 'sugar'}) you have a collection of documents, not a single element. So you need to explore all the collection. You need to iterate inside the cursor:
try:
cursorObject = db.collectionIngredient.find({'name': 'sugar'})
except:
print "Unexpected error:", sys.exc_info()[0]
for doc in cursorObject:
print doc
Here you have the Pymongo Tutorial
I have the following Mongoengine document:
class MyEmbed(EmbeddedDocument):
embedField = StringField(primary_key=True)
varField = StringField()
class TestDoc(Document):
myField = StringField()
embed_list = ListField(EmbeddedDocumentField(MyEmbed))
So I keep a list of embedded documents, to which I wish to add new documents if they don't exist already. The problem is that when I use the atomic update operator add_to_set things don't turn out the way I want them to.
This is what I am trying to do:
embed1 = models.MyEmbed(embedField="F1")
parent = models.TestDoc(myField="ParentField")
embed_list = []
embed_list.append(embed1)
parent.embed_list = embed_list
parent.save()
embed2 = models.MyEmbed(embedField="F1", varField="varField")
TestDoc.objects(id=parent.id).update_one(add_to_set__embed_list=embed2)
The problem is that after doing this, I have in the DB a list of embedded documents with 2 elements. And what I want is to decide upon one field (embedField in this case) whether 2 EmbeddedDocuments are equal or not, and not by taking into account all the properties. My questions are:
What are the default criteria according to which Mongoengine decides whether 2 EmbeddedDocuments are equal or not?
How can I redefine the function that makes Mongoengine decide when 2 EmbeddedDocuments are equal or not?
Thanks!
The actual checking is done inside MongoDB and not mongoengine.
The object sent to mongodb should be the same, but this is where it gets tricky as with BSON order is important and in python with dictionaries its not. When converting to send to mongodb mongoengine just passes a dictionary. This is a bug - so I've added #296 and will fix for 0.8
See https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/document.py#L51 and https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/base/document.py#L52:
def __eq__(self, other):
if isinstance(other, self.__class__):
return self._data == other._data
return False
It compare dicts of Embedded documents data. So you can override this method.
If you look at Document update that calls QuerySet update (find add_to_set and addToSet) you can find that mongoengine doesnt't check exists document in list and just call mongo $addToSet operation: https://github.com/MongoEngine/mongoengine/blob/master/mongoengine/queryset/transform.py#L156.
In your code you have document MyEmbed(embedField="F1") and try add another document MyEmbed(embedField="F1", varField="varField") so logic right: it add new document. If you try next code:
embed1 = models.MyEmbed(embedField="F1")
parent = models.TestDoc(myField="ParentField")
embed_list = []
embed_list.append(embed1)
parent.embed_list = embed_list
parent.save()
embed2 = models.MyEmbed(embedField="F1", varField="varField")
TestDoc.objects(id=parent.id).update_one(add_to_set__embed_list=embed2)
embed3 = models.MyEmbed(embedField="F1")
TestDoc.objects(id=parent.id).update_one(add_to_set__embed_list=embed3)
embed4 = models.MyEmbed(embedField="F1", varField="varField")
TestDoc.objects(id=parent.id).update_one(add_to_set__embed_list=embed4)
you can find that parent contains only embed1 and embed2.
So, to resolve you problem you can override __eq__ method and check document in list, but you must find another solution for update document list, because it have direct call of mongo method.