I can execute update commands likes below.
update #28:51 put sessionStatus = 'session1', true
and
update #28:51 put sessionStatus = 'session2.s', true
This means the key of embedded map can contain dot, and at this point I have the data as below.
If the key does not contain dot, I can query by following statement.
select from gw_true where sessionStatus.session1 = true
Please help me the case that key contains a dot character.
Related
We have following structure in MongoDB documents.
{
"id":"1111",
"keys":[
{
"name":"Country",
"value":"USA"
},
{
"name":"City",
"value":"LongIsland"
},
{
"name":"State",
"value":"NewYork"
}
]
}
Now using Springframework Query object, I figured out a way to pull the details using below syntax
query.addCriteria(
Criteria.where("keys.value").is(countryparam).
andOperator(
Criteria.where("keys.value").is(stateparam)
)
);
Two issue with this query model.
First issue is it is irrelevant if countryparam and stateparam are actually meant to match Country key name and State key name respectively. If just the values matches, the query returns the document. Means, if I have Country and City params, this just works if user passes Country and City values, even if they are swapped. So how can I exactly compare City to cityparam and State to Stateparam?
More complexity is if I have to extract the document basing on multiple key value pairs, I should be correspondingly able to match key name with respective value and query the document. How can I do this?
Thanks in advance!
In my CRUD Rest Service I do an insert into a DB and want to respond to the caller with the created new record. I am looking for a nice way to convert the map to json.
I am running on ballerina 0.991.0 and using a postgreSQL.
The return of the Update ("INSERT ...") is a map.
I tried with convert and stamp but i did not work for me.
import ballerinax/jdbc;
...
jdbc:Client certificateDB = new({
url: "jdbc:postgresql://localhost:5432/certificatedb",
username: "USER",
password: "PASS",
poolOptions: { maximumPoolSize: 5 },
dbOptions: { useSSL: false }
}); ...
var ret = certificateDB->update("INSERT INTO certificates(certificate, typ, scope_) VALUES (?, ?, ?)", certificate, typ, scope_);
// here is the data, it is map<anydata>
ret.generatedKeys
map should know which data type it is, right?
then it should be easy to convert it to json like this:
{"certificate":"{certificate:
"-----BEGIN
CERTIFICATE-----\nMIIFJjCCA...tox36A7HFmlYDQ1ozh+tLI=\n-----END
CERTIFICATE-----", typ: "mqttCertificate", scope_: "QARC", id_:
223}"}
Right now i do a foreach an build the json manually. Quite ugly. Maybe somebody has some tips to do this in a nice way.
It cannot be excluded that it is due to my lack of programming skills :-)
The return value of JDBC update remote function is sql:UpdateResult|error.
The sql:UpdateResult is a record with two fields. (Refer https://ballerina.io/learn/api-docs/ballerina/sql.html#UpdateResult)
UpdatedRowCount of type int- The number of rows which got affected/updated due to the given statement execution
generatedKeys of type map - This contains a map of auto generated column values due to the update operation (only if the corresponding table has auto generated columns). The data is given as key value pairs of column name and column value. So this map contains only the auto generated column values.
But your requirement is to get the entire row which is inserted by the given update function. It can’t be returned with the update operation if self. To get that you have to execute the jdbc select operation with the matching criteria. The select operation will return a table or an error. That table can be converted to a json easily using convert() function.
For example: Lets say the certificates table has a auto generated primary key column name ‘cert_id’. Then you can retrieve that id value using below code.
int generatedID = <int>updateRet.generatedKeys.CERT_ID;
Then use that generated id to query the data.
var ret = certificateDB->select(“SELECT certificate, typ, scope_ FROM certificates where id = ?”, (), generatedID);
json convertedJson = {};
if (ret is table<record {}>) {
var jsonConversionResult = json.convert(ret);
if (jsonConversionResult is json) {
convertedJson = jsonConversionResult;
}
}
Refer the example https://ballerina.io/learn/by-example/jdbc-client-crud-operations.html for more details.?
Is there a search all filter when trying to search through the db? I have multiple select boxes that I would like to get the value of and then according to the values, it searches for the data. I found that I would need to have an if statement for each select box to check if it is the default value(String) or a selected value(Integer).
items.find({
"person.age": obj.age, //is 'All'
"person.gender": obj.gender // is 'male'
},{
limit: this.state.limit
}).fetch()
In my example above, one is the default (all) and the other is the selected value. When I try to search, it would return nothing because of the 'all'. I am looking for a symbol or something to replace the 'all'.
#masteram is right - you want to avoid the keys that contain All - there is no * in mongodb. You can simplify your logic at the same time.
The following will copy only the keys of obj that are not equal to All then run the search on those keys:
const person = {};
Object.keys(obj).forEach((k) => {
if (obj[k] !== 'All') person[k] = obj[k];
});
items.find(person, { limit: this.state.limit }).fetch()
i have a collection of financial data stored in mongodb. each company symbol has its data. the question is how to iterate over the collection and change the value of the key which is the symbol company to print out the whole collection and this is my list of companies ['TSLA','TYO','C','LULU','DTV','SHS',' ZNGA'] and this is my cod which return the data of one company:
from pymongo import MongoClient
import csv
host = "localhost"
port = 27017
databaseName = "finance000"
collection_name = "income"
client = MongoClient(host, port)
database = client[databaseName]
collection = database[collection_name]
def finance_data(symbol):
Earnings_BeforeInterestAndTaxes = symbol['statement'[0]EarningsBeforeInterestAndTaxes']['content']
Total_Revenue = symbol['statement'][0]['TotalRevenue']['content']
return Earnings_BeforeInterestAndTaxes,Total_Revenue
i =collection.find({'symbol':'C'})
with open('D:/INCOMEdata.csv', "w") as output:
writer = csv.writer(output, lineterminator='\n')
for key in i :
print finance_data(key)
writer.writerow(finance_data(key))
If I understood you correctly. You want to retrieve the document/data for a given company. The company is denoted by a symbol example 'TYSLA'.
If that is what you need. Here is how you do it (assuming each symbol is unique)
company_data = collection.find({'symbol':'TYSLA'})
The above will return a dictionary. To access an element within the document you just use:
company_data['profit'] #profit is an example of an attribute name. You can use any attribute you like.
Assuming you have multiple companies with the same symbol. If you used the above command. you will get a cursor. to get each company just do a for loop example:
company_data = collection.find({'symbol':'TYSLA'})
Now to loop:
for one in company_date:
print one['profit'] #python 2.7
Now to edit the say for example the profit attribute use $set
collection.update_one({'symbol':'TYSLA'},{'profit':100})
the above will change TYSLA's company profit to 100
Update
Assuming you have a collection with the symbol being any value of ['TSLA','TYO','C','LULU','DTV','SHS',' ZNGA']. to get the data for any of the symbols you use (assuming symbol contains any of the names (only one name)):
you use the $or as:
collection.find({"$or":[ {'symbol':'TSLA},{'symbol':'TYO},... ]},{}) #the ... are the rest of the names you need
The above returns the whole data for a given symbol. To return the specifics Total_Revenue and Earnings_BeforeInterestAndTaxes you use the following:
collection.find({"$or":[ {'symbol':'TSLA},{'symbol':'TYO},... ]},{'Total_Revenue ':1,'Earnings_BeforeInterestAndTaxes ':1, _id:0 }) #if you remove _id it will always be returned
I hope that helps.
How to use dot in field name ?
I see error in example:
db.test2.insert({ "a.a" : "b" })
can't have . in field names [a.a]
You can replace dot symbols of your field name to Unicode equivalent of \uff0E
db.test.insert({"field\uff0ename": "test"})
db.test.find({"field\uff0ename": "test"}).forEach(printjson)
{ "_id" : ObjectId("5193c053e1cc0fd8a5ea413d"), "field．name" : "test" }
See more:
http://docs.mongodb.org/manual/faq/developers/#faq-dollar-sign-escaping
http://docs.mongodb.org/manual/core/document/#dot-notation
Actualy you may use dots in queries. See: http://www.mongodb.org/display/DOCS/Dot+Notation+%28Reaching+into+Objects%29
Because of this special dot symbol mean you cannot use it in field names. Like you cannot use dot symbol in identifiers in most of programming languages.
You may write query db.test2.find({ "a.a" : "b" }) but if you want to be able to write such a query you need to insert your object like so: db.test2.insert({"a": {"a": "b"}}). This will create document with the field named "a" with the value of embeded document containing the field named "a" (again) with the value "b".
You can also write a SONManipulator using the pymongo library that transforms the data going to and back out of mongodb. There are downsides; there is a performance hit (impact depends on your use case) and you have to transform your keys when you do searches using find.
Here's code with an example of how to use it in the comment for the KeyTransform class:
from pymongo.son_manipulator import SONManipulator
class KeyTransform(SONManipulator):
"""Transforms keys going to database and restores them coming out.
This allows keys with dots in them to be used (but does break searching on
them unless the find command also uses the transform).
Example & test:
# To allow `.` (dots) in keys
import pymongo
client = pymongo.MongoClient("mongodb://localhost")
db = client['delete_me']
db.add_son_manipulator(KeyTransform(".", "_dot_"))
db['mycol'].remove()
db['mycol'].update({'_id': 1}, {'127.0.0.1': 'localhost'}, upsert=True,
manipulate=True)
print db['mycol'].find().next()
print db['mycol'].find({'127_dot_0_dot_0_dot_1': 'localhost'}).next()
Note: transformation could be easily extended to be more complex.
"""
def __init__(self, replace, replacement):
self.replace = replace
self.replacement = replacement
def transform_key(self, key):
"""Transform key for saving to database."""
return key.replace(self.replace, self.replacement)
def revert_key(self, key):
"""Restore transformed key returning from database."""
return key.replace(self.replacement, self.replace)
def transform_incoming(self, son, collection):
"""Recursively replace all keys that need transforming."""
for (key, value) in son.items():
if self.replace in key:
if isinstance(value, dict):
son[self.transform_key(key)] = self.transform_incoming(
son.pop(key), collection)
else:
son[self.transform_key(key)] = son.pop(key)
elif isinstance(value, dict): # recurse into sub-docs
son[key] = self.transform_incoming(value, collection)
return son
def transform_outgoing(self, son, collection):
"""Recursively restore all transformed keys."""
for (key, value) in son.items():
if self.replacement in key:
if isinstance(value, dict):
son[self.revert_key(key)] = self.transform_outgoing(
son.pop(key), collection)
else:
son[self.revert_key(key)] = son.pop(key)
elif isinstance(value, dict): # recurse into sub-docs
son[key] = self.transform_outgoing(value, collection)
return son
def remove_dots(data):
for key in data.keys():
if type(data[key]) is dict: data[key] = remove_dots(data[key])
if '.' in key:
data[key.replace('.', '\uff0E')] = data[key]
del data[key]
return data
this recursive method replaces all dot characters from keys of a dict with \uff0E
as suggested by Fisk
I replaced the key value using myString.replace(".","\u2024") before inserting it into the JsonObject.
Initially I used a simple recursion to replace all "." characters with its unicode equivalent but figured it out that even the dots in the values was getting replaced. So I thought that we should replace the dots only from keys and made the changes accordingly in case "if isinstance(input, dict)".
I thought it should be a sufficient condition to do the magic but I forgot that dict value can also be a dict or a list and then I finally added that check that if value of a dict was not string then, go inside recursively and was finally able to come up with this solution which eventually did the trick.
def remove_dots(data):
if isinstance(data, dict):
return {remove_dots(key): value if isinstance(value, str) else remove_dots(value) for key,value in data.iteritems()}
elif isinstance(data, list):
return [remove_dots(element) for element in data]
elif isinstance(data, str):
return data.replace('.','\u002e')
else:
return data
I've only really come across this problem when trying to serialize Dictionaries and such where the offending dot can appear as a key name.
Edited to show the references.
The quick and dirty C# approach:
using MongoDB.Bson;
using Newtonsoft.Json.Linq;
using System.Text.RegularExpressions;
public static T Sanitize<T>(T obj)
{
var str = JObject.FromObject(obj).ToJson();
var parsed = Regex.Replace(str, #"\.(?=[^""]*"":)", "_"); //i.e. replace dot with underscore when found as a json property name { "property.name": "don't.care.what.the.value.is" }
return JObject.Parse(parsed).ToObject<T>();
}