Adding elements to JSON array using circe and scala - scala

I have a JSON string as the following:
{
"cars": {
"Nissan": [
{"model":"Sentra", "doors":4},
{"model":"Maxima", "doors":4},
{"model":"Skyline", "doors":2}
],
"Ford": [
{"model":"Taurus", "doors":4},
{"model":"Escort", "doors":4}
]
}
}
I would like to add a new cars brand (in addition to Nissan and Ford), using circe at scala.
How could I do it?
Thank you in advance.

You can modify JSON using cursors. One of the possible solutions:
import io.circe._, io.circe.parser._
val cars: String = """
{
"cars": {
"Nissan": [
{"model":"Sentra", "doors":4},
{"model":"Maxima", "doors":4},
{"model":"Skyline", "doors":2}
],
"Ford": [
{"model":"Taurus", "doors":4},
{"model":"Escort", "doors":4}
]
}
}"""
val carsJson = parse(cars).getOrElse(Json.Null)
val teslaJson: Json = parse("""
{
"Tesla": [
{"model":"Model X", "doors":5}
]
}""").getOrElse(Json.Null)
val carsCursor = carsJson.hcursor
val newJson = carsCursor.downField("cars").withFocus(_.deepMerge(teslaJson)).top
Here we just go down to cars field, "focus" on it and pass the function for modifying JSON values. Here deepMerge is used.
newJson will be look as follows:
Some({
"cars" : {
"Tesla" : [
{
"model" : "Model X",
"doors" : 5
}
],
"Nissan" : [
{
"model" : "Sentra",
"doors" : 4
},
{
"model" : "Maxima",
"doors" : 4
},
{
"model" : "Skyline",
"doors" : 2
}
],
"Ford" : [
{
"model" : "Taurus",
"doors" : 4
},
{
"model" : "Escort",
"doors" : 4
}
]
}
})

Related

graphene-mongo and nested json data

i have the follow two documents in mongo:
> db.user.find();
{ "_id" : ObjectId("623d12f5ee5204c41f028944"), "uid" : "you", "uid_number" : 5678, "eppns" : [ "you#x.com", "y.com" ], "props" : { "one" : 1, "two" : 2 } }
{ "_id" : ObjectId("623d1310ee5204c41f028945"), "uid" : "me", "uid_number" : 123, "eppns" : [ "me#x.com", "me#y.com" ], "props" : { "one" : 3, "two" : 3 } }
defined with
from mongoengine import Document
from graphene_mongo import MongoengineObjectType
from mongoengine.fields import (
FloatField,
IntField,
DictField,
StringField,
EmailField,
ListField,
URLField,
ObjectIdField,
)
from graphene import ObjectType, Schema, List, Field
from graphene.relay import Node
class User(Document):
meta = { 'collection': 'user' }
ID = ObjectIdField()
uid = StringField(required=True)
uid_number = IntField(required=True)
eppns = ListField( EmailField() )
props = DictField()
class UserType(MongoengineObjectType):
class Meta:
model = User
class Query(ObjectType):
node = Node.Field()
users = List(UserType)
def resolve_users(self, info, **kwargs):
return User.objects.all()
yet, when i query as such:
{ users { id eppns uid uidNumber props } }
i get the following:
{
"data": {
"users": [
{
"id": "623d12f5ee5204c41f028944",
"eppns": [
"you#x.com",
"you#y.com"
],
"uid": "you",
"uidNumber": 5678,
"props": "{\"one\": 1.0, \"two\": 2.0}"
},
{
"id": "623d1310ee5204c41f028945",
"eppns": [
"me#x.com",
"me#y.com"
],
"uid": "me",
"uidNumber": 123,
"props": "{\"one\": 3.0, \"two\": 3.0}"
}
]
}
}
ie, it does not render props as json, but as a string. how can i get props to render/resolve as a dict? i would prefer not to define props as another Document and $ref it.
getting the same problem and I found a possible solution with graphene_django and it solved my problem.
You just need to override your props field with GenericScalar in your UserType:
from graphene.types.generic import GenericScalar # solution
class UserType(MongoengineObjectType):
props = GenericScalar()
class Meta:
model = User

How can I access the nested values in JsValue?

I have the following Json string
{
"results" : [
{
"address_components" : [
{
"long_name" : "1600",
"short_name" : "1600",
"types" : [ "street_number" ]
},
{
"long_name" : "Amphitheatre Parkway",
"short_name" : "Amphitheatre Pkwy",
"types" : [ "route" ]
},
{
"long_name" : "Mountain View",
"short_name" : "Mountain View",
"types" : [ "locality", "political" ]
}
]}]}
I parse it using the following statement
val payload = Json.parse(results)
I then get the following results
payload: play.api.libs.json.JsValue = {"results":[{"address_components":[{"long_name":"1600","short_name":"1600","types":["street_number"]},{"long_name":"Amphitheatre Parkway","short_name":"Amphitheatre Pkwy","types":["route"]},{"long_name":"Mountain View","short_name":"Mountain View","types":["locality","political"]}]}
When I try and run this command
val extract = (payload \ "results.address_components")
I get the following output
play.api.libs.json.JsLookupResult = JsUndefined('results.address_components' is undefined on object
How do I access the element "address_components" ?
I'd recommend using case classes kind of like GamingFelix's suggestion (an alternative to that is below the line in my answer), but if you're wondering why your example didn't work here's why:
You didn't traverse the array properly. Given the following JSON:
{
"foo": [
{
"bar": 123,
"baz": "abc"
},
{
"bar": 456,
"baz": "def"
}
]
}
you'd get all of the bar values like this: Json.parse(result) \ "foo" \\ "bar" - ie, use double backslashes. You can chain this together too, like (Json.parse(result) \\ "field1").map(_ \\ "field2") etc.
So in your example, if you want all of the long_name fields you'd do it like this:
import play.api.libs.json.Json
val json = Json.parse(
"""{"results" : [{"address_components" : [{"long_name" : "1600","short_name" : "1600","types" : [ "street_number" ]},{"long_name" : "Amphitheatre Parkway","short_name" : "Amphitheatre Pkwy","types" : [ "route" ]},{"long_name" : "Mountain View","short_name" : "Mountain View","types" : [ "locality", "political" ]}]},{"address_components" : [{"long_name" : "16400","short_name" : "1600","types" : [ "street_number" ]},{"long_name" : "Amphitheatre 5Parkway","short_name" : "Amphitheatre Pkwy","types" : [ "route" ]},{"long_name" : "Mounta6in View","short_name" : "Mountain View","types" : [ "locality", "political" ]}]}]}"""
)
(json \ "results" \\ "address_components").map(_ \\ "long_name")
This returns an array of arrays, since you're going through multiple arrays to get your value. If you put a .foreach(_.foreach(println)) on the end, you'll get this:
"1600"
"Amphitheatre Parkway"
"Mountain View"
"16400"
"Amphitheatre 5Parkway"
"Mounta6in View"
See it working on Scastie.
If you're curious, GamingFelix's suggestion to almost combine the two approaches isn't really the best - it's easier to follow what's going on if you just convert the entire thing into case classes instead. Also, if you create a companion object to each case class to put the formatters in, instead of keeping everything in the same formatter object, you won't need to explicitly import formatters every time you need them (Scala will find them automatically).
This is how I'd do it:
import play.api.libs.json.{Json, OFormat}
val json = Json.parse(
"""{"results" : [{"address_components" : [{"long_name" : "1600","short_name" : "1600","types" : [ "street_number" ]},{"long_name" : "Amphitheatre Parkway","short_name" : "Amphitheatre Pkwy","types" : [ "route" ]},{"long_name" : "Mountain View","short_name" : "Mountain View","types" : [ "locality", "political" ]}]},{"address_components" : [{"long_name" : "16400","short_name" : "1600","types" : [ "street_number" ]},{"long_name" : "Amphitheatre 5Parkway","short_name" : "Amphitheatre Pkwy","types" : [ "route" ]},{"long_name" : "Mounta6in View","short_name" : "Mountain View","types" : [ "locality", "political" ]}]}]}"""
)
case class MyClass(results: Seq[Result])
object MyClass {
implicit val format: OFormat[MyClass] = Json.format[MyClass]
}
case class Result(address_components: Seq[AddressComponent])
object Result {
implicit val format: OFormat[Result] = Json.format[Result]
}
case class AddressComponent(long_name: String, short_name: String, types: Seq[String])
object AddressComponent {
implicit val format: OFormat[AddressComponent] = Json.format[AddressComponent]
}
val model = json.as[MyClass] // `.as` is unsafe, use `.asOpt` or `.validate` or something like that in a real scenario
model.results.flatMap(_.address_components.map(_.long_name)) // get values out like you normally would in a class
Here's that working on Scastie.
So since you're using play api, you can use implicits and case classes.
I usually use this website: https://json2caseclass.cleverapps.io/
to turn the json into scala case classes. And then I create implicits for the case classes.
Here's how I do it:
import play.api.libs.json._
case class Address_components(
long_name: String,
short_name: String,
types: List[String]
)
case class Results(
address_components: List[Address_components]
)
case class RootJsonObject(
results: List[Results]
)
implicit val a: Format[Address_components] = Json.format[Address_components]
implicit val b: Format[Results] = Json.format[Results]
implicit val c: Format[RootJsonObject] = Json.format[RootJsonObject]
Then you can easily parse the json and access them.
val test = Json.parse("""{
"results" : [
{
"address_components" : [
{
"long_name" : "1600",
"short_name" : "1600",
"types" : [ "street_number" ]
},
{
"long_name" : "Amphitheatre Parkway",
"short_name" : "Amphitheatre Pkwy",
"types" : [ "route" ]
},
{
"long_name" : "Mountain View",
"short_name" : "Mountain View",
"types" : [ "locality", "political" ]
}
]}]}""")
println(test.validate[RootJsonObject])
val mylist = (test \ "results").get.as[List[Results]]
mylist.map(a => println(a))
I also created a Scalafiddle for you if you wanna test it out in the browser:
https://scalafiddle.io/sf/J5dDfFo/6

I am new to mongoDB need a query to delete the collections

I have two collections.
1.Equipment
db.getCollection("Equipment").find({
$and: [
{ $where: 'this._id.length <= 7' },
{ "model": "A505"}
]})
{
"_id" : "1234567",
"locationId" : "DATALOAD",
"model" : "A505",
"subscriberId" : "",
"status" : "Stock",
"headendNumber" : "4"
}
{
"_id" : "P13050I",
"locationId" : "1423110302801",
"model" : "A505",
"subscriberId" : "37",
"status" : "Stock",
"headendNumber" : "4"
}
I will get more than 100 documents (rows) Equipment collection.
2.Subscriber
db.getCollection('Subscriber').find({})
{
"_id" : "5622351",
"equipment" : [
"0018015094E6",
"1234567",
"ADFB70878422",
"M10610TCB052",
"MA1113FHQ151"
]
}
{
"_id" : "490001508063",
"equipment" : [
"17616644510288",
"P13050I",
"M91416EA4251",
"128552270280560"
]
}
In the Subscriber collection, I need to remove (get all the id from Equipment collection loop it) only the matches equipment field.
Forex from the above result, I need to remove only "1234567", and "P13050I"
Expected output.
db.getCollection('Subscriber').find({})
{
"_id" : "5622351",
"equipment" : [
"0018015094E6",
"ADFB70878422",
"M10610TCB052",
"MA1113FHQ151"
]
}
{
"_id" : "490001508063",
"equipment" : [
"17616644510288",
"M91416EA4251",
"128552270280560"
]
}
Please help me, anyone.
You can use the following to update records.
Let's find records which need to deleted and store them in array
var equipments = [];
db.getCollection("Equipment").find({ $and: [
{ $where: 'this._id.length <= 7' },
{ "model": "A505"}
]}).forEach(function(item) => {
equipments.push(item._id)
})
Now, iterate over records of the second collection and update if required.
db.getCollection('Subscriber').find({}).forEach(function(document) => {
var filtered = document.equiment.filter(id => equipments.indexOf(id) < 0);
if(filtered.length < document.equipment.length){
db.getCollection('Subscriber').update({"_id": document.id }, { $set: {'equipment': filtered}})
}
})
.filter(id => equipments.indexOf(id) < 0) will keep entries which is not present in initially populated array equipments and it will persist if there is any change.

How to search in nested string arrays

I am a beginner in mongo db, I have this sample in my database
{
"_id" : ObjectId("5a95cef390bd8fbf1c699d73"),
"dr_asin" : "0439394422",
"dr_description" : "Product that encourages families to learn, explore, and create in new and exciting ways.",
"dr_price" : 12.96,
"dr_imUrl" : "http://ecx.images-amazon.com/images/I/51Zx2bIwWcL._SX300_.jpg",
"dr_related" : {
"also_bought" : [
"B0002667BI",
"B00005JKTY",
"B0002667B8"
],
"buy_after_viewing" : [
"B00005JKTY",
"B00000DGSW",
"B0002667BI"
]
},
"dr_salesRank" : {
"Video Games" : 36531
},
"dr_categories" : [
[
"Video Games",
"Mac",
"Games"
],
[
"Video Games",
"PC",
"Games"
]
]
}
I want to find all products that are in the Mac category, I tried all these
db.P15242570_products.find({dr_categories: {$in: ["Mac"]}})
db.P15242570_products.find( { 'dr_categories.Mac': { $exists: true } } )
db.P15242570_products.find({ dr_categories: "Mac" }
db.P15242570_products.find({ dr_categories : { $all : ['Mac'] }});
but nothing is working ? any ideas ?
This worked for me
db.15242570_products.find({"dr_categories":{$elemMatch:{$elemMatch:{$in:["PC"]}}});

Entity Framework/Linq group related data

I understand GroupBy and Include more or less, but can I group together the records that are included?
I have this working in my controller:
public async Task<IActionResult> Index()
{
return View(await _context.Organizations.Include(x => x.Members).ToListAsync());
}
Which gives me my Organizations and their Members... but I want the members grouped in to their teams. I thought something like:
public async Task<IActionResult> Index()
{
return View(await _context.Organizations.Include(
x => x.Members.GroupBy(m => m.Team)).ToListAsync());
}
But that is incorrect.
I'd like the returned data to be something around (but not necessarily exactly):
[
{
"ID" : "some-guid-string"
"Name" : "MyOrganization",
"Members" : {
"Team1" : [
"MemberName" : "John",
"MemberName" : "Jess",
"MemberName" : "Joe",
],
"Team2" : [
"MemberName" : "Jake",
"MemberName" : "Jeff"
]
}
}
]
I don't think you need to explicitly include fields which are coded in your query. But If I'm wrong, then you just need to replace _context.Organizations with _context.Organizations.Include(o => o.Members).Include(o => o.Members.Select(m => m.Team)) and the reset is the same.
To get a JSON output like this:
[
{
"Id": "some-guid-string",
"Name": "MyOrganization",
"Members":
{
"Team1":
[
"John",
"Jess",
"Joe"
],
"Team2":
[
"Jake",
"Jeff"
]
}
}
]
Could be done with:
_context.Organizations.Select(o => new
{
Id = o.Id,
Name = o.Name,
Members =
o.Members.GroupBy(m => m.Team)
.ToDictionary(kvp => kvp.Key.Name, kvp => kvp.Select(p => p.Name))
});