Posting to Consecutive Commands to Solr - Data Missing from 1st Call - perl

I am sending the following commands with
my $url = "http://xxxxx/solr/inventory/update?commitWithin=1000";
I am using perl to send to a solr setup on another server.
Please excuse the formatting. I really did try.
Thanks
Mike
RESULTING DATA - The data from the first command is not here. All subsequent calls are.
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"*:*",
"fq":"id:3-159682",
"_":"1529984183431"
}
},
"response":{
"numFound":1,
"start":0,
"docs":[
{
"checklist_id":249746,
"brand_s":"Pinnacle",
"featured":"",
"sf_set_sort":"Baseball1992Pinnacle",
"sf_set_sort_s":"Baseball1992Pinnacle",
"sport_s":"Baseball",
"cardnumber":"308",
"issue_s":"",
"id":"3-159682",
"year_s":"1992",
"team":"Los Angeles Dodgers",
"set_name_s":"",
"has_image":1,
"amazon_sku":"159682",
"amazon_sync":1,
"sf_id":378827,
"sf_ending_time":2222222222,
"sf_sort_id":199230875,
"sf_listing_type":"buy",
"shopify_id":"1302493397094",
"_version_":1604345060355211264
}
]
}
}
COMMANDS AND RESPONSES
[
{
"inv_location":"",
"ean":"",
"site_id":"3",
"category_id":[
"1",
"55",
"2162220",
"2715086",
"306",
"2352370",
"2413461"
],
"cp_id":"159682",
"isbn":"",
"id":"3-159682",
"consigner":"",
"upc_code":"0",
"quantity":"1",
"created_date":"2018-06-26T10:17:55Z",
"mpn":"",
"description":"",
"inv_num":"",
"cp_listing_type":"1",
"price":"0.69",
"title":"1992 Pinnacle #308 Darryl Strawberry NM-MT ",
"live_status":""
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"checklist_id":"249746",
"brand_s":"Pinnacle",
"featured":"",
"sf_set_sort":"Baseball1992Pinnacle",
"sport_s":"Baseball",
"cardnumber":"308",
"issue_s":"",
"id":"3-159682",
"year_s":"1992",
"team":"Los Angeles Dodgers",
"set_name_s":""
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"has_image":{
"set":"1"
},
"id":"3-159682"
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"amazon_sku":{
"set":"159682"
},
"amazon_sync":{
"set":"1"
},
"id":"3-159682"
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"sf_id":{
"set":"378827"
},
"sf_ending_time":{
"set":"2222222222"
},
"sf_sort_id":{
"set":"199230875"
},
"id":"3-159682",
"sf_listing_type":{
"set":"buy"
}
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"id":"3-159682",
"shopify_id":{
"set":"1302493397094"
}
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}

The two documents you submit have the same id. The second document overwrites the first one, since the id has to uniquely identify a document. If the id doesn't do that, change which field is defined as a uniqueKey, or use a UUID generator to get a new id each time the document is submitted. The latter will cause issues if you're trying to do updates without having the new uuid readily available, tho.
Another solution would be to prefix the id with the document type, or (depending on your use case) merge it into a single document before indexing.

The answer to my problem was that the 1st and 2nd commands were full updates, and the rest were partial updates (using "set"
I change the second command to a format like this
[{"checklist_id":{"set":"249725"},"brand_s":{"set":"Pinnacle"},"featured":{"set":""},"sf_set_sort":{"set":"Baseball1992Pinnacle"},"sport_s":{"set":"Baseball"},"cardnumber":{"set":"287"},"issue_s":{"set":""},"id":"3-159694","year_s":{"set":"1992"},"team":{"set":"Milwaukee Brewers"},"set_name_s":{"set":""}}]
And all was right with the code, no lionger overwriting the first query.
Maybe this will help someone else!
Thanks
Mike

Related

Query finding a group whose members match exactly with a given string

First of all, my English is not good, I apologize if I am rude.
I need your help with a query. I tried many things but failed.
I’ve written a little database example below.
I want to find the group whose members match exactly with the given username array.
groups.findOne( { 'members.username': { $in: [ 'john22', 'david7' ] } } )
As a result, the first group is returning. But there is another user in the first group. Whereas the array matches exactly with the second group. How should I write the query for exact match. Please help me.
Thank you
{ groups: [
{
id:1,
name:"aliens",
members:[
{
username:"john22",
joineddate:222222222,
removed: false
},
{
username:"david7",
joineddate:3333333333,
removed: false
},
{
username:"william4",
joineddate:444444444444,
removed: false
},
]
},
{
id:2,
name:"stars",
members:[
{
username:"john22",
joineddate:111111111111,
removed: false
},
{
username:"david7",
joineddate:111111111111,
removed: false
},
]
}
] }

MongoDB $addToSet inside another $addToSet

I am trying to achieve the below response from query.
{
users:[
user:{
name:"",
email:[
]
},
....user2
]
}
I have something similar to below.
I will put this way. One user can have n number of devices. He may have more than one email per device. I want to group at devices. And user field should have common information for that device. As we think, name will always be same. Device specific attributes also needed like whatKindOfDevice, howManyIssuesAreThereInThatDevice, howManyCanBeAddressedByUpgrade, howManyAreRare,etc.. along with this i need to get all the emails used in that device like owner,user,associate - all emails put.
Think my document Id is not associated with single user, single device. One user can have any number of devices. One device can have n Number of documents.
$group:{
"_id":"user_device_id",
"user": {
"$addToSet": {
"name":"$name",
"deviceName":"$deviceName",
"email": {"$addToSet":{}} //Something Similar I am expecting
}
}
}
If I add email outer user it works - but it affects the response format required.
Is it possible or any other way to get it the similar response through the query?
Let's assume one user can have more than one document. In each doc, there could be same or duplicate email IDs. I am trying to get that together.
Please advise.
Sample Doc:
{
_id:ObjectId,
name:"user1",
email:"a#yahoo.com"
},
{
_id:ObjectId,
name:"user1",
email:"a#device.com"
},
..user2Doc
..user1Doc with another category, duplicate email i.e a#yahoo.com
..user2Doc with new email
..
Well, it seems like you want to get all the email for the particular user and then group all the users.
So, to achieve that you have to do consecutive grouping stages.
I used the below documents:
[
{
name:"user1",
email:"a#yahoo.com"
},
{
name:"user1",
email:"a#device.com"
},
{
name:"user2",
email:"b#yahoo.com"
},
{
name:"user1",
email:"c#device.com"
}
]
Here is the query:
db.collection.aggregate([
{
$group:{
"_id":"$name",
"emails":{
$addToSet:"$email"
},
"name":{
$first:"$name"
}
}
},
{
$group:{
"_id":null,
"users":{
$addToSet:{
"name":"$name",
"email":"$emails"
}
}
}
},
{
$project:{
"_id":0
}
}
]).pretty()
Output:
{
"users" : [
{
"name" : "user1",
"email" : [
"c#device.com",
"a#device.com",
"a#yahoo.com"
]
},
{
"name" : "user2",
"email" : [
"b#yahoo.com"
]
}
]
}
For mare about $group refer here.
Hope this will help :)

find only matching elements from a JSON array in Ne DB/react native-local-mongo/mongo db

i am trying to read only matching keys from a JSON with a nested array.
here is my JSON.
data:[
{
"locale":"en_US",
"pages":[
{
pageName:"testpage1",
"messages":{
"m1":"v1",
"m2":""
}
},
{
pageName:"testpage2",
"messages":{
"m1":"v3",
"m2":"v4"
}
}
]
},
{
"locale":"in_L1",
"pages":[
{
pageName:"testpage1",
"messages":{
"m1":"i1",
"m2":"i2"
}
},
{
pageName:"testpage2",
"messages":{
"m1":"i3",
"m2":"i4"
}
}
]
}
]
I am trying the below query:
db['collec1'].find({locale:"en_US", pages:{$elemMatch:{pageName:"testpage1"},
{locale:1,"pages.pageName":1}}})
also tried,
db['collec1'].find({locale:"en_US", "pages.pageName":"testpage1"},{locale:1,"pages.pageName":1}}})
both are returning, the all the elements in the array as below:
[
{
"locale":"en_US",
pages:{pageName:["testpage1", "testpage2"]}]
}
]
Expected output is:
[
{
"locale":"en_US",
pages:{pageName:["testpage1"]}]
}
]
can someone help me where i am doing it wrong and what needs to be changed?
i am using this library which is a clone of mongodb called Ne DB.
db['collec1'].find({locale:"en_US", pages:{$elemMatch:{pageName:"testpage1"}}).
try with an empty project in the find method {} and then try like
the above example

Graphcool (Graphql): how to make exact filter in array of Id

I have chat model
type Chat #model {
id: ID! #isUnique
name: String
messages: [Message!]! #relation(name: "ChatMessages")
users: [User!]! #relation(name: "ChatUser")
createdAt: DateTime!
}
And I've created a chat with two people ["123", "234"] and all was ok. But when I want to create a new group chat with three people where two of them already have a chat with themselves ["123", "234", "345"]
I make a query for check is this chat exist with three people
query {
allChats(filter:{
users_every: {
id_in: ["123", "234", "345"]
}
}
){
id
users{
id
}
}
}
and I got response that this chat already exist, but here I have just two users not all of them
{
"data": {
"allChats": [
{
"id": "cjgxuub2351uj0187qeil548m",
"users": [
{
"id": "123"
},
{
"id": "234"
}
]
}
]
}
}
The every suffix in users_every makes sure it matches all chats that have only users with ids from the id_in array. That's why you're getting chats with fewer users, because subsets match.
The solution:
query {
allChats(
filter: {
AND: [
{ users_some: { id: "123" } },
{ users_some: { id: "234" } },
{ users_some: { id: "345" } },
{ users_every: { id_not_in: ["678", "910", ...] } }
]
}
) {
id
users {
id
}
}
}
The users_some filters make sure to select chats that have at least all those 3 users attached. The users_every - id_not_in is needed to exclude the chats with those three plus any other users.
In all honesty this seems a bit convoluted to me, and probably kinda unfeasible for an app with a large number of users. I'd love for graph.cool to implement some easier solution in the api.
As an alternative, you could just omit the users_every filter, and then, in your client app, pick from the results only the chat that has exactly 3 users.

How do I use the Echonest API Start Parameter?

I am following the examples located on the following page:
http://developer.echonest.com/docs/v4/genre.html#artists
I'd like to offset the results from a search for artists by genre. The example they provide on the page listed "results" and "start". I assume "start" is the offset. The example query they provide is:
http://developer.echonest.com/api/v4/genre/artists?api_key=JEXNQ223JXCCQEINO&format=json&results=5&start=0&bucket=hotttnesss&name=jazz
But I get any error stating the "start" is an invalid parameter. Has anyone been able to use the "start" parameter with success?
This looks like a bug in their example. If you read the documentation, "start" and "results" are not valid for the genre/artists endpoint. Changing the example to remove these to parameters works.
Calling:
http://developer.echonest.com/api/v4/genre/artists?api_key=*********&format=json&bucket=hotttnesss&name=jazz
(replace the *** with your Key)
Yields:
{
"response":{
"status":{
"version":"4.2",
"code":0,
"message":"Success"
},
"artists":[
{
"name":"John Coltrane",
"hotttnesss":0.588225,
"id":"ARIOZCU1187FB3A3DC"
},
{
"name":"Thelonious Monk",
"hotttnesss":0.649332,
"id":"AR9PLH11187FB58A87"
},
{
"name":"Miles Davis",
"hotttnesss":0.697302,
"id":"AR7RTGF1187FB38793"
},
{
"name":"Miles Davis Quintet",
"hotttnesss":0.489603,
"id":"AR5DF1C1187FB4E94C"
},
{
"name":"Cannonball Adderley",
"hotttnesss":0.560071,
"id":"ARQ5TM41187FB3E97D"
},
{
"name":"Wayne Shorter",
"hotttnesss":0.548165,
"id":"ARO3CKW1187B9905A8"
},
{
"name":"Wynton Marsalis",
"hotttnesss":0.566708,
"id":"ARV3VEI1187B9AD5C9"
},
{
"name":"Sonny Rollins",
"hotttnesss":0.577764,
"id":"AR6Q4T91187B995616"
},
{
"name":"The Dave Brubeck Quartet",
"hotttnesss":0.570099,
"id":"ARLKR161187FB50694"
},
{
"name":"Kenny Burrell",
"hotttnesss":0.543388,
"id":"ARQYH461187FB3E975"
},
{
"name":"Stan Getz",
"hotttnesss":0.559735,
"id":"ARMGQLA1187B9AEBF8"
},
{
"name":"Dizzy Gillespie",
"hotttnesss":0.561122,
"id":"ARXA17J1187FB3B507"
},
{
"name":"Yusef Lateef",
"hotttnesss":0.513261,
"id":"ART95BW1187FB3AF79"
},
{
"name":"Bill Evans",
"hotttnesss":0.581819,
"id":"ARTLL9E1187FB4436F"
},
{
"name":"Freddie Hubbard",
"hotttnesss":0.524227,
"id":"ARU1K2U1187FB48529"
}
]
}
}
As far as I can tell, there isn't a way to page through the artists associated with a genre...