How do I use the Echonest API Start Parameter? - echonest

I am following the examples located on the following page:
http://developer.echonest.com/docs/v4/genre.html#artists
I'd like to offset the results from a search for artists by genre. The example they provide on the page listed "results" and "start". I assume "start" is the offset. The example query they provide is:
http://developer.echonest.com/api/v4/genre/artists?api_key=JEXNQ223JXCCQEINO&format=json&results=5&start=0&bucket=hotttnesss&name=jazz
But I get any error stating the "start" is an invalid parameter. Has anyone been able to use the "start" parameter with success?

This looks like a bug in their example. If you read the documentation, "start" and "results" are not valid for the genre/artists endpoint. Changing the example to remove these to parameters works.
Calling:
http://developer.echonest.com/api/v4/genre/artists?api_key=*********&format=json&bucket=hotttnesss&name=jazz
(replace the *** with your Key)
Yields:
{
"response":{
"status":{
"version":"4.2",
"code":0,
"message":"Success"
},
"artists":[
{
"name":"John Coltrane",
"hotttnesss":0.588225,
"id":"ARIOZCU1187FB3A3DC"
},
{
"name":"Thelonious Monk",
"hotttnesss":0.649332,
"id":"AR9PLH11187FB58A87"
},
{
"name":"Miles Davis",
"hotttnesss":0.697302,
"id":"AR7RTGF1187FB38793"
},
{
"name":"Miles Davis Quintet",
"hotttnesss":0.489603,
"id":"AR5DF1C1187FB4E94C"
},
{
"name":"Cannonball Adderley",
"hotttnesss":0.560071,
"id":"ARQ5TM41187FB3E97D"
},
{
"name":"Wayne Shorter",
"hotttnesss":0.548165,
"id":"ARO3CKW1187B9905A8"
},
{
"name":"Wynton Marsalis",
"hotttnesss":0.566708,
"id":"ARV3VEI1187B9AD5C9"
},
{
"name":"Sonny Rollins",
"hotttnesss":0.577764,
"id":"AR6Q4T91187B995616"
},
{
"name":"The Dave Brubeck Quartet",
"hotttnesss":0.570099,
"id":"ARLKR161187FB50694"
},
{
"name":"Kenny Burrell",
"hotttnesss":0.543388,
"id":"ARQYH461187FB3E975"
},
{
"name":"Stan Getz",
"hotttnesss":0.559735,
"id":"ARMGQLA1187B9AEBF8"
},
{
"name":"Dizzy Gillespie",
"hotttnesss":0.561122,
"id":"ARXA17J1187FB3B507"
},
{
"name":"Yusef Lateef",
"hotttnesss":0.513261,
"id":"ART95BW1187FB3AF79"
},
{
"name":"Bill Evans",
"hotttnesss":0.581819,
"id":"ARTLL9E1187FB4436F"
},
{
"name":"Freddie Hubbard",
"hotttnesss":0.524227,
"id":"ARU1K2U1187FB48529"
}
]
}
}
As far as I can tell, there isn't a way to page through the artists associated with a genre...

Related

How to search in ElasticSearch the most common word of a single field in a single document?

How to search in ElasticSearch the most common word of a single field in a single document? Lets say I have a document that have a field "pdf_content" of type keyword containing:
"good polite nice good polite good"
I would like a return of
{
word: good,
occurences: 3
},
{
word: polite,
occurences: 2
},
{
word: nice,
occurences: 1
},
How is this possible using ElasticSearch 7.15?
I tried this in the Kibana console:
GET /pdf/_search
{
"aggs": {
"pdf_contents": {
"terms": { "field": "pdf_content" }
}
}
}
But it only returns me the list of PDFs i have indexed.
Have you ever tried term_vector?:
Basically, you can do:
Mappings:
{
"mappings": {
"properties": {
"pdf_content": {
"type": "text",
"term_vector": "with_positions_offsets_payloads"
}
}
}
}
with your sample document:
POST /pdf/_doc/1
{
"pdf_content": "good polite nice good polite good"
}
Then you can do:
GET /pdf/_termvectors/1
{
"fields" : ["pdf_content"],
"offsets" : false,
"payloads" : false,
"positions" : false,
"term_statistics" : false,
"field_statistics" : false
}
If you want to see other information, you can set them to true. Set all to false give you what you want.

Index not picked with nested field hierarchy but gets picked in the flatten mode

so i am struggling for 2 weeks on why does not my indexes get picked when i “explain” my queries.
i have this query:
{ “$and”: [
{ "extraProperties.class": "Residential" }, { "extraProperties.type": "Sale" }, { "extraProperties.propertyType": "Condo Apartment" }, { "extraProperties.propertyTypeStyle": "Apartment" } ] }
the above query wont pick this index :
{ “extraProperties.class”:1 , “extraProperties.type” : 1, “extraProperties.propertyType”:1,“extraProperties.propertyTypeStyle”:1}
i have been testing everything these days and finally i decided to flatten the hierarchy and now my query looks like this:
{ “$and”: [
{ “class”: “Residential” }, { “type”: “Sale” }, { “propertyType”: “Condo Apartment” }, { “propertyTypeStyle”: “Apartment” }
] }
now the above query will pick this index :
{ “class”:1 , “type” : 1, “propertyType”:1,“propertyTypeStyle”:1}
could someone explain what the hell is going on there?!?!
explain result:
https://drive.google.com/file/d/1bs_mqO-1FEBHQ_FsBWgP2TQ2jeiQz4q_/view?usp=sharing

Posting to Consecutive Commands to Solr - Data Missing from 1st Call

I am sending the following commands with
my $url = "http://xxxxx/solr/inventory/update?commitWithin=1000";
I am using perl to send to a solr setup on another server.
Please excuse the formatting. I really did try.
Thanks
Mike
RESULTING DATA - The data from the first command is not here. All subsequent calls are.
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"*:*",
"fq":"id:3-159682",
"_":"1529984183431"
}
},
"response":{
"numFound":1,
"start":0,
"docs":[
{
"checklist_id":249746,
"brand_s":"Pinnacle",
"featured":"",
"sf_set_sort":"Baseball1992Pinnacle",
"sf_set_sort_s":"Baseball1992Pinnacle",
"sport_s":"Baseball",
"cardnumber":"308",
"issue_s":"",
"id":"3-159682",
"year_s":"1992",
"team":"Los Angeles Dodgers",
"set_name_s":"",
"has_image":1,
"amazon_sku":"159682",
"amazon_sync":1,
"sf_id":378827,
"sf_ending_time":2222222222,
"sf_sort_id":199230875,
"sf_listing_type":"buy",
"shopify_id":"1302493397094",
"_version_":1604345060355211264
}
]
}
}
COMMANDS AND RESPONSES
[
{
"inv_location":"",
"ean":"",
"site_id":"3",
"category_id":[
"1",
"55",
"2162220",
"2715086",
"306",
"2352370",
"2413461"
],
"cp_id":"159682",
"isbn":"",
"id":"3-159682",
"consigner":"",
"upc_code":"0",
"quantity":"1",
"created_date":"2018-06-26T10:17:55Z",
"mpn":"",
"description":"",
"inv_num":"",
"cp_listing_type":"1",
"price":"0.69",
"title":"1992 Pinnacle #308 Darryl Strawberry NM-MT ",
"live_status":""
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"checklist_id":"249746",
"brand_s":"Pinnacle",
"featured":"",
"sf_set_sort":"Baseball1992Pinnacle",
"sport_s":"Baseball",
"cardnumber":"308",
"issue_s":"",
"id":"3-159682",
"year_s":"1992",
"team":"Los Angeles Dodgers",
"set_name_s":""
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"has_image":{
"set":"1"
},
"id":"3-159682"
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"amazon_sku":{
"set":"159682"
},
"amazon_sync":{
"set":"1"
},
"id":"3-159682"
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"sf_id":{
"set":"378827"
},
"sf_ending_time":{
"set":"2222222222"
},
"sf_sort_id":{
"set":"199230875"
},
"id":"3-159682",
"sf_listing_type":{
"set":"buy"
}
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
[
{
"id":"3-159682",
"shopify_id":{
"set":"1302493397094"
}
}
]
Success: {
"responseHeader":{
"status":0,
"QTime":1
}
}
The two documents you submit have the same id. The second document overwrites the first one, since the id has to uniquely identify a document. If the id doesn't do that, change which field is defined as a uniqueKey, or use a UUID generator to get a new id each time the document is submitted. The latter will cause issues if you're trying to do updates without having the new uuid readily available, tho.
Another solution would be to prefix the id with the document type, or (depending on your use case) merge it into a single document before indexing.
The answer to my problem was that the 1st and 2nd commands were full updates, and the rest were partial updates (using "set"
I change the second command to a format like this
[{"checklist_id":{"set":"249725"},"brand_s":{"set":"Pinnacle"},"featured":{"set":""},"sf_set_sort":{"set":"Baseball1992Pinnacle"},"sport_s":{"set":"Baseball"},"cardnumber":{"set":"287"},"issue_s":{"set":""},"id":"3-159694","year_s":{"set":"1992"},"team":{"set":"Milwaukee Brewers"},"set_name_s":{"set":""}}]
And all was right with the code, no lionger overwriting the first query.
Maybe this will help someone else!
Thanks
Mike

Tricky MongoDB search challenge

I have a tricky mongoDB problem that I have never encountered.
The Documents:
The documents in my collection have a search object containing named keys and array values. The keys are named after one of eight categorys and the corresponding value is an array containing items from that category.
{
_id: "bRtjhGNQ3eNqTiKWa",
/* */
search :{
usage: ["accounting"],
test: ["knowledgetest", "feedback"]
},
test: {
type:"list",
vals: [
{name:'knowledgetest', showName: 'Wissenstest'},
{name:'feedback', showName: '360 Feedback'},
]
},
usage: {
type:"list",
vals: [
{name:'accounting', showName: 'Accounting'},
]
}
},
{
_id: "7bgvegeKZNXkKzuXs",
/* */
search :{
usage: ["recruiting"],
test: ["intelligence", "feedback"]
},
test: {
type:"list",
vals: [
{name:'intelligence', showName: 'Intelligenztest'},
{name:'feedback', showName: '360 Feedback'},
]
},
usage: {
type:"list",
vals: [
{name:'recruiting', showName: 'Recruiting'},
]
}
},
The Query
The query is an object containing the same category - keys and array - values.
{
usage: ["accounting", "assessment"],
test : ["feedback"]
}
The desired outcome
If the query is empty, I want all documents.
If the query has one category and any number of items, I want all the documents that have all of the items in the specified category.
If the query has more then one category, I want all the documents that have all of the items in all of the specified categorys.
My tries
I tried all kinds of variations of:
XX.find({
'search': {
"$elemMatch": {
'tool': {
"$in" : ['feedback']
}
}
}
No success.
EDIT
Tried: 'search.test': {$all: (query.test ? query.test : [])} which gives me no results if I have nothing selected; the right documents when I am only looking inside the test category; and nothing when I additionally look inside the usage category.
This is at the heart of my app, thus I historically put up a bounty.
let tools = []
const search = {}
for (var q in query) {
if (query.hasOwnProperty(q)) {
if (query[q]) {
search['search.'+q] = {$all: query[q] }
}
}
}
if (Object.keys(query).length > 0) {
tools = ToolsCollection.find(search).fetch()
} else {
tools = ToolsCollection.find({}).fetch()
}
Works like a charm
What I already hinted at in the comment: your document structure does not support efficient and simple searching. I can only guess the reason, but I suspect that you stick to some relational ideas like "schemas" or "normalization" which just don't make sense for a document database.
Without digging deeper into the problem of modeling, I could imagine something like this for your case:
{
_id: "bRtjhGNQ3eNqTiKWa",
/* */
search :{
usage: ["accounting"],
test: ["knowledgetest", "feedback"]
},
test: {
"knowledgetest" : {
"showName": "Wissenstest"
},
"feedback" : {
"showName": "360 Feedback"
}
},
usage: {
"accounting" : {
"values" : [ "knowledgetest", "feedback" ],
"showName" : "Accounting"
}
}
},
{
_id: "7bgvegeKZNXkKzuXs",
/* */
search : {
usage: ["recruiting"],
test: ["intelligence", "feedback"]
},
test: {
"intelligence" : {
showName: 'Intelligenztest'
},
"feedback" : {
showName: '360 Feedback'
}
},
usage: {
"recruiting" : {
"values" : [ "intelligence", "feedback" ],
"showName" : "Recruiting"
}
}
}
Then, a search for "knowledgetest" and "feedback" in "accounting" would be a simple
{ "usage.accounting.values" : { $all : [ "knowledgetest", "feedback"] } }
which can easily be used multiple times in an and condition:
{
{ "usage.accounting.values" : { $all : [ "knowledgetest", "feedback"] } },
{ "usage.anothercategory.values" : { $all [ "knowledgetest", "assessment" ] } }
}
Even the zero-times-case matches your search requirements, because an and-filter with none of these criteria yields {} which is the find-everything filter expression.
Once more, to make it absolutely clear: when using mongo, forget everything you know as "best practice" from the relational world. What you need to consider is: what are your queries, and how can my document model support these queries in an ideal way.

Exclude nested subdocuments in Mongodb without arrays

Here is how data are inserted in a "Products" MongoDB collection (using Meteor):
Products.insert(
{
productOne:
{
publicData:
{
pricePerUnit : 1,
label : "The first product"
},
privateData:
{
test1: "xxxxx",
test2: "xxxxx"
}
},
productTwo:
{
publicData:
{
pricePerUnit : 2,
label : "The second product"
},
privateData:
{
test1: "yyyyy",
test2: "yyyyy"
}
}
}
);
I would like to retrieve all the products, but without the "privateData" subdocuments, to get this:
{
productOne:
{
publicData:
{
pricePerUnit : 1,
label : "The first product"
}
},
productTwo:
{
publicData:
{
pricePerUnit : 2,
label : "The second product"
}
}
}
I tried several things with "$elemMatch" but honnestly I didn't succeed in anything, I have trouble understanding how I am even supposed to do that.
Would anyone have a suggestion? Any help would be greatly appreciated.
Thanks!
Your Query would be something Similar to this
Products.find({},{
fields: {
privateData: 0
}
}
privateData:0 will make sure that the field is omitted.
please Refer https://docs.mongodb.org/manual/tutorial/project-fields-from-query-results/ for more info
if you can use the aggregation framework you can use the $project operator:
db.<colletion_name>.aggregate( { $project: { publicData: 1} } );
And you will get back all of your documents with only the publicData field