I have a jsonpath respone like:
{
"pageable": {
"currentPage": 1,
"totalPages": 1,
"pageSize": 20,
"last": true,
"first": true
},
"sort": {
"orders": [],
"sorted": false,
"unsorted": true,
"empty": true
},
"totalElements": 6,
"data": [
{
"id": 1,
"roleName": "test1",
"userCount": 5
},
{
"id": 2,
"roleName": "test2",
"userCount": 5
}
]
}
I want to check that the data with id == 1 exist.
I tried some combinations like:
checks ::= jsonPath("$..data.*[?(#.id=='1')]").exists
checks ::= jsonPath("$.data.*[?(#.id=='1')]").exists
checks ::= jsonPath("$..data[?(#.id=='1')]..id").exists
checks ::= jsonPath("$..data[?(#.id=='1')].id").exists
checks ::= jsonPath("$.data[?(#.id=='1')].id").exists
But no path works, I'm getting error:
jsonPath($..data..*[?(#.id=='1')]).find.exists, found nothing
Any idea?
Instead of trying to use JsonPath whose syntax is unclear and lacks a standard, you should try JMESPath instead, that Gatling supports as well.
jmesPath("data[?id == `1`]")
Related
The problem I am facing is that I want to develop an autocomplete search bar using Mean Stack like the one in this site, but when I type, for example, 'ag' it's not returning the right location that should be 'Aguascalientes'.
I have two different search indexes set up and a different query for each.
First Index:
{
"mappings": {
"dynamic": false,
"fields": {
"name": {
"foldDiacritics": false,
"maxGrams": 7,
"minGrams": 3,
"tokenization": "edgeGram",
"type": "autocomplete"
},
"searchName": {
"foldDiacritics": false,
"maxGrams": 7,
"minGrams": 3,
"tokenization": "edgeGram",
"type": "autocomplete"
}
}
}
}
First Query:
[
{
$search: {
index: "autocomplete2",
compound: {
must: [
{
text: {
query: search,
path: "searchName",
fuzzy: {
maxEdits: 2,
},
},
},
],
},
},
},
{
$limit: 10,
},
]
The first ones are not returning any document at all. But the second example is:
{
"mappings": {
"dynamic": false,
"fields": {
"name": {
"analyzer": "lucene.standard",
"type": "string"
},
"searchName": {
"analyzer": "lucene.standard",
"type": "string"
}
}
}
}
Query:
[
{
$search: {
index: 'default',
compound: {
must: [
{
text: {
query: search,
path: 'name',
fuzzy: {
maxEdits: 1,
},
},
},
{
text: {
query: search,
path: 'searchName',
fuzzy: {
maxEdits: 1,
},
},
},
],
},
},
},
{
$limit: 5,
},
]
The second example is only returning documents if the search term 'aguascalient' but is not returning any document if the search term is shorter like the site. Maybe it has something to do with the fuzzy edits but if I set it up to greater than 2 I get an error.
Also the order is not right, it returns first the CITY and second the STATE but I need the STATE first because the search term is more similar than the city. Let me explain, search field for STATE is only 'Aguascalientes' but search field cities is 'Aguascalientes Aguascalientes' so I don't know why is not working properly. Maybe in that case I should give weights accordingly but I'm not sure if it's the right approach to solve this.
My data structure:
{
"_id": "638d0ffc34ad076c6bd12cb6",
"depth": 2,
"label": "CITY",
"location_id": "V1-C-247",
"name": "Aguascalientes",
"parent": "Aguascalientes",
"fullName": "Aguascalientes, Aguascalientes",
"parentId": "V1-B-61",
"searchName": "Aguascalientes Aguascalientes",
}
{
"_id": "638d0ffc34ad076c6bd12cb6",
"depth": 1,
"label": "STATE",
"location_id": "V1-C-248",
"name": "Aguascalientes",
"parent": null,
"fullName": "Aguascalientes",
"parentId": null,
"searchName": "Aguascalientes",
}
For the first index + query setup:
First, you are indexing the name field but are not searching on it. I will remove it from the code snippets for readability, but you can add it back to your index definition if you find you need to search on it.
There are two problems with the this index + query setup if you want to return results with a query for "ag". You have searchName defined as a field mapping of type autocomplete, but you also need to use the autocomplete operator in your query:
[
{
$search: {
index: "autocomplete2",
compound: {
must: [
{
autocomplete: {
query: search,
path: "searchName",
},
},
],
},
},
},
{
$limit: 10,
},
]
Second, in your index definition field mapping for searchName, you have minGram set to 3 and maxGram set to 7. Based on the documentation for the autocomplete field mapping, this means that your data will be tokenized into sequences of character lengths between 3 to 7, using the selected tokenization strategy. Since you have selected edgeGram, the tokens generated by the text "Aguascalientes" will be tokenized starting from the left edge, resulting in tokens "agu", "agua", "aguas", "aguasc", "aguasca". Since the search term "ag" does not match any of the tokens, nothing is returned. So, you must change the minGram to 2 to get the token "ag":
{
"mappings": {
"dynamic": false,
"fields": {
"searchName": {
"foldDiacritics": false,
"maxGrams": 7,
"minGrams": 2,
"tokenization": "edgeGram",
"type": "autocomplete"
}
}
}
}
Finally, if you want the document with an exact match to return over a partial match, ie. "Aguascalientes" should return before "Aguascalientes Aguascalientes", you need to implement exact matching. Here is a MongoDB blog post outlining a few options.
One option that I tried: In the index, use a keyword analyzer on the "searchName" field typed as a string data type. In the query, use the text operator nested in a should clause so that exact matches will return higher than other results.
Index:
{
"mappings": {
"dynamic": false,
"fields": {
"searchName": [
{
"foldDiacritics": false,
"maxGrams": 7,
"type": "autocomplete"
},
{
"analyzer": "lucene.keyword",
"searchAnalyzer": "lucene.keyword",
"type": "string"
}
]
}
}
}
Query:
[
{
$search: {
compound: {
must: [
{
autocomplete: {
query: search,
path: "searchName"
}
}
],
should:[
{
text: {
query: search,
path: "searchName"
}
}
],
},
},
},
]
I'm very new to mongodb. I'm trying to do an aggregation pipeline for lookup kinda like SQL left join. Given the following document's schema:
Mongo playground: https://mongoplayground.net/p/yAIwH5V2yv8
Characters:
{
"_id": 1,
"account_id": 1,
"world_id": 0,
"name": "hello"
}
Inventories:
{
"_id": 7,
"character_id": 2,
"type": "EQUIPPED"
}
Items:
{
"_id": 1,
"inventory_id": 7
}
I want to query for characters and look up inventories as well as items in inventories. I was able to achieve this however I would like to separate the inventories field in characters result document.
Current result:
{
"_id": 2,
"account_id": 1,
"world_id": 0,
"name": "hello",
"inventories: [
{
"_id": 1,
"character_id": "2",
"type": "EQUIPPED",
"items: [...]
}
]
}
What I want is based on the type of inventory I want it to be a separate field of the resulted character document something like this:
{
"_id": 2,
"account_id": 1,
"world_id": 0,
"name": "hello",
"equippedInventory: {
"_id": 1,
"character_id": "2",
"type": "EQUIPPED",
"items: [...]
},
"equipInventory: {
"_id": 2,
"character_id": "2",
"type": "EQUIP",
"items: [...]
},
}
Also, is my pipeline the best way to achieve this?
I got a collection of 10000 ca. docs, where each doc has the following format:
{
"_id": {
"$oid": "631edc6e207c89b932a70a26"
},
"name": "Ethereum",
"auditInfoList": [
{
"coinId": "1027",
"auditor": "Fairyproof",
"auditStatus": 2,
"reportUrl": "https://www.fairyproof.com/report/Covalent"
}
],
"circulatingSupply": 122335921.0615,
"cmcRank": 2,
"dateAdded": "2015-08-07T00:00:00.000Z",
"id": 1027,
"isActive": 1,
"isAudited": true,
"lastUpdated": 1662969360,
"marketPairCount": 6085,
"quotes": [
{
"name": "USD",
"price": 1737.1982544180462,
"volume24h": 14326453277.535921,
"marketCap": 212521748520.66168,
"percentChange1h": 0.62330307,
"percentChange24h": -1.08847937,
"percentChange7d": 10.96517745,
"lastUpdated": 1662966780,
"percentChange30d": -13.49374496,
"percentChange60d": 58.25153862,
"percentChange90d": 42.27475921,
"fullyDilluttedMarketCap": 212521748520.66,
"marketCapByTotalSupply": 212521748520.66168,
"dominance": 20.0725,
"turnover": 0.0674117,
"ytdPriceChangePercentage": -53.9168
}
],
"selfReportedCirculatingSupply": 0,
"slug": "ethereum",
"symbol": "ETH",
"tags": [
"mineable",
"pow",
"smart-contracts",
"ethereum-ecosystem",
"coinbase-ventures-portfolio",
"three-arrows-capital-portfolio",
"polychain-capital-portfolio",
"binance-labs-portfolio",
"blockchain-capital-portfolio",
"boostvc-portfolio",
"cms-holdings-portfolio",
"dcg-portfolio",
"dragonfly-capital-portfolio",
"electric-capital-portfolio",
"fabric-ventures-portfolio",
"framework-ventures-portfolio",
"hashkey-capital-portfolio",
"kenetic-capital-portfolio",
"huobi-capital-portfolio",
"alameda-research-portfolio",
"a16z-portfolio",
"1confirmation-portfolio",
"winklevoss-capital-portfolio",
"usv-portfolio",
"placeholder-ventures-portfolio",
"pantera-capital-portfolio",
"multicoin-capital-portfolio",
"paradigm-portfolio",
"injective-ecosystem"
],
"totalSupply": 122335921.0615
}
Im pulling updated version of it and, to aviod duplicates, im doing the following by using 'update_one'
for doc in new_doc_list:
CRYPTO_TEMPORARY_LIST.update_one(
{ "name" : doc['name']},
{ "$set": {
"lastUpdated": doc['lastUpdated']
}
},
upsert=True)
The problem is it's too slow.
I'm trying to figure out how to improve speed by using update_many but can't figure out how to set it up.
I Basically want to update every document x name. Completely change the doc and not the "lastUpdated" field would b even better.
Thanks guys <3
I am running a simple query using mongoDB compass using filter and project and I'm having a behaviour I can't explain.
Here is my filter:
{
"$and": [
{
"_id": ObjectId('611ee5ee6b93815ee436969e')
},
{
"type": "article"
}
]
}
I get the following result:
{
"_id": {
"$oid": "611ee5ee6b93815ee436969e"
},
"type": "article",
"history": [],
"liked": [],
"parentId": "61105f00cc11ec10406fd1c4",
"permissionList": [],
"title": "Test",
"wikiId": "610de623fbfa1e58cdba9d2c"
}
As expected I get all the fields, in particular type and wikiId
However if i add the following projection:
{
"_id": 1,
"wikiId": 1,
"parentId": 1,
"title": 1,
"type": 1,
"permissionList": 1,
"liked": 1,
"history": 1
}
I would expect the same result, however i get:
{
"_id": {
"$oid": "611ee5ee6b93815ee436969e"
},
"type": "article",
"history": [],
"liked": [],
"parentId": "61105f00cc11ec10406fd1c4",
"permissionList": [],
"title": "Test"
}
This time i don't have the field wikiId, however it was requested in the projection.
And what's bug me is that if I do this projection instead:
{
"_id": 1,
"wikiId": 1,
"parentId": 1,
"title": 1,
"permissionList": 1,
"liked": 1,
"history": 1
}
Then I got the wikiId field as expected in the result again.
Anyone can provide me an insight of what is going on with those queries and where i'm mistaken.
Edit 1: The fact I want to use a projection is that depending on the type field I can use different document with different field.
In my Java code I'm using #BsonDiscriminator(key = "type") but when I explicitly want a kind of document I'm creating the appropriate projection to be sure. However in this case I just wanted to simplify the issue I'm facing to the simplest.
Thanks
in compass by default filters are having and condition so your filter can be refactored as below:
{ "_id": ObjectId('611ee5ee6b93815ee436969e'), "type": "article" }
And regarding projection you can just simply mention fields to Exclude or include like below:
{
"_id": 1,
"wikiId": 1,
"parentId": 1,
"title": 1,
"type": 1,
"permissionList": 1,
"liked": 1,
"history": 1
}
if you need all fields then there is no need to mention anything by default in compass you get all fields, but lets say you want all but few fields(_id,wikiId) then use below projection:
{
"_id": 0,
"wikiId": 0,
}
Also in compass you can try clicking find again or refresh button to see as sometimes it does not reflects current filter as as we change filter it fetches data so just hit refresh or find and see it should work.
I'm trying to get the GitHub data using Talend big data. The thing is, i have multiple URLs,because used each URL to take some values & stored into single mongoDB. The below order only i'm going to try & get the informations,
https://api.github.com/users/sample/repos
https://api.github.com/repos/sample/awesome-ciandcd/commits
https://api.github.com/repos/sample/awesome-ciandcd/contributors
Each URLs are giving the single JSONArray with multiple data format.Please give some suggestion to do this. I've already tried with sub-jobs component. But not get clear job.
My Output Should be like,
{
"gitname": "sample",
"gitType" : "User",
"password": "password",
"repoCount": 3,
"repositoryDetails": [
{
"repoName": "MergeCheckRepository",
"fileCount": 10,
"branchCount": 6,
"releaseCount": 2,
"commitsCount": 10,
"contributorsCount": 3,
"totalPulls": 1,
"mergeCount": 1,
"totalIssues": 12,
"closedIssueCount": 3,
"watchersCount": 1,
"stargazersCount": 4,
"contributorsDetails": [
{
"login": "sample",
"avatarURL": "https://avatars2.githubusercontent.com/u/30261572?v=4",
"contributions": 3
}
],
"commitDetails": [
{
"name": "sample",
"email": "sampletest#test.com",
"date": "2017-07-20T09:09:09Z"
}
]
},
{
"repoName": "Dashboard",
"filecount": 19,
"branchCount": 4,
"releasecount": 2,
"commitsCount": 5,
"contributorsCount": 3,
"totalPulls": 1,
"totalIssues": 2,
"closedIssueCount": 3,
"watchersCount": 1,
"stargazersCount": 4,
"contributorsDetails": [
{
"login": "sample",
"avatarURL": "https://avatars2.githubusercontent.com/u/30261572?v=4",
"contributions": 3
},
{
"login": "Dashboard",
"avatarURL": "https://avatars2.githubusercontent.com/u/30261572?v=4",
"contributions": 3
}
],
"commitDetails": [
{
"name": "sample",
"email": "sampletest#test.com",
"date": "2017-07-14T09:09:09Z"
},
{
"name": "Dashboard",
"email": "prakash.thangasamy#test.com",
"date": "2017-07-19T09:09:09Z"
},
{
"name": "testrepo",
"email": "test.dashboard#test.com",
"date": "2017-07-20T09:09:09Z"
}
]
}
]
}
How to achieve this one with sub-job? Is there any other way to do this?