I saw that you can atomically increment a value in IronCache, but what if you have many IronWorkers trying to put a value into a single cache key? Would it be better to put those value updates on a Message Queue in order to synchronize updates to the cache or is there another idiomatic way?
There is currently no idiomatic way to update a non-integer Cache item without provoking the race condition gods. There are a lot of different hacks to get around the limitation, but your MQ solution (assuming only one worker is writing the changes) is probably your best bet.
We are aware of the shortcoming, and we're working on a fix, but we have nothing to announce at this time.
One way to do this would be to split up your value into multiple cache entries. Say you have your json hierarchy:
{
"x": "y",
"sub1": {
"a": "b"
},
"sub2": {
"c": "d"
}
}
Change it to:
{
"x": "y",
"sub1": "cache_key_a",
"sub2": "cache_key_b"
}
Then in cache_key_a:
{
"a": "b"
}
And do the same for cache_key_b and so on. Would that solve your problem?
Related
Ok, new to RESTapi so not sure if I am using the correct terminology for what I want to ask so bear with me. I believe what I am asking about is nested resources in a service but I want to ask specifically about using it for separating a blob of "closely related" content. It may be easier to provide an example. Let's say I have the following service that could output the following:
/Policy
"data": [ {
"name": "PolicyName1",
"description": "",
"size": 25000,
.... (bunch of other fields)
"specialEnablement": true,
“specialEnablementOptions”: { <-- options below valid only if specialEnablement is true
“optionType”: “TypeII”,
“optionFlagA”: false,
“optionFlagB”: true,
“optionFlagC”: false,
...(bunch of other options here)
}
},
{ . . . }],
The specialEnablementOptions are only used if specialEnablement is 'true'. It is all part of this /Policy service so has no primary key other than the policy "name" (and doesnt make sense to have to generate one) so does not fall under some of the other questions I have been reading about nested resources.
It does make it more readable to separate this set of information since there are 12 or so options but, this is REST so, maybe human readability does not weigh heavily here.
I am being told that, if we do it this way, it makes it more complex to work with during POST/PUT/PATCH commands. Specifically, it is being said in my group that if we do this, we should require two calls....one that creates the policy main information then the user must call a second time to PATCH the specialEnablementOptions (assuming specialEnablement is true). This seems kludgy to me.
I am looking for expert advise on what the best practice is.
My questions:
Does having the specialEnablementOptions nested in this way cause a
lot of complexity. Seems to me that either way we have to verify
that the settings are valid?
Does having the specialEnablementOptions nested in this way require
two calls? In other words, can a user not do a POST/PATCH/PUT for
all the fields including those in the specialEnablementOptions in
one call? We are planning to provide a way for the user to do a
PATCH of just the specialEnablementOptions options without changing
any of the first level for ease of use but is there something that
prevents them from creating or modifying all settings in one call?
Another option is to just get rid of the nested
specialEnablementOptions and put everything at the same level. I
dont have a problem with this but wasn't sure if this was just being
lazy. I dont mind doing this if the consensus is it is the best way
to do it....but I also have a second example that is similar to this
scenario but is a bit more complex where putting everything under the parent level is not really optimal (I will show in the next example)
So, my second example is as follows:
/anotherPolicy
"data": [ {
"name": "APolicyName1",
"description": "",
"count": 123,
"lastModified": "2022-05-17-20.37.27.000000",
[{
"ownerId": 1
"ownerCount": 1818181
"specialFlags": 'ABA'
},
{ . . . }]
},
{ . . . }],
The above 'count' is the total number associated to that policy and then there is a nested resource by owner where the count by owner can be seen..plus maybe other information specific to that owner. The SUM(ownerCount) would equal "count" above it. Does this scenario change any of the answers to the questions above?
I appreciate your help. I found a ton of information and reference on when to use or not use nested endpoints but all the examples seem to orient around subjects that seem like they could easily be separated into two resource...for instance whether to nest /employees under /departments or /comments under /posts. Also, they didn't deal with the complexities of having nested endpoints vs avoiding them. And last, if using nesting is unnecessary as a readability standpoint.
given the following set of selects :-
select JSON_MODIFY('{"A": {"X": 1}}', 'lax $.A.B',JSON_QUERY('{"K":1}'))
select JSON_MODIFY('{"A": {"X": 1, "B":{"E":1 }}}', 'lax $.A.B',JSON_QUERY('{"K":1}'))
select JSON_MODIFY('{"D": { "J": 1 }}', 'lax $.A.B',JSON_QUERY('{"K":1}'))
select JSON_MODIFY('{}', 'lax $.A.B',JSON_QUERY('{"K":1}'))
How do you modify the original json Object such that it adds or merges in
{"A": {"B": {"K" : 1}}}
with the existing JSON which may or may not have any or all of the path already existing.
from my examples :-
first one is correct
second one incorrectly overwrites the B tag
third one doesn't add anything
fourth doesn't add anything
Note: these are just examples, "K" in reality could be a whole other chunk of JSON that I want to merge into the "B" object without affecting other existing things on the "B" object.
Now I can imagine you could use "JSON_PATH_EXISTS" in a bunch of IIF clauses, but as the nesting gets deeper, this is going to be painful. I'm just wondering if I've missed some easy way to do this, or there is whole other approach to do this?
I have a ton of documents (around 10 million) and I need to change their field type. The usual forEach function (just looping through every value) seems to take forever and is clearly not viable in the timeframe I have (it basically took all night for one out of four updates)
I've heard that bulkwrites may be able to do it but I'm getting mixed messages. I saw a confusing answer on this site, for example, says that there's no written function to do it (you would have to do some workaround), others say that it can be done with updates in Python, using pymongo.
I was wondering if there was a quicker way to mass changes of field type (string->double, string -> int) using python? I can also work from the console but I find even less solutions there.
Thanks
You can try using aggregation query in the mongo shell
Something like
db.your_collection.aggregate([
{
$addFields: {
field1: {
$convert: {
input: "$field1",
to: "string"
}
}
}
},
{ $out: "your_collection" }
])
More info here https://docs.mongodb.com/manual/reference/operator/aggregation/convert/
In My social network I want to get the feed for member A , member A is following lets say 20 category/member.
when a category/member(followed by member A) do an activity it is inserted into a collection called recent_activity :
{
"content_id": "6", // content id member A is following
"content_type_id": "6",// content type (category , other member)
"social_network_id": "2", // the action category did (add/like/follow)
"member_id": "51758", //Member A
"date_added": ISODate("2014-03-23T11:37:03.0Z"),
"platform_id": NumberInt(2),
"_id": ObjectId("532ec75f6b1f76fa2d8b457b"),
"_type": {
"0": "Altibbi_Mongo_RecentActivity"
}
}
I want when member A login into the system to get last 10 activities for the categories/member
my problem :
How to get Only 10 activities for all categories/members.
It is better to do it in one query or to do a for loop.
For this use case, I'd suggest to invert the logic and keep a separate object of the last 10 activities for member A that is kept up-to-date all the time. While that solution is more write-heavy, it makes reading trivially simple and it can be extended very easily. I'd like to blatantly advertise a blog post I wrote a while ago about news feeds with mongodb which outlines this approach.
This 'fan-out' approach might seem overly complex at first, but when you think about importance filtering / ranking (a la facebook), push messages for particularly important events (facebook, twitter) or regular digest emails (practically all), you will get one location in your code to perform all this logic.
I think I commented that T'm not really seeing the selection criteria. So if you are "outside" of a single collection, then you have problems. But if your indicated fields are the things you want to "filter" by, then just do this:
db.collection.find({
"social_network_id": "2",
"content_type_id": "6",
"content_id": "6",
"member_id": { "$ne": "51758" }
})
.sort({ "$natural": -1 })
.limit(10);
So what does that do? You match the various conditions in the data to do the "category match" (if I understood what was meant), then you make sure you are not matching entries by the same member.
The last parts do the "natural" sort. This is important because the ObjectId is monotinic, or math speak for "ever increasing". This means the "newest" entries are always the "highest" value. So descending order is "latest" to "oldest".
And the very final part is a basic "limit". So just return the last 10 entries.
As long as you can "filter" within the same collection in whatever way you want, then this should be fine.
For learning purposes, I am building an app with basic functionality. This is the first time I am using MongoDB and I am wondering whether my data model makes sense or whether I should model it differently.
Basically, this is what should happen:
I have user accounts, they can create "tasks" and execute them. Each time they execute a task, the timestamp should be logged. They can then view a log (journal) which shows all the dates at which they executed a particular task.
My data model looks like this. Does this make sense?
I am especially wondering about the array of dates - is this the right way to go? (the log-Array might become quite long)
{
_id: "51d951922cf9f30200000004",
user: "jon doe",
password: "xxxxxxxxxx",
tasks: [
{
name: "one task",
log: [
ISODate("2010-10-10T20:55:36Z"),
ISODate("2011-10-10T20:55:36Z"),
ISODate("2012-10-10T20:55:36Z")
]
},
{
name: "another task",
log: [
ISODate("2010-10-10T20:55:36Z"),
ISODate("2011-10-10T20:55:36Z"),
ISODate("2012-10-10T20:55:36Z")
]
},
{
name: "one last task",
log: [
ISODate("2010-10-10T20:55:36Z"),
ISODate("2011-10-10T20:55:36Z"),
ISODate("2012-10-10T20:55:36Z")
]
}
]
}
What you have is probably fine. As you grow the sizes of the arrays you might run into some delays in copying large arrays, but there isn't really a space-saving alternative for the logs. The only thing I can think of is a nested document. That will probably update slightly faster once the size is huge, but it will take up more space and won't be nearly as nice or as readable. Rather than having an array with the logs, you would have a document like
{ 0: ISODate("2010-10-10T20:55:36Z" ,
1: ISODate("2010-10-10T20:55:36Z" ,
2: ISODate("2010-10-10T20:55:36Z" }
etc. This will update faster than creating a whole new (larger) array when a log is added, but also take up more space.
If it helps, MongoDB has a page on data modeling with some links you might find useful. http://docs.mongodb.org/manual/core/data-modeling/