Using Fn::Join with a "Ref" - aws-cloudformation

I'm at a roadblock with this one, I've been trying to find a way to simplify my work so as to avoid using a giant if statement block. I came to the idea that I could use s "Ref" With a "Fn::Join" but obviously it doesn't work or at least not the way I'd visually think it would work.
"Type": "AWS::WAFRegional::Rule",
"Properties":{
"Predicates": [{
"DataId":{
"Ref" : {"Fn::Join" : ["", [{"Ref" : "IpSets"}, "IPSet"]]},
So how I visually see this working is that "Ref" : IpSets is a parameter that we set when uploading the file to CloudFormation. So it will pull down that text, and combine it with IpSet to create something like DevIPSet. Since that's a String after a join, it would come out to be "Ref" : "DevIPSet" and then that would pull the set of IP's I've put aside.
This obviously doesn't work and was reaching out to the community if there is a trick I can use. I have looked into Mapping and can't see to make that work with the dataID either. The only real solution I've found is just using Fn::If and make a giant block but that is ugly.

What you are trying to do, I think, is not possible. Because there is a duplication of tag property.

Related

RESTapi nesting endpoint

Ok, new to RESTapi so not sure if I am using the correct terminology for what I want to ask so bear with me. I believe what I am asking about is nested resources in a service but I want to ask specifically about using it for separating a blob of "closely related" content. It may be easier to provide an example. Let's say I have the following service that could output the following:
/Policy
"data": [ {
"name": "PolicyName1",
"description": "",
"size": 25000,
.... (bunch of other fields)
"specialEnablement": true,
“specialEnablementOptions”: { <-- options below valid only if specialEnablement is true
“optionType”: “TypeII”,
“optionFlagA”: false,
“optionFlagB”: true,
“optionFlagC”: false,
...(bunch of other options here)
}
},
{ . . . }],
The specialEnablementOptions are only used if specialEnablement is 'true'. It is all part of this /Policy service so has no primary key other than the policy "name" (and doesnt make sense to have to generate one) so does not fall under some of the other questions I have been reading about nested resources.
It does make it more readable to separate this set of information since there are 12 or so options but, this is REST so, maybe human readability does not weigh heavily here.
I am being told that, if we do it this way, it makes it more complex to work with during POST/PUT/PATCH commands. Specifically, it is being said in my group that if we do this, we should require two calls....one that creates the policy main information then the user must call a second time to PATCH the specialEnablementOptions (assuming specialEnablement is true). This seems kludgy to me.
I am looking for expert advise on what the best practice is.
My questions:
Does having the specialEnablementOptions nested in this way cause a
lot of complexity. Seems to me that either way we have to verify
that the settings are valid?
Does having the specialEnablementOptions nested in this way require
two calls? In other words, can a user not do a POST/PATCH/PUT for
all the fields including those in the specialEnablementOptions in
one call? We are planning to provide a way for the user to do a
PATCH of just the specialEnablementOptions options without changing
any of the first level for ease of use but is there something that
prevents them from creating or modifying all settings in one call?
Another option is to just get rid of the nested
specialEnablementOptions and put everything at the same level. I
dont have a problem with this but wasn't sure if this was just being
lazy. I dont mind doing this if the consensus is it is the best way
to do it....but I also have a second example that is similar to this
scenario but is a bit more complex where putting everything under the parent level is not really optimal (I will show in the next example)
So, my second example is as follows:
/anotherPolicy
"data": [ {
"name": "APolicyName1",
"description": "",
"count": 123,
"lastModified": "2022-05-17-20.37.27.000000",
[{
"ownerId": 1
"ownerCount": 1818181
"specialFlags": 'ABA'
},
{ . . . }]
},
{ . . . }],
The above 'count' is the total number associated to that policy and then there is a nested resource by owner where the count by owner can be seen..plus maybe other information specific to that owner. The SUM(ownerCount) would equal "count" above it. Does this scenario change any of the answers to the questions above?
I appreciate your help. I found a ton of information and reference on when to use or not use nested endpoints but all the examples seem to orient around subjects that seem like they could easily be separated into two resource...for instance whether to nest /employees under /departments or /comments under /posts. Also, they didn't deal with the complexities of having nested endpoints vs avoiding them. And last, if using nesting is unnecessary as a readability standpoint.

Mongo query with regex fails when backslash\newline is there in a field

Hi I have a field in a user collection called "Address".User saving their address from a textarea in my application. mongodb convert it to new line like following.
{
"_id": ObjectId("56a9ba0ffbe0856d4f8b456d"),
"address": "HOUSE NO. 3157,\r\nSECTOR 50-D",
"pincode": "",
},
{
"_id": ObjectId("56a9ba0ffbe0856d4f8b456d"),
"address": "HOUSE NO. 3257,\r\nSECTOR 50-C",
"pincode": "",
}
So now When I am running a search query on the basis of "address".Like following:
guardianAdd = $dm->getRepository('EduStudentBundle:GuardianAddress')->findBy(array(
'address' => new \MongoRegex('/.*' .$data['address'] . '.*/i'),
'isDelete' => false
));
echo count($guardianAdd);die;
it does not give any result. My Searchi key word is : "HOUSE NO.3157 SECTOR 50-D".
However if I am searching using like: HOUSE NO. 3157 its giving correct result.
Please advice how to fix this.Thanks in advance
First of all, trailing .* are redundant. regexps /.*aaa.*/ and /aaa/ are identical and match the same pattern.
Second, you probably need to use multiline modifier /pattern/im
Finally, it is not quite clear what you want to fix. The best think you can do is to provide some basic explanation of regex syntax in the search form, so users can search properly, e.g. HOUSE NO.*3157.*SECTOR 50-D to get best results.
You can make some bold assumptions and build the pattern with something like
$pattern = implode('\W+',preg_split('/\W+/', $data['address']))
which will give you a regexp HOUSE\W+NO\W+3157\W+SECTOR\W+50\W+D for different kind of HOUSE NO.3157 SECTOR 50-D requests, but it will cut all the regex flexibility available with bare input, and eventually will result with unexpected response anyway. You can follow this slippery slope and end up with your own query DSL to compile to regex, but I doubt it can be any better or more convenient than pure regex. It will be more error prone for sure.
Asking right question to get right answers is true not only on SO, but also in your application. Unfortunately there is no general solution to search for something that people have in mind, but fail to ask. I believe that in your particular case best code is no code.

Elasticsearch mongodb river script in index doesn't work

I'm trying to change few fields strings using javascript.
For example take only the last part of the URL taken from mongo through the river so in elasticsearch I'll have only the end of it.
When creating the index (using curl) I added under "options" the following script:
"script": "ctx.document.shorturl = ctx.document.url.substr(-4);delete ctx.document.url;
I tried some manipulations such as adding \"...\" or use ctx['doc']['url'] and others but nothing seems to work.
I always get only url field with the full url (shorturl is not created at all).
Can anyone suggest what is the right syntax to make it work?
Another thing I need to do is combine to fields - lat & long, to one "location" field in order to use it in Kibana, can anyone suggest the right script for that? (create new field called "location" which contain both field "lat" & "long" with comma between them).
Thanks.
You did substring(-4), hence it will return the whole string. You should use substring(4) instead:
ctx.document.shorturl = ctx.document.url.substr(4);delete ctx.document.url;

Is there any way to force a schema to be respected?

First, I'd like to say that I really love NoSQL & MongoDB but I've got some major concerns with its schema-less aspect.
Let's say I have 2 tables. Employees and Movies.
And... I have a very stupid data layer / framework that sometimes like to save objects in the wrong tables.
So one day, a Movie gets saved in the Employees table. Like this:
> use mongoTests;
switched to db mongoTests
> db.employees.insert({ name : "Max Power", sex : "Male" });
> db.employees.find();
{ "_id" : ObjectId("4fb25ce6420141116081ae57"), "name" : "Max Power", "sex" : "Male" }
> db.employees.insert({ title : "Fight Club", actors : [{ name : "Brad Pitt" }, { name : "Edward Norton" }]});
> db.employees.find();
{ "_id" : ObjectId("4fb25ce6420141116081ae57"), "name" : "Max Power", "sex" : "Male" }
{ "_id" : ObjectId("4fb25db834a31eb59101235b"), "title" : "Fight Club", "actors" : [ { "name" : "Brad Pitt" }, { "name" : "Edward Norton" } ] }
This is VERY wrong.
Let's switch the context, think about Movies, and CreditCards (for whatever reason, in this context credit cards would be stored in clear text inside the DB). This is SUPER WRONG?
The code would probably explode because it's trying to use an object
structure and receives another totally unknown structure.
Even worst, the code actually works and the webstore visitors
actually see credit cards information in the "Rent a movie" list.
Is there anything, built-in that would prevent such threat to ever happen? Like some way to "force" a schema to be respected for only some tables?
Or is there any way to force MongoDB to make a schema mandatory? (Can't create new fields in a table, etc)
EDIT: For those who thinks I'm trolling, I'm really not, this is an important question for me and my team because this is a big decision whether or not we're going to use NoSQL.
Thanks and have a nice day.
The schema-less aspect is one of the major positives.
A DB with a schema doesn't fully remove this kind of issue - e.g. there could be a bug in a system that uses a RDBMS that puts the wrong data in the wrong field/table.
IMHO, the bigger concern would be, how did that kind of bug make it through dev, testing and out into production?!
Having said that, you could set up a process that checks the "schema" of documents within a collection (e.g. look at newly added documents, check whether they have fields you would expect to see in there) - then flag up for investigation. There is such a tool (node.js) here (I think, I've never used it):
http://dhendo.github.com/node-mongodb-schema-validator/
Edit:
For those finding this question in future, so the link in my comment doesn't go overlooked, there's a jira item for this kind of thing here:
http://jira.mongodb.org/browse/SERVER-3536

MongoDB $ne Explanation

The official MongoDB api wrote very little about $ne
http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24ne
So when I encountered something like
db.papers.update({"authors cited" : {"$ne" : "Richie"}},
... {$push : {"authors cited" : "Richie"}})
I have no choice but to become utterly confused. Can someone please explain it to me?
This would add "Richie" to the list of authors cited for a paper that does not already have "Richie" as an author.
An alternative would be to use $addToSet.
But then how would I know whether {"authors cited" : {"$ne" : "Richie"}} means the elements in the list corresponding to "author cited", vs the value corresponding to "author cited"?
That is a bit confusing. Generally (I am sure there are exceptions, but those should be documented), all selectors target the individual values for multi-values fields. In Mongo this is called "multikeys" .
Note that this led me to assume initially that your query would target all papers that have at least one author who is not Richie. Then I checked and this turned out to be wrong. +1 for your question, because this really needs to be documented better.