How to go about planning an USSD applicaiton(using nodejs sailsjs) - rest

I have been looking for a way to organize an USSD application without success, I have so many unanswered questions making me basically stalled. I have read many questions on stackoverflow about state machine pattern or strategy pattern but it's still blur in my mind.
Let's say I have a ussd application to provide the following contrived services:
1 - get free joke
2 - register for paid joke (ie 10 jokes/day)
3 - get balance for paid joke (remaining )
Under 1 I have other options like categories :
1 - satire
2 - nerdy
3 - explicit
Under 2 I could have options like:
1 - tumblr
2 - yahoo
3 - hotmail
Under 2-1 for example it can show payment options like :
1 - paypal
2 - stipe
Then under 2-1-1 , it can ask for email, then password or so etc.
This can get other nested structure where some payment method are enabled for some vendors or not etc.
Problem 1 Considering USSD provider uses POST expect us to use a REST webservice to receive input from users' phone. This is server making API call to another server. How can session been used to pass information between requests. USSD can result is long conversation(back and forth ) with server. how is session maintained ? cookie (client is not a browser) or database? (sails has support for redis and mongo session storage).But again which mechanism is used to ID each request. That's one fuzzy area I am struggling with.
Problem 2 If problem 1 is solved how to apply it to the application. I mean by that say, user chooses flow 2 by typing number 2 on phone, the app UssdrouteController got the input 2 and send the request to RegisterpaidjokeController to process. How is RegisterpaidjokeController is going to differentiate between first number 2 and maybe second number which means yahoo option? How UssdrouteController doesn't confuse this with flow 1 option 2 which is nerdy category. I would like to find a way not to hard code this and track user's progress in each flow. This assumes some session is used. how to document this flow and make it maintainable. It's all confusing for me.
I would be most grateful if I could get some pointers around this. Thanks in advance

Related

Converting a 'list-manage.com' daily update of url's into actual url's

I receive a daily listing of url's as part of a MailChimp based emailing and I want to find out if it is possible to automate the process of retrieving the end url.
For the example, I receive ‘Outpouring of love’: Justin Welby addresses mourners at Queen’s funeral | Queen Elizabeth II | The Guardian'
Which when clicked directs to:
us2.list-manage.com/track/click?u=50eac70851c7245ce1ce00c45&id=08efb7cfcc&e=852b2b1fd4
Which then seems to redirect to the endpoint:
https://www.theguardian.com/uk-news/2022/sep/19/justin-welby-sermon-queen-funeral?utm_source=Daily+media+digest&utm_campaign=c6b9dd75ec-EMAIL_CAMPAIGN_2019_11_27_02_01_COPY_01&utm_medium=email&utm_term=0_296e14724b-c6b9dd75ec-248630149&mc_cid=c6b9dd75ec&mc_eid=852b2b1fd4
I am thinking it's not possible as I don't have any access to the MailChimp account but it would be good to have this confirmed by someone cleverer than me...
I am a bit of a newbie so please try keep things simple. Very many thanks,

Facebook pixel events call from server

I have absolutelly the same question as dan here - Facebook conversion pixel with "server to server" option . There was written, that there was no way, but it was 2013, so I hope something changed.
So, is there any way to call facebook pixel events (e.g. CompleteRegistration) from server side now?
I can describe situation in more details. Imagine, that user visits our site, where fb pixel tracks 'PageView' of course. When user passes form and sends his phone number, we call 'Lead' event. But then we need to track one more event, when our manager successfully confirmes this user! Of course, it happens on other computer and so on, so there is no idea, how to "connect" to base user.
I've seen a lot of documentation departments like this, but I can't fully understand even if it's possible or not.
Logically, we need to generate specific id for user (or it can be phone number really), when 'Lead' event is called. Then, we should use this id to 'CompleteRegistration' for that user. But I can't understand, how to do it technically.
It would be gratefull, if somebody could explain it.
P.S. As I understand, it is fully available in API for mobile apps. Is it ok idea to use it for our situation, if there is no other solution?
Use Offline Conversions to record events that happen after a user has left your website. Logging these conversions, technically, is very easy. Setting everything up takes a little effort
tldr; check the code below
Follow setup steps in the FB docs (Setup steps 1-5) which are:
Setup facebook Business Manager account
Add a new app to Business Manager account
Create an Ad account, if you don't already have one
Create a System User for the ad account
After the setup, follow Upload Event Data steps on the same page, steps 1-3 to create an offline event set and associate it with your ad. These can be carried out in the Graph API Explorer by following the links in the examples. These can be done programmatically, but is out of the scope of making the event calls from the server for one campaign.
Once you have created the event set, then you can upload your CompleteRegistration events!
You will need to make a multipart form data request to FB, the data key will be an array of your conversion events. As #Cbroe mentioned, you must hash your match keys (the data you have available about your user to match them with a FB user) before sending to FB. The more match keys you are able to provide, the better chance at matching your user. So if you can get their email and phone at the same time, you're much more likely to match your user.
Here's an example of the call to FB using node.js:
var request = require('request')
// The access token you generated for your system user
var access_token = 'your_access_token'
// The ID of the conversion set you created
var conversionId = 'your_conversion_set_id'
var options = {
url: 'https://graph.facebook.com/v2.12/' + conversionId + '/events',
formData: {
access_token: access_token,
upload_tag: 'registrations', //optional
data: [{
match_keys: {
"phone": ["<HASH>", "<HASH>"]
},
currency: "USD",
event_name: "CompleteRegistration",
event_time: 1456870902,
custom_data: { // optional
event_source: "manager approved"
},
}]
}
}
request(options, function(err, result) {
// error handle and check for success
})
Offline Conversion Docs
Facebook has now a Server-Side API: https://developers.facebook.com/docs/marketing-api/server-side-api/get-started
Implementing this is similar to implementing the offline events outlined in the accepted answer.
Keep in mind that it will always be cumbersome to track and connect events from the browser and from your server. You need to share a unique user id between the browser and server, so that Facebook (or any other analytics provider) will know that the event belongs to the same user.
Tools like mixpanel.com and amplitude.com may be more tailored to your needs, but will get very expensive once you move out of the free tier (100+ EUR at mixpanel, 1000+ EUR at Amplitude, monthly). Those tools are tailored towards company success, whereas Facebook is tailored towards selling and measuring Facebook ads.

GWT RequestFactory: check if members have been set without permission

I am working with GWT / RequestFactory and a set of customer requirements regarding permissions. Let me explain a basic example:
Every user is assigned to a company. Every user should be able to edit company's core data - but only e.g contact information, website etc. Security-relevant ones like BIC/SWIFT, IBAN, Company name and so on can only be changed if the user has a certain permission XY.
So far so good, on the client side I can check the permissions and disable those fields the user is not allowed to edit. But what would be the most elegant way to ensure on the server side that those fields have not been set without permission?
My problem is that I cannot track changes on the server side. Having #PreAuthorize on every setter is not an option too, because it would end in an authorization-massacre in each and every entity.
At the moment I am following a workaround: every field that is secured / depends on a given permission is passed as an argument to the entity-method and is excluded from the proxy. That way, values cannot be set using the proxy and I can check in my server code if the user has permissions. If not, nothing happens. If user has permissions, I set the values manually. But that produces a lot of boilerplate-code and ugly method signatures because the number of values passed to the method could get large.
I hope you understand my issue. I'm looking forward for your opinions and tips. Thank you in advance.
Well, you can receive many answers (different each other), and all of them could be right, so, at the end is your call. Wait for others answers. I am going to give you the approach that I followed (and it worked pretty well). :D.
Under my opinion, the server should do less as possible, so keep the logic for allowing modify each param on the server I think it is not a scalable solution (if your system has 1M users modifying everything at the same time, will your server work fluent?). I prefer let the client do the job (like Roomba :D).
For solving that problem, in our system we implemented an Access Control List solution. You can store in your db, on each user entity, a list with granted permissions. So, when that information arrives to the client (after user's log in, for example), you can get them, and show the fields that he/she is allow to modify.
Something like:
if (canModifyPersonalDetails(user.getAcls(), ...) ) {
//show labels ...
}
if (canModifyBankDetails(user.getAcls(), ...) ) {
//show labels
}
You can not avoid server call for log in, so it is not a big deal send the extra information (think about the ACLs could be simple list of integers 0 means personal details, 1 bank details....).
If you are dealing with very compromised information and you prefer do some stuff on the server, in that case probably I'd set up a security level, when you are persisting/updating your proxy, I'd do something like:
if (isAllowForPersonalDetails(user.getSecurityCode()) {
//update the modified personal details
}
if (isAllowForBankDetails(user.getSecurityCode()) {
//update the modified bank details
}
user.update();
I am a big fan of clear User GUI's, and a very big fan of let the server free as much as possible, so I prefer the first option. But if you have constraints for modifying user entity in db, or you prefer do not modify your views, or any constraint with security, maybe the second option is the best one for you.
Hope that helps!

Magento Strange Redirect Behaviour on OnePage Checkout

My Magento Verison is - 1.4.1.1
I am having two problems:
1) When I am going through various steps of Onepage checkout (registration, billing, shipping, and payment tabs), sometimes during this process I am redirected to the cart page. There is no error, no exception, no report gets generated in var/report. I dont know how to debug it. Aren’t there any logs I can look for?
2) In same Onepage Checkout process after clicking on place the order (last step) , some times it redirects to the cart page, sends an email saying that the order failed with the message:
Quote totals must be collected before this operation.
To resolve it I commented this line in prepareRecurringPaymentProfiles in the file magento/app/code/core/Mage/Sales/Model/Quote.php, which solved the problem:
throw new Exception("Quote totals must be collected before this operation.");
I dont know if these 2 problems are related or not. But I am not having the 2nd problem now but having 1st one quite frequent. What could be the reason and how to resolve it?
further update-I checked the firebug trace, It is 500 internal server error which comes sometimes in any of the step in one page checkout. I was able to dig down into savebillingaction, saveshippingaction functions in onepagecontroller.php and found that error comes when $this->getRequest()->isPost() is blank , If it is 1 then it goes ahead, and goes to the next step else it redirects to cart, No I dont know why this is not 1 or is it because ajax is not able to send post data but I checked XHR request, Ajax send the post data every time (checked with firebug extension). Can Someone tell me What I could do next to troubleshoot. Where I can look for these Ajax Calls?
Shipping.phtml (any step.phtml) has JS at the bottom , How does it call OnePagecontroller saveshippingaction function ?
Since it's internal server error, try to access the error log of the server. Will tell you where is the problem. I' ve had the same problem in 1.7.0. In my example the problem was at /app/code/core/Mage/Usa/Model/Shipping/Carrier/Fedex.php
For anyone else coming across the "Quote totals must be collected before this operation." error, check your Apache logs for the reason of the 500 internal server error. If it's something like this:
mod_fcgid: read data timeout in 40 seconds
Premature end of script headers: index.php
process 26126 graceful kill fail, sending SIGKILL
..PHP is taking too long to respond. Usually its the onepage/checkout saveOrder action because it's quite heavy and often needs to connect to third party services (payment gateways, newsletter services like mailchimp, etc.). These calls to third party services can take a while, depending on the network state and might be the reason of PHP timing out.
You can start by increasing the timeout, but it's not a good permanent solution because you want to find out why this is happening in the first place.
New Relic is a good tool to monitor these calls (and a good tool to monitor your Magento store in general).
Firstly, commenting an error message is almost never the way to solve a problem, as you are just covering up some issue that may have severe consequences for your system.
Nailing down errors like this can be hard, but there are a few places to look first:
Did you install this system on a lower version and then upgrade? If so, how?
Are you using any extensions that modify the sales/checkout portion of the site?
Have you overridden any of the models concerned with this part of the site?
Have you changed the JS or HTML for the checkout?
If one of those is the case, you should review those changes for bugs. If not, try turning on the default theme for the site and checking out again. If the bug disappears, there is a problem with the theme that you are using. If it still appears, the problem is in code.
In that latter case, use Firebug to verify that the offending page requests result in Magento sending back "redirect" commands to the frontend. If that isn't the case, it may be some kind of JS error, but more likely you are ending up with invalid data in the system somewhere that causes Magento to choke during checkout.
Also (just thought of this, haven't tried it), try the multi-address checkout. As I recall, it uses regular page posts, and may even have more useful messaging than the OnePage checkout. Please edit your post with your findings from the above so that we can help more if that doesn't do it.
Hope that helps!
Just in case someone gets the "Quote totals must be collected before this operation." error, and none of these solutions fixes their particular problem, I'll mention that mine was an issue with this:
skin/frontend/base/default/js/opcheckout.js
var params = Form.serialize(payment.form);
There was a JS error unique to this site which was clearing the Payment form and preventing JS from reading it's contents. The module or theme that you use will differ, but check to make sure that the payment form can serialize correctly. If not, then that could be your problem.
I had the very same problem on my store Princessly:
It takes about 20 to 130 seconds or even longer for "Submitting order information ..." to go through and redirect to the payment gateway such as PayPal, if at all, after clicking the Place Order button, last step of one page checkout.
If it doesn't go through, very probably because something timed out since it took too long, it will redirect back to shopping cart, leaving the customer an empty cart and a Pending Payment order, OR, it will give the exception of:
Quote totals must be collected before this operation.
Since obviously, well, something timed out and the script ends before quote totals are collected (which is just my theory), thus sending the Payment Transaction Failed Reminder email.
After 12 hours of research and debug, I finally found the culprit and the solution.
Magento enables RSS stock and new order notification by default, so every time Place Order is pressed ('sales/order' resources are then saved), cache is refreshed so RSS will be published. Cache cleaning can be very time-expensive for Magento. Therefore the solution is simple. Just disable RSS notification for save of 'sales/order' resources.
Find /app/code/core/Mage/Rss/etc/config.xml and locate this block:
<sales_order_item_save_after>
<observers>
<notifystock>
<class>rss/observer</class>
<method>salesOrderItemSaveAfterNotifyStock</method>
</notifystock>
</observers>
</sales_order_item_save_after>
<sales_order_item_save_after>
<observers>
<ordernew>
<class>rss/observer</class>
<method>salesOrderItemSaveAfterOrderNew</method>
</ordernew>
</observers>
</sales_order_item_save_after>
Simply remove or comment it out and refresh Magento cache in System => Cache Management => Select All => Submit.
Now it only takes 1 second or even less for my store to go through Place Order and redirect to payment gateway.

Yet more questions on RESTful URIs

Numerical IDs vs names
As an example, which of these would you choose for identifying a single transaction, from a single bank account, for a single company:
/companies/freds-painting-ltd/accounts/savings/transactions/4831
/companies/freds-painting-ltd/accounts/1/transactions/4831
/companies/62362/accounts/1/transactions/4831
You idiot, something totally different! Crikey, did you even READ Fielding's dissertation?
Now, I think the 1st one is the most readable. If I have more than one company, or if I'm someone like an accountant managing multiple companies, it's immediately clear which company, and which account, I'm looking at. It's also more bookmarkable/emailable and would prevent 'fishing' for other companies by changing the company ID. I would want transaction IDs to be unique to an account (I.e. Both 'savings' and 'current' accounts could have transaction '1'
A 'company' will be my 'top-level', or 'first class' resource. Nothing at all would ever be shared between companies. As such, it would be the ideal candidate for a shard (or 'ancestor'/'namespace' in Google App Engine parlance). So I'd only have to worry about the account names being unique within one company. Every company could have an account called 'savings'.
Not sure what the situation in the rest of the world is, though LTDs or PLCs in UK would have a unique name, there could be many 'Dave's Window Cleaning' businesses (what's know as a trading name).
The business owner(s) could potentially opt for the top level /company/company-name URI to be public, and contain some basic details like their website, contact details etc, but everything below that would NEVER be accessible by search engines.
So my thoughts/concerns are:
1) Is it reasonable, when someone signs in to add their business, to say "Sorry, 'Dave's Window Cleaning' business is taken. How about 'Dave's Window Cleaning Portsmouth' (Having taken their location in another field)? My worry with this is that, for a more well known company, you're giving away the fact that they have an account with you. Or that someone could use that form to search for names. Perhaps not a biggie.
2) The size of the company name. Would it be reasonable for a name like 'Dave's Window cleaning, gardening, and loads of other stuff'? Thus creating a URL like 'daves-window-cleaning-gardening-and-loads-of-other-stuff/'
3) How to deal with someone changing their business name - I would approach it by creating a new company with that string ID, copying over everything, then deleting the old resource. The original URI would return 404 rather than redirecting - as you can't guarantee someone else won't want to take the now unused name, or even if more than one person has used the same name in the past.
4) Should the 'real' unique ID be an number in the back end, and for every request to be handled by first doing a query for what company ID this name actually related to.
5) The impact of searching for a transaction in the persistence layer.
6) The possibility of URL rewriting, but then that wouldn't work cleanly in GAE, nor would it solve the issue of ensuring company names are unique.
RESTful webservice vs RESTful website
So, we potentially have this lovely RESTful webservice that the latest snazzy iphone/android app can use (delusions of grandeur). But what about the main website itself? I note, right now, that the URL I see at the top of my page is not 'RESTful': /questions/ask is an action. There is no 'ask' resource on the server. It's more the state of the page, the preparation for POSTing to /questions/ - or if I'm editing, PUTing to /questions/{id}
I also note that Stackoverflow has URIs like /questions/362352/name-of-the-question, and that the latter part can be omitted, and one will be redirected to it.
Should I host a completely separate webapp that consumes my lovely webservice (from the same domain)? Do I even need a separate REST server, or can I rely on content negotiation (JSON/XML) and HTTP verb to select the right method (I'm using Jersey), and return the right representation?
So I could have /companies/aboxo/ return the whole HTML page (using stringtemplate.org) if it's a GET /,text/plain or test/html, and JSON/XML for others?
But what happens for 'add/edit/delete' transaction? Would GET / /companies/freds-painting-ltd/savings/transactions/?template=add be ok (or GET ../transactions/352?template=edit), and that would return the right HTML?
Thinking about this last detail is driving me mad for some reason.
Comments, suggestions, outright ridicule - all welcome!
Marcos
Rails solves the "id vs name" problem by displaying both in the URL but using only the id to actually identify eg:
/companies/62362-freds-painting-ltd/accounts/1-savings/transactions/4831
ie - for the ones that have a "pretty url" the function that generates your path write both the id and the name... but for your router, where relevant: you strip off everything thats not the id.
incidentally, it means your customer could actually write whatever they like into the URL and it'd make no difference:
/companies/62362-i_luv_blue_turtles/accounts/1-your_mum/transactions/4831
and your router still just sees:
/companies/62362/accounts/1/transactions/4831
:)
For a cannonical URI I suggest just /transactions/{id} as I presume the transaction knows what the company and account is. Therefore, #4 :-)
Is SEO a concern? I presume you don't want random folks off the internet googling for X company's transactions?! Therefore, I would just keep names (which may change) out of the URI.