Body params disappear on some POST requests routes - rest

I have some rest and GraphQL services works with ruby and typescript (with nestjs framework).
I've noticed that a couple of days ago some of the POST requests failed due to params validation error.
After further investagaion, it seems like on some requests the body params sphoradicly drops from the requests somewhere on the network. It happen for small amount of requests (less than 1%).
This issue started when no changes deployed on the server or clients side. My clients are iOS and android apps and it happens for both platforms.
I've tried to find the exact point on the network that the body dropped, with no success.I've also tried to find a solutions to similar issue on the net.
Does anyone have any idea what can it be? I haven'e found any relevant information about similar issues.
Thanks!

Related

Unable to investigate on ZEIT Now 502 eror for a NextJS app

I'm investigating by days with no results about this exception that my NextJS app is currently throwing, in particular when I try to open a single specific URL:
502: BAD_GATEWAY
Code: NO_STATUS_CODE_FROM_FUNCTION
ID: zrh1:4zx5l-1572269318137-64d401b5d058
Here's the screenshot:
Basically, I have on https://lucacattide.dev/about/en a page that this app should open. This is linked to a MongoDB third-party cloud API platform - Squidex - which is responsible to populate the page itself, via GraphQL queries. The app uses Apollo as GraphQL client. The app instead, is hosted on ZEIT.co serverless cloud, with Now 2.0 version.
During the development process, everything works fine. The page loads up and data is fetched in the right way. Notice that for development, I'm working on now-dev environment instead of a custom Express server, in order to reproduce the production one, as suggested by ZEIT itself.
The exception is being thrown on the production environment - the live one on the hosting platform, not on localhost; the main problem is that no errors are being shown on live logs or local development. So I'm literally going mad in inspecting the possible cause.
I've already tried to test the involved page, by splitting it in sections and trying to exclude child components, or focusing the inspection on the GraphQL query. But the first hasn't produced results and the latter works fine in every environment.
As last try, I deleted and re-created the back-end contents related to that page, because in the past I had a similar issue due to an old GraphQL edited schema that didn't reflected its modifications through the API - so in that case I was still receiving 502 errors. But this time it didn't worked.
Anyone could help me to understand what's going on, please?
Thanks everyone in advance
The issue was caused by an incompatibility between the d3-cloud library and the Now environment.
By replacing it with the react-wordcloud one, the error has been solved.
Thanks everyone for your assistance.

GWT-RPC and the infamous sporadic "StatusCodeException: 0" exception revisited

My problem is the infamous "StatusCodeException: 0" problem happening when using GWT 2.6.1 when accessing page via subdomain https://sub.site.com/.
Now, this happens quite sporadically for one customer using IE11 and I can't reproduce this from several distinct computers using IE11, IE10, IE9 or IE8 (not to talk about Chrome or Firefox).
Accessing exactly the same webapp from https://site.com/ seems to work fine for that customer.
This obviously lead me to conclusion that I'm having problem with Same Origin Policy.
What is strange though is that my webapp is designed in the way that no cross-domain or cross-subdomain requests are made. Same goes for no cross-protocol as well no cross-port requests. In other words, Same Origin Policy is not violated in this situation. As a confirmation of that, I can provide following proof:
While being at customer site I've seen how this is reproduced: customer starts using application and everything works fine - all requests are returning response normally. Then, after several minutes of working, exactly the same requests on the same page (without reloads) starts to fail with StatusCodeException: 0.
Basically, both https://sub.site.com and https://site.com points to the same IP, and there is only one Tomcat webapp serving exactly the same resources both for https://sub.site.com and https://site.com.
Another proof would be the codebase of the single GWT module itself: there I use only one instance of one service called DashboardService:
public class DashboardModule extends EntryPoint implements IDashboardModule {
private final DashboardServiceAsync dashboardService = createDashboardService();
#Override
public void onModuleLoad() {
// loading of module elements
// dashboardService is passed as a parameter so only one instance is used
}
/**
* PLEASE SEE QUESTION #1 BELOW CODE SNIPPET
*/
private static final String DASHBOARD_REQUEST_URL = "request";
private static DashboardServiceAsync createDashboardService() {
final DashboardServiceAsync service = GWT.create(DashboardService.class);
((ServiceDefTarget) service).setServiceEntryPoint(DASHBOARD_REQUEST_URL);
return service;
}
}
=================================== EDIT ====================================
After looking in the console at customer location, the error was always the following:
SCRIPT7002: XmlHttpRequest: network error 0x2ee4, ...
so it seems that this has nothing to do with Same-Origin Policy, because as per this article it is described as ERROR_INTERNET_INTERNAL_ERROR An internal error has occurred.
It's a pity, but I've found only 2 mentions of this error which were not resolved:
Error under IE10 and Error under IE11.
I have an assumption that customer is very probably accessing site through some proxy which slightly changes the requests and IE can't handle them.
Question 1: does anybody knows how can I simulate or reproduce mentioned error locally?
Question 2: does anybody knows how this problem can be gracefully worked around?
Question 3: is it ok to simply retry the request, or this request may have reached the server and modify it, so retrying it may produce duplicate modification?
Will try to setup forwarding proxy to simulate possible customer setup to at least reproduce mentioned error...
I greatly appreciate any help!
Ok, so after bugging with this problem for a workweek I finally managed to solve it.
Actually, I was able to reproduce very similar problem locally when I installed Apache2 server in front of Tomcat and accessed it from another VirtualBox Win7 host with IE11. This gave me sporadic StatusCodeException: 0 with Network error 0x2ef3 though but the behaviour was very similar: GWT-RPC requests started to fail after a minute or so. This was reproducable in IE10 and IE11 but working fine in IE8 and IE9 :) (is IE getting crappier with new versions?)
Locally I was able to fix that problem by simply disabling Keep-alive functionality for IE browsers by adding following lines to /etc/apache2/sites-enabled/default-ssl.conf Apache2 ssl configuration file:
# following line was added
BrowserMatch "Trident" nokeepalive ssl-unclean-shutdown downgrade-1.0 force-response-1.0
</VirtualHost>
</IfModule>
This basically tells Apache2 not to use keep-alive, use special SSL handling and generally downgrade to HTTP 1.0 standard whenever user-agent string in request has Trident word (matches IE11 and IE10 and possibly earlier IEs)
This added Connection: close HTTP header to each response and seemed to work fine locally.
On customers site this wasn't still working and produced the same Network error: 0x2ee4.
It may be worth noting that customer was using McAfee Web Gateway as forwarding proxy which stood in the middle of browser <-> server communication.
Long story short, I found out that the problem was in the following: when page loads there are multiple GET requests being sent to server to get the page, resources etc. Then after 10 seconds of using it (my webapp is single-page-application, so user may spend more than 10 minutes on same page) only GWT-RPC requests are being made to the server which are POST requests. And after a minute of using this page (I suspect 1 minute = keep-alive timeout of proxy server) these POST requests start randomly fail with 0x2ee4 network error.
After I implemented GWT-RPC retry functionality, I found out that after 30 seconds of retries simply ALL GWT-RPC requests fail with above error. Refreshing the page was solving this problem again for a minute or so and then same story happened.
So, I figured out that CRAPPY IE11 and IE10 are incorrectly handling combination of SSL, Keep-alive and POST requests. It seems that сrappy IE10 and IE11 simply can't renew keep-alive ssl connection using POST requests and only do this using GET requests.
Please note that Chrome, Firefox and other normal browsers are handling this situation quite well. When inspecting how Firefox behaves in such situation in Firebug: it can be clearly seen that POST request is made, then it is shown as aborted for like 0.5s and then this it is shown as successful (I suspect that Firefox handles this specific situation and makes GET request to server itself to renew SSL keep-alive connection and then retries POST request)
So, to fix this problem in IE I simply implemented functionality which "pings" server with GET request every 5 seconds (be ready to experiment with this time since this is most probably related to customer's proxy keep-alive timeout).
This made it work (please note that above Apache2 configuration hack is not needed in this case)
I really hope that this will help people with similar issue and save their time
Resources used:
IE Network Error 0x2ef3 question 1
IE Network Error 0x2ef3 question 2
IE Network Error 0x2ef3 question 3
Awesome q&a on how to implement transparent GWT-RPC retry functionality
P.S. Will I report this IE10 and IE11 issue to Microsoft? - really I'm not eager spending 30+ minutes of my time reporting issue on commercial crappy IE browser issue after I've already spent more than a week of finding out the problem.
I insist on recommending Chrome or Firefox or other normal browser to customers as viable alternative and I still think that IE11 is not suited for modern websites with AJAX

Twython 401 on stream, but REST works fine

NO KIDDING. My code was working yesterday
So I'm writing a script to get streaming data using twython. But today it's giving me 401 errors. Even with the example code
I tried new keys, and a new app, but it still gives me 401.
However, normal REST api interface in Twython (see next code block) works just fine.
twitter = Twython(APP_KEY, APP_SECRET, OAUTH_TOKEN, OAUTH_TOKEN_SECRET)
print twitter.get_home_timeline()
Some Googlefu led me to this SO thread which suggested using ntpd -q, but this doesn't solve my issue!
This affects only some of the servers behind stream.twitter.com. Perhaps their clocks are off.
Some kind of geo-aware load balancing appears to be used and can the domain name resolves to a number of different IP addresses, depending on where you are doing the resolving.
Connecting to 199.59.148.229 or 199.59.148.138 gives the 401 error for me (these I get resolving from AWS Oregon), but 199.16.156.20 works (this I get resolving from my home network).
Therefore, my temporary workaround for this was to add this to /etc/hosts:
199.16.156.20 stream.twitter.com
I expect the original problem to go away in a few hours, but this works for me, for now.

Blackberry ksoap2 request issues

First time posting a question. I'm trying to call some SOAP webservices from inside a blackberry app using the ksoap2 library. I've successfully managed to get a response from the one service, which uses an HTTP url, but now that I'm trying to get response from a (different) HTTPS url, I've run up against a brick wall.
The response dump I'm getting has the following fault message:
"An error occurred while routing the message for element value : (country option I specified in my request). Keep-Alive and Close may not be set using this property. Parameter name: value."
The weird thing is that using Oxygen XML's SOAP tools with the XML request dump works just fine. Any ideas where to start looking? This has taken up a full day already.
Update:
Responding to your comment below - it turns out the double quoting is part of the SOAP spec. Some servers are more relaxed in their implementation, and will work without the quotes.
ksoap2 doesn't force the quotes onto your actions - you may want to patch your ksoap2 library to ensure the quotes are always there.
ymmv
Original:
I don't think this is a SOAP related problem, nor with BlackBerry.
I think the problem lies on the server side, since that error string is not a common error (just google it to see no hits on the whole internet other than this question).
Looks like this is a job for the network guy on the server side to tell you what he's seeing on his end.
Only other thing I can think of is to make the call using HTTP instead of HTTPS. You can then use some network sniffer to see what the difference between the messages is. Alternatively, install an SSL proxy with something like "Charles" and sniff the packets like that.

Problems with making web service requests with custom headers via MonoTouch

My team and I are working against a few webservices that require SOAP Message Headers to be available when making a request. We are not in control of these webservices so we can't change the implementation, even if we wanted to (or at least not without a lot of pain). We just need to be able to have authentication related information & a couple of other items passed through our message headers.
I've read of a few people who've had this problem in the past with no clear indication on if they succeeded in pulling it off on Monotouch.
Here's what I've read: http://forums.monotouch.net/yaf_postsm2104.aspx so far.
Any ideas on what we can do to overcome this on the Monotouch framework?
Here's what i'm trying to do for now:
using (var scope = new OperationContextScope (client.InnerChannel))
{
client.GetHistories += handler;
OperationContext.Current.OutgoingMessageHeaders.Add (MessageHeader.CreateHeader ("EnvironmentInfo", "http://schemas.contoso.com",
ServiceContext.Current.OperatingEnvironment));
OperationContext.Current.OutgoingMessageHeaders.Add (MessageHeader.CreateHeader ("AuthenticationToken", "http://schemas.contoso.com",
ServiceContext.Current.Token));
client.GetHistoriesAsync (ServiceContext.Current.OperatingEnvironment, ServiceContext.Current.Token, request);
}
Thanks for your time.
JM
I was not able to get Message Headers to work with WCF in Mono 2.6. I tried several different ways (including how you do it in your example) - it just doesn't work in Mono 2.6.
I raised a bug for this, which I then closed after discovering it is fixed in the latest trunk. So if you run against Mono 2.7 or greater, this should work.