all
I've been using AutoHotkey to perform a number of batch processes at work. It's going pretty well, but I keep running into an issue. The scripts (more than one, doing very different things) will all crash after about 4 hours of running. It's pretty consistent with the time, but only kind of consistent with the location in the program where it's running.
I'm not getting an error message. The script will just stop, and when I try to open it up to see the last line, the program closes without any info.
I know it's vague, but any ideas what may be happening?
Edit: added Event Viewer data. Two other crash events were almost identical.
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Level>2</Level>
<Task>100</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2013-10-11T04:20:51.000000000Z" />
<EventRecordID>11519</EventRecordID>
<Channel>Application</Channel>
<Computer>My Computer</Computer>
<Security />
</System>
- <EventData>
<Data>CDR to CGM-PDF 3-0-1.exe</Data>
<Data>1.1.11.2</Data>
<Data>51f4589f</Data>
<Data>CDR to CGM-PDF 3-0-1.exe</Data>
<Data>1.1.11.2</Data>
<Data>51f4589f</Data>
<Data>c00000fd</Data>
<Data>00000000000ca117</Data>
<Data>1c44</Data>
<Data>01cec61d2e612892</Data>
<Data>C:\CDR to CGM-PDF 3-0-1.exe</Data>
<Data>C:\CDR to CGM-PDF 3-0-1.exe</Data>
<Data>82958232-322c-11e3-ab1f-d067e5491d83</Data>
</EventData>
</Event>
Related
Windows Server 2012 - IIS 8.x
I have a WCF web service (DLL) deployed in IIS that supports SOAP and REST over https. There is a certificate installed from Comodo for the SSL.
When I create a proxy for the service in C# apps everything works fine. Also, when I call most of the REST methods from a browser or from Postman they also work fine.
But there are a few REST methods that fail with no exception. In Chrome browser I get an error that says: "This site can't be reached." In Postman I get an error that says: "Could not get any response."
When I check the code in the WCF service I can see that the method gets called and actually executes with no exception, but nothing is returned. I've also checked the IIS logs and I don't seem to see any errors.
Here is an example of a method that works fine:
[OperationContract]
[WebGet(ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Wrapped, UriTemplate = "/GetActivityRules?agencyID={agencyID}")]
List<ActivityRule> GetActivityRules(int agencyID);
Yet, this method fails:
[OperationContract]
[WebGet(ResponseFormat = WebMessageFormat.Json, BodyStyle = WebMessageBodyStyle.Wrapped, UriTemplate = "/GetActivity?agencyID={agencyID}&userID={userID}&date={date}&view={view}")]
List<Activity> GetActivity(int agencyID, int userID, string date, string view);
Has anyone experienced this before or have any ideas why this is happening or how best to troubleshoot?
UPDATE #1
I added code to the GetActivity function that writes to a log file each time its called. I can see the function executes correctly without error each time, but still there is no response. Makes me wonder if the problem is not related to WCF. I also added a trace listener to web config, but the file never gets created.
<system.diagnostics>
<sources>
<source name="System.ServiceModel"
switchValue="Information, ActivityTracing"
propagateActivity="true">
<listeners>
<add name="traceListener"
type="System.Diagnostics.XmlWriterTraceListener"
initializeData= "C:\Samadhi\TraceLog.svclog" />
</listeners>
</source>
</sources>
</system.diagnostics>
UPDATE #2
I spent a week of mind numbing searching, reading, and checking everything I possibly could with no result. I raised a support issue with Microsoft, which was escalated to one of their senior engineers. They have spent two days looking into the problem with still no solution. They tell me they have never seen this before!
I tried using your code and testing it locally, and it looked like everything was working fine. Even if method 2 contains multiple parameters, WCF can receive them. I think the problem is somewhere else. For example, if the data volume transfer is too large, please try to configure the following code in the binding.
<binding name="mybinding" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647" maxBufferSize="2147483647" sendTimeout="00:10:00" receiveTimeout="00:10:00">
<readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" />
Or you can construct the collection return directly to eliminate problems with backend data queries.
to enable wcf tracing, you have to create a source configuration for System.ServiceModel. This should trace the service serialization:
<system.diagnostics>
<sources>
<source name="System.ServiceModel" switchValue="Verbose" propagateActivity="true">
<listeners>
<add name="svc" />
</listeners>
</source>
</sources>
<sharedListeners>
<add name="svc" type="System.Diagnostics.XmlWriterTraceListener" traceOutputOptions="DateTime" initializeData="c:\trace.svclog" />
</sharedListeners>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="svc" />
<remove name="Default" />
</listeners>
</trace>
<system.diagnostics>
How frustrating software development can be. I swear I'm going to need hair replacement therapy soon. I finally found what was causing the issue, and as is most often the case, it boils down to my programming error. The frustrating part is when you have no error message, or when a framework throws a generic error that gives no clue as to what's wrong.
The issue came down to the fact that WCF was unable to return the JSON result - unable to parse the list of objects into JSON. This happened because an uninitialised date value was outside the allowed min and max values. The net result was no response from the server, not even an error code. When calling the same method using SOAP, the XML parsed fine - only the REST call was failing.
I initially had problems setting up WCF trace. For some reason it was causing the service to stop working. And because it was a production server, I had only limited time frames to work with when making config changes. Finally after seeing the JSON error message I knew what to do to fix the problem.
I'm working a long time with Mule application and AMQPConnector.
Before some days I saw an ERROR: Too many open files, crash, and can't send request till I'm restarted the Mule application.
Mule code is something like this: (critical parts)
<amqp:connector name="AMQPConnector" validateConnections="true"
doc:name="AMQPConnector" host="x.x.x.x" port="5672"
password="xxxxx" username="xxx">
<reconnect-forever frequency="1000" blocking="false" />
From flows:
<flow name="first">
<http:listener config-ref="HTTP_Listener_Configuration" path="order/submit" doc:name="HTTP"/>
<byte-array-to-object-transformer doc:name="Byte Array to Object"/>
<json:object-to-json-transformer doc:name="Object to JSON"/>
<json:json-to-object-transformer returnClass="java.lang.Object" doc:name="JSON to Object"/>
<set-session-variable value="#[payload]" variableName="order" doc:name="Session Variable" />
<set-payload doc:name="order" value="#[payload.info]" />
<amqp:outbound-endpoint queueName="xxxx"
responseTimeout="100000" exchange-pattern="request-response"
connector-ref="AMQPConnector" doc:name="XXXX" />
<object-to-string-transformer doc:name="Object to String"/>
<logger message="response from queue #[payload]"
level="INFO" doc:name="Logger" />
</flow>
<flow name="second">
<amqp:inbound-endpoint queueName="xxxx"
responseTimeout="10000" exchange-pattern="request-response"
connector-ref="AMQPConnector" doc:name="AMQP-0-9" />
<byte-array-to-object-transformer doc:name="Byte Array to Object" />
<json:object-to-json-transformer doc:name="Object to JSON" />
<http:request config-ref="HTTP_Request_XXXX" host="#[sessionVars['partUrl']]" path="#[sessionVars['path']]" method="#[sessionVars['method']]" doc:name="HTTP">
<http:request-builder>
<http:header headerName="#[sessionVars['key']]" value="#[sessionVars['value']]"/>
</http:request-builder>
</http:request>
</flow>
The error exception log is:
2016-11-27 04:30:12,272 [amqpReceiver.1018] ERROR org.mule.exception.CatchMessagingExceptionStrategy -
Message : Error sending HTTP request. Message payload is of type: String
Code : MULE_ERROR--2
Exception stack is:
1. Too many open files (java.net.SocketException) sun.nio.ch.Net:-2 (null)
2. java.net.SocketException: Too many open files (java.util.concurrent.ExecutionException)
org.glassfish.grizzly.impl.SafeFutureImpl$Sync:349 (null)
3. java.util.concurrent.ExecutionException: java.net.SocketException: Too many open files (java.io.IOException)
org.mule.module.http.internal.request.grizzly.GrizzlyHttpClient:223 (null)
4. Error sending HTTP request. Message payload is of type: String (org.mule.api.MessagingException)
org.mule.module.http.internal.request.DefaultHttpRequester:287 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/api/MessagingException.html)
Root Exception stack trace:
java.net.SocketException: Too many open files
at sun.nio.ch.Net.socket0(Native Method)
at sun.nio.ch.Net.socket(Net.java:411)
at sun.nio.ch.Net.socket(Net.java:404)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
Request was accepted by http-listner and system stopped.
Maybe miss one configuration, or close connection....
(I saw same questions with solutions to increase system or add code in java class etc... - it wasn't help me, I don't have java classes. )
Someone can help me?
This is an issue with Anypoint Studio. Unfortunately your operating system can only open so many files. Depending on your operating system this limit varies, however I experienced this same issue in both Ubuntu and macOS using the latest Anypoint Studio (5 to 6).
The best thing I can suggest is to restart and try and keep your open files to a minimum at any one point whilst using Anypoint. When I'm only working on 4-5 files at a time I can usually go an entire day without needing to restart.
With heavy usage I end up needing to restart within 5 hours.
There seems to be a lot of people asking questions about this, and yet the folks over there have decided to close my question before it's resolved; the perception being that I'm a lazy developer and haven't read documentation - not the case: https://github.com/IdentityServer/IdentityServer3/issues/3083
I've followed the instructions here: https://identityserver.github.io/Documentation/docsv2/configuration/logging.html
but I cannot get logging to happen.
My question really at this point in time is, assuming I haven't done something wrong, does it matter that I'm firing things from a unit test method?
I have a separate unit test project which is just requesting a token and writing the response out, but I'm getting a 500 error somewhere and I'd assume logging would tell me why.
The test was working before I had some repo issues a while ago and lost some work, and I THINK I am back to where I was, but I'm sure the root cause will be something simple that I've overlooked - it usually is.
Anyway, I really hope someone can help, and not just be snotty.
Many thanks!
First adding logger to IdentityServer Configuration (if Owin, under Startup public void Configuration(IAppBuilder app) method;
Serilog.Log.Logger = new LoggerConfiguration()
.WriteTo.Trace(outputTemplate: "{Timestamp} [{Level}] ({Name}){NewLine} {Message}{NewLine}{Exception}")
.CreateLogger();
Then adding webconfig the following under configuration;
<system.diagnostics>
<sources>
<source name="Thinktecture.IdentityServer" switchValue="Information, ActivityTracing">
<listeners>
<add name="xml" type="System.Diagnostics.XmlWriterTraceListener" initializeData="trace.svclog" />
</listeners>
</source>
</sources>
<trace autoflush="true" indentsize="4">
<listeners>
<add name="sybsListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="Trace.log" />
<remove name="Default" />
</listeners>
</trace>
After running, and trying to interact with the IndentityServer, you can check the Trace.log file under your project. Don't forget selecting "Show All Files"
Based on Trace.log, can you specify the problem again if you can't solve?
I managed get logging working by wrapping my unit test in
using (var logger = new LoggerConfiguration()
.MinimumLevel.Debug()
.WriteTo.File(#"C:\Users\Richard Terris\Desktop\idsLogs.txt")
.CreateLogger())
{
It's not the best solution possibly, but it works!
Thanks for the replies!
I have a MVC3 web application consisting Bing Translate API. Everything works fine on the emulator, but when I deploy it to Windows Azure, I encounter this issue again and again ( I retry more than twice):
Instance 0 of role Website is busy
Instance 0 of role Website is cycling
It stops at cycling for very long.
And in portal, I see this message:
Waiting for role to start... System is initializing
I already selected the "Add deployable assemblies", and check true all the reference Copy Local = true
I also checked connection string to my Account Storage, I set my project to work on cloud through an Account Storage.
And here my web.config for Bing Translate API
<system.serviceModel>
<bindings>
<basicHttpBinding>
<binding name="BasicHttpBinding_LanguageService" closeTimeout="00:25:00"
openTimeout="00:25:00" receiveTimeout="00:25:00" sendTimeout="00:25:00"
allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"
maxBufferSize="65536" maxBufferPoolSize="524288" maxReceivedMessageSize="65536"
messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"
useDefaultWebProxy="true">
<readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"
maxBytesPerRead="4096" maxNameTableCharCount="16384" />
<security mode="None">
<transport clientCredentialType="None" proxyCredentialType="None"
realm="" />
<message clientCredentialType="UserName" algorithmSuite="Default" />
</security>
</binding>
</basicHttpBinding>
</bindings>
<client>
<endpoint address="http://api.microsofttranslator.com/V2/soap.svc"
binding="basicHttpBinding" bindingConfiguration="BasicHttpBinding_LanguageService"
contract="BingTranslator.LanguageService" name="BasicHttpBinding_LanguageService" />
</client>
</system.serviceModel>
I have searched for a long time but nothing works for me. Please give me your help.
Best Regards
I can hit your something about how to translate these error to some information:
Case One: You status is looping within the Web Role:
WebRole Starting
WebRole Busy
WebRole Cycling
WebRole Unknown State
WebRole Restarting
This is potentially a role specific problem and Your best bet is to RDP to your instances and look for several things i.e. Azure Diagnostics Log for potential exception, Event Logs, Azure BootStrapper Logs, IIS Configurator Logs, etc. This will help you to find the problem faster then any other method, unless you can get Azure Diagnostics logs from Azure Storage having all the details about your potential problem.
Case Two: Your status is looping within the Full Machine:
Starting...
Initializing...
WebRole Starting
WebRole Busy
WebRole Cycling
WebRole Restarting
This could be a machine specific problem where the VM itself is cycling, this is good candidate to contact Windows Azure Support team to investigate for you.
We have a jboss esb server which is reading files from the file system in a scheduled way (schedule frequency of 20sec) and convert them into the esb message then we parse the message.
There are some other providers/listeners (jms) and services configured on the esb servers. When there is an error in one of the services it effects the above process. File system provider (gateway) is working fine but the jms-listener who takes the gateway messages are not working and lots of messages are accumulated in the jbm queue (jbm_msg Oracle DB table).
Here is the problem, when my server is restarted messages in the jbm-queue is parsed in the esb for just 20 seconds which is the scheduled frequency of fs-provider, never process messages again and cpu usage goes up to 100% and stays there. We believe somehow fs-providers interrupts the jms-provider.
Is there any configuration we have been missing out.
Here are the configuration files that we have:
jboss-esb.xml
<?xml version = "1.0" encoding = "UTF-8"?>
<jbossesb xmlns="http://anonsvn.labs.jboss.com/labs/jbossesb/trunk/product/etc/schemas/xml/jbossesb-1.0.1.xsd" parameterReloadSecs="5">
<providers>
<fs-provider name="SitaIstProvider">
<fs-bus busid="gw_sita_ist" >
<fs-message-filter
directory="/ikarussita/IST/IN"
input-suffix=".RCV"
work-suffix=".lck"
post-delete="false"
post-directory="/ikarussita/IST/OK"
post-suffix=".ok"
error-delete="false"
error-directory="/ikarussita/IST/ERR"
error-suffix=".err"/>
</fs-bus>
</fs-provider>
<jms-provider name="SitaESBQueue" connection-factory="ConnectionFactory">
<jms-bus busid="esb_sita_queue">
<jms-message-filter dest-type="QUEUE" dest-name="queue/esb_sita_queue"/>
</jms-bus>
</jms-provider>
</providers>
<services>
<service category="SITA" name="SITA_IST" description="SITA Daemon For ISTCOXH">
<listeners>
<fs-listener name="Sita_Ist_Gateway" busidref="gw_sita_ist" is-gateway="true" schedule-frequency="20" />
<jms-listener name="Jms_Sita_EsbAware" busidref="esb_sita_queue" />
</listeners>
<actions mep="OneWay">
<action name="parse_msg" class="com.celebi.integration.action.sita.inbound.SitaHandler" process="parseMessage" />
<action name="send_ikarus" class="com.celebi.integration.action.ikarus.outbound.fis.FlightJmsSender" />
</actions>
</service>
</services>
</jbossesb>
jbm-queue-service.xml
<?xml version="1.0" encoding="UTF-8"?>
<server>
<mbean code="org.jboss.jms.server.destination.QueueService"
name="jboss.messaging.destination:service=Queue,name=esb_sita_queue"
xmbean-dd="xmdesc/Queue-xmbean.xml">
<depends optional-attribute-name="ServerPeer">jboss.messaging:service=ServerPeer</depends>
<depends>jboss.messaging:service=PostOffice</depends>
</mbean>
<server>
deployment.xml
<jbossesb-deployment>
<depends>jboss.messaging.destination:service=Queue,name=esb_sita_queue</depends>
</jbossesb-deployment>
Thanx
Split the service into 2 separate services, one handling the JMS queue, the other the file poller. Specify the same action pipeline. That way you get the same functionality but without the threading issue. Also use max-threads attr on the listener to specify the number of reading threads.