I am working on some in which the speed and time are of high importance. I am using profiler to find the bottleneck of my code, but i cannot understand some things in profiler.
first, what does self and total time mean?
second, it has something called workspacefunc>local_min and workspacefunc>local_max, what are they?
self time is the total time spent in a function, not including any spent in any child functions called. As an example, if you had a function which was calling a whole bunch of other functions, the profiler only includes the time spent in the main function called from the profiler and not in any of the other functions defined inside the main function.
total time is the total time spent on a function (makes sense, right?). This includes the timing in all of the child functions called. Also, you need to be careful where the profiler itself can take some time to execute as well, which is included in the results. One small thing as well: the total time can be zero for functions whose running time are inconsequential.
Reference: http://www.mathworks.com/help/matlab/matlab_prog/profiling-for-improving-performance.html
workspacefunc... there doesn't seem to be any documentation on it, but this is the help text that I get when checking what it does:
workspacefunc Support function for Workspace browser component.
The Workspace browser is a window that shows you all of the variables that are defined in your workspace. If I were to take an educated guess, profiler does some analysis on your workspace variables, which include the min and max of certain variables in your workspace. I can't really say much more as there is absolutely no documentation on this, but it's safe to ignore. Simply focus on the functions that you are calling from your own code.
Related
I am working on a complex Modelica model that contains a large set of data, and I need the simulation to keep going until I terminate the simulation process, maybe even for days, so the .mat file could get very large, I got trouble with how to do data processing. So I'd like to ask if there are any methods that allow me to
output the data I need after a fixed time step during simulation, but not using the .mat file after simulation. I am considering using Modelica.Utilities.Stream.Print` function to print the data I need into a CSV file, but I have to write a huge amount of code that prints every variable I need, so I think there should be a better solution.
delete the .mat file during a fixed time step, so the .mat file stored on my PC wouldn't get too large, and don't affect the normal simulation of Dymola.
Long time ago I wrote a small C-program that runs the executable of Dymola with two threads. One of them is responsible for terminating the whole simulation after exceeding an input time limit. I used the executable of this C-program within the standard given mfiles from Dymola. I think with some hacking capabilities, one would be able to conduct the mentioned requirements.
Have a look at https://github.com/Mathemodica/dymmat however I need to warn that the associated mfiles were for particular type of models and the software is not maintained since long time. However, the idea of the C-program would be reproducible.
I didn't fully test this, so please think of this more like "source of inspiration" than a full answer:
In Section "4.3.6 Saving periodic snapshots during simulation" of the Dymola 2021 Release Notes you'll find a description to do the following:
The simulator can be instructed to print the simulation result file “dsfinal.txt” snapshots during simulation.
This can be done periodically using the Simulation Setup options "Complete result snapshots", but I think for your case it could be more useful to trigger it from the model using the function Dymola.Simulation.TriggerResultSnapshot(). A simple example is given as well:
when x > 0 then
Dymola.Simulation.TriggerResultSnapshot();
end when;
Also one property of this function could help, as it by default creates multiple files without overwriting them:
By default, a time stamp is added to the snapshot file name, e.g.: “dsfinal_0.1.txt”.
The format of the created dsfinal_[TIMESTAMP].txt is a bit overwhelming at first, as it contains all information for initializing the model, but there should be everything you need...
So some effort is shifted to the post processing, as you will likely need to read multiple files, but I think this is an acceptable trade-off.
I am modeling ticket system with various SLA. The model must contain several service blocks with different reaction time ( from 2 to 32 hours). In the service block only working hours should be taken into account. So in the service block timeout should stop when non-workong hours and on the weekend. Could you please kindly tell me how i can realize it?
Thank you very much in advance!
I can think of two answers, one simplified but works in many cases, the other more advanced and probably more accurate:
Simplified approach: I would set the model in hours and keep everything running as is without any stop. So, at the end of the simulation, if the total time is 100 hours and you know that you have 8 hours/day with 5 days/week, then you'd know the total duration is 2.5 weeks. Of course, this might have limitations or might become more complex later on if you want day-specific actions (e.g. you want to differentiate between Monday, Tuesday, etc.)
Advanced more accurate approach: Create resources whose capacities are defined by schedule and assigned them to your services. Create a schedule and specify the working hours in that schedule. Check the below link to learn more about schedules. I call this the more advanced approach because you need to make sure the schedule is defined correctly and make sure all elements in the model are properly controlled (e.g. non-service blocks such as source, delays, etc.).
https://help.anylogic.com/topic/com.anylogic.help/html/data/schedule.html?resultof=%22%73%63%68%65%64%75%6c%65%73%22%20%22%73%63%68%65%64%75%6c%22%20
I personally would use the first approach if the model is rather simple and modeling working hours is enough for analysis. Otherwise, I'd go for option 2.
Finally, another option I'd like to highlight is the "suspend/resume" functions. I am only adding this because you asked "how to stop timeout". So these functions specifically stop and resume timeout. But you'll need to define the times at which they are executed (through an event for example).
I have a semi-slow memory leak in a Talend joblet. I am using a tHashOutput/tHashInput pair in the middle of a joblet because I need to find out how many rows are in the flow. Therefore, I push them into a tHashOutput and later on reference tHashOutput_1_NB_LINE from the globalMap.
I have what I think are the proper options:
allRows - "append" is FALSE
tHashinput_1 - "Clear after reading" is TRUE
Yet, when I run this for a period of time, and analyzing with the Eclipse Memory Analyzer, I see objects building up over time. This is what I get after 12 hours:
This usage (64MB/12 hours) increases steadily and is unrelated to what the job is doing (i.e. actively pumping data or just idling - and this code while invoked for idling also). If I look inside the memory references in MAT, I can see strings that point me to this place in the code, like
tHashFile_DAAgentProductAccountCDC_delete_BPpuaT_jsonToDataPump_1_tHashOutput_2
(jsonToDataPump being the name of the joblet). Am I doing something wrong in using these hash components?
i believe you should set your garbage collector pace to minimum time duration so that it will take care of unused object in application
I have a Matlab code (from a journal paper) and I'm trying to re-simulate their data.
I executed the code one week ago. I think the code is taking so long time to run. Matlab is still busy and taking 50% of my cpu.
I was wondering if the process has ended with some errors somewhere in the code. My question is:
When I see no errors, can I be sure that everything is fine with this running process? And I can wait until it is finished?
Is there any way to check which part of code is being run now ( without stopping the execution)?
Or I should stop the program and try something else?
Actually I don't want to loose this 1 week and if you think everything is fine, I would wait until the code stops.
(The authors of the paper didn't reply to my question and I don't know how long should it naturally take... They just mentioned it may take a long time to simulate the data).
Unfortunately, there is little we can do for you.
When I see no errors, can I be sure that everything is fine with this running process?
That's pretty much the definition of an error. If no error is raised, then it means that the program is still running.
Is there any way to check which part of code is being run now (without stopping the execution)?
Unfortunately no. For long-lasting execution times like that, a good developing practice is to display some information from time to time to inform the end user of the execution status.
However, if the programs produces files all along the way (like for instance at every step in an iterative simulation) you can check on your computer that the files are well-produced, and the production rate will more or less inform you on the total execution time.
For all your other questions, well, it's up to you to decide what to do (stop it or let it run). Be aware that the execution time can differ significantly from one machine to another, so the time it took on the author's machine may not be really informative to you.
In the future, I would advise you to react faster than within a week. When you launch a code that has a long execution time and see that there is no display within the first hour, you should stop it, modify it such that it regulatly displays information, and re-run it. It's better to loose one hour than one week.
Best,
I'm completely confused about Devel::NYTProf reports generated by nytprofhtml. I'm using old version of NYTProf 1.90. I know it is very old version but should use it for number of reasons.
So these HTML reports look something
like this (when looking on particular *.pl file report):
|Line|Stmts.| Time | Avg. |Code|
|42 | 6804 | 0.04506 | 7e-06 | }; |
I have never seen reports from new version of nytprofhtml, so not sure if they look the same.
In, my case this line is most slow part in whole program (it not a small program).
So my question is how can statement like this '};' be slowest part in the program with lot more complex statements. I think misunderstand what NYTProf reports.
If my question is confusing just give me definitions of each column from these reports this will help? This will help a lot.
Especially I'm interested what Stmts. mean. I'm guessing, but I don't want to guess!
Thanks in advance.
Stmts. is the number of times the statement was executed or, more precisely, the number of times execution moved from a statement associated with that line (which is not always accurate) to whichever statement was executed next.
Time is the sum of the time spent executing statements associated with that line.
Avg. is simply Time divided by Stmts.
These extracts from the current Devel::NYTProf documentation may help:
The statement profiler measures the time between entering one perl statement and entering the next. Whenever execution reaches a new statement, the time since entering the previous statement is calculated and added to the time associated with the line of the source file that the previous statement starts on. [...]
For example, given:
while (<>) {
...
1;
}
After the first time around the loop, any further time spent evaluating the condition (waiting for input in this example) would be recorded as having been spent on the last statement executed in the loop.
More recent versions of NYTProf, of which there are many, offer much more accurate timings for this situation by intercepting the appropriate internal loop opcodes, plus many other significant improvements.