I have a queue block where agents enter and exit. Each agent has the following attributes (parameters): ID, processing time and due date. After one hour, I want to use an event block to collect the info (ID,processing time and due date) of the agents that are still waiting in the queue at that moment, and write this info to Excel. In Excel, I would like 3 columns, one with the ID's, one with the processing times and one with the due dates of each order in the queue.
I have tried adding and removing info to a LinkedList, but this did not work. Does anyone know how I could get the information I need?
In your event, simply loop across all agents in the queue and write a dbase query to insert data for them using the insertInto syntax.
Could look like this:
for (int i=0; i<queue.size(); i++) {
MyAgent currentAgent = queue.get(i);
insertInto(myDbaseTable)
.columns(myDbaseTable.column1, myDbaseTable.column2)
.values(currentAgent.someInfo, currentAgent.otherInfo)
.execute();
}
Related
I want to have a periodic celery task to do the following:
Get the list of receipt items not yet scraped from the data base
Call a task to scrap a specific website for every item on the previous list. Each task will return a list of products
Save the products on the data base and update the receipts so it won't be used on the next call of the periodic task
I think I could accomplish that by using chain, chunks and group, but I don't know how to do it. My idea was to chain the list from item 1 as input to item 2, but I would like every item on the list to be a separate task. It seems that "chunks" is a good candidate, but how can I pipe the list from item 1 to it? And how can I use the length of this list to define the number of tasks? Is it possible?
Than the returned values of each one of those tasks would be pipe to a "group" composed of the tasks to save the products to the db and update the receipts.
I am solving a job shop scheduling problem resorting to anylogic. I have 20 jobs (agents) and 5 machines(resources) and each job as a specific order to visit the machines. My question is: how can I make sure that each job follows its order.
This is what I have done. One agent called 'jobs' and 5 agents, each one corresponding to a machine. One resource pool associated to each one of the service blocks. In the collection enterblocks I selected the 5 enter blocks.
In the agent 'jobs' I have this. The parameters associated to each job, read from the database file, and the collection 'enternames' where I selected the machine(1,2,3,4,5) parameters and the collection 'ptimes' where I put the processing times of the job (This two colletions is where I am not sure I have done it correctly)
My database file
I am not sure how to use the counter used here How to store routings in job shop production in Anylogic. In the previous link the getNextService function is used in the exit blocks but I am also not sure how to use it in my case due to the counter.
Firstly, to confirm that based on the Job agent and database view, the first line in the database will result in a Job agent with values such as:
machine1 = 1 and process1=23
machine2 = 0 and process2=82 and so on
If that is the intent, then a better way is to restructure the database, so there are two tables:
Table of jobs to machine sequence looking something like this:
job
op1
op2
op3
op4
op5
1
machine2
machine1
machine4
machine5
machine3
2
machine4
machine3
machine5
machine1
machine2
3
...
...
...
...
...
Table of jobs to processing time
Then, add a collection of type ArrayList of String to Job (let's call this collection col_machineSequence) and when the Job agents get created their on startup code should be:
for (String param : List.of("op1","op2","op3","op4","op5")) {
col_machineSequence.add(getParameter(param));
}
As a result, col_machineSequence will contain sequence of machines each job should visit in the order defined in the database.
NOTE: Please see help on getParameter() here.
Also:
Putting a Queue in front of the Service isn't necessary
Repeating Enter-Queue-Service-Exit isn't necessary, this can be simplified using this method
Follow-up clarifications:
Collections - these will be enclosed in each Job agent
Queue sorting - Service block has Priorities / preemption which governs the ordering on the queue
Create another agent for the second table (call the agent ProcessingTime and table processing_time) and add it to the Job agent and then load it from database filtering on p_jobid as shown in the picture
I have a spark-streaming job in which I receive data from a message queue and process a bunch of records. In the process, I have a take() method on a dataset. Although the take action is happening in an expected manner, In the DAG visualization, I see multiple job ids created and all of them have the same take action. This is happening only when the data is in the order of a hundreds of thousand records. I didn't observe redundant jobs while running with tens of records in my local machine. Can anyone help me understand the reasoning behind this behavior?
The job ids - (91 to 95) are basically running the same action. Following is the code snippet corresponding to the mentioned action above.
val corruptedMessageArray: Array[ String ] = corruptedMessageDs.take(1);
if ( !corruptedMessageArray.isEmpty ) {
val firstCorruptedMessage: String = corruptedMessageArray( 0 )
}
Your question seems to be whether duplicate jobs are created by Spark.
If you look at the screenshot you will see that the jobs have a different number of tasks, hence it is not a simple matter of duplication.
I am not sure exactly what is happening, but it seems that for large datasets take() needs several quick subsequent jobs. Perhaps because it devises work, or perhaps because it needs to try how much work needs to be done.
I have a very short question regarding the batching process in Anylogic.
I would like to print out the IDs of the agents that already exited the previous batch element where they were batched together. As a result, they are at a different element (Release to be precise) and I am struggling to reach their ID inside the batch. The only idea I have is to first unbatch and then print out the IDs.
Is there a way to do it without unbatching them?
Thank you very much in advance.
Kind regards
All batched (not permanently) or picked up agents are stored in the collection named 'contents' inside a batch/container agent.
So, you can access IDs of the agents stored in this collection using the code like:
for(int i = 0; i < agent.contents().size(); i++)
traceln(((MyAgent)agent.contents().get(i)).id);
My requirement is
Parallel Job1 --I extract data from a table, when row count is more than 0
Parallel job 2 should be triggered in the sequencer only when the row count from source query in Job1 is greater than 0
I want to achieve this without creating any intermediate file in job1.
So basically what you want to do is using information from a data stream (of your Job1) and use it in the "above" sequence as a parameter.
In your case you want to decide on sequence level to run subsequent jobs (if more than 0 rows get returned) or not.
Two options for that:
Job1 writes information to a file which is a value file of a parameterset. These files are stored in a fixed directory. The parameter of the value file could then be used in your sequence to decide your further processing. Details for parameter sets can be found here.
You could use a server job for Job1 and set a user status (basic function DSSetUserStatus) in a transfomer. This is also passed back to the sequence and could be referenced in subsequent stages of the sequence. See the documentation but you will find many other information on the internet as well regarding this topic.
There are more solution to this problem - or let us call it challenge. Other ways may be a script called at sequence level which queries the database and will avoid Job1...