No sure if there's a lot of act-r programmer on here, but I can't seem to find a forum/group for that anywhere so...
I'm writing a program which as a chunk defined as (and the goal below):
(chunk-type position position-x position-y)
(chunk-type goal state last-pos)
In a production, I'm fetching the position of a thing on screen from the visual-location and then I would need to create a position chunk and put that in my goal's last-pos slot. Here's the production rule:
(P attend-projectile
=goal>
ISA goal
state nil
=visual-location>
screen-x =pos-x
screen-y =pos-y
?visual>
state free
==>
+visual>
cmd move-attention
screen-pos =visual-location
=goal>
state attended
last-pos (position pos-x screen-x pos-y screen-y)
)
Or something like that. I've tried various syntaxe. The problem boils down to:
I need to instantiate a chunk within a production (the position chunk) based on values recovered in the lhs,
then assign that chunk to a goal's slot.
Somehow I can't seem to find an equivalent example in the doc...
EDIT:
I do need this to be a chunk, not just storing the x & y position. Eventually this chunk will be extended to include an ID (which will be obtained from the visual location, e.g. a different letter will be assigned to each moving object). I will be tracking those object through time. Because I'm tracking through time, another chunk (trajectory) will contain 3 position chunks (with their IDs).
Other productions will expect to find this chunk (trajectory, once I have 3 position chunks) and make decisions based on that.
Obviously the above is a snippet of the code. But the conceptual difficulty I have is manipulating (instantiating/creating however it's called in actr nomentalture) chunks at runtime, essentially.
Why do you need another chunk? You have the chunk in the visual-location buffer with that information so why not use it:
(P attend-projectile
=goal>
ISA goal
state nil
=visual-location>
?visual>
state free
==>
+visual>
cmd move-attention
screen-pos =visual-location
=goal>
state attended
last-pos =visual-location
)
Of course, that doesn't answer the question that was asked.
To create a new chunk, the proper way to do so is through a request to the imaginal buffer which would then require a following production to harvest the result and place it into the slot of the goal buffer's chunk. Assuming that the slots you want in the new chunk are from the position chunk-type you show, and that the values are from the similarly named slots of the chunk in the visual-location buffer, this would create the new chunk in the imaginal buffer:
(P attend-projectile
=goal>
ISA goal
state nil
=visual-location>
screen-x =pos-x
screen-y =pos-y
?visual>
state free
==>
+visual>
cmd move-attention
screen-pos =visual-location
=goal>
state attended
+imaginal>
position-x =pos-x
position-y =pos-y
)
Related
I've been given a task to set up an openai toy gym which can only be solved by an agent with memory. I've been given an example with two doors, and at time t = 0 I'm shown either 1 or -1. At t = 1 I can move to correct door and open it.
Does anyone know how I would go about starting out? I want to show that a2c or ppo can solve this using an lstm policy. How do I go about setting up environment, etc?
To create a new environment in gym format, it should have the 5 functions mentioned in the gym.core file.
https://github.com/openai/gym/blob/e689f93a425d97489e590bba0a7d4518de0dcc03/gym/core.py#L11-L35
To lay this down in steps-
Define observation space and action space for your environment, preferably using gym.spaces module.
Write down the step function which performs agent's action and returns a 4 tuple containing - next set of observations from the environment , reward ,
done - a boolean indicating whether the episode is over , and some extra info if you want.
Write a reset function for the environment to reinitialise the episode to a random start state and return a 4 tuple similar to step.
These functions are enough to be able to run an RL agent on your environment.
You can skip the render, seed and close functions if you want.
For the task you have defined,you can model the observation and action space using Discrete(2). 0 for first door and 1 for second door.
Reset would return in it's observation which door has the reward.
Then agent would choose either of the door - 0 or 1.
Then perform a environment step by calling step(action), which will return agent's reward and done flag as true - signifying that the episode is over.
Frankly, the problem you describe seems too simple to accomplish for any reinforcement learning algorithm, but I assume you have provided that as an example.
Remembering for longer horizons is usually harder.
You can read their documentation and toy environments to understand how to create one.
I have many data and I have experimented with partitions of cardinality [20k, 200k+].
I call it like that:
from pyspark.mllib.clustering import KMeans, KMeansModel
C0 = KMeans.train(first, 8192, initializationMode='random', maxIterations=10, seed=None)
C0 = KMeans.train(second, 8192, initializationMode='random', maxIterations=10, seed=None)
and I see that initRandom() calls takeSample() once.
Then the takeSample() implementation doesn't seem to call itself or something like that, so I would expect KMeans() to call takeSample() once. So why the monitor shows two takeSample()s per KMeans()?
Note: I execute more KMeans() and they all invoke two takeSample()s, regardless of the data being .cache()'d or not.
Moreover, the number of partitions doesn't affect the number takeSample() is called, it's constant to 2.
I am using Spark 1.6.2 (and I cannot upgrade) and my application is in Python, if that matters!
I brought this to the mailing list of the Spark devs, so I am updating:
Details of 1st takeSample():
Details of 2nd takeSample():
where one can see that the same code is executed.
As suggested by Shivaram Venkataraman in Spark's mailing list:
I think takeSample itself runs multiple jobs if the amount of samples
collected in the first pass is not enough. The comment and code path
at GitHub
should explain when this happens. Also you can confirm this by
checking if the logWarning shows up in your logs.
// If the first sample didn't turn out large enough, keep trying to take samples;
// this shouldn't happen often because we use a big multiplier for the initial size
var numIters = 0
while (samples.length < num) {
logWarning(s"Needed to re-sample due to insufficient sample size. Repeat #$numIters")
samples = this.sample(withReplacement, fraction, rand.nextInt()).collect()
numIters += 1
}
However, as one can see, the 2nd comment said it shouldn't happen often, and it does happen always to me, so if anyone has another idea, please let me know.
It was also suggested that this was a problem of the UI and takeSample() was actually called only once, but that was just hot air.
I'm using a STM32F401VCT6U "discovery" board, and I need to provide a way for the user to write addresses in memory at runtime.
I wrote what can be simplified to the following function:
uint8_t Write(uint32_t address, uint8_t* values, uint8_t count)
{
uint8_t index;
for (index = 0; index < count; ++index) {
if (IS_FLASH_ADDRESS(address+index)) {
/* flash write */
FLASH_Unlock();
if (FLASH_ProgramByte(address+index, values[index]) != FLASH_COMPLETE) {
return FLASH_ERROR;
}
FLASH_Lock();
} else {
/* ram write */
((uint8_t*)address)[index] = values[index]
}
}
return NO_ERROR;
}
In the above, address is the base address, values is a buffer of size at least count which contains the bytes to write to memory and count the number of bytes to write.
Now, my problem is the following: when the above function is called with a base address in flash and count=100, it works normally the first few times, writing the passed values buffer to flash. After those first few calls however, I cannot write just any value anymore: I can only reset bits in the values in flash, eg an attempt to write 0xFF to 0x7F will leave 0x7F in the flash, while writing 0xFE to 0x7F will leave 0x7E, and 0x00 to any value will be successful (but no other value will be writable to the address afterwards).
I can still write normally to other addresses in the flash by changing the base address, but again only a few times (two or three calls with count=100).
This behaviour suggests that the maximum write count of the flash has been reached, but I cannot imagine it can be so fast. I'd expect at the very least 10,000 writes before exhaustion.
So what am I doing wrong?
You have missunderstood how flash works - it is not for example as straight forward as writing EEPROM. The behaviour you are discribing is normal for flash.
To repeatidly write the same address of flash the whole sector must be first erased using FLASH_EraseSector. Generally any data that needs to preserved during this erase needs to be either buffered in RAM or in another flash sector.
If you are repeatidly writing a small block of data and are worried about flash burnout do to many erase write cycles you would want to write an interface to the flash where each write you move your data along the flash sector to unwriten flash, keeping track of its current offset from the start of sector. Only then when you run out of bytes in the sector would you need to erase and start again at start of sector.
ST's "right way" is detailed in AN3969: EEPROM emulation in STM32F40x/STM32F41x microcontrollers
This is more or less the process:
Reserve two Flash pages
Write the latest data to the next available location along with its 'EEPROM address'
When you run out of room on the first page, write all of the latest values to the second page and erase the first
Begin writing values where you left off on page 2
When you run out of room on page 2, repeat on page 1
This is insane, but I didn't come up with it.
I have a working and tested solution, but it is rather different from #Ricibob's answer, so I decided to make this an answer.
Since my user can write anywhere in select flash sector, my application cannot handle the responsability of erasing the sector when needed while buffering to RAM only the data that need to be preserved.
As a result, I transferred to my user the responsability of erasing the sector when a write to it doesn't work (this way, the user remains free to use another address in the sector to avoid too many write-erase cycles).
Solution
Basically, I expose a write(uint32_t startAddress, uint8_t count, uint8_t* values) function that has a WRITE_SUCCESSFUL return code and a CANNOT_WRITE_FLASH in case of failure.
I also provide my user with a getSector(uint32_t address) function that returns the id, start address and end address of the sector corresponding to the address passed as a parameter. This way, the user knows what range of address is affected by the erase operation.
Lastly, I expose an eraseSector(uint8_t sectorID) function that erase the flash sector whose id has been passed as a parameter.
Erase Policy
The policy for a failed write is different from #Ricibob's suggestion of "erase if the value in flash is different of FF", as it is documented in the Flash programming manual that a write will succeed as long as it is only bitreset (which matches the behavior I observed in the question):
Note: Successive write operations are possible without the need of an erase operation when
changing bits from ‘1’ to ‘0’.
Writing ‘1’ requires a Flash memory erase operation.
If an erase and a program operation are requested simultaneously, the erase operation is
performed first.
So I use the macro CAN_WRITE(a,b), where a is the original value in flash and b the desired value. The macro is defined as:
!(~a & b)
which works because:
the logical not (!) will transform 0 to true and everything else to false, so ~a & b must equal 0 for the macro to be true;
any bit at 1 in a is at 0 in ~a, so it will be 0 whatever its value in b is (you can transform a 1 in 1 or 0);
if a bit is 0 in a, then it is 1 in ~a, if b equals 1 then ~a & b != 0 and we cannot write, if bequals 0 it's OK (you can transform a 0 to 0 only, not to 1).
List of flash sector in STM32F4
Lastly and for future reference (as it is not that easy to find), the list of sectors of flash in STM32 can be found on page 7 of the Flash programming manual.
The following space allocation is giving me an sB37 JCL error. The cobol size of the output file is 100 bytes and the lrecl size is 100 bytes. What do you think is causing this error? I have tried increase the size to 500,100 and still get the same error.
Code:
//OUTPUT1 DD DSN=A.B.C,DISP=(NEW,CATLG,DELETE),
// DCB=(LRECL=100,BLKSIZE=,RECFM=FBM),
// SPACE=(CYL,(10,5),RLSE)
Try to increase not only the space, but the volume as well.
Include VOL=(,,,#) in your DD. # is the numbers of values you want to allocate
Ex: SPACE=(CYL,(10,5),RLSE),VOL=(,,,3) - includes 3 volumes.
Additionally, you can increase the size, but try to stay within reasonable limits :)
The documentation for B37 says the application programmer should respond as indicated for message IEC030I. The documentation for IEC030I says, in part...
Probable user error. For all cases, allocate as many units as volumes
required.
...as noted in another answer. However, be advised that the documentation for the VOL parameter of the DD statement says...
If you omit the volume count or if you specify 1 through 5, the system
allows up to five volumes; if you specify 6 through 20, the system
allows 20 volumes; if you specify a count greater than 20, the system
allows 5 plus a multiple of 15 volumes. You can override the maximum
volume count in data class by using the volume-count subparameter. The
maximum volume count for an SMS-managed mountable tape data set or a
Non-managed tape data set is 255.
...so for DASD allocations you are best served specifying a volume count greater than 5 (at least).
//OUTPUT1 DD DSN=A.B.C,DISP=(NEW,CATLG,DELETE),
// DCB=(LRECL=100,BLKSIZE=,RECFM=FBM),
// SPACE=(CYL,(10,5),RLSE)
Try this instead. Notice that the secondary will take advantage of a large dataset whereas without that parameter the largest secondary that makes any sense is < 300. Oh, and if indeed it is from a COBOL program make sure that the FD says "BLOCK 0"!!!!! If it isn't "BLOCK 0" then you might not even need to change your JCL because it wasn't fixed block machine. It was merely fixed and unblocked so the space would almost never be enough. And finally you may wish to revisit why you have the M in the RECFM to begin with. Notice also that I took out the LRECL, the BLKSIZE and the RECFM. That is because the FD in the COBOL program is all you need and putting it in the JCL is not only redundant but dangerous because any change will have to be now done in multiple places.
//OUTPUT1 DD DSN=A.B.C,DISP=(NEW,CATLG,DELETE),
// DSNTYPE=LARGE,UNIT=(SYSALLDA,59),
// SPACE=(CYL,(10,1000),RLSE)
There is a limit of 65,535 tracks per one volume. So if you will specify a SPACE that exceeds that limit - system will simply ignore it.
You can increase this limit to 16,777,215 tracks by adding DSNTYPE=LARGE paramter.
Or you can specify that your dataset is a multi volume by adding VOL=(,,,3)
You can also use DATACLAS=xxxx paramter here, however first of all you need to find it. Easy way is to contact your local Storage Team and ask for one. Or If you are familiar with ISPF navigation, you can enter ISMF;4 command to open a panel
use bellow paramters before hitting enter.
CDS Name . . . . . . 'ACTIVE'
Data Class Name . . *
It should produce a list of all available data classes. Find the one that suits you ( has enougth amount of volume count, does not limit primary and secondary space
I am currently developing a basic image processing application on MATLAB. I have to implement undo to previous state feature. I searched net there is uiundo but it only undos GUI actions. Is there a simple command to undo. Thanks.
If you have plenty of memory to spare, you could store all of your program states in a structure and then push this structure into a circular buffer. The number of elements in the buffer would determine the number of levels of undo.
It seems to me that you would have to implement your own multi-level (or one-level) undo by using the Command pattern.
This would require you to wrap your operations into objects that contain the logic to perform the action and to restore the state. No silver bullet, hard work needed.
General undo will be complicated, but if you are only interested in saving the state of MATLAB variables and returning to a saved state, this might be one possible solution:
save_state.m:
SavedStateFolder = '/home/user/matlab_saved_states_folder/';
save([ SavedStateFolder, 'saved_state_', sprintf('%06d', (size(dir(SavedStateFolder), 1) - 1)) ]);
fprintf('state saved in saved_state_%s.mat\n', sprintf('%06d', (size(dir(SavedStateFolder), 1) - 2)));
undo_index.m:
function undo_index()
SavedStateFolder = '/home/user/matlab_saved_states_folder/';
FilesStruct = dir(SavedStateFolder);
LastSavedStateIndex = size(FilesStruct, 1) - 2;
if (LastSavedStateIndex < 1)
fprintf('No saved states available.\n');
else
fprintf('Index of last saved state is %06d\n', LastSavedStateIndex);
end
return
undo.m:
SavedStateFolder = '/home/user/matlab_saved_states_folder/';
load([ SavedStateFolder, 'saved_state_', sprintf('%06d', input('Enter saved state index ')) ]);
Then you can use save_state to save the state of MATLAB variables or you can even prefix all your commands with save_state; if you want to keep track of all changes. When you want to return to a previous state of variables, you can run undo_index to find out the index of last saved state and then you can run undo and input the index given by undo_index, or alternatively any smaller positive integer to return to an earlier state of variables. The first saved state file will be named saved_state_000001.mat, then saved_state_000002.mat and so on... Note that save_state saves everything to disk, so using SSD or RAM disk might be a good idea if you want to try this in a loop with a lot of data. Note also that the previous content of variable called SavedStateFolder is lost when you run save_state or undo. To avoid this, you can replace all instances of SavedStateFolder in save.state.m and undo.m with hard-coded folder names, for example:
save_state.m:
save([ '/home/user/matlab_saved_states_folder/', 'saved_state_', sprintf('%06d', (size(dir('/home/user/matlab_saved_states_folder/'), 1) - 1)) ]);
fprintf('state saved in saved_state_%s.mat\n', sprintf('%06d', (size(dir('/home/user/matlab_saved_states_folder/'), 1) - 2)));
undo.m:
load([ '/home/user/matlab_saved_states_folder/', 'saved_state_', sprintf('%06d', input('Enter saved state index ')) ]);
Note also that save_state, undo_index and undo assume that in SavedStateFolder there are no other files except ., .. and saved_state_*.mat files. Also the number of saved_state_*.mat files and the running index to be saved is found out this way, so if you delete some previous saved_state_*.mat files without eg. creating equal number of empty files to replace them, the counts don't match and save_state might save the state on top of an already existing saved state file.