After enabling In-Memory in Database, I was trying to send a table to I-Memory by using ALTER TABLE EMP INMEMORY;
The Table was altered and then I had executed a simple query to check whether the table is actually altered. My query was as below;
SELECT ENAME, COUNT(*)
FROM EMP
WHERE ENAME='JAMES'
GROUP BY ENAME;
Query executed successfully, but when I execute a statement on V$IM-SEGMENTS, no rows had been selected.
If I'm not mistaken, When a table had moved to ORACLE IN-MEMORY, all segments should have been recorded with V$IM-SEGMENTS. If it is so why does my query does not showing in the v$im-segments?
Can someone help me out on this please?
This is because of the default priority clause "priority none".
The documentation says:
On-demand population
By default, the INMEMORY PRIORITY parameter is set to NONE. In this
case, the database only populates the object when it is accessed
through a full table scan. If the object is never accessed, or if it
is accessed only through an index scan or fetch by rowid, then
population never occurs.
So, in your case you need to do full scan (select query covering all the rows in the table) to induce the population of table into inmemory segments. Following lists the altered steps:
Rem -- Enable table for inmemory
alter table emp inmemory;
Rem -- Perform full scan
select count(*) from emp;
Rem -- Sleep for few seconds
Rem -- This is to ensure segments are fully
Rem -- loaded into inmemory area before
Rem -- query v$im_segments
exec dbms_lock.sleep(30);
Rem -- Now query v$im_segments
select substr(segment_name,1,20), populate_status, bytes_not_populated from v$im_segments;
Related
I execute several processes to continuously (update + select) data from postgresql (with autocommit option enabled), and get data which should already be filtered out by executing previous update done by other processes.
This is essentially a queue, with all rows initially having status=0, and selected row-by-row in some complicated (not shown) order. Each pending row is set status=1, pid=(process id) and uniqueid=(some iterating id), but different processes for some reason share same rows.
Oversimplified (but still incorrectly working) example is below:
Table definition:
create table Todq (
id bigserial primary key,
url varchar(1024),
status integer not null,
pid integer,uniqueid bigint,
priority integer
);
create index Iodq1 on Todq (status,priority,id);
create index Iodq2 on Todq (pid,uniqueid);
Update (gets next element in queue, sets process pid and iterating uniqueid):
update Todq odq2
set status=1,pid=?,uniqueid=?
from (
select odq.id
from Todq odq
where odq.status=0
order by odq.priority desc,odq.id
limit 1
) odq1
where odq2.id=odq1.id
Select (selects by process id and incrementable uniqueid):
select odq.id,odq.url,odq.pid,odq.uniqueid
from Todq odq
where odq.status=1 and pid=? and uniqueid=?
I made a test bench in perl, which writes values selected to logs/$$.log and as a result different logs share same entries (though with different pid and uniqueid).
The answer is to use (select .. for update) like:
update Todq odq2
set status=1,pid=?,uniqueid=?
from (
select odq.id
from Todq odq
where odq.status=0
order by odq.priority desc,odq.id
limit 1
for update
) odq1
where odq2.id=odq1.id
I have a situation where I have multiple (potentially hundreds) threads repeating the same task (using a java scheduled executor, if you are curious). This task entails selecting rows of changes (from a table called change) that have not yet been processed (processed changes are kept track in a m:n join table called process_change_rel that keeps track of the process id, record id and status) processing them, then updating back the status.
My question is, how is the best way to prevent two threads from the same process from selecting the same row? Will the below solution (using for update to lock rows ) work? If not, please suggest a working solution
Create table change(
—id , autogenerated pk
—other fields
)
Create table change_process_rel(
—change id (pk of change table)
—process id (pk of process table)
—status)
Query I would use is listed below
Select * from
change c
where c.id not in(select changeid from change_process_rel with cs) for update
Please let me know if this would work
You have to "lock" a row which you are going to process somehow. Such a "locking" should be concurrent of course with minimum conflicts / errors.
One way is as follows:
Create table change
(
id int not null generated always as identity
, v varchar(10)
) in userspace1;
insert into change (v) values '1', '2', '3';
Create table change_process_rel
(
id int not null
, pid int not null
, status int not null
) in userspace1;
create unique index change_process_rel1 on change_process_rel(id);
Now you should be able to run the same statement from multiple concurrent sessions:
SELECT ID
FROM NEW TABLE
(
insert into change_process_rel (id, pid, status)
select c.id, mon_get_application_handle(), 1
from change c
where not exists (select 1 from change_process_rel r where r.id = c.id)
fetch first 1 row only
with ur
);
Every such a statement inserts 1 or 0 rows into the change_process_rel table, which is used here as a "lock" table. The corresponding ID from change is returned, and you may proceed with processing of the corresponding event in the same transaction.
If the transaction completes successfully, then the row inserted into the change_process_rel table is saved, so, the corresponding id from change may be considered as processed. If the transaction fails, the corresponding "lock" row from change_process_rel disappears, and this row may be processed later by this or another application.
The problem of this method is, that when both tables become large enough, such a sub-select may not work as quick as previously.
Another method is to use Evaluate uncommitted data through lock deferral.
It requires to place the status column into the change table.
Unfortunately, Db2 for LUW doesn't have SKIP LOCKED functionality, which might help with such a sort of algorithms.
If, let's say, status=0 is "not processed", and status<>0 is some processing / processed status, then after setting these DB2_EVALUNCOMMITTED and DB2_SKIP* registry variables and restart the instance, you may "catch" the next ID for processing with the following statement.
SELECT ID
FROM NEW TABLE
(
update
(
select id, status
from change
where status=0
fetch first 1 row only
)
set status=1
);
Once you get it, you may do further processing of this ID in the same transaction as previously.
It's good to create an index for performance:
create index change1 on change(status);
and may be set this table as volatile or collect distribution statistics on this column in addition to regular statistics on table and its indexes periodically.
Note that such a registry variables setting has global effect, and you should keep it in mind...
So I have added a table to inmemory and have scanned the table after that. But it is still not appearing in V$IM_SEGMENTS. In EXPLAIN PLAN it is showing INMEMORY ACCESS FULL. So not sure if it is using the column store.
Did these:
ALTER TABLE INMEMORY;
SELECT * FROM ;
SELECT * FROM V$IM_SEGMENTS;
no rows
To start with inmemory_size should be around 100M.
Following command should show appropriate size value for parameter inmemory_size:
show parameter inmemory_size
Loading of table segments into inmemory area kicks when there is a full scan on the table or inmemory priority clause is other than none, so we need to be sure the select query you had done went through table access full path.
So, one more way to initiate full table scan is to do select count(*) from table.
Or you can use populate procedure from dbms_inmemory package to load the table manually into inmemory area.
Example usage (for user inmem_user, table t1):
exec dbms_inmemory.populate('INMEM_USER','T1');
One more thing to consider here with respect to querying v$im_segments is; bytes_not_populated and populate_status columns also to be queried for correctness.
When v$im_segments returns rows, bytes_not_populated should be 0 and populate_status should be COMPLETED.
More information about inmemory population can be foune here
When you want to use postgres's SELECT FOR UPDATE SKIP LOCKED functionality to ensure that two different users reading from a table and claiming tasks do not get blocked by each other and also do not get tasks already being read by another user:
A join is being used in the query to retrieve tasks. We do not want any other table to have row-level locking except the table that contains the main info. Sample query below - Lock only the rows in the table -'task' in the below query
SELECT v.someid , v.info, v.parentinfo_id, v.stage FROM task v, parentinfo pi WHERE v.stage = 'READY_TASK'
AND v.parentinfo_id = pi.id
AND pi.important_info_number = (
SELECT MAX(important_info_number) FROM parentinfo )
ORDER BY v.id limit 200 for update skip locked;
Now if user A is retrieving some 200 rows of this table, user B should be able to retrieve another set of 200 rows.
EDIT: As per the comment below, the query will be changed to :
SELECT v.someid , v.info, v.parentinfo_id, v.stage FROM task v, parentinfo pi WHERE v.stage = 'READY_TASK'
AND v.parentinfo_id = pi.id
AND pi.important_info_number = (
SELECT MAX(important_info_number) FROM parentinfo) ORDER BY v.id limit 200 for update of v skip locked;
How best to place order by such that rows are ordered? While the order would get effected if multiple users invoke this command, still some order sanctity should be maintained of the rows that are being returned.
Also, does this also ensure that multiple threads invoking the same select query would be retrieving a different set of rows or is the locking only done for update commands?
Just experimented with this a little bit - multiple select queries will end up retrieving different set of rows. Also, order by ensures the order of the final result obtained.
Yes,
FOR UPDATE OF "TABLE_NAME" SKIP LOCKED
will lock only TABLE_NAME
Due to a concurrency issue in my project which happend due to 2 threads coming in together to do a select at same time, both recieve the same values which ideally should not happen.
After selecting a value it should perform a update and then second thread should select the updated value.
Am using DB2
I thought of using this approach of using
select number from final table(update tablename set columnanme=""
where )
.
My question is would this approach lock the db when the other thread comes in to select the value as theere is an update within select? and solve my concurrency issue.
OR
I was browsing and found another approach
update table (.....) select col from table where wait for
outcome
Would this select wait until the first thread finishes the select?
One thing you can certainly do to avoid multiple reads of the same value before it gets updated by one of the readers:
LOCK TABLE tablename IN EXCLUSIVE MODE;
SELECT id, ... FROM tablename WHERE ...;
UPDATE tablename SET id=newval WHERE ...;
COMMIT;
This will of course block the full table, which is maybe not what you want!
Alternative approach (relatively standard, but somewhat more involved programming logic):
SELECT id, ... FROM tablename WHERE ...
SELECT count(1) FROM FINAL TABLE (UPDATE tablename SET id=newval
WHERE ... AND id=newval);
while this count(1) is zero (meaning: someone else meanwhile updated
it) repeat from 1)
--Peter Vanroose,
ABIS Training & Consulting,
Leuven, Belgium.