I am writing an application that exports data and serializes it to file for archiving old data.
There may be occasions where for some reason select data needs to be re-imported. This has been causing me a problem because of an identity column.
To get around this I am performing the work inside a transaction scope. Setting the Identity Insert On for that table and then updating my transaction e.g.
using (TR.TransactionScope scope = new TR.TransactionScope(TR.TransactionScopeOption.RequiresNew))
{
// allow transaction nbr to be inserted instead of auto generated
int i = context.ExecuteStoreCommand("SET IDENTITY_INSERT dbo.Transactions ON");
try
{
// check if it already exists before restoring
var matches = context.Transactions.Where(tr => tr.transaction_nbr == t.transaction_nbr);
if (matches.Count() == 0)
{
Transaction original = t;
context.Transactions.AddObject(original);
context.SaveChanges();
restoreCount++;
But I receive an exception saying:
Explicit value must be specified for identity column in table either when IDENTITY_INSERT >is set to ON or when a replication user is inserting into a NOT FOR REPLICATION identity >column.
I am assuming the entity framework is trying to do some sort of block insert without specifying the columns. Is there anyway to do this in the entity framework.
The object is large and has a number of associated entities that are also deserialized and need inserting so I want to let the entity framework do this if possible as it will save me a lot of extra work.
Any help is appreciated.
Gert Arnold - that was the answer to my question. Thank you.
I did read that elsewhere and had set the value in the object browser so thought this was suffice. I also double checked the value by right clicking the .edmx and Open With to see the details in an XML editor as suggested in another post.
When I checked the value in the XML editor initially it too was "None" so assumed this wasn't my problem. But I guess just going in there and saving rectified the problem first time around.
The second time round after I must have updated the model from the database I had to repeat this step upon your suggestion. But the second time round the StoreGeneratorPattern was different to the one set in the object browser so I needed to manually change it in the XML.
In my circumstance this is fine since normally the records are inserted via another mechanism so the identity is just getting in the way for me as the identity will always be inserted as an old (used to exist) identity value which is being temporarily restored.
Thanks for your help.
Related
I use SQL Server with ASP.NET Core and EF Core. After each record is added, the identity column's value jumps about by 1000 and creates a gap between current row and the last previously added row.
Questions
Is there any way to prevent this?
How to delete those gaps that have been created before?
If I use GUID for key columns to prevent that issue, is there a problem (performance or each other problem)?
Is it way on the server side that with EF Core could handle it (each some way)?
Thank you in advance for your helps...
For the reason for 1000-value gaps, see Aaron Bertrand's answer
It doesn't really make sense to "want" to delete the gaps. The content of an identity column contains no semantic information. It correlates to nothing "in the world" outside the database. The gaps are as meaningless as the values themselves.
I don't see how a uniqueidentifier would "prevent" that issue. A uniqueidentifier may be "meaningfully" sortable (if you use newsequentialid()), but there's no sense in which any particular value is "one more" than a previous value.
You can certainly try to build your own key generating algorithm that does not produce gaps, but you will run into concurrency issues (also mentioned by Mr Bertrand).
workaround trick:
CREATE OR ALTER TRIGGER TGR_Transaction_Identity_Fix
ON [dbo].[TBL_Transaction]
FOR INSERT
AS
BEGIN
DECLARE #RESEEDVAL INT
SELECT #RESEEDVAL = MAX(TransactionId) FROM [dbo].[TBL_Transaction]
DBCC CHECKIDENT([TBL_Transaction], RESEED, #RESEEDVAL)
END
this triger will reset identity on each insert
I am referencing the 2 step newsletter example at http://agiletoolkit.org/codepad/newsletter. I modified the example into a 4 step process. The following page class is step 1, and it works to insert a new record and get the new record id. The problem is I don't want to insert this record into the database until the final step. I am not sure how to retrieve this id without using the save() function. Any ideas would be helpful.
class page_Ssp_Step1 extends Page {
function init(){
parent::init();
$p=$this;
$m=$p->add(Model_Publishers);
$form=$p->add('Form');
$form->setModel($m);
$form->addSubmit();
if($form->isSubmitted()){
$m->save();//inserts new record into db.
$new_id=$m->get('id');//gets id of new record
$this->api->memorize('new_id',$new_id);//carries id across pages
$this->js()->atk4_load($this->api->url('./Step2'))->execute();
}
}
}
There are several ways you could do this, either using atk4 functionality, mysql transactions or as a part of the design of your application.
1) Manage the id column yourself
I assume you are using an auto increment column in MySQL so one option would be to not make this auto increment but use a sequence and select the next value and save this in your memorize statement and add it in the model as a defaultValue using ->defaultValue($this->api->recall('new_id')
2) Turn off autocommit and create a transaction around the inserts
I'm from an oracle background rather than MySQL but MySQL also allows you to wrap several statements in a transaction which either saves everything or rollsback so this would also be an option if you can create a transaction, then you might still be able to save but only a complete transaction populating several tables would be committed if all steps complete.
In atk 4.1, the DBlite/mysql.php class contains some functions for transaction support but the documentation on agiletoolkit.org is incomplete and it's unclear how you change the dbConnect being used as currently you connect to a database in lib/Frontend.php using $this->dbConnect() but there is no option to pass a parameter.
It looks like you may be able to do the needed transaction commands using this at the start of the first page
$this->api->db->query('SET AUTOCOMMIT=0');
$this->api->db->query('START TRANSACTION');
then do inserts in various pages as needed. Note that everything done will be contained in a transaccion so if the user doesnt complete the process, nothing will be saved.
On the last insert,
$this->api->db->query('COMMIT');
Then if you want to, turn back on autocommit so each SQL statement is committed
$this->api->db->query('SET AUTOCOMMIT=1');
I havent tried this but hopefully that helps.
3) use beforeInsert or afterInsert
You can also look at overriding the beforeInsert function on your model which has an array of the data but I think if your id is an auto increment column, it won't have a value until the afterInsert function which has a parameter of the Id inserted.
4) use a status to indicate complete record
Finally you could use a status column on your record to indicate it is only at the first stage and this only gets updated to a complete status when the final stage is completed. Then you can have a housekeeping job that runs at intervals to remove records that didn't complete all stages. Any grid or crud where you display these records would be limited with AddCondition('status','C') in the model or added in the page so that incomplete ones never get shown.
5) Manage the transaction as non sql
As suggested by Romans, you could store the result of the form processing in session variables instead of directly into the database and then use a SQL to insert it once the last step is completed.
I'm new to postgreSQL, so would really appreciate any pointers from the community.
I am updating some functionality in the CMS of a pretty old site I've just inherited. Basically, I need the ID of an article before it is inserted into the database. Is there anyway anyway to check the next value that will be used by a sequence before a database session (insert) has begun?
At first I thought I could use SELECT max(id) from tbl_name, however as the id is auto incremented from a sequence and articles are often deleted, it obviously won't return a correct id for the next value in the sequence.
As the article isn't in the database yet, and a database session hasn't started, it seems I can't use the currval() functionality of postgreSQL. Furthermore if I use nextval() it auto increments the sequence before the data is inserted (the insert also auto-incrementing the sequence ending up with the sequence being doubly incremented).
The way I am getting around it at the moment is as follows:
function get_next_id()
{
$SQL = "select nextval('table_id_seq')";
$response = $this->db_query($SQL);
$arr = pg_fetch_array($query_response, NULL, PGSQL_ASSOC);
$id = (empty($arr['nextval'])) ? 'NULL' : intval($arr['nextval']);
$new_id = $id-1;
$SQL = "select setval('table_id_seq', {$new_id})";
$this->db_query($SQL);
return $id;
}
I use SELECT nextval('table_id_seq') to get the next ID in the sequence. As this increments the sequence I then immediately use SELECT setval('table_id_seq',$id) to set the sequence back to it's original value. That way when the user submits the data and the code finally hits the INSERT statement, it auto increments and the ID before the insert and after the insert are identical.
While this works for me, I'm not too hot on postgreSQL and wonder if it could cause any problems down the line, or if their isn't a better method? Is there no way to check the next value of a sequence without auto-incrementing it?
If it helps I'm using postgresql 7.2
Folks - there are reasons to get the ID before inserting a record. For example, I have an application that stores the ID as part of the text that is inserted into another field. There are only two ways to do this.
1) Regardless of the method, get the ID before inserting to include in my INSERT statement
2) INSERT, get the the ID (again, regardless of how (SELECT ... or from INSERT ... RETURNING id;)), update the record's text field that includes the ID
Many of the comments and answers assumed the OP was doing something wrong... which is... wrong. The OP clearly stated "Basically, I need the ID of an article before it is inserted into the database". It should not matter why the OP wants/needs to do this - just answer the question.
My solution opted to get the ID up front; so I do nextval() and setval() as necessary to achieve my needed result.
Disclaimer: Not sure about 7.2 as I have never used that.
Apparently your ID column is defined to get its default value from the sequence (probably because it's defined as serial although I don't know if that was available in 7.x).
If you remove the default but keep the sequence, then you can retrieve the next ID using nextval() before inserting the new row.
Removing the default value for the column will require you to always provide an ID during insert (by retrieving it from the sequence). If you are doing that anyway, then I don't see a problem. If you want to cater for both scenarios, create a before insert trigger (does 7.x have them?) that checks if the ID column has a value, if not retrieve a new value from the sequence otherwise leave it alone.
The real question though is: why do you need the ID before insert. You could simply send the row to the server and then get the generated id by calling curval()
But again: you should really (I mean really) talk to the customer to upgrade to a recent version of Postgres
I am obtaining a json array from a url and inserting data into a table. Since the contents of the url are subject to change, I want to make a second connection to a url and check for updates and insert new records in y table using sqlite3.
The issues that I face are:
1) My table doesn't have a primary key
2) The url lists the changes on the same day. Hence, if I run my app multiple times, when I insert values in my database, I get duplicate entries. I want to keep a check for the day duplicated entries that should be removed. The problem can be solved by adding a constraint, but since the url itself has duplicated values, I find it difficult.
The only way I can see you can do it if you have no primary key or something you can use that is unique to each record, is when you get your new data in you go through the new entries where for each one you check if the exact same data exists in the database already. If it doesn't then you add it, if it does then you skip over it.
You could even do something like create a unique key yourself for each entry which is a concatenation of each column of the table. That way you can quickly do the check for if the entry already exists in the database.
I see two possibilities depending on your setup:
You have a column setup as UNIQUE (this can be through a PRIMARY KEY or not). In this case, you can use the ON CONFLICT clause:
http://www.sqlite.org/lang_conflict.html
If you find this construct a little confusing, you can instead use "INSERT OR REPLACE" or "INSERT OR IGNORE" as described here:
http://www.sqlite.org/lang_insert.html
You do not have a column setup as UNIQUE. In this case, you will need to SELECT first to verify for duplicate data, and based on the result INSERT, UPDATE, or do nothing.
A more common & robust way to handle this is to associate a timestamp with each data item on the server. When your app interrogates the server it provides the timestamp corresponding to the last time it synced. The server then queries its database and returns all values that are timestamped later than the timestamp provided by the app. Then it also returns a new timestamp value for the app to store, to use on the next sync.
Can any one help me why this error occurs when i update using sqlDataadapter with join query
Dynamic SQL generation is not supported against multiple base tables.
You have a "join" in your main query for your dataset (The first one in the TableAdapter with a check by it). You can't automatically generate insert/update/delete logic for a TableAdapter when the main query has multiple tables referenced in the query via a join. The designer isn't smart enough to figure out which table you want to send updates to in that case, that is why you get the error message.
Solution. Ensure that your main query only references the table you want the designer to write insert/update/delete code for. Your secondary queries may reference as many tables as you want.
It was in the case that i was trying to set value for identity column in my datarow. Simply i deleted the code to set value for identity column and it will work.
My Scenario:
Database:
uin [primary, identity]
name
address
Whenever i tried to set the datarow("uin") the error occurs. But works fine with datarow("name") and datarow("address").
Hope it works for you too