I have a huge mdf file on my SQL Server and doubt concerning its data integrity. I am performing manipulations with tables in this DB and noticed about many disk bad block reports in my Windows System Event Log (the File System is NTFS). I suspect that this errors may be connected to my operation with this DB. My questing is in the case some part of data in the mdf file is corrupted does SQL Server have possibility to detect this data integrity problem? Is there some error checking mechanism in the mdf: per-record or per-table crc, etc? In the case it is not performed automatically how could I test it manually?
This is not a programming question but.
DBCC CHECKDB (Transact-SQL)
DBCC CHECKTABLE (Transact-SQL)
If you are throwing bad block errors then most likely you have a disk going bad. You need to fix you hardware problem.
Related
Referring to the official documentation, I now find that there are two ways:
set param zero_damaged_pages on, However, it is not recommended because data loss may occur, and I do not Know if the database is corrupt;
set checksum on, This can cause significant performance costs, and You can find the database is corrupted when you query
I want to restore data using a backup database when find current database is corrupted on embedded device. Is there more convenient way to find out whether postgresql database is corrupt just like sqlite? In sqlite, Database corruption can be detected by the API return value:
#define SQLITE_CORRUPT 11 /* The database disk image is malformed */
Both ways won't reliably detect all kinds of data corruption. zero_damaged_pages won't help at all, and checksums will only detect data corruption on disk (as opposed to corruption in RAM or caused by database bugs).
Some data corruption can be detected by nasty error messages, for example if you dump the database (which selects all data). Other types of data corruption causes no errors, but bad results.
I often have to execute complex sql scripts in a single transaction on a large PostgreSQL database and I would like to verify everything that was changed during the transaction.
Verifying each single entry on each table "by hand" would take ages.
Dumping the database before and after the script to plain sql and using diff on the dumps isn't really an option since each dump would be about 50G of data.
Is there a way to show all the data that was added, deleted or modified during a single transaction?
Dude, What are you looking for is the most searchable thing on the internet when it comes to capturing Database changes. It is a kind of version control we can say.
But as long as I know, sadly there are no in-built approaches are available in PostgreSQL or MySql. But you can overcome it by setting/adding some triggers for your most usable operations.
You can create some backup schemas, and tables to capture your changes that are changed(updated), created, or deleted.
In this way you can achieve what you want. I know this process is fully manual, But really effective.
If you need to analyze the script's behaviour only sporadically, then the easiest approach would be to change server configuration parameter log_min_duration_statement to 0 and then back to any value it had before the analysis. Then all of the script activity will be written to the instance log.
This approach is not suitable if your storage is not prepared to accommodate this amount of data, or for systems in which you don't want sensitive client data to be written to a plain-text log file.
The PostgreSQL database server stores "change data" in WAL log file, and I wanted to parse the archive log file to sql like mysqlbinlog parse binlog file to sql, That I can find the application execute sql. Does anyone have a tool like this?
You can't. It's the changes to the actual disk blocks.
You can set the server to log all the SQL statements to file if you would like though. Not sure you'd be able to replay them without being very clear about transaction boundaries though.
This feature is currently under development. (Look for "logical replication" patches by Andres Freund.) It's a huge project, so don't hold your breath. The short answer is: It's currently not possible.
If you are feeling adventurous, xlogdump might get you part way to extracting data from your WAL segments. If you truly only need the SQL that gets executed in your cluster, then set log_min_duration_statement = 0 to log all statements.
Now you can replicate with SQL. Look at pglogical. However, it doesn't cover schema changes.
What the title says -
Msg 9002, Level 17, State 4, Line 1
The transaction log for database 'tempdb' is full. To find out why space in the log cannot be reused, see the log_reuse_wait_desc column in sys.databases
The query in question first pulls out some rows from a databased on the linked server (matching strings in the server I'm querying from), stores them in a table, then uses this table to pull out more matching records from the linked server. This is what I get on the second part.
The basic question is, is there something else hiding in this error message? Which tempdb is it, my tempdb, or the linked server's tempdb? I don't think mine can be a problem, as increasing the size doesn't help, the recover mode is simple, autogrowth is on, etc.
You first need to check your SQL Server's tempDB.... is the drive that TempDB and its log got lots of free disc space? It might be on two different drives. I would expect such an error to write a message in the SQL Server error log - have you checked there as well at the time of the problem? You then need to do the same on the remote server, if you have access.
Whether it's tempDB or a user/application database, just because the recovery model is simple doesn't mean that the transaction log won't grow - and take all the disk space! It does make it less likely, but long transactions can cause it to "blow".
i know this has been asked here. But my question is slightly different. When the dataset was designed keeping the disconnected principle in mind, what was provided as a feature which would handle unexpected termination of the application, say a power failure or a windows hang or system exception leading to restart. Say the user has entered some 100 rows and it is modified at the dataset alone. Usually the dataset is updated at the application close or at a timely period.
In old times which programming using vb 6.0 all interaction used to take place directly with the database, thus each successful transaction was committing itself automatically. How can that be done using datasets?
DataSets are never for direct access to database, they are a disconnected model only. There is no intent that they be able to recover from machine failures.
If you want to work live against the database you need to use DataReaders and issue DbCommands against the database live for changes. This of course will increase your load on the database server though.
You have to balance the two for most applications. If you know a user just entered vital data as a new row, execute an insert command to the database, and put a copy in your local cached DataSet. Then your local queries can run against the disconnected data, and inserts are stored immediately.
A DataSet can be serialized very easily, so you could implement your own regular backup to disk by using serialization of the DataSet to the filesystem. This will give you some protection, but you will have to write your own code to check for any data that your application may have saved to disk previously and so on...
You could also ignore DataSets and use SqlDataReaders and SqlCommands for the same sort of 'direct access to the database' you are describing.