Monday, 30 September 2013

SQL Server :Part 1 : Architecture of Transaction Log

In our earlier post, we have discussed in detail about the architecture of the data file and different types of pages. In this post, we will be discussing about the architecture of log file.

Logging is a mechanism used in RDBMS to support various ACID (Atomicity,Consistency,Isolation and Durability) properties of transactions. A transaction log is a physical file in which SQL server stores the details of all transactions and data modifications performed on the database.In the event of of disaster, that causes SQL server to shutdown unexpectedly (Power failure/hardware failure), the transaction log is used to bring the database in a consistent state while restarting the server.On restarting the server, database goes through the recovery process.During this recovery process , the transaction log is used to make sure that all committed transactions are written to respective data pages (rolled forward) and revert the uncommitted transaction that were written to data pages.

Logically transaction log is a set of log records.Each records is identified by a log sequence number (LSN). The new log records is always written at the logical end of log file with a LSN which is greater than the previous one.Each LSN will be associated with a transaction id , which can be used to group the log records of a transaction. As log file store the log records in the sequential order as it happens, It is not necessary that, log records of a transaction are always available in sequence in the log file.Each log records will have the LSN of previous log as a backward pointer and that will help in rollback of transaction.

Transaction log will store separate log entries for each operation.For example, while inserting a record into a table, transaction log will store separate log entry for inserting into clustered index and other non clustered index. In the same way, if a single update statement is updating 10 records, transaction log will capture 10 separate log entries.For data modification, transaction log store either the logical operation performed or the before and after image of the record.

To understand it better, let us look into the transaction log using the sys.fn_dblog. It is an undocumented function which will help us to read the active portion of the log. we are using the below query to create two tables and insert some records into that.We will delete the records from these table to keep the table clean. This insert and delete operation is to make sure that the pages are allocated to the table and we will not get unnecessary entries in the transaction log while examining it.We have issued a manual checkpoint to force the SQL server to write the log information to data file and clear the log. Note that, one table is heap table and other one is clustered index table.

USE Mydb
id INT)


INSERT INTO LoginfoHeap VALUES(1),(2)
DELETE FROM LoginfoHeap 
[Current LSN]
Operation  ,Context    ,
[Transaction ID],
[Previous LSN],AllocUnitName,[
Previous Page LSN],
[Begin Time],[End Time]
FROM sys.fn_dblog (NULL, NULL)

From the output it is very clear that , we have only two active log entries.First one is written ,when the checkpoint started. The second one is written,  when the checkpoint completed the process.

Now we will insert,update and delete records to these tables through two session 


   INSERT INTO LoginfoHeap VALUES(1),(2)


   UPDATE LoginfoHeap   SET id =WHERE id=1

   UPDATE LoginfoCI  SET id =WHERE id=2

DELETE FROM LoginfoHeap    WHERE id=2

   DELETE FROM LoginfoCI   WHERE id=4
   SELECT FROM sys.dm_tran_current_transaction

   SELECT FROM sys.dm_tran_current_transaction

The DMV sys.dm_tran_current_transcation returns a single row that displays the state information of the current transaction in the current session.We are interested only in the transaction_id, which will help us to filter the output of sys.fn_dblog. Let us see the output of sys.fn_dblog.

[Current LSN]
Operation  ,
Context    ,
[Transaction ID],
[Previous LSN],
[Previous Page LSN],
[Page ID],[XACT ID],
[Begin Time],
[End Time]
FROM sys.fn_dblog (NULL, NULL) 
WHERE [Transaction ID] IN 
   SELECT [Transaction ID] FROM sys.fn_dblog (NULL, NULL) 
   WHERE [XACT ID] IN (856960,856981)

The values 856960 and 856981 are the transaction_id returned from sys.dm_tran_current_transaction.We have filter the output to get only the relevant rows in which we are interested.

In our script, we have opened two transaction and all our transaction log entries are grouped to 
one of these transaction_id marked in red and green.Let us analyse what we did and how it is captured in the transaction log.

In the session 1, we have started the transaction and inserted a single record.The first records in the output map to the BEGIN TRAN command. This is the starting point of the transaction and created a new transaction_id.The previous LSN column value is 0 as this is the first log entry in this transaction.In the same log records,it stores the XACT_ID,login and transaction start time.The second record represent the insert into the clustered table.The transaction_id is used to group the entries associated with a transaction. The previouse LSN column, is a pointer to the previous log entry in the same transaction which help SQL server to move backwards in case of rollback.Page id column refer the the page number where this LSN made the change.Previous Page LSN column refer the last log sequence number(LSN) which modify this page.When LSN modify a page, it will also update the corresponding LSN number in the page header (m_lsn field in the header. For more detail refer this post)

In the session 2, we have opened another transaction and inserted two records through single insert statement to the heap table. You can map these operations to row number 3,4, and 5 in the transaction log output. Third row represent the Begin tran command. Even if we inserted two records in single insert statement , SQL server recorded two separate entry in the transaction log. 

As a next step, in session 1 we have added 1 record to the clustered index table.We can map this operation to the 6th record in the transaction log output.

In the next statement , we have modified a record in heap table through Session 2. You can map this to the 7th record in the transaction log output.If you look into the previous LSN column , it will be current LSN column value of the last record associated with this transaction.

In the same way, as a next statement we have modified a record in the clustered table through session 1. We can map the 8th and 9th records in the transaction log output to the update operation on the clustered table. You might have noticed that, when we modified a record in the heap table, transaction log recorded operation in a single row. Where as the same operation in a clustered table has two record in the transaction log. One for delete and other one for insert. When you modify the clustered index key, SQL server internally delete the existing record and insert a new record. This is because, the record need to be stored in the new location based on the modified value(based on the order of clustered index column). The easiest way for SQL server to achieve this is , delete the existing record and insert it as new records with modified clustered column value.

In the next two statement, we are deleting one record from heap table and clustered table.This can be mapped to the 10th and 11th records in the output.Finally we have issued the commit statement in both sessions.12th and 13th record in the transaction log output can be mapped to the commit operation.The Previous LSN column refer the Current LSN column of corresponding  begin tran statement. It will also capture the transaction end time in the End time column.

Understanding the VLF(Virtual Log File)

A database can have one or more log file. In general there will be only one log file as there is no performance improvement by having multiple log file. SQL server uses the transaction log in sequential manner.As the data file divided into pages,log files are divided into virtual log file(VLF).The size of the VLFs  in a log file may not be in equal size. SQL server decide the size and number of VLF in a log file based on the size of the log file growth as given below.

Growth upto 64 MB          = 4  VLF
From 64 MB to 1 GB       = 8   VLF
Larger than 1 GB             = 16 VLF

Let us create a database with 64 MB initial log size and later increase it to 1 GB. As per above calculation the log file should have 12 VLFs.  4 VLF based on initial size and 8 VLF due to changing the log size to 1 GB.

(      NAME = MyDb_dat,  FILENAME = 'D:\MyDb\Mydb.mdf',
       SIZE = 10MB, MAXSIZE = 3072MB,   FILEGROWTH = 5MB )
LOG ON ( NAME = MyDb_log,FILENAME = 'D:\MyDb\MyDB.ldf',


MODIFY FILE ( NAME = MyDb_Log,FILENAME = 'D:\MyDb\MyDB.ldf',    SIZE = 1024MB)

Now Let us see how many VLF got created. To find out the number of VLF in database log file, we can make use of DBCC Loginfo.

DBCC loginfo('mydb')

The output is given below.

There are 12  records in the output each represent a VLF.Let us try to understand the result

FileId: This is the file id of the log file and will be same for all 12 records as we have only one log file.If we have multiple log file , we can multiple numbers here

FileSize: This is the size of the VLF. If you look into the first four, have same size except the fourth one. This because first 8KB of the log file is used for file header. If you add filesize value of first four records along with 8192(8KB) , you will get 64MB which is the initial size of the log file.
16711680+16711680+16711680+16965632 =67100672+8192 =67108864bytes =64MB
In the same if you add the last 8 records it will account the 960 MB (1024-64) , the growth happened due to the alter statement.

StartOffSet: This values is also in bytes, and is the sort column of the output. The first VLF alwasy start from 8192, which is the number of bytes in a page.As mentioned above, the first 8KB is used for file header and will not store any log.

FSeqNo: The file sequence number indicates the order of usage of the VLFs. The row with the highest FSeqNo value is the VLF where current log records are being written.FSeqNo values are not consistent. It will keep changing each time when VLF are getting reused. We will discuss more about this later in this post. A value of 0 in this column means that this VLF has never been used at all. That is the reason we have 0 for all records except one where it is currently logging.

Status: Status has two possible values : 0 and 2. A value of 2 means the VLF is not reusable and a value 0 means it can be reused.It will be more clear as we go further.

Parity: Parity has three possible values 0 ,64 and 128. If the VLF is not used yet, it will have a value 0 and will be set to 64 on first use.Every time a VLF is reused, the parity value is switched between 64 and 128.

CreateLSN: The value indicates when the VLF is created or to group the VLF based on the creation. A values 0 indicates, those VLFs are created as part of database creation. In our case first four records has a value 0 which indicate these VLFs are created as part of database creation with 64MB log size. The remaining 8 records has the same value. These VLF are created as part of our alter database statement to increase the size of the log file from 64 MB to 1024MB

The above output description is referred from Kalen Delaney Blog Post

Now our transaction log will looks like below

Now we have learned about the LSN and VLF. we will discuss more about transaction log in the next post.

If you liked this post, do like my page on FaceBook


  1. Magnificent article a debt of gratitude is in order for the post on sql server dba. I propose you to look our site for indepth data on sql server dba training and demo additionally accessibleRead more...

    Check this site mindmajix for indepth Sql server dba blogs.
    Go here if you’re looking for information on Sql server dba training.

  2. This is the excellent one for all sqlserver learning & working professionals......

    For best Oracle Apps Technical Training In Hyderabad with job assistance... for all graduates ... in India , U.S.A , U.K ...
    Many professionals were placed in MNCs through RCP Technologies.
    Lots of openings on oracle technical... contact us for more details::::::::::
    Oracle Apps Technical Training In Hyderabad

  3. This comment has been removed by the author.


  4. such a good website and given to more information thanks! and more visit
    sas online training


  5. Spot on with this write-up, I absolutely believe this web site needs a lot more attention. I’ll probably be back again to read through more, thanks for the information!
    SAP APO Online Training

  6. About the SQl server information was more useful at the my studies and the easily observe all given information,thanks for sharing that valuable information.
    html5 training in chennai

  7. Dear friend. I truly just like your posting and your current web page all in all! That write-up is really plainly composed and without difficulty understandable.ok about SQL inforamtion sap sql

  8. Many professionals were placed in MNCs through RCP Technologies.
    Lots of openings on oracle technical... contact us for more details
    SAP GTS Training In Hyderabad

  9. Wow amazing i saw the article with execution models you had posted. It was such informative. Really its a wonderful article. Thank you for sharing and please keep update like this type of article because i want to learn more relevant to this topic.

    SAS Training

  10. This content creates a new hope and inspiration with in me. Thanks for sharing article like this. The way you have stated everything above is quite awesome. Keep blogging like this. Thanks.

    SAP training in Chennai

  11. Being new to the blogging world I feel like there is still so much to learn. Your tips helped to clarify a few things for me as well as giving guidance...

    Linux Training in Chennai

  12. Excellent post! I must thank you for this informative read. I hope you will post again soon. Warehousing of merchandise .
    While general look the things you ought to utilize little stockpiling zone space stockpiling range compartments to program the aides. Books get to be heavier when pressed 7 or more aides in a crate.
    Packers And Movers Chennai

  13. Hey guys, surbhi Sharma here, listen,i need to gather some ppl for some stuff that's going down soon...
    Packers And Movers Hyderabad
    We have effectively balanced by frameworks for 21st century; rapidly extending our routines for organizations provided for you by much of the time updating ourselves.

  14. This comment has been removed by the author.

  15. Thanks a lot,your every article is very helpful for an interview even great real time concepts. I have found another post for the same see here:

  16. Really nice to know about the SQL. and i am much interested to know more about this. So please keep update like this.

    Web Designing Training in Chennai Adyar

  17. Thanku for sharing..
    Hadoop online training in hyderabad.All the basic and get the full knowledge of hadoop.
    hadoop online training in hyderbad

  18. Big data and data warehousing related information is always updated to me at hadoop online training in hyderabad. Nice insight on the topic refer the details at
    hadoop online training

  19. Updating with the latest technology and implementing it is the only way to survive in our niche. Thanks for making me this article. You have done a great job by sharing this content in here. Keep writing article like this.
    SAS Training in Chennai | SAS Course in Chennai


  20. Thanks for posting this useful content, Good to know about new things here, Let me share this, . LINUX training in pune

  21. i love this website. your site is good and your work is very good for social work.. keep work.
    Packers And Movers Ahmedabad

  22. An excellent information provided thanks for all the information i must say great efforts made by you. thanks a lot for all the information you provided.
    Packers And Movers Pune

  23. Wow amazing i saw the article with execution models you had posted. It was such informative. Really its a wonderful article. Thank you for sharing and please keep update like this type of article because i want to learn more relevant to this topic.
    Jio TV