Tuesday, August 31, 2010

Troubleshooting Relay Log Corruption

Did you ever encountered the below error in the slave status.

mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 10.x.x.x.
Master_User: abcd
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: master-bin.000875
Read_Master_Log_Pos: 163966855
Relay_Log_File: slave-relay-bin.002638
Relay_Log_Pos: 238770262
Relay_Master_Log_File: master-bin.000874
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error: Could not parse relay log event entry. The possible reasons are: the master's binary log is corrupted (you can check this by running 'mysqlbinlog' on the binary log), the slave's relay log is corrupted (you can check this by running 'mysqlbinlog' on the relay log), a network problem, or a bug in the master's or slave's MySQL code. If you want to check the master's binary log or slave's relay log, you will be able to know their names by issuing 'SHOW SLAVE STATUS' on this slave.
Skip_Counter: 0
Exec_Master_Log_Pos: 238770119
Relay_Log_Space: 1237710237
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
1 row in set (0.00 sec)

The error messages says that the possibilities are slave relay log corruption, master binlog corruption, network problem, or a bug.

But, as far as I investigated this error is occured due to slave relay log corruption. Try to run mysqlbinlog on the relay log file which will most probably throw the below error.

-bash-3.00$ /usr/local/mysql/bin/mysqlbinlog slave-relay-bin.002638 > /tmp/slave-relay-bin.002638_log
ERROR: Error in Log_event::read_log_event(): 'Event too small', data_len: 0, event_type: 0
Could not read entry at offset 238770262:Error in log format or read error

The above error ensures that the slave relay log file is corrupted. In this case you need to execute the below in your slave to restart your replication.

mysql> slave stop;
Query OK, 0 rows affected (0.00 sec)
mysql> CHANGE master TO master_log_file='Relay_Master_Log_File',master_log_pos=Exec_Master_Log_Pos;
Query OK, 0 rows affected (1.16 sec)
mysql> slave start;
Query OK, 0 rows affected (0.00 sec)

Hope this post is useful for someone.

No comments:

Post a Comment