How to investigate message glibc detected *** dbvrep APPLY TEST: malloc(): memory corruption: 0x000000000d7fc420 ***

Owned by Clive Otten (Unlicensed)
Jan 21, 2020
1 min read

Problem Description

The main purpose of this article is to help investigate dbvisit support and development team problem with memory corruption. The article can be used in case you are able to reproduce the problem with memory corruption. 

Steps Performed 

  • set in the session export export MALLOC_CHECK_=2 [http://www.novell.com/support/kb/doc.php?id=3113982]

  • check value for command ulimit -c and dbvisit user. If there will be zero, You can execute ulimit -c unlimited. It should enable core dump. (optional)

  • Check if trace file was generated in replication home i.e. /u01/dbvisit/replicate/rep01/log/trace. Don't worry if trace file won't exists. 

  • Let's try to start apply process without daemon variable (start it together with strace command). Beware the session shouldn't be closed, because the process is running in the foreground. In case it takes some time to reproduce the problem use screen program for the session.

EXEC="/home/oracle/dbvrep/280/dbvrep" $EXEC --daemon --ddcfile /home/oracle/dbvrep/280/LOAD/LOAD-APPLY.ddc start APPLY change to: EXEC="/home/oracle/dbvrep/280/dbvrep" $EXEC --ddcfile /home/oracle/dbvrep/280/LOAD/LOAD-APPLY.ddc start APPLY

  • use strace: strace $EXEC --ddcfile /home/oracle/dbvrep/280/LOAD/LOAD-APPLY.ddc start APPLY

  • Write us what's your OS version (cat /etc/*issue*) and kernel version - uname -r.

  • check what's written /etc/log/messages after apply process crash

 

Create or update the ticket with above information. 

Related to this issue could happen the plog opened by the apply process is corrupted. In this case, you can try to 1) stop apply processrename

2. rename plog from apply process

3. copy the plog with equal sequence from mine process

4. start the apply process and see apply log if issue happens again 

 

Keywords: glibc; dbvisit replicate; memory corruption