Hadoop:FATAL-8001: Internal error: load of libhdfs.so failed: Inappropriate ioctl for device at expression handle at apply_file.c:961

The reported error may occur when you're creating a replication for Hadoop. You will be able to start the replication, but it suddenly crashes after the first redo is applied to Hadoop. The error might be related to environment variables. Below article explains how to check/fix this issue.

Problem Details

Problem Statement

FATAL-8001: Internal error: load of libhdfs.so failed: Inappropriate ioctl for device at expression handle at apply_file.c:961

Affected Platforms

Hadoop

Description

The error is caused by wrongly load library libhdfs.so for Hadoop. Dbvisit Replicate uses Hadoop binaries and Hadoop relies on Java. Please, ensure the Java and Hadoop are correctly set.

Solutions

There might be few problems in your environment and here is a list of possible solutions to this error:

Make sure, that your LD_LIBRARY_PATH is set to <HADOOP_HOME>/lib/native and /lib64 directories.

2. Upgrade to version 2.9.02. In version 2.9.00, the dbvrep is looking for the library to /tmp/par-* directory regardless of LD_LIBRARY_PATH settings

3. Copy Hadoop directory from Hadoop if an APPLY process is elsewhere and copy all record related to Hadoop settings to .bash_profile file. Install Java 64 bit to the server. If there already is Java, check if it's 64 bit (if you're using 64 bit OS, if not you need to have 32 bit).

Check libhdfs.so library if all shared libraries could be found:


[oracle@middle hadoop]$ ldd /opt/hadoop/lib/native/libhdfs.so
        linux-vdso.so.1 =>  (0x00007ffd13351000)
        libjvm.so => not found
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007feb67248000)
        libc.so.6 => /lib64/libc.so.6 (0x00007feb66e86000)
        /lib64/ld-linux-x86-64.so.2 (0x00007feb6767d000)
 
[oracle@middle ~]$ find / -mount -name "libjvm*" 2>/dev/null
/opt/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so
/opt/hadoop/opt/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so
/u01/app/oracle/product/11.2.0/dbhome_1/jdk/jre/lib/amd64/server/libjvm.so

[root@middle ~]# ln -s /opt/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so /lib64/libjvm.so 
 
[oracle@middle hadoop]$ ldd /opt/hadoop/lib/native/libhdfs.so
        linux-vdso.so.1 =>  (0x00007ffeb83ca000)
        libjvm.so => /lib64/libjvm.so (0x00007fb3edbc6000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fb3ed9a9000)
        libc.so.6 => /lib64/libc.so.6 (0x00007fb3ed5e7000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fb3eedd1000)
        libm.so.6 => /lib64/libm.so.6 (0x00007fb3ed2e5000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007fb3ed0e0000)

Add library libjvm.so to LD_LIBRARY_PATH variable to .bash_profile. Example:


export LD_LIBRARY_PATH=/lib:/opt/jdk1.8.0_131/jre/lib/amd64/server

4. Copy the library directly to /tmp/par-* directory. It is just a workaround, if you need to move further, not a proper solution.