Ошибка при запуске namenode в hadoop 2.4.1

Когда я пытаюсь запустить dfs, используя:

start-dfs.sh

Я получаю сообщение об ошибке:

14/07/03 11:03:21 WARN util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes
where applicable Starting namenodes on [OpenJDK 64-Bit Server VM
warning: You have loaded library
/usr/local/hadoop/lib/native/libhadoop.so.1.0.0 which might have
disabled stack guard. The VM will try to fix the stack guard now. It's
highly recommended that you fix the library with 'execstack -c
<libfile>', or link it with '-z noexecstack'. localhost] sed: -e
expression #1, char 6: unknown option to `s' Server: ssh: Could not
resolve hostname Server: Name or service not known
-c: Unknown cipher type 'cd' stack: ssh: Could not resolve hostname stack: Name or service not known 64-Bit: ssh: Could not resolve
hostname 64-Bit: Name or service not known guard.: ssh: Could not
resolve hostname guard.: Name or service not known The: ssh: Could not
resolve hostname The: Name or service not known guard: ssh: Could not
resolve hostname guard: Name or service not known might: ssh: Could
not resolve hostname might: Name or service not known stack: ssh:
Could not resolve hostname stack: Name or service not known will: ssh:
Could not resolve hostname will: Name or service not known the: ssh:
Could not resolve hostname the: Name or service not known fix: ssh:
Could not resolve hostname fix: Name or service not known VM: ssh:
Could not resolve hostname VM: Name or service not known You: ssh:
Could not resolve hostname You: Name or service not known which: ssh:
Could not resolve hostname which: Name or service not known It's: ssh:
Could not resolve hostname It's: Name or service not known disabled:
ssh: Could not resolve hostname disabled: Name or service not known
try: ssh: Could not resolve hostname try: Name or service not known
localhost: namenode running as process 4463. Stop it first. library:
ssh: Could not resolve hostname library: Name or service not known
with: ssh: Could not resolve hostname with: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service
not known VM: ssh: Could not resolve hostname VM: Name or service not
known now.: ssh: Could not resolve hostname now.: Name or service not
known loaded: ssh: Could not resolve hostname loaded: Name or service
not known library: ssh: Could not resolve hostname library: Name or
service not known <libfile>',: ssh: Could not resolve hostname
<libfile>',: Name or service not known to: ssh: connect to host to
port 22: Connection refused OpenJDK: ssh: Could not resolve hostname
OpenJDK: Name or service not known have: ssh: Could not resolve
hostname have: Name or service not known have: ssh: Could not resolve
hostname have: Name or service not known with: ssh: Could not resolve
hostname with: Name or service not known fix: ssh: Could not resolve
hostname fix: Name or service not known noexecstack'.: ssh: Could not
resolve hostname noexecstack'.: Name or service not known that: ssh:
Could not resolve hostname that: Name or service not known you: ssh:
Could not resolve hostname you: Name or service not known or: ssh:
Could not resolve hostname or: Name or service not known highly: ssh:
Could not resolve hostname highly: Name or service not known
recommended: ssh: Could not resolve hostname recommended: Name or
service not known 'execstack: ssh: Could not resolve hostname
'execstack: Name or service not known link: ssh: Could not resolve
hostname link: Name or service not known it: ssh: Could not resolve
hostname it: Name or service not known '-z: ssh: Could not resolve
hostname '-z: Name or service not known localhost: datanode running as
process 4561. Stop it first. Starting secondary namenodes [OpenJDK
64-Bit Server VM warning: You have loaded library
/usr/local/hadoop/lib/native/libhadoop.so.1.0.0 which might have
disabled stack guard. The VM will try to fix the stack guard now. It's
highly recommended that you fix the library with 'execstack -c
<libfile>', or link it with '-z noexecstack'.
0.0.0.0] sed: -e expression #1, char 6: unknown option to `s' OpenJDK: ssh: Could not resolve hostname OpenJDK: Name or service not known
-c: Unknown cipher type 'cd' VM: ssh: Could not resolve hostname VM: Name or service not known The authenticity of host '0.0.0.0 (0.0.0.0)'
can't be established. ECDSA key fingerprint is
dd:64:53:7e:c0:62:40:c0:63:2b:5c:6d:1e:b6:cd:23. Are you sure you want
to continue connecting (yes/no)? might: ssh: Could not resolve
hostname might: Name or service not known Server: ssh: Could not
resolve hostname Server: Name or service not known guard.: ssh: Could
not resolve hostname guard.: Name or service not known have: ssh:
Could not resolve hostname have: Name or service not known You: ssh:
Could not resolve hostname You: Name or service not known The: ssh:
Could not resolve hostname The: Name or service not known which: ssh:
Could not resolve hostname which: Name or service not known have: ssh:
Could not resolve hostname have: Name or service not known disabled:
ssh: Could not resolve hostname disabled: Name or service not known
VM: ssh: Could not resolve hostname VM: Name or service not known
It's: ssh: Could not resolve hostname It's: Name or service not known
fix: ssh: Could not resolve hostname fix: Name or service not known
the: ssh: Could not resolve hostname the: Name or service not known
warning:: ssh: Could not resolve hostname warning:: Name or service
not known will: ssh: Could not resolve hostname will: Name or service
not known the: ssh: Could not resolve hostname the: Name or service
not known library: ssh: Could not resolve hostname library: Name or
service not known that: ssh: Could not resolve hostname that: Name or
service not known highly: ssh: Could not resolve hostname highly: Name
or service not known 'execstack: ssh: Could not resolve hostname
'execstack: Name or service not known try: ssh: Could not resolve
hostname try: Name or service not known guard: ssh: Could not resolve
hostname guard: Name or service not known 64-Bit: ssh: Could not
resolve hostname 64-Bit: Name or service not known loaded: ssh: Could
not resolve hostname loaded: Name or service not known library: ssh:
Could not resolve hostname library: Name or service not known fix:
ssh: Could not resolve hostname fix: Name or service not known to:
ssh: connect to host to port 22: Connection refused link: ssh: Could
not resolve hostname link: Name or service not known stack: ssh: Could
not resolve hostname stack: Name or service not known '-z: ssh: Could
not resolve hostname '-z: Name or service not known you: ssh: Could
not resolve hostname you: Name or service not known with: ssh: Could
not resolve hostname with: Name or service not known with: ssh: Could
not resolve hostname with: Name or service not known recommended: ssh:
Could not resolve hostname recommended: Name or service not known
stack: ssh: Could not resolve hostname stack: Name or service not
known now.: ssh: Could not resolve hostname now.: Name or service not
known <libfile>',: ssh: Could not resolve hostname <libfile>',: Name
or service not known or: ssh: Could not resolve hostname or: Name or
service not known noexecstack'.: ssh: Could not resolve hostname
noexecstack'.: Name or service not known it: ssh: Could not resolve
hostname it: Name or service not known ^C0.0.0.0: Host key
verification failed. ^C

Мой файл core-site.xml содержит следующее:

<configuration>
    <property>
       <name>fs.default.name</name>
       <value>hdfs://localhost:9000</value>
    </property>
</configuration>

Мой .profile (замена .bashrc) содержит такие строки:

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64
export HADOOP_INSTALL=/usr/local/hadoop
export PATH=$PATH:$HADOOP_INSTALL/bin
export PATH=$PATH:$HADOOP_INSTALL/sbin
export HADOOP_MAPRED_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_HOME=$HADOOP_INSTALL
export HADOOP_HDFS_HOME=$HADOOP_INSTALL
export YARN_HOME=$HADOOP_INSTALL
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"

И я могу легко ssh свой локальный хост сказать:

ssh localhost

Welcome to Linux Mint 16 Petra (GNU/Linux
3.11.0-12-generic x86_64)

Welcome to Linux Mint  * Documentation:  http://www.linuxmint.com Last
login: Wed Jul  2 16:51:15 2014 from localhost

hadoop jvm-arguments hadoop2
person gonephishing    schedule 02.07.2014    source источник
comment
Пожалуйста, включите в свое сообщение содержимое вашего файла core-site.xml.   -  person Balduz    schedule 02.07.2014
comment
@Balduz: я добавил содержимое файла core-site.xml.   -  person gonephishing    schedule 02.07.2014
comment
Проверьте файл hadoop-env.sh и посмотрите, устанавливается ли HADOOP_CONF_DIR и на что он указывает.   -  person Balduz    schedule 02.07.2014
comment
покажи строку export HADOOP_HOME= которую ты вставил в $HOME/.bashrc   -  person A J    schedule 02.07.2014
comment
@Balduz: извините за задержку ... я отредактировал свой вопрос, включая текущие сообщения об ошибках.   -  person gonephishing    schedule 03.07.2014
comment
Для возможности поиска: эта проблема также относится к Hadoop 2.4.0.   -  person Greg Dubicki    schedule 20.09.2015
comment
возможный дубликат hadoop 2.2.0 64-разрядная установка, но не могу запустить   -  person Greg Dubicki    schedule 20.09.2015

Ответы (4)


arrow_upward
7
arrow_downward

Не позволяйте JVM печатать предупреждение защиты стека в stdout/stderr, потому что это нарушает стартовый скрипт HDFS.


Сделайте это, заменив в строке etc/hadoop/hadoop-env.sh:

export HADOOP_OPTS="$HADOOP_OPTS -Djava.net.preferIPv4Stack=true"

с участием:

export HADOOP_OPTS="$HADOOP_OPTS -XX:-PrintWarnings -Djava.net.preferIPv4Stack=true"


(Это решение было найдено на блог Сумита Чавлы)

person Greg Dubicki    schedule 20.09.2015

arrow_upward
1
arrow_downward

Отредактируйте файл .bashrc и добавьте следующие строки:

export HADOOP_HOME=path_to_your_hadoop_folder
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"

И хотя ваш ssh должен работать по тому, что вы только что сказали, повторите это на всякий случай:

ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
person Balduz    schedule 03.07.2014
comment
Я использую linux mint, поэтому у меня есть .profile вместо .bashrc. Как я уже упоминал выше, все вышеупомянутые 3 строки уже присутствуют в моем файле .profile (проверьте предпоследнюю кавычку блока). - person gonephishing; 03.07.2014

arrow_upward
1
arrow_downward

Похоже, вы не добавили строку $HADOOP_INSTALL в свой файл .profile, которая указывает на вашу основную папку hadoop. Как предполагает Балдуз, использование HADOOP_HOME будет работать вместо переменной $HADOOP_INSTALL. Я бы воспользовался его предложением, но вы также можете исправить это, добавив...

export HADOOP_INSTALL=/path/to/hadoop/
person Brian Olsen    schedule 06.08.2014

arrow_upward
0
arrow_downward

пожалуйста, проверьте свой HADOOP_CONF_DIR (скорее всего, в bashrc). Он должен указывать на $HADOOP_HOME/etc/hadoop

person SIva    schedule 16.02.2019