FreeBSDでHadoop(Standalone mode)

まずは,最も動きそうなSun JDK+スタンドアローンモード.
特に障害は無く動きました.

JavaとOSは以下のものを使用.

> java -version
java version "1.6.0_03-p4"
Java(TM) SE Runtime Environment (build 1.6.0_03-p4-09_may_2010_19_47-b00)
Java HotSpot(TM) Client VM (build 1.6.0_03-p4-09_may_2010_19_47-b00, mixed mode)
> uname -v
FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:48:17 UTC 2009     root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC

bashが入っていなかったので,それだけ新たにインストール.

> pkg_add -r bash

設定ファイルで書き換えたのは一箇所だけ.

export JAVA_HOME=/usr/local/jdk1.6.0

サンプルプログラムの実行.

> ./bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
10/08/13 10:17:54 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName
=JobTracker, sessionId=
10/08/13 10:17:55 INFO mapred.FileInputFormat: Total input paths to process : 5
10/08/13 10:17:55 INFO mapred.FileInputFormat: Total input paths to process : 5
10/08/13 10:17:55 INFO mapred.JobClient: Running job: job_local_0001
10/08/13 10:17:55 INFO mapred.MapTask: numReduceTasks: 1
10/08/13 10:17:55 INFO mapred.MapTask: io.sort.mb = 100
10/08/13 10:17:56 INFO mapred.MapTask: data buffer = 79691776/99614720
10/08/13 10:17:56 INFO mapred.MapTask: record buffer = 262144/327680
10/08/13 10:17:56 INFO mapred.MapTask: Starting flush of map output
10/08/13 10:17:56 INFO mapred.JobClient:  map 0% reduce 0%

(中略)

10/08/13 10:18:04 INFO mapred.JobClient:  map 100% reduce 100%
10/08/13 10:18:04 INFO mapred.JobClient: Job complete: job_local_0002

(後略)

無事終了.


ところで,JavaはSunのものが推奨されているけど,それ以外のものでも動くのか試してみました.
設定ファイルを書き換え.

export JAVA_HOME=/usr/local/diablo-jdk1.6.0

前の実行結果が残っているディレクトリを削除後にサンプルプログラムを実行.

> rm -fR output
> ./bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'
10/08/13 12:40:13 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName
=JobTracker, sessionId=
10/08/13 12:40:14 INFO mapred.FileInputFormat: Total input paths to process : 5
10/08/13 12:40:15 INFO mapred.FileInputFormat: Total input paths to process : 5
10/08/13 12:40:15 INFO mapred.JobClient: Running job: job_local_0001
10/08/13 12:40:15 INFO mapred.MapTask: numReduceTasks: 1
10/08/13 12:40:15 INFO mapred.MapTask: io.sort.mb = 100
10/08/13 12:40:15 INFO mapred.MapTask: data buffer = 79691776/99614720
10/08/13 12:40:15 INFO mapred.MapTask: record buffer = 262144/327680
10/08/13 12:40:16 INFO mapred.MapTask: Starting flush of map output
10/08/13 12:40:16 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is
done. And is in the process of commiting
10/08/13 12:40:16 INFO mapred.LocalJobRunner: file:/usr/home/tetu-s/hadoop-0.20.
2/input/mapred-site.xml:0+178
10/08/13 12:40:16 INFO mapred.TaskRunner: Task 'attempt_local_0001_m_000000_0' d
one.
10/08/13 12:40:16 INFO mapred.JobClient:  map 100% reduce 0%

(中略)

10/08/13 12:40:22 INFO mapred.JobClient:  map 100% reduce 100%
10/08/13 12:40:22 INFO mapred.JobClient: Job complete: job_local_0002

(後略)

普通に動いている感じです.