FreeBSDでHadoop(Standalone mode)
まずは,最も動きそうなSun JDK+スタンドアローンモード.
特に障害は無く動きました.
JavaとOSは以下のものを使用.
> java -version java version "1.6.0_03-p4" Java(TM) SE Runtime Environment (build 1.6.0_03-p4-09_may_2010_19_47-b00) Java HotSpot(TM) Client VM (build 1.6.0_03-p4-09_may_2010_19_47-b00, mixed mode) > uname -v FreeBSD 8.0-RELEASE #0: Sat Nov 21 15:48:17 UTC 2009 root@almeida.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC
bashが入っていなかったので,それだけ新たにインストール.
> pkg_add -r bash
設定ファイルで書き換えたのは一箇所だけ.
export JAVA_HOME=/usr/local/jdk1.6.0
サンプルプログラムの実行.
> ./bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+' 10/08/13 10:17:54 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName =JobTracker, sessionId= 10/08/13 10:17:55 INFO mapred.FileInputFormat: Total input paths to process : 5 10/08/13 10:17:55 INFO mapred.FileInputFormat: Total input paths to process : 5 10/08/13 10:17:55 INFO mapred.JobClient: Running job: job_local_0001 10/08/13 10:17:55 INFO mapred.MapTask: numReduceTasks: 1 10/08/13 10:17:55 INFO mapred.MapTask: io.sort.mb = 100 10/08/13 10:17:56 INFO mapred.MapTask: data buffer = 79691776/99614720 10/08/13 10:17:56 INFO mapred.MapTask: record buffer = 262144/327680 10/08/13 10:17:56 INFO mapred.MapTask: Starting flush of map output 10/08/13 10:17:56 INFO mapred.JobClient: map 0% reduce 0% (中略) 10/08/13 10:18:04 INFO mapred.JobClient: map 100% reduce 100% 10/08/13 10:18:04 INFO mapred.JobClient: Job complete: job_local_0002 (後略)
無事終了.
ところで,JavaはSunのものが推奨されているけど,それ以外のものでも動くのか試してみました.
設定ファイルを書き換え.
export JAVA_HOME=/usr/local/diablo-jdk1.6.0
前の実行結果が残っているディレクトリを削除後にサンプルプログラムを実行.
> rm -fR output > ./bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+' 10/08/13 12:40:13 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName =JobTracker, sessionId= 10/08/13 12:40:14 INFO mapred.FileInputFormat: Total input paths to process : 5 10/08/13 12:40:15 INFO mapred.FileInputFormat: Total input paths to process : 5 10/08/13 12:40:15 INFO mapred.JobClient: Running job: job_local_0001 10/08/13 12:40:15 INFO mapred.MapTask: numReduceTasks: 1 10/08/13 12:40:15 INFO mapred.MapTask: io.sort.mb = 100 10/08/13 12:40:15 INFO mapred.MapTask: data buffer = 79691776/99614720 10/08/13 12:40:15 INFO mapred.MapTask: record buffer = 262144/327680 10/08/13 12:40:16 INFO mapred.MapTask: Starting flush of map output 10/08/13 12:40:16 INFO mapred.TaskRunner: Task:attempt_local_0001_m_000000_0 is done. And is in the process of commiting 10/08/13 12:40:16 INFO mapred.LocalJobRunner: file:/usr/home/tetu-s/hadoop-0.20. 2/input/mapred-site.xml:0+178 10/08/13 12:40:16 INFO mapred.TaskRunner: Task 'attempt_local_0001_m_000000_0' d one. 10/08/13 12:40:16 INFO mapred.JobClient: map 100% reduce 0% (中略) 10/08/13 12:40:22 INFO mapred.JobClient: map 100% reduce 100% 10/08/13 12:40:22 INFO mapred.JobClient: Job complete: job_local_0002 (後略)
普通に動いている感じです.