Cloudera提供HTTP方式连接HDFS工具

Hoop是Apache Hadoop贡献者Cloudera公司推出的一款新工具。Hoop可通过REST API提供以HTTP方式访问Hadoop 分布式文件系统（HDFS）的功能。

Hoop是Hadoop NameNode的一个单独服务。例如在UNIX下：

$ curl http://myhoop:14000/user/foo/README.txt returns the contents of the HDFS /user/foo/README.txt file. 
$ curl http://myhoop:14000/user/foo?op=list returns the contents of the HDFS /user/foo directory in JSON format. 
$ curl -X POST http://myhoop:14000/user/foo/bar?op=mkdirs creates the HDFS /user/foo.bar directory. 

Hoop遵循Apache License 2.0，并可用于平台间不同版本Hadoop集群之间的数据交换，或访问防火墙后面的数据。

Hoop是一个完全重写的Hadoop HDFS代理。Cloudera表示其具有以下优点：

●支持所有HDFS的操作（读、写）

●JSON格式的数据状态（文件的状态、操作状态、错误信息）

●Kerberos HTTP SPNEGO client/server authentication and pseudo authentication out of the box (using Alfredo)

●Hadoop代理用户的支持

●提供在任何集群运行的工具，如DistCP（李智/译）

from：
http://cloud.csdn.net/a/20110722/302086.html
http://www.readwriteweb.com/hack/2011/07/access-hadoop-hdfs-over-http.php

What is Hoop?

Hoop provides access to all Hadoop Distributed File System (HDFS) operations (read and write) over HTTP/S.

Hoop can be used to:

Hoop has a Hoop client and a Hoop server component:

Hoop and Hadoop HDFS Proxy

Hoop server is a full rewrite of Hadoop HDFS Proxy. Although it is similar to Hadoop HDFS Proxy (runs in a servlet-container, provides a REST API, pluggable authentication and authorization), Hoop server improves many of Hadoop HDFS Proxy shortcomings by providing:

Accessing HDFS files -via Hoop- using Unix ‘curl’ command

Assuming Hoop is running on http://hoopbar:14000, the following examples show how the Unix ‘curl’ command can be used to access data in HDFS via Hoop using pseudo authentication.

Getting the home directory:

$ curl -i "http://hoopbar:14000?op=homedir&user.name=babu"
HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked
{"homeDir":"http:\/\/hoopbar:14000\/user\/babu"}
$

Reading a file:

$ curl -i "http://hoopbar:14000?/user/babu/hello.txt&user.name=babu"
HTTP/1.1 200 OK
Content-Type: application/octet-stream
Transfer-Encoding: chunked
Hello World!
$

Writing a file:

$ curl -i -X POST "http://hoopbar:14000/user/babu/data.txt?op=create" --data-binary @mydata.txt --header "content-type: application/octet-stream"
HTTP/1.1 200 OK
Location: http://hoopbar:14000/user/babu/data.txt
Content-Type: application/json
Content-Length: 0
$

Listing the contents of a directory:

$ curl -i "http://hoopbar:14000?/user/babu?op=list&user.name=babu"
HTTP/1.1 200 OK
Content-Type: application/json
Transfer-Encoding: chunked

[
  {
    "path" : "http:\/\/hoopbar:14000\/user\/babu\/data.txt"
    "isDir" : false,
    "len" : 966,
    "owner" : "babu",
    "group" : "supergroup",
    "permission" : "-rw-r--r--",
    "accessTime" : 1310671662423,
    "modificationTime" : 1310671662423,
    "blockSize" : 67108864,
    "replication" : 3
  }
]
$

Click this link for more details about the Hoop HTTP REST API.

Getting Hoop

Hoop is distributed with an Apache License 2.0.

The source code is available at http://github.com/cloudera/hoop.

Instructions on how to build, install and configure Hoop server and the rest of documentation is available at http://cloudera.github.com/hoop.

Contributing Hoop to Apache Hadoop

The goal is to contribute Hoop to Apache Hadoop as the next generation of Hadoop HDFS proxy. We are just waiting on the Mavenization of Hadoop Common and Hadoop HDFS which will make integration easier.

from: http://www.cloudera.com/blog/2011/07/hoop-hadoop-hdfs-over-http/