Apache Chukwa

Apache Chukwa can be used to log the activity that happens into a complex Hadoop distributed computing system.

Chukwa will help developers track down problems and solve bugs in the data processing applications, by providing a toolkit for not only recording all the computations that occur but also analyzing, classifying and then displaying the resulted data.

The project is tightly integrated with the MapReduce philosophy and since it natively uses HDFS (Hadoop Distributed File System) it will work out of the box with Hadoop powered projects.

What is new in this release:

• NEW FEATURES:
• Migrate from ant to maven.
• Added Metrics 2 support for Hadoop 0.20.203+.
• Added cluster summary pig script.
• Added digested system metrics to SystemMetrics demux parser.
• Added configuration to skip HBase schema validation for HBaseWriter.
• Update default Chukwa configuration files to use SocketAdaptor for Hadoop metrics streaming.
• Configuration spellcheck.
• Implemented graphing rest api and graph explorer for plotting HBase data to graph.
• Implemented Syslog Adaptor for mappig Syslog facility name to Chukwa data type.
• Replaced system metrics collection with Sigar.
• Added HBaseWriter for storing time series data in HBase for faster random read/write.
• Make default processor configurable.
• Support HTTP trigger actions.
• Added JMSAdaptor.
• Support post-demux trigger.

What is new in version 0.6.0:

• NEW FEATURES:
• Migrate from ant to maven.
• Added Metrics 2 support for Hadoop 0.20.203+.
• Added cluster summary pig script.
• Added digested system metrics to SystemMetrics demux parser.
• Added configuration to skip HBase schema validation for HBaseWriter.
• Update default Chukwa configuration files to use SocketAdaptor for Hadoop metrics streaming.
• Configuration spellcheck.
• Implemented graphing rest api and graph explorer for plotting HBase data to graph.
• Implemented Syslog Adaptor for mappig Syslog facility name to Chukwa data type.
• Replaced system metrics collection with Sigar.
• Added HBaseWriter for storing time series data in HBase for faster random read/write.
• Make default processor configurable.
• Support HTTP trigger actions.
• Added JMSAdaptor.
• Support post-demux trigger.

What is new in version 0.4.0:

• The collection of shell scripts in /bin has been significantly pared down. For instance, instead of bin/agent, you should now say bin/chuka agent.
• It is now possible to disable remote connections to the Chukwa agent. This makes Chukwa more suitable for use on a customer-facing deployment
• Chukwa now includes an adaptor for reading data via a UDP port, the UDPAdaptor. This should facilitate integration with syslog and similar legacy tools.
• There have been a number of bug fixes and code cleanups since the last release; check the changelog and JIRA for details.

Requirements:

• Java 1.6 or higher
• Apache Hadoop 0.20.205.1 or higher
• Apache HBase 0.90.4 or higher