groonga

groonga is an open source, freely distributed, accurate and extremely fast full-text search engine and column store utility. It is based on an inverted index and lets users write high-performance apps, which require full-text search.

Features at a glance

Key features include full-text search and instant update, column store and aggregate query, inverted index and tokenizer, sharable storage and read lock-free, as well as geo-location (latitude and longitude) search.

Using the built-in command-line options, which can be viewed at a glance by running the ‘groonga --help’ command after installation (see the getting started section below for details), you can force groonga to run in client, server or daemon mode, as, by default, the software runs only in standalone mode.

Various command-line options are available to users and organized in categories, such as database creation options, standalone/client options, server/daemon options, logging options, as well as common options.

Designed for Mroonga

Also part of this project, is the Mroonga application, which is designed as an open source command-line storage engine powered by the MySQL database. groonga is distributed as a C library and a built-in server.

Getting started with groonga

This program must be installed on your system in order to use it. Therefore, you will have to open your package manager, search for groonga and install it. If the software is not available on the main software repositories of your distribution, or the version is older than the one distributed on Softoware, we strongly recommend to follow the next instructions to install groonga.

Download the tarball (TAR archive) using the dedicated download section above, save the file somewhere on your PC, unpack it, open a terminal emulator and go to the location of the extracted archive files using the ‘cd’ command (e.g. cd /home/softoware/groonga-4.0.8).

Then, you must run the ‘./configure && make’ command to configure and compile the program, followed by the ‘make install’ command, as root, or the ‘sudo make install’ command, as a privileged user, to install it system wide.

What is new in this release:

• Improvements:
• [deb] Supported Ubuntu 12.04 again because Travis-CI runs on it.
• [mruby] Supported pretty print - "pp" for debugging. And supported to show internal objects more descriptive format when inspecting it.
• Supported full text search query without index if one column is specified for --match_columns even though sequential search is executed by --query and --match_columns combination. [groonga-dev,02902] [Reported by Atsushi Shinoda]
• [experimental] in_values supports auto sequential search mode. There is a case that sequential search is faster than index search when the number of narrowed down records is small enough in contrast to the number of expected records to narrow down by in_values with AND operation which use indexes. The value of GRN_IN_VALUES_TOO_MANY_INDEX_MATCH_RATIO is used as threshold whether Groonga execute sequential search or search with indexes in such a case.
• [mruby] Supported to load relative path of script.
• Supported test suite to write query optimizer test more easily with mruby enabled build.
• Supported offline index construction for token filters. [GitHub#239] [Patch by Naoya Murakami]
• [experimental] between supports auto sequential search mode. Use the value of GRN_BETWEEN_TOO_MANY_INDEX_MATCH_RATIO as threshold. Basic concept is similar to GRN_IN_VALUES_TOO_MANY_INDEX_MATCH_RATIO.
• [experimental] Supported multiple drilldowns for :doc:/reference/command/select command. The syntax is extended to support this feature. The new one is --drilldown[LABEL1].keys COLUMN1,COLUMN2 for keys, --drilldown[LABEL1].output_columns _value.COLUMN1,_value.COLUMN2,_nsubrecs for output columns. In the previous versions, a client application must implement by oneself for drilldown by multiple keys. Use _key[N] for --drilldown[LABEL1].output_columns to show original value of key in records.
• Supported to search 'reference_column @ "query"' without index.
• [experimental] Added range_filter command. It is faster than select command to fetch specific range of records in most cases, because it doesn't returns the number of matched records. There are limitations which doesn't support drilldown and so on. Thus, range_filter is suitable for extracting first N records in specific range for ranking or something else.
• [httpd] Supported groonga-httpd on windows.
• Supported vector column for drilldown.
• [deb] Changed groonga-httpd as default HTTP server package. groonga-server-http is deprecated. Note that groonga HTTPサーバー functionality itself is still exists in Groonga. If you just want to use simple HTTP server in traditional way, specify proper arguments by command line for groonga command.
• Added configuration file for Windows CI on AppVeyor. [groonga-dev,02970,02981] [Patch by Hiroshi Hatake]
• Fixes:
• [doc] Fixed duplicated execution example path about table_create. [GitHub groonga.org#14] [Reported by Masafumi Yokoyama]
• Fixed a bug that nested indexed full text search against X._key returns empty results.
• Fixed flags parameter which is strict in BSD mkostemp(3). [GitHub#240] [Patch by Jun Kuriyama]
• Fixed a bug that select command reports invalid value type (table) as index column value type. It was fixed to UInt32. Note that it affects only the value of response header. The search results is not changed at all.
• Fixed a bug that compare operations such as =, != with Time and String type returns invalid results.
• Fixed a bug that prefix search can't be executed against single character such as "x*". [GitHub#242] [Patch by Naoya Murakami]
• Fixed a memory leak on drilldown ShortText value with --filter.
• Fixed missing "," for "DDL:set_token_filters" log. [Reported by Naoya Murakami]

What is new in version 4.0.0:

• Improvements:
• [normalizer] Supported to show "checks" which is used for calculating next character position. Use WITH_CHECKS flag to enable this feature.
• [deb] Dropped Ubuntu 13.04 support.
• Fixes:
• Fixed a crash bug that an object in grn_expr is used after it is freed. Normally Groonga server users aren't affected this bug. This bug mainly affects Rroonga users. Because this bug is occured by specifying column name including pseudo column name - such as '_key' - Rroonga users may use the usage.
• Fixed not to execute unexpected cascade delete which is introduced Groonga 3.0.8 release. If source's range and index's domain are different, Groonga doesn't execute cascade delete. [groonga-dev,02073] [Reported by yoku]
• Fixed not to publish grn_snip structure. Use grn_obj instead of grn_snip. If you use grn_snip_close, please replace grn_snip_close to grn_obj_close.
• [snippet_html] Fixed a crash bug when --query is empty. [groonga-dev,02097] [Reported by Naoya Murakami]
• [snippet_html] Fixed to suppress ALERT level message when contents of column is empty text. [groonga-dev,02097] [Reported by Naoya Murakami]
• [groonga-httpd] Fixed a bug "off" is used as path name in groonga_query_log_path. [groonga-dev,02113] [Reported by Ryoji Yamamoto]

What is new in version 3.1.2:

• Improvements:
• [doc] Updated to caplitalized "Groonga" terms in documentation. [Patch by cosmo0920] [GitHub#136, #137, #138, #139, #140, #141, #142, #143, #144, #145, #146, #147, #148, #149, #150, #151]
• Supported to customize the value of lock timeout. See Global configurations about details. [groonga-dev,02017] [Suggested by yoku]
• [doc] Added description about the value of lock timeout.
• Enabled GRN_JA_SKIP_SAME_VALUE_PUT by default. In the previous releases, the value of this configuration is 'no'. This change affects reducing the size of Groonga database.
• Supported multiple indexes including a nested index and multiple keywords query. This change improves missing search results isssue when narrowing down by multiple keywords query.
• Added API to customize normalizer for snippet.
• Fixes:
• Fixed not to use index for empty query. This change enables you to search even though empty query. Note that this means that there is performance penalty if many empty records exist. [groonga-dev,02052] [Reported by Naoya Murakami]
• Fixed the behaviour about return value of "X || Y" and "X && Y" for adjusting to ECMAScript. In "X || Y" case, if either X or Y satisfy the condition, it returns X itself or Y itself instead of 1 or 0.
• In "X && Y" case, if X and Y satisfy the condition, it returns X itself instead of 1. if X doesn't satisfy the condition, it returns false instead of 0.
• Fixed to return null when no snippet is found. This change enables you to set the default value of snippet_html. In such a purpose, use "snippet_html(XXX) || 'default value'".

What is new in version 3.1.1:

• Improvements:
• [deb] Refined Groonga packages for Debian policy. [groonga-dev,01930] [Suggested by Hideki Yamane]
• [deb] Dropped Debian 6.0 (squeeze) support.
• [deb] Dropped Ubuntu 10.04 (lucid) support.
• Supported geo_in_rectangle in the whole world.
• Supported error report when failed to casting invalid value of geo point type.
• Exported some macros which is used for accessing pseudo column names. [groonga-dev,1999] [Suggested by whombx]
• [doc] Refined drilldown documentation. [Reported by @Yahppo]
• Supported between function which is used for filtering the column value in specific range.
• [doc] Updated Travis CI envrinment information. [Patch by cosmo0920]
• [rpm][fedora] Dropped Fedora 19.
• [rpm][fedora] Supported Fedora 20.
• [doc] Updated "Groonga" notation. [Patch by cosmo0920] [GitHub#122, #123, #124, #125, #125, #126, #127, #128, #129, #130, #131, #131, #132, #133, #134]
• Fixes:
• [munin] Fixed the wrong port number which is specified for GQTP.
• [geo_in_circle] Fixed a bug that if you specify "LONGITUDExLATITUDE" as the 3rd argument of geo_in_circle, Groonga crashes.

What is new in version 3.1.0:

• Improvements:
• [experimental] Supported an option to skip updating for the same value. It is enabled by setting GRN_JA_SKIP_SAME_VALUE_PUT=yes as environment variable.
• Improved to show tokenizer and original text in empty token warning message. It shows actual token, so it helps you to investigate token related issue.
• Dropped broken AIO support. [#2054] [Reported by sho MINAGAWA]
• [windows] Supported build with Visual Studio Express 2008. [#1964 the 21th comment] It is aimed to promote bundling Mroonga with MariaDB in the future.
• Fixes:
• Fixed a bug that type of "_value" column is treated as Int32. It causes the case that the value of "_value" column is casted unexpectedly. [Reported by Genki Takiuchi]
• Fixed to require GLib 2.14.0 or later for benchmark program. The benchmark program is changed to use GRegex which requires GLib 2.14.0 or later. It affects users who compile Groonga with '--enable-benchmark' configuration. [groonga-dev,01890] [Reported by WING]
• Fixed a memory leak on updating an empty column vector with the same value.
• Fixed a memory leak on updating index for column vector. It affects the case if you store the value of column which has invalid UTF-8 byte sequence.
• [plugin][windows] Fixed the issue that register command fails without 'lib/groonga/plugins'. [Reported by @yito]

What is new in version 3.0.9:

• Improvements:
• [doc][httpd] Added documentation about groonga_database_auto_create directive.
• [httpd] Added groonga_cache_limit directive.
• [doc] Added description why zlib/lzo compression are disabled by default. [groonga-dev, 01845] [Suggested by Naoya Murakami]
• Remove a restriction related to RLIMIT_NOFILE on HTTP server. It enables HTTP server process to handle over 4096 files.
• [experimental] Added some API to integrate mruby into groonga. [GitHub#109, #110, #111, #112, #113, #114, #115, #116, #117, #118] [Patch by wanabe]
• [travis] Supported CMake build on Travis-CI.
• [doc] Added documentation about Tuning.
• [experimental] Added ruby_load command.
• [httpd] Supported multiple buffered body of POST request. The load command caused timeout in such a case. [GitHub #120] [Patch by Motoi Washida]
• [gqtp] Supported error message. The body is changed to return raw text insted of double quoted text. This is incompatible changes.
• [http] Supported "400 Bad request" for invalid argument error.
• [doc] Added examples for Completion
• Supported Ubuntu 13.10 (Saucy Salamander).
• Fixes:
• Fixed a bug that it cause SEGV by arithmetic overflow. This problem occurs when the size of indexes exceeds the size of virtual memory. [groonga-dev,01661] [Reported by Naoya Murakami]
• Fixed needless lock for creating table. This change fixes a performance regression instroduced at Groonga 3.0.5.
• Fixed a bug that database which has reference column can't be removed. It affects the case that indexed tables/columns and referenced tables are removed at first, then remove operation to index columns and reference columns fails. [Reported by Eito Katagiri] [GitHub Rroonga #13]
• Fixed a bug that removing database which uses DAT key may remain garbage.
• [http] Fixed a small memory leak which is caused by shared HTTP header buffer in the same worker thread.
• [httpd] Fixed a bug that groonga_database_auto_create parameter isn't inherited. It means that these parameters are just ignored in non location blocks.
• Fixed a bug that status command may returns invalid cache hit rate.
• Fix a bug that casting to Time may causes overflow issue. It affects the case that assignment of the Time value which indicate over 32bit value.

What is new in version 3.0.8:

• Improvements:
• [admin] Improved error handling of select functionality. This change shows error message and hides loading dialog. [Patch by orangain]
• [admin] Improved to submit search form with the enter key. [Patch by orangain]
• [admin] Improved to show placeholder in search form. This change shows usage examples. [Patch by orangain]
• [httpd] Supported groonga_query_log_path directive
• [doc] Updated select documentation about the number of hits in select response.
• Improved to ignore an empty token which is generated by continuous spaces from tokenizer. [groonga-dev,01729] [Suggested by Naoya Murakami]
• [token delimit] Improved to treat continuous spaces as a space. The tokenizer regards 2 or more spaces are valid input.
• [doc] Added note about indexes for each column vs indexes for multiple column.
• [doc] Added language annotation to source label. [groonga-dev,01751] [Suggested by Kazuhiro Isobe]
• Supported Cascade delete to referenced records. In previous version, You got an error for the operation.
• [doc] Updated documentation location in README. [groonga-dev,01800] [Reported by Kazuhiro Isobe]
• [experimental] Added ruby_eval command.
• [doc] Added note about GQTP and HTTP for server use. [groonga-dev,01810] [Reported by Kazuhiro Isobe]
• Fixes:
• [admin] Fixed a bug that add value button of COLUMN_VECTOR does not work.

What is new in version 3.0.7:

• Improvements:
• [doc] Added API documentation. [groonga-dev,01593] [GitHub#79,#80,#81,#82,#83,#84,#85,#86,#87,#88,#89,#90,#91,#92,#93,#95,#96,#97,#98,#99,#100] [Patch by whombx]
• [table_remove] Improved not to remove patricia trie table or hash table
• which is still referenced. This change avoids that referenced table is removed accidentally.
• Improved to just ignore too long token (4096 bytes or more). This change affects when it is registered. In the previous version, it is treated as an error instead of a warning. [groonga-dev,01571] [Suggested by Naoya Murakami]
• Improved to show actual discarded posting term in log message. [groonga-dev,01621] [Reported by Naoya Murakami]
• [httpd] Supported loading data using POST method. [GitHub#101] [Patch by Motoi Washida]
• [httpd] Supported groonga log as groonga_log_path directive.
• [httpd] Supported groonga log level as groonga_log_level directive.
• [httpd] Supported customizing cache limit as groonga_cache_limit directive.
• Fixes:
• Fixed a bug that error in expression may cause stack overflow. This change avoids that groonga crashes suddenly as a result of error accumlation.
• Fixed a crash bug that offline index construction for vector column which has orphan reference causes.
• Fixed a but that groonga doesn't exit on error while daemon mode starts. For example, groonga -d /tmp/non-existence.db reproduce this bug.
• [dump] Fixed a bug that wrong table type is dumped. This bug occurs when table type is TABLE_PAT_KEY and key is reference type.
• Fixed a bug that the default --cache-limit value is 0. The default value is changed to '100'.
• Fixed a memory leak when sub_filter is used.
• [doc] Fixed description of defrag command. [Reported by @naoina]
• [doc] Fixed description about the way to contribute. [GitHub#77] [Patch by Masafumi Yokoyama]
• [plugin] Fixed a crash bug on multiple DB open case. This bug affects the case that multiple plugin is used and the same database is opened from multiple process. [groonga-dev,01596] [Reported by Naoya Murakami]
• Fixed a bug that nested match operation after AND operator doesn't work. The select ... --filter '... && nested.column @ "keyword"' query reproduces this bug. [groonga-dev,01599] [Reported by Motoi Washida]
• [doc] Fixed a typo about documentation of special characters. [Reported by Genki Takiuchi]
• Fixed a typo in error message when invalid character is given. [Reported by YOSHIDA Mitsuo]

What is new in version 3.0.6:

• Improvements:
• [doc] Added documentation about parameter of sub_filter function.
• [suggest] Supported 0MQ 3.x. [GitHub#63] [Reported by Naoya Inada]
• Added availability check for NetBSD about pthread_{mutex,cond}attr_setpshared() [Reported by OBATA Akio]
• Supported backslash escape in query syntax. [groonga-dev,01520] [Reported by Kouhei Tanabe]
• [rpm][fedora] Supported Fedora 19.
• [rpm][fedora] Dropped Fedora 18 support.
• [httpd] Updated bundled nginx version to 1.4.2.
• Fixes:
• Fixed a crash bug when specific internal function (grn_obj_path()) is called for built-in proc function such as select. [Reported by Genki Takiuchi]