Chapter 5. Storing mnoGoSearch data

Table of Contents
SQL storage types
Cache mode storage
mnoGoSearch performance issues
SearchD support
Oracle notes
IBM DB2 notes

SQL storage types

Various modes of words storage

There are different modes of word storage which are currently supported by mnoGoSearch: "single", "multi", "blob". Default mode is "single". Mode can be selected using DBMode part of DBAddr command in both indexer.conf and search.htm files.


Examples:
DBAddr mysql://localhost/test/?DBMode=single
DBAddr mysql://localhost/test/?DBMode=multi
DBAddr mysql://localhost/test/?DBMode=blob

Storage mode - single

When "single" is specified, all words are stored in a signle table with structure (url_id,word,weight), where url_id is the ID of the document which is referenced by rec_id field in "url" table. Word has variable char(32) SQL type. Each appearance of the same word in a document produces a separate record in the table.

Storage mode - multi

If "multi" is selected, words are located in 256 separate tables using hash function for distribution. Structures of these tables are almost the same with "single" mode, but all word appearance are groupped into a single binary array, instead of producing multiple records. This fact makes "multi" mode much faster comparing with "single" mode.

Storage mode - blob

If "blob" is selected, words are located in a single table with structure (word, secno, intag), where intag is binary array of coords. All word appearance for current section are grouped into single binary array. This mode is highly optimized for search, indexing is not supported. You should index your data with "multi" mode and then run "indexer -Eblob" to convert "multi" tables into "blob". Note: this mode work only with MySQL for now, but will be extended to work with other databases in the future.


Example:

# indexer.conf
DBAddr mysql://localhost/test/?DBMode=multi

# search.htm
DBAddr mysql://localhost/test/?DBMode=blob

Substring search notes

"single", "multi" and "blob" modes support substring search. An SQL query containing LIKE predicate is executed internally in order to do substring search.