A guide on configuration file

Manticore search uses a configuration file for customizing settings and declaratation indexes and sources.

The configuration file is in a plain text format and can be edited with any text editor. The configuration is logically split into sections. It’s content is delimited by { and }. There are 5 types of sections as follow:

  • searchd - mandatory and can be declared only once, contains settings of searchd daemon
  • indexer - optional, can be declared only once, contains settings for indexer tool
  • common - optional, can be declared only once, contains common settings for searchd and indexer
  • index - can be declared multiple times,supports inheritance, requires declaration of an index name, contains configuration of an index. At least one index must be declared.
  • source - can be declared multiple times,supports inheritance, requires declaration of a source name, contains configuration of a source, optional

Indexes are sources are parsed as the configuration file is read. In case of interited sections, child sections must come after parent declarations. The same applies to distributed indexes with local indexes.

There is no rule for the settings sections. They can declared at the start or end of the file or even mixed between index/source declarations.

Scripting support

The configuration support shebang syntax - this means the configuration can be written in a programming language and interpreted at loading, allowing dynamic settings.

For example, indexes can be generated by querying a database table, various settings can be modified depending on external factors or external files can be included (which contain indexes and/sources).

The configuration file is parsed by declared declared interpreter and the output is used as the actual configuration. This is happening each time the configuration is read (not only at searchd startup).

This facility is not available on Windows platform.

In the following example, we are using php to create multiple indexes with different name and we also scan a specific folder for file containing extra declarations of indexes.

#!/usr/bin/php
...
<?php for ($i=1; $i<=6; $i++) { ?>
index test_<?=$i?> {
  type = rt
  path = /var/lib/manticore/data/test_<?=$i?>
  rt_field = subject
  ...
 }
 <?php } ?>
 ...

 <?php
 $confd_folder='/etc/manticore.conf.d/';
 $files = scandir($confd_folder);
 foreach($files as $file)
 {
         if(($file == '.') || ($file =='..'))
         {}else{
                 $fp = new SplFileInfo($confd_folder.$file);
                 if('conf' == $fp->getExtension()){
                         include ($confd_folder.$file);
                 }
         }
 }
 ?>

Comments

The configuration file supports comments, with # character used as start comment section. The comment character can be present at the start of the line or inline.

Extra care should be considered when using # in character tokenization settings as everything after it will not be taken into consideration. To avoid this, use # UTF-8 which is U+23.

# can also be escaped using \. Escaping is required if # is present in database credential in source declarations.

Inheritance of index and source declarations

Both index and source declarations support inheritance. This allows a better organization of indexes having similar settings or structure and reduce the size of the configuration.

For a parent index/source nothing needs to be specified.

For the child index/source the declaration will contain the index/source name followed by : and the parent name.

index parent {
path = /var/lib/manticore/parent
...
}

index child:parent {
path = /var/lib/manticore/child
...
}

The child will inherit the entire configuration of the parent. In the child declaration any setting declared will overwrite the inherited values. Please note that in case of multi value settings, defining a single value in child will clear out all inherited values. For example in the parent there are several sql_query_pre declaration and the child has a single sql_query_pre declaration, all the sql_query_pre inherited declarations are cleared. If you need to override some of the inherited values from parent, they need to be explicit declared in the child. This is also available if you don’t need a value from parent. For example if the value of sql_query_pre from parent is not needed, then in the child we will declare the directive with an empty value like sql_query_pre=. This also means that existing values of a multi value setting will not be copied if the child declares one value for that setting. The inheritance bevahiour applies to fields and attributes and not just index options. If, for example, the parent has 2 integer attributes and the child needs a new integer attribute, the integer attributes declaration from parent must be copied in the child configuration.