afs_search_build - AFS - Reference Guides

AFS Filters Description

Product
AFS
Platform
7.11
Category
Reference Guides
Language
English

The afs_search_build filter generates reply databases from PaF execution result.

The filter is declared with the afs_search_build type. It is in the antidot-paf package. It is a visitor filter.

This filter will only work if instanced after the afs_doc_index filter.

This filter can be instantiated only once at any given moment. It will not read the "instances" parameter in the configuration.

The Search Build filter specifications are described in the following table:

Parameter name

Mandatory

Type

Default

Description

from_PaF_id

No

string

current PaF id

Process documents with PaF id greater or equal to this value.
  • Set parameter to current_paf_id to process all documents of the current PaF identifier (useful when PaF has been launched with -S option)
  • Set parameter to any positive integer value to process all documents with PaFid greater or equal to that value.

max_doc_per_dir

No

integer

100000

The desired maximum number of documents per directory.

max_memory_percent

No

integer

80% of host memory

Define the maximal memory (in % of host memory) the afs_search_build process should use. Defaults to 80% of the host memory.

nb_dirs_threshold

No

integer

32

No defragmentation unless the number of shards reaches this limit.

min_shard_fill_percentage

No

integer

50

Looks for the first shard (from ordered list of shards) which has less than this percentage of the maximum number of valid documents per directory. The defragmentation process occurs on the reply database starting from this shard until the last generated shard.

to_PaF_id

No

integer

Current PaF id

Process documents with PaF id in [from PaF id, to PaF id].

The Search Build filter deprecated specifications are described in the following table:

Parameter name

Deprecated since

Replaced by

Description

max_memory_mb

7.8

max_memory_percent

Define the maximal memory (in megabytes) the afs_search_build process should use. Defaults to 80% of the host memory.

nb_dirs

7.9.0

N/A

The number of desired output directories. Note that documents size, memory usage, and other factors may have an influence on actual output directories numbers.

By default, all deleted/failed documents and valid documents that reach this filter are used to generate reply databases. In order to process all the files of the current PaF id, the from_PaF_id parameter must be set to current_paf_id.
Note: If a negative value is set for the from_PaF_id parameter, value is automatically set to 1. If a negative value is set for the to_PaF_id parameter, value is automatically set to current PaF id.
Note: This filter will only work if instanced after an afs_doc_index filter
.
Attention: This filter can be instantiated only once at any given moment. It will not read the "instances" parameter in the configuration.
Attention: max_memory_percent and nb_dirs parameters are mutually exclusive.