Yioop_V9.5_Source_Code_Documentation

WordIterator extends IndexBundleIterator

in package

• Application

Used to iterate through the documents associated with a word in an IndexArchiveBundle. It also makes it easy to get the summaries of these documents.

Tags

author

Chris Pollett

see

IndexArchiveBundle

Table of Contents

DOC_RANK_WEIGHT  = 50

Weighting factor to multiply to make a doc-rank (approximate score of document based on its position in the index (when crawled)).

HOST_KEY_POS  = 17

Host Key position + 1 (first char says doc, inlink or external link)

KEY_LEN  = 8

Length of a doc key part

RESULTS_PER_BLOCK  = 200

Default number of documents returned for each block (at most)

$archive_file  : int

$avg_items_per_partition  : int

$base64_word_key  : string

Word key above in our modified base 64 encoding

$count_block  : int

The number of documents in the current block

$current_block_fresh  : bool

Says whether the value in $this->count_block is up to date

$current_doc_offset  : int

The current value of the doc_offset of current posting if known

$current_generation  : int

Numeric number of current shard

$current_offset  : int

The current byte offset in the IndexShard (if older index)

$dictionary_info  : array<string|int, mixed>

An array of shard generation and posting list offsets, lengths, and numbers of documents

$direction  : int

Whether the iterator iterates forward or backward through documents in bundle

$empty  : int

Keeps track of whether the word_iterator list is empty because the word does not appear in the index shard

$filter  : SearchfiltersModel

Model responsible for keeping track of edited and deleted search results

$generation_pointer  : int

Index into dictionary_info corresponding to the current shard

$index_name  : string

The timestamp of the index is associated with this iterator

$index_version  : int

The index version affects how the iterator cycles through documents There was a big change in index format between version 3 and prior formats

$is_meta  : string

Whether word key corresponds to a meta word

$last_offset  : int

Last Offset of word occurrence in the IndexShard

$max_items_per_partition  : int

$next_offset  : int

The next byte offset in the IndexShard

$no_more_generations  : bool

Used to keep track of whether getWordInfo might still get more data on the search terms as advance generations

$num_docs  : int

Estimate of the number of documents that this iterator can return

$num_generations  : int

The total number of shards that have data for this word

$num_occurrences  : int

$pages  : array<string|int, mixed>

Cache of what currentDocsWithWord returns

$ranking_factors  : array<string|int, mixed>

How url, keywords, and title words should influence relevance and doc rank calculations

$results_per_block  : int

Number of documents returned for each block (at most)

$seen_docs  : int

The number of documents already iterated over

$start_generation  : int

First shard generation that word info was obtained for

$start_offset  : int

Starting Offset of word occurrence in the IndexShard

$term_info_computed  : int

$threshold_exceeded  : int

$total_num_docs  : int

$total_num_docs_and_links  : int

$total_number_of_partitions  : int

$word_key  : string

hash of word or phrase that the iterator iterates over

__construct()  : mixed

Creates a word iterator with the given parameters.

advance()  : mixed

Forwards the iterator one group of docs

advanceGeneration()  : mixed

Switches which index shard is being used to return occurrences of the word to the next shard containing the word

advanceSeenDocs()  : mixed

Updates the seen_docs count during an advance() call

currentDocsWithWord()  : mixed

Gets the current block of doc ids and score associated with the this iterators word

currentGenDocOffsetWithWord()  : mixed

Gets the doc_offset and generation for the next document that would be return by this iterator

findDocsWithWord()  : mixed

Hook function used by currentDocsWithWord to return the current block of docs if it is not cached

frequencyNormalizationPrefaceScoring()  : array<string|int, mixed>

Normalizes the frequencies of a term within a document with respect to the length of the document, the positions of the term with the document and the overall importance score for a given position within the document Also computes the score of the posting for the host keywords, title keywords, and path keywords.

genDocOffsetCmp()  : int

Compares two arrays each containing a (generation, offset) pair.

getCurrentDocsForKeys()  : array<string|int, mixed>

Gets the summaries associated with the keys provided the keys can be found in the current block of docs returned by this iterator

getDirection()  : int

Returns CrawlConstants::ASCENDING or CrawlConstants::DESCENDING depending on the direction in which this iterator ttraverse the underlying index archive bundle.

getDocKeyPositionsScoringInfo()  : mixed

Add to a set of postings from a partition scoring information, position list information and info about the relative weights of given position based on the position list file and doc_map file.

getGenerationPostings()  : array<string|int, mixed>

Given a partition number in the the index's PartitionDocumentBundle retrieves all the posting for the word iterator's term in that partition.

getPostingsSliceResults()  : mixed

Given the current_offset, result_per_block, and index used get the result_per_block postings starting from current_offset in the current direction (ascending or descending) for the term word iterator iterates over from the index.

nextDocIndexOffsetPair()  : array<string|int, mixed>

Computes a pair [posting_slice_offset, $doc_index], such that the $doc_index when shift to make a doc_offset is greater than $doc_offset and posting_slice_offset is the offset of the first posting with this property.

nextDocsWithWord()  : array<string|int, mixed>

Get the current block of doc summaries for the word iterator and advances the current pointer to the next block of documents. If a doc index is the next block must be of docs after this doc_index

plainAdvance()  : mixed

Forwards the iterator one group of docs. This is what's called by @see advance($gen_doc_offset) if $gen_doc_offset is null

plan()  : string

Returns a string representation of a plan by which the current iterator finds its results

reset()  : mixed

Resets the iterator to the first document block that it could iterate over

setResultsPerBlock()  : mixed

Sets the value of the result_per_block field. This field controls the maximum number of results that can be returned in one go by currentDocsWithWord()

termInfoIteratorFields()  : mixed

Used to compute fields such as $this->total_num_docs for this iterator on term $word_key for index $index_name