|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectit.unimi.dsi.fastutil.ints.AbstractIntIterator
it.unimi.dsi.mg4j.search.AbstractDocumentIterator
it.unimi.dsi.mg4j.search.AbstractCompositeDocumentIterator
it.unimi.dsi.mg4j.search.AbstractUnionDocumentIterator
it.unimi.dsi.mg4j.index.MultiTermIndexIterator
public class MultiTermIndexIterator
A virtual index iterator that merges several component index iterators.
This class adds to AbstractUnionDocumentIterator
an interval iterator generating the OR of the intervals returned for each of the documents involved.
The main difference with an OrDocumentIterator
built on the same array of component iterators
is that this class implements IndexIterator
and hence provides a count()
(the sum
of counts of those component iterators positioned on the current document) and a frequency()
. The
latter is by default the maximum frequency of a component iterator, but it can be set
at construction time
.
The main raison d'être of this class is support for query expansion: a blind application
of OrDocumentIterator
to an array of index iterators would mislead scorers such as BM25Scorer
because low-frequency terms (e.g., hapax legomena) would be responsible for most of the score.
Note that DocumentIteratorVisitor has a visit method for generic index iterator
and a visit method for instances of this class.
This approach provides additional flexibility—a scorer, for instance, might treat an instance of
this class as a standard IndexIterator
, or it might choose to query which terms actually appear
and do something more sophisticated (for instance, using weights).
Nested Class Summary |
---|
Nested classes/interfaces inherited from class it.unimi.dsi.mg4j.search.AbstractCompositeDocumentIterator |
---|
AbstractCompositeDocumentIterator.AbstractCompositeIndexIntervalIterator, AbstractCompositeDocumentIterator.AbstractCompositeIntervalIterator |
Nested classes/interfaces inherited from class it.unimi.dsi.mg4j.search.AbstractDocumentIterator |
---|
AbstractDocumentIterator.AbstractIntervalIterator |
Field Summary | |
---|---|
protected int |
id
The id of this iterator. |
protected String |
term
The term of this iterator. |
Fields inherited from class it.unimi.dsi.mg4j.search.AbstractUnionDocumentIterator |
---|
curr, front, frontSize, queue |
Fields inherited from class it.unimi.dsi.mg4j.search.AbstractCompositeDocumentIterator |
---|
documentIterator, indexIterator, indices, n, soleIndex |
Fields inherited from class it.unimi.dsi.mg4j.search.AbstractDocumentIterator |
---|
last, next, weight |
Constructor Summary | |
---|---|
protected |
MultiTermIndexIterator(int defaultFrequency,
IndexIterator... indexIterator)
Creates a new document iterator that merges the given array of iterators. |
Method Summary | ||
---|---|---|
|
accept(DocumentIteratorVisitor<T> visitor)
Accepts a visitor. |
|
|
acceptDeep(DocumentIteratorVisitor<T> visitor)
|
|
|
acceptDeepOnTruePaths(DocumentIteratorVisitor<T> visitor)
|
|
|
acceptOnTruePaths(DocumentIteratorVisitor<T> visitor)
Invokes AbstractUnionDocumentIterator.acceptOnTruePaths(DocumentIteratorVisitor) only on component
iterators positioned on the current document. |
|
int |
count()
The count is the sum of counts of those component iterators positioned on the current document. |
|
int |
frequency()
The frequency is either the default frequency set at construction time, or the maximum frequency of the component iterators. |
|
int |
front(IndexIterator[] indexIterator)
Fills the given array with the index iterators composing the current front. |
|
protected IntervalIterator |
getComposedIntervalIterator(Index index)
|
|
static IndexIterator |
getInstance(Index index,
IndexIterator... indexIterator)
Returns an index iterator that merges the given array of iterators. |
|
static IndexIterator |
getInstance(IndexIterator... indexIterator)
Returns an index iterator that merges the given array of iterators. |
|
static IndexIterator |
getInstance(int defaultFrequency,
Index index,
IndexIterator... indexIterator)
Returns an index iterator that merges the given array of iterators. |
|
static IndexIterator |
getInstance(int defaultFrequency,
IndexIterator... indexIterator)
Returns an index iterator that merges the given array of iterators. |
|
int |
id()
Returns the id of this index iterator. |
|
IndexIterator |
id(int id)
Sets the id of this index iterator. |
|
Index |
index()
Returns the index over which this iterator is built. |
|
int |
nextDocument()
Returns the next document provided by this document iterator, or -1 if no more documents are available. |
|
Payload |
payload()
This method is not implemented by this class. |
|
int[] |
positionArray()
Returns the positions at which the term appears in the current document in an array. |
|
IntIterator |
positions()
Returns the positions at which the term appears in the current document. |
|
int |
positions(int[] position)
Stores the positions at which the term appears in the current document in a given array. |
|
int |
skipTo(int n)
Skips all documents smaller than n . |
|
String |
term()
Returns the term whose inverted list is returned by this index iterator. |
|
IndexIterator |
term(CharSequence term)
Sets the term whose inverted list is returned by this index iterator. |
|
int |
termNumber()
Returns the number of the term whose inverted list is returned by this index iterator. |
|
IndexIterator |
weight(double weight)
Sets the weight of this index iterator. |
Methods inherited from class it.unimi.dsi.mg4j.search.AbstractUnionDocumentIterator |
---|
computeFront, intervalIterator, intervalIterators |
Methods inherited from class it.unimi.dsi.mg4j.search.AbstractCompositeDocumentIterator |
---|
dispose, indices, intervalIterator, toString |
Methods inherited from class it.unimi.dsi.mg4j.search.AbstractDocumentIterator |
---|
document, hasNext, iterator, nextInt, weight |
Methods inherited from class it.unimi.dsi.fastutil.ints.AbstractIntIterator |
---|
next, remove, skip |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Methods inherited from interface it.unimi.dsi.mg4j.search.DocumentIterator |
---|
dispose, document, indices, intervalIterator, intervalIterator, intervalIterators, iterator, nextInt, weight |
Methods inherited from interface it.unimi.dsi.fastutil.ints.IntIterator |
---|
skip |
Methods inherited from interface java.util.Iterator |
---|
hasNext, next, remove |
Field Detail |
---|
protected String term
protected int id
Constructor Detail |
---|
protected MultiTermIndexIterator(int defaultFrequency, IndexIterator... indexIterator) throws IOException
defaultFrequency
- the default term frequency (or Integer.MIN_VALUE
for the max).indexIterator
- the iterators to be joined.
IOException
Method Detail |
---|
public static IndexIterator getInstance(IndexIterator... indexIterator) throws IOException
index()
will return the result of the same method on the first iterator.
indexIterator
- the iterators to be joined (at least one).
IllegalArgumentException
- if no iterators were provided.
IOException
public static IndexIterator getInstance(Index index, IndexIterator... indexIterator) throws IOException
Note that the special case of the empty and of the singleton arrays
are handled efficiently. The frequency is computed as a max, and
index()
will return index
.
index
- the index that wil be returned by index()
.indexIterator
- the iterators to be joined.
IOException
public static IndexIterator getInstance(int defaultFrequency, IndexIterator... indexIterator) throws IOException
defaultFrequency
- the default term frequency (or Integer.MIN_VALUE
for the max).indexIterator
- the iterators to be joined (at least one).
IllegalArgumentException
- if no iterators were provided, or they run on different indices.
IOException
public static IndexIterator getInstance(int defaultFrequency, Index index, IndexIterator... indexIterator) throws IOException
Note that the special case of the empty and of the singleton arrays are handled efficiently.
defaultFrequency
- the default term frequency (or Integer.MIN_VALUE
for the max).index
- the index that wil be returned by index()
.indexIterator
- the iterators to be joined.
IllegalArgumentException
- if there is some iterator on an index different from index
.
IOException
protected IntervalIterator getComposedIntervalIterator(Index index)
getComposedIntervalIterator
in class AbstractUnionDocumentIterator
public int skipTo(int n) throws IOException
DocumentIterator
n
.
Define the current document k
associated with this document iterator
as follows:
DocumentIterator.nextDocument()
and this method have never been called;
Integer.MAX_VALUE
, if a call to this method returned Integer.MAX_VALUE
;
DocumentIterator.nextDocument()
or this method, otherwise.
If k
is larger than or equal to n
, then
this method does nothing and returns k
. Otherwise, a
call to this method is equivalent to
while( ( k = nextDocument() ) < n && k != -1 ); return k == -1 ? Integer.MAX_VALUE : k;
Thus, when a result k
≠ Integer.MAX_VALUE
is returned, the state of this iterator
will be exactly the same as after a call to DocumentIterator.nextDocument()
that returned k
.
In particular, the first document larger than or equal to n
(when returned
by this method) will not be returned by the next call to
DocumentIterator.nextDocument()
.
skipTo
in interface DocumentIterator
skipTo
in class AbstractUnionDocumentIterator
n
- a document pointer.
n
if available, Integer.MAX_VALUE
otherwise.
IOException
public int nextDocument() throws IOException
DocumentIterator
Warning: the specification of this method has significantly changed as of MG4J 1.2.
The special return value -1 is used to mark the end of iteration (a NoSuchElementException
would have been thrown before in that case, so ho harm should be caused by this change). The reason
for this change is providing fully lazy iteration over documents. Fully lazy iteration
does not provide an hasNext()
method—you have to actually ask for the next
element and check the return value. Fully lazy iteration is much lighter on method calls (half) and
in most (if not all) MG4J classes leads to a much simpler logic. Moreover, DocumentIterator.nextDocument()
can be specified as throwing an IOException
, which avoids the pernicious proliferation
of try/catch blocks in very short, low-level methods (it was having a detectable impact on performance).
nextDocument
in interface DocumentIterator
nextDocument
in class AbstractUnionDocumentIterator
IOException
public int count() throws IOException
count
in interface IndexIterator
IOException
public int front(IndexIterator[] indexIterator)
This method is essentially a safe exposure of the front of the queue
merging the component index iterators.
After a call to nextDocument()
, you can use this method to know
which terms actually appear in the current document. You can use the public
field AbstractCompositeDocumentIterator.n
to size the argument
array appropriately.
indexIterator
- an array, at least large as the number of component index iterators,
that will be partially filled with the index iterators corresponding to terms appearing in the current document.
indexIterator
.public int frequency() throws IOException
frequency
in interface IndexIterator
IOException
public IndexIterator term(CharSequence term)
IndexIterator
Usually, the term is automatically set by Index.documents(CharSequence)
or by IndexReader.documents(CharSequence)
, but you can
use this method to ensure that IndexIterator.term()
doesn't throw
an exception.
term
in interface IndexIterator
term
- a character sequence (that will be defensively copied)
that will be assumed to be the term whose inverted list is returned by this index iterator.
public String term()
IndexIterator
Usually, the term is automatically set by IndexReader.documents(CharSequence)
or IndexReader.documents(int)
, but you can
supply your own term with IndexIterator.term(CharSequence)
.
term
in interface IndexIterator
IndexIterator.termNumber()
public int termNumber()
IndexIterator
Usually, the term number is automatically set by IndexReader.documents(CharSequence)
or IndexReader.documents(int)
.
termNumber
in interface IndexIterator
IndexIterator.term()
public IndexIterator id(int id)
IndexIterator
The id is an integer associated to each index iterator. It has no specific semantics, and can be used differently in different contexts. A typical usage pattern, for instance, is using it to assign a unique number to the index iterators contained in a composite document iterator (say, numbering consecutively the leaves of the composite).
id
in interface IndexIterator
id
- the new id for this index iterator.
public int id()
IndexIterator
id
in interface IndexIterator
IndexIterator.id(int)
public Index index()
IndexIterator
index
in interface IndexIterator
public Payload payload()
payload
in interface IndexIterator
public int[] positionArray() throws IOException
IndexIterator
Implementations are allowed to return the same array across different calls to this method.
positionArray
in interface IndexIterator
IOException
public IntIterator positions() throws IOException
IndexIterator
positions
in interface IndexIterator
IOException
public int positions(int[] position) throws IOException
IndexIterator
If the array is not large enough (i.e., it does not contain IndexIterator.count()
elements),
this method will return a negative number (the opposite of the count).
positions
in interface IndexIterator
position
- an array that will be used to store positions.
positions
cannot
hold all positions.
IOException
public IndexIterator weight(double weight)
DocumentIterator
weight
in interface IndexIterator
weight
in interface DocumentIterator
weight
in class AbstractDocumentIterator
weight
- the weight of this index iterator.
DocumentIterator.weight(double)
public <T> T accept(DocumentIteratorVisitor<T> visitor) throws IOException
DocumentIterator
A document iterator is usually structured as composite,
with operators as internal nodes and IndexIterator
s
as leaves. This method implements the visitor pattern.
accept
in interface DocumentIterator
accept
in class AbstractCompositeDocumentIterator
visitor
- the visitor.
null
if the visit was interrupted.
IOException
public <T> T acceptOnTruePaths(DocumentIteratorVisitor<T> visitor) throws IOException
AbstractUnionDocumentIterator
AbstractUnionDocumentIterator.acceptOnTruePaths(DocumentIteratorVisitor)
only on component
iterators positioned on the current document.
acceptOnTruePaths
in interface DocumentIterator
acceptOnTruePaths
in class AbstractUnionDocumentIterator
visitor
- a visitor.
IOException
DocumentIterator.accept(DocumentIteratorVisitor)
,
CounterCollectionVisitor
public <T> T acceptDeep(DocumentIteratorVisitor<T> visitor) throws IOException
IOException
public <T> T acceptDeepOnTruePaths(DocumentIteratorVisitor<T> visitor) throws IOException
IOException
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |