Class Lucene91HnswVectorsFormat
java.lang.Object
org.apache.lucene.codecs.KnnVectorsFormat
org.apache.lucene.backward_codecs.lucene91.Lucene91HnswVectorsFormat
- All Implemented Interfaces:
NamedSPILoader.NamedSPI
Lucene 9.1 vector format, which encodes numeric vector values and an optional associated graph
connecting the documents having values. The graph is used to power HNSW search. The format
consists of three files:
.vec (vector data) file
This file stores all the floating-point vector data ordered by field, document ordinal, and vector dimension. The floats are stored in little-endian byte order.
.vex (vector index)
Stores graphs connecting the documents for each field organized as a list of nodes' neighbours as following:
- For each level:
- For each node:
- [int32] the number of neighbor nodes
- array[int32] the neighbor ordinals
- array[int32] padding from empty integers if the number of neighbors less than the maximum number of connections (maxConn). Padding is equal to ((maxConn-the number of neighbours) * 4) bytes.
- For each node:
.vem (vector metadata) file
For each field:
- [int32] field number
- [int32] vector similarity function ordinal
- [vlong] offset to this field's vectors in the .vec file
- [vlong] length of this field's vectors, in bytes
- [vlong] offset to this field's index in the .vex file
- [vlong] length of this field's index data, in bytes
- [int] dimension of this field's vectors
- [int] the number of documents having values for this field
- [int8] if equals to -1, dense – all documents have values for a field. If equals to 0, sparse – some documents missing values.
- array[int] for sparse case, the docids of documents having vectors, in order
- [int] the maximum number of connections (neighbours) that each node can have
- [int] number of levels in the graph
- Graph nodes by level. For each level
- [int] the number of nodes on this level
- array[int] for levels greater than 0 list of nodes on this level, stored as the the level 0th nodes ordinals.
- WARNING: This API is experimental and might change in incompatible ways in the next release.
-
Field Summary
Modifier and TypeFieldDescriptionstatic final int
Default number of the size of the queue maintained while searching during a graph construction.static final int
Default number of maximum connections per nodeFields inherited from class org.apache.lucene.codecs.KnnVectorsFormat
DEFAULT_MAX_DIMENSIONS, EMPTY
-
Constructor Summary
ConstructorDescriptionA constructor for vectors format with default parametersLucene91HnswVectorsFormat
(int maxConn, int beamWidth) A constructor for vectors format -
Method Summary
Modifier and TypeMethodDescriptionfieldsReader
(SegmentReadState state) fieldsWriter
(SegmentWriteState state) toString()
Methods inherited from class org.apache.lucene.codecs.KnnVectorsFormat
availableKnnVectorsFormats, forName, getMaxDimensions, getName, reloadKnnVectorsFormat
-
Field Details
-
DEFAULT_MAX_CONN
public static final int DEFAULT_MAX_CONNDefault number of maximum connections per node- See Also:
-
DEFAULT_BEAM_WIDTH
public static final int DEFAULT_BEAM_WIDTHDefault number of the size of the queue maintained while searching during a graph construction.- See Also:
-
-
Constructor Details
-
Lucene91HnswVectorsFormat
public Lucene91HnswVectorsFormat()A constructor for vectors format with default parameters -
Lucene91HnswVectorsFormat
public Lucene91HnswVectorsFormat(int maxConn, int beamWidth) A constructor for vectors format
-
-
Method Details
-
fieldsWriter
- Specified by:
fieldsWriter
in classKnnVectorsFormat
- Throws:
IOException
-
fieldsReader
- Specified by:
fieldsReader
in classKnnVectorsFormat
- Throws:
IOException
-
toString
-