Lucene90NormsFormat

Lucene 9.0 Score normalization format.

Encodes normalization values by encoding each value with the minimum number of bytes needed to represent the range (which can be zero).

Files:

1. .nvd: Norms data

2. .nvm: Norms metadata

The Norms metadata or .nvm file.

For each norms field, this stores metadata, such as the offset into the Norms data (.nvd)

Norms metadata (.dvm) --> Header,^NumFields,Footer

• Header -->CodecUtil.writeIndexHeader

• Entry --> FieldNumber, DocsWithFieldAddress, DocsWithFieldLength, NumDocsWithField, BytesPerNorm, NormsAddress

• FieldNumber -->DataOutput.writeInt

• DocsWithFieldAddress -->DataOutput.writeLong

• DocsWithFieldLength -->DataOutput.writeLong

• NumDocsWithField -->DataOutput.writeInt

• BytesPerNorm -->DataOutput.writeByte

• NormsAddress -->DataOutput.writeLong

• Footer -->CodecUtil.writeFooter

FieldNumber of -1 indicates the end of metadata.

NormsAddress is the pointer to the start of the data in the norms data (.nvd), or the singleton value when BytesPerValue = 0. If BytesPerValue is different from 0 then there are NumDocsWithField values to read at that offset.

DocsWithFieldAddress is the pointer to the start of the bit set containing documents that have a norm in the norms data (.nvd), or -2 if no documents have a norm value, or -1 if all documents have a norm value.

DocsWithFieldLength is the number of bytes used to encode the set of documents that have a norm.

The Norms data or .nvd file.

For each Norms field, this stores the actual per-document data (the heavy-lifting)

Norms data (.nvd) --> Header,< Data >^NumFields,Footer

• Header -->CodecUtil.writeIndexHeader

• DocsWithFieldData -->Bit set of MaxDoc bits

• NormsData -->DataOutput.writeByte^{NumDocsWithField * BytesPerValue}

• Footer -->CodecUtil.writeFooter