core/org.gnit.lucenekmp.util.quantization/ScalarQuantizer/Companion

Companion

object Companion

Properties

SCALAR_QUANTIZATION_SAMPLE_SIZE

const val SCALAR_QUANTIZATION_SAMPLE_SIZE: Int = 25000

SCRATCH_SIZE

const val SCRATCH_SIZE: Int = 20

Functions

fromVectors

fun fromVectors(floatVectorValues: FloatVectorValues, confidenceInterval: Float, totalVectorCount: Int, bits: Byte): ScalarQuantizer

This will read the float vector values and calculate the quantiles. If the number of float vectors is less than .SCALAR_QUANTIZATION_SAMPLE_SIZE then all the values will be read and the quantiles calculated. If the number of float vectors is greater than .SCALAR_QUANTIZATION_SAMPLE_SIZE then a random sample of .SCALAR_QUANTIZATION_SAMPLE_SIZE will be read and the quantiles calculated.

fun fromVectors(floatVectorValues: FloatVectorValues, confidenceInterval: Float, totalVectorCount: Int, bits: Byte, quantizationSampleSize: Int): ScalarQuantizer

fromVectorsAutoInterval

fun fromVectorsAutoInterval(floatVectorValues: FloatVectorValues, function: VectorSimilarityFunction, totalVectorCount: Int, bits: Byte): ScalarQuantizer

getUpperAndLowerQuantile

fun getUpperAndLowerQuantile(arr: FloatArray, confidenceInterval: Float): FloatArray

Takes an array of floats, sorted or not, and returns a minimum and maximum value. These values are such that they reside on the (1 - confidenceInterval)/2 and confidenceInterval/2 percentiles. Example: providing floats [0..100] and asking for 90 quantiles will return 5 and 95.