Type Alias CriteriaEvalChainConfig

Type declaration

• evaluatorType: "criteria"

• formatEvaluatorInputs: EvaluatorInputFormatter

  Convert the evaluation data into formats that can be used by the evaluator. This should most commonly be a string. Parameters are the raw input from the run, the raw output, raw reference output, and the raw run.

  Example

  // Chain input: { input: "some string" }
  // Chain output: { output: "some output" }
  // Reference example output format: { output: "some reference output" }
  const formatEvaluatorInputs = ({
    rawInput,
    rawPrediction,
    rawReferenceOutput,
  }) => {
    return {
      input: rawInput.input,
      prediction: rawPrediction.output,
      reference: rawReferenceOutput.output,
    };
  };
  Copy

  Returns

  The prepared data.

• OptionalagentTools?: undefined | StructuredToolInterface[]

  A list of tools available to the agent, for TrajectoryEvalChain.

• OptionalchainOptions?: undefined | Partial<Omit<LLMEvalChainInput<EvalOutputType, BaseLanguageModelInterface>, "llm">>

• Optionalcriteria?: undefined | (CriteriaLike & (Record<string, string> | Criteria))

  The criteria to use for the evaluator.

• OptionaldistanceMetric?: undefined | EmbeddingDistanceType

  The distance metric to use for comparing the embeddings.

• Optionalembedding?: any

  The embedding objects to vectorize the outputs.

• OptionalfeedbackKey?: undefined | string

  The feedback (or metric) name to use for the logged evaluation results. If none provided, we default to the evaluationName.

• Optionalllm?: any