EvaluateRagFaithfulness 2025.10.9.21¶
Bundle¶
com.snowflake.openflow.runtime | runtime-rag-evaluation-processors-nar
Description¶
Evaluates the faithfulness of generated answers in a Retrieval-Augmented Generation (RAG) system by analyzing responses using an LLM (e.g., OpenAI’s GPT). The processor enriches each FlowFile record with faithfulness metrics and detailed analysis.
Tags¶
ai, evaluation, faithfulness, llm, nlp, openai, openflow, rag
Input Requirement¶
REQUIRED
Supports Sensitive Dynamic Properties¶
false
Properties¶
| Property | Description |
|---|---|
| Context Identifier Record Path | The RecordPath to the array of contexts IDs in the record. |
| Context Record Path | The RecordPath to the array of contexts in the record. |
| Evaluation Results Record Path | The RecordPath to write the results of the evaluation to. |
| Generated Answer Record Path | The path to the answer field in the record |
| LLM Provider Service | The provider service for sending evaluation prompts to LLM |
| Question Record Path | The RecordPath to the question field in the record. |
| Record Reader | The Record Reader to use for reading the FlowFile. |
| Record Writer | The Record Writer to use for writing the results. |
Relationships¶
| Name | Description |
|---|---|
| failure | FlowFiles that cannot be processed are routed to this relationship |
| success | FlowFiles that are successfully processed are routed to this relationship |
Writes attributes¶
| Name | Description |
|---|---|
| average.answer.faithfulness | The average faithfulness score computed over all records. |
| json.parse.failures | Number of JSON parse failures encountered. |
Use cases¶
| Use this processor to assess the faithfulness of answers generated by an LLM compared to the provided context. It provides metrics that can be used for monitoring and improving the performance of RAG systems. |
|---|