📘 Overview

The RMSampleEvaluator is an operator that scores text quality using a reward model (OpenAssistant/reward-model-deberta-v3-large-v2) trained on human preference data. It takes an instruction-response pair as input and outputs a reward score, where a higher score indicates better quality. This is useful for evaluating the quality of generated text in a human-aligned way.

__init__

def __init__(self, device='cuda', model_cache_dir='./dataflow_cache')

Parameter	Type	Default	Description
device	str	'cuda'	The device to run the model on (e.g., 'cuda', 'cpu').
model_cache_dir	str	'./dataflow_cache'	The directory to cache the downloaded Hugging Face model.

run

def run(self, storage: DataFlowStorage, input_instruction_key: str = 'instruction', input_output_key: str = 'output', output_key: str = 'RMScore')

Parameter	Type	Default	Description
storage	DataFlowStorage	Required	The DataFlowStorage instance for reading and writing data.
input_instruction_key	str	'instruction'	The column name in the input dataframe for the instruction text.
input_output_key	str	'output'	The column name in the input dataframe for the response text.
output_key	str	'RMScore'	The column name in the output dataframe for the generated reward score.

📝 Prompt Template Descriptions

🧠 Example Usage

🧾 Default Output Format

Field	Type	Description
instruction	str	The input instruction text.
output	str	The input response text.
RMScore	float	The reward model score generated by the model.

Example Input:

{
  "instruction": "Explain the importance of photosynthesis in simple terms.",
  "output": "Photosynthesis is crucial because it's how plants make their own food using sunlight, water, and air. This process also releases the oxygen that we need to breathe."
}

Example Output:

{
  "instruction": "Explain the importance of photosynthesis in simple terms.",
  "output": "Photosynthesis is crucial because it's how plants make their own food using sunlight, water, and air. This process also releases the oxygen that we need to breathe.",
  "RMScore": 0.952
}