Skip to content

DataFlow中文文档

Main Navigation

基本信息

快速开始

流水线

简体中文

简体中文

外观

Sidebar Navigation

API文档入口

serving

serving_main

operators

agentic_rag

eval

AgenticRAGQAF1SampleEvaluator

generate

AgenticRAGAtomicTaskGenerator

AgenticRAGDepthQAGenerator

AgenticRAGWidthQAGenerator

code

eval

CodeAutoGeneratedSampleEvaluator

CodeDocumentQualitySampleEvaluator

CodeEncodedDataSampleEvaluator

CodeLengthSampleEvaluator

CodeQualitySampleEvaluator

CodeSandboxSampleEvaluator

CodeTextCompositionSampleEvaluator

filter

CodeAutoGeneratedFilter

CodeDocumentQualityFilter

CodeEncodedDataFilter

CodeFileTypeContentFilter

CodeGenericScoreFilter

CodeLengthSampleFilter

CodeQualityScoreFilter

CodeTextCompositionFilter

generate

CodeCodeToInstructionGenerator

CodeEnhancementInstructionGenerator

CodeInstructionGenerator

CodeInstructionToCodeGenerator

chemistry

eval

EvaluateSmilesEquivalence

generate

ExtractSmilesFromText

conversations

eval

FuncCallConversationSampleEvaluator

filter

CompositionTaskFilter

generate

AtomTaskGenerator

ConsistentChatGenerator

FunctionGenerator

MultiTurnConversationGenerator

ParaSeqTaskGenerator

ScenarioExpandGenerator

ScenarioExtractGenerator

SequentialTaskGenerator

core_speech

generate

Speech2TextGenerator

core_text

eval

BenchDatasetEvaluator

PromptedEvaluator

Text2QASampleEvaluator

filter

GeneralFilter

KCenterGreedyFilter

PromptedFilter

generate

PairedPromptedGenerator

PromptedGenerator

RandomDomainKnowledgeRowGenerator

Text2MultiHopQAGenerator

Text2QAGenerator

refine

PandasOperator

PromptedRefiner

core_vision

generate

PromptedVQAGenerator

db

generate

DBOperator

general_text

eval

BertSampleEvaluator

BleuSampleEvaluator

CiderSampleEvaluator

LangkitSampleEvaluator

LexicalDiversitySampleEvaluator

NgramSampleEvaluator

PerspectiveSampleEvaluator

PresidioSampleEvaluator

Task2VecDatasetEvaluator

VendiDatasetEvaluator

filter

AlphaWordsFilter

BlocklistFilter

CapitalWordsFilter

CharNumberFilter

ColonEndFilter

ContentNullFilter

CurlyBracketFilter

HashDeduplicateFilter

HtmlEntityFilter

IDCardFilter

LLMLanguageFilter

LangkitFilter

LanguageFilter

LexicalDiversityFilter

LineEndWithEllipsisFilter

LineStartWithBulletpointFilter

LineWithJavascriptFilter

LoremIpsumFilter

MeanWordLengthFilter

MinHashDeduplicateFilter

NgramFilter

NgramHashDeduplicateFilter

NoPuncFilter

PerspectiveFilter

PresidioFilter

SemDeduplicateFilter

SentenceNumberFilter

SimHashDeduplicateFilter

SpecialCharacterFilter

StopWordFilter

SymbolWordRatioFilter

UniqueWordsFilter

WatermarkFilter

WordNumberFilter

refine

HtmlEntityRefiner

HtmlUrlRemoverRefiner

LowercaseRefiner

NERRefiner

PIIAnonymizeRefiner

ReferenceRemoverRefiner

RemoveContractionsRefiner

RemoveEmojiRefiner

RemoveEmoticonsRefiner

RemoveExtraSpacesRefiner

RemoveImageRefsRefiner

RemoveNumberRefiner

RemovePunctuationRefiner

RemoveRepetitionsPunctuationRefiner

RemoveStopwordsRefiner

SpellingCorrectionRefiner

StemmingLemmatizationRefiner

TextNormalizationRefiner

knowledge_cleaning

generate

FileOrURLToMarkdownConverterBatch

KBCChunkGenerator

KBCChunkGeneratorBatch

KBCMultiHopQAGeneratorBatch

KBCTextCleaner

KBCTextCleanerBatch

MathBookQuestionExtract

reasoning

eval

ReasoningCategoryDatasetEvaluator

ReasoningDifficultyDatasetEvaluator

ReasoningQuestionCategorySampleEvaluator

ReasoningQuestionDifficultySampleEvaluator

ReasoningTokenDatasetEvaluator

filter

ReasoningAnswerFormatterFilter

ReasoningAnswerGroundTruthFilter

ReasoningAnswerModelJudgeFilter

ReasoningAnswerNgramFilter

ReasoningAnswerPipelineRootFilter

ReasoningAnswerTokenLengthFilter

ReasoningQuestionFilter

generate

ReasoningAnswerExtractionQwenMathEvalGenerator

ReasoningAnswerGenerator

ReasoningPretrainFormatConvertGenerator

ReasoningPseudoAnswerGenerator

ReasoningQuestionGenerator

text2sql

eval

SQLComponentClassifier

SQLExecutionClassifier

filter

SQLConsistencyFilter

SQLExecutionFilter

generate

SQLByColumnGenerator

SQLGenerator

SQLVariationGenerator

Text2SQLCoTGenerator

Text2SQLPromptGenerator

Text2SQLQuestionGenerator

text_pt

eval

DebertaV3SampleEvaluator

FineWebEduSampleEvaluator

MetaSampleEvaluator

PairQualSampleEvaluator

PerplexitySampleEvaluator

QuratingSampleEvaluator

TextbookSampleEvaluator

filter

CCNetDeduplicateFilter

DebertaV3Filter

FineWebEduFilter

PairQualFilter

PerplexityFilter

QuratingFilter

TextbookFilter

generate

Phi4QAGenerator

text_sft

eval

AlpagasusSampleEvaluator

DeitaComplexitySampleEvaluator

DeitaQualitySampleEvaluator

InstagSampleEvaluator

RMSampleEvaluator

SuperfilteringSampleEvaluator

TreeinstructSampleEvaluator

filter

AlpagasusFilter

DeitaComplexityFilter

DeitaQualityFilter

InstagFilter

RMFilter

SuperfilteringFilter

TreeinstructFilter

generate

CondorGenerator

SFTGeneratorSeed

refine

CondorRefiner

此页内容

首页
API文档入口

API文档入口

22 字小于 1 分钟

2025-06-13

这里是API文档

贡献者: Sunnyhaze

下一页serving_main

Powered by VuePress & vuepress-theme-plume