Extractor

pipeline pipeline

The Extractor pipeline joins a prompt, context data store and generative model together to extract knowledge.

The data store can be an embeddings database or a similarity instance with associated input text. The generative model can be a prompt-driven large language model (LLM), an extractive question-answering model or a custom pipeline. This is known as prompt-driven search or retrieval augmented generation (RAG).

Example

The following shows a simple example using this pipeline.

  1. from txtai.embeddings import Embeddings
  2. from txtai.pipeline import Extractor
  3. # LLM prompt
  4. def prompt(question):
  5. return f"""
  6. Answer the following question using the provided context.
  7. Question:
  8. {question}
  9. Context:
  10. """
  11. # Input data
  12. data = [
  13. "US tops 5 million confirmed virus cases",
  14. "Canada's last fully intact ice shelf has suddenly collapsed, " +
  15. "forming a Manhattan-sized iceberg",
  16. "Beijing mobilises invasion craft along coast as Taiwan tensions escalate",
  17. "The National Park Service warns against sacrificing slower friends " +
  18. "in a bear attack",
  19. "Maine man wins $1M from $25 lottery ticket",
  20. "Make huge profits without work, earn up to $100,000 a day"
  21. ]
  22. # Build embeddings index
  23. embeddings = Embeddings({"content": True})
  24. embeddings.index([(uid, text, None) for uid, text in enumerate(data)])
  25. # Create and run pipeline
  26. extractor = Extractor(embeddings, "google/flan-t5-base")
  27. extractor([{"query": "What was won?", "question": prompt("What was won?")}])

See the links below for more detailed examples.

NotebookDescription
Prompt-driven search with LLMsEmbeddings-guided and Prompt-driven search with Large Language Models (LLMs)Open In Colab
Prompt templates and task chainsBuild model prompts and connect tasks together with workflowsOpen In Colab
Extractive QA with txtaiIntroduction to extractive question-answering with txtaiOpen In Colab
Extractive QA with ElasticsearchRun extractive question-answering queries with ElasticsearchOpen In Colab
Extractive QA to build structured dataBuild structured datasets using extractive question-answeringOpen In Colab

Configuration-driven example

Pipelines are run with Python or configuration. Pipelines can be instantiated in configuration using the lower case name of the pipeline. Configuration-driven pipelines are run with workflows or the API.

config.yml

  1. # Allow documents to be indexed
  2. writable: True
  3. # Content is required for extractor pipeline
  4. embeddings:
  5. content: True
  6. extractor:
  7. path: google/flan-t5-base
  8. workflow:
  9. search:
  10. tasks:
  11. - task: extractor
  12. template: |
  13. Answer the following question using the provided context.
  14. Question:
  15. {text}
  16. Context:
  17. action: extractor

Run with Workflows

Built in tasks make using the extractor pipeline easier.

  1. from txtai.app import Application
  2. # Create and run pipeline with workflow
  3. app = Application("config.yml")
  4. app.add([
  5. "US tops 5 million confirmed virus cases",
  6. "Canada's last fully intact ice shelf has suddenly collapsed, " +
  7. "forming a Manhattan-sized iceberg",
  8. "Beijing mobilises invasion craft along coast as Taiwan tensions escalate",
  9. "The National Park Service warns against sacrificing slower friends " +
  10. "in a bear attack",
  11. "Maine man wins $1M from $25 lottery ticket",
  12. "Make huge profits without work, earn up to $100,000 a day"
  13. ])
  14. app.index()
  15. list(app.workflow("search", ["What was won?"]))

Run with API

  1. CONFIG=config.yml uvicorn "txtai.api:app" &
  2. curl \
  3. -X POST "http://localhost:8000/workflow" \
  4. -H "Content-Type: application/json" \
  5. -d '{"name": "search", "elements": ["What was won"]}'

Methods

Python documentation for the pipeline.

Builds a new extractor.

Parameters:

NameTypeDescriptionDefault
similarity

similarity instance (embeddings or similarity pipeline)

required
path

path to model, supports Questions, Generator, Sequences or custom pipeline

required
quantize

True if model should be quantized before inference, False otherwise.

False
gpu

if gpu inference should be used (only works if GPUs are available)

True
model

optional existing pipeline model to wrap

None
tokenizer

Tokenizer class

None
minscore

minimum score to include context match, defaults to None

None
mintokens

minimum number of tokens to include context match, defaults to None

None
context

topn context matches to include, defaults to 3

None
task

model task (language-generation, sequence-sequence or question-answering), defaults to auto-detect

None
output

output format, ‘default’ returns (name, answer), ‘flatten’ returns answers and ‘reference’ returns (name, answer, reference)

‘default’
kwargs

additional keyword arguments to pass to pipeline model

{}

Source code in txtai/pipeline/text/extractor.py

  1. def __init__(
  2. self,
  3. similarity,
  4. path,
  5. quantize=False,
  6. gpu=True,
  7. model=None,
  8. tokenizer=None,
  9. minscore=None,
  10. mintokens=None,
  11. context=None,
  12. task=None,
  13. output="default",
  14. **kwargs,
  15. ):
  16. """
  17. Builds a new extractor.
  18. Args:
  19. similarity: similarity instance (embeddings or similarity pipeline)
  20. path: path to model, supports Questions, Generator, Sequences or custom pipeline
  21. quantize: True if model should be quantized before inference, False otherwise.
  22. gpu: if gpu inference should be used (only works if GPUs are available)
  23. model: optional existing pipeline model to wrap
  24. tokenizer: Tokenizer class
  25. minscore: minimum score to include context match, defaults to None
  26. mintokens: minimum number of tokens to include context match, defaults to None
  27. context: topn context matches to include, defaults to 3
  28. task: model task (language-generation, sequence-sequence or question-answering), defaults to auto-detect
  29. output: output format, 'default' returns (name, answer), 'flatten' returns answers and 'reference' returns (name, answer, reference)
  30. kwargs: additional keyword arguments to pass to pipeline model
  31. """
  32. # Similarity instance
  33. self.similarity = similarity
  34. # Question-Answer model. Can be prompt-driven LLM or extractive qa
  35. self.model = self.load(path, quantize, gpu, model, task, **kwargs)
  36. # Tokenizer class use default method if not set
  37. self.tokenizer = tokenizer if tokenizer else Tokenizer() if hasattr(self.similarity, "scoring") and self.similarity.scoring else None
  38. # Minimum score to include context match
  39. self.minscore = minscore if minscore is not None else 0.0
  40. # Minimum number of tokens to include context match
  41. self.mintokens = mintokens if mintokens is not None else 0.0
  42. # Top n context matches to include for context
  43. self.context = context if context else 3
  44. # Output format
  45. self.output = output

Finds answers to input questions. This method runs queries to find the top n best matches and uses that as the context. A model is then run against the context for each input question, with the answer returned.

Parameters:

NameTypeDescriptionDefault
queue

input question queue (name, query, question, snippet), can be list of tuples or dicts

required
texts

optional list of text for context, otherwise runs embeddings search

None

Returns:

TypeDescription

list of answers matching input format (tuple or dict) containing fields as specified by output format

Source code in txtai/pipeline/text/extractor.py

  1. def __call__(self, queue, texts=None):
  2. """
  3. Finds answers to input questions. This method runs queries to find the top n best matches and uses that as the context.
  4. A model is then run against the context for each input question, with the answer returned.
  5. Args:
  6. queue: input question queue (name, query, question, snippet), can be list of tuples or dicts
  7. texts: optional list of text for context, otherwise runs embeddings search
  8. Returns:
  9. list of answers matching input format (tuple or dict) containing fields as specified by output format
  10. """
  11. # Save original queue format
  12. inputs = queue
  13. # Convert dictionary inputs to tuples
  14. if queue and isinstance(queue[0], dict):
  15. # Convert dict to tuple
  16. queue = [tuple(row.get(x) for x in ["name", "query", "question", "snippet"]) for row in queue]
  17. # Rank texts by similarity for each query
  18. results = self.query([query for _, query, _, _ in queue], texts)
  19. # Build question-context pairs
  20. names, queries, questions, contexts, topns, snippets = [], [], [], [], [], []
  21. for x, (name, query, question, snippet) in enumerate(queue):
  22. # Get top n best matching segments
  23. topn = sorted(results[x], key=lambda y: y[2], reverse=True)[: self.context]
  24. # Generate context using ordering from texts, if available, otherwise order by score
  25. context = " ".join(text for _, text, _ in (sorted(topn, key=lambda y: y[0]) if texts else topn))
  26. names.append(name)
  27. queries.append(query)
  28. questions.append(question)
  29. contexts.append(context)
  30. topns.append(topn)
  31. snippets.append(snippet)
  32. # Run pipeline and return answers
  33. answers = self.answers(names, questions, contexts, [[text for _, text, _ in topn] for topn in topns], snippets)
  34. # Apply output formatting to answers and return
  35. return self.apply(inputs, queries, answers, topns)