使用 Hugging Face 模型

使用 Hugging Face 模型之前，需要先设置环境变量

  1. import os
  2. os.environ['HUGGINGFACEHUB_API_TOKEN'] = ''

使用在线的 Hugging Face 模型

  1. from langchain import PromptTemplate, HuggingFaceHub, LLMChain
  3. template = """Question: {question}
  4. Answer: Let's think step by step."""
  6. prompt = PromptTemplate(template=template, input_variables=["question"])
  7. llm = HuggingFaceHub(repo_id="google/flan-t5-xl", model_kwargs={"temperature":0, "max_length":64})
  8. llm_chain = LLMChain(prompt=prompt, llm=llm)
  10. question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"
  11. print(llm_chain.run(question))

将 Hugging Face 模型直接拉到本地使用

  1. from langchain import PromptTemplate, LLMChain
  2. from langchain.llms import HuggingFacePipeline
  3. from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline, AutoModelForSeq2SeqLM
  5. model_id = 'google/flan-t5-large'
  6. tokenizer = AutoTokenizer.from_pretrained(model_id)
  7. model = AutoModelForSeq2SeqLM.from_pretrained(model_id)
  9. pipe = pipeline(
  10.     "text2text-generation",
  11.     model=model,
  12.     tokenizer=tokenizer,
  13.     max_length=100
  14. )
  16. local_llm = HuggingFacePipeline(pipeline=pipe)
  17. print(local_llm('What is the capital of France? '))
  20. template = """Question: {question} Answer: Let's think step by step."""
  21. prompt = PromptTemplate(template=template, input_variables=["question"])
  23. llm_chain = LLMChain(prompt=prompt, llm=local_llm)
  24. question = "What is the capital of England?"
  25. print(llm_chain.run(question))

将模型拉到本地使用的好处：

  • 训练模型
  • 可以使用本地的 GPU
  • 有些模型无法在 Hugging Face 运行