mirror of
https://github.com/tiennm99/danswer-docs.git
synced 2026-08-06 20:24:32 +00:00
25 lines
1.3 KiB
Plaintext
25 lines
1.3 KiB
Plaintext
---
|
|
title: Q&A Transformers
|
|
description: 'Configure Danswer to use last generation Transformers trained for Q&A'
|
|
---
|
|
|
|
Refer to [Model Configs](https://docs.danswer.dev/gen_ai_configs/overview#model-configs) for how to set the
|
|
environment variables for your particular deployment.
|
|
|
|
## What are Q&A Transformers
|
|
Before the billion+ parameter Generative AI models became possible/popular, there was a class of models trained
|
|
specifically to answer questions based on provided context. These models are not general purpose and much weaker at
|
|
generalizing compared to the latest LLMs. They mostly function by extracting answers from the passage and presenting a
|
|
confidence score and are not capable of combining this with internal knowledge.
|
|
|
|
However, these models are able to be run on CPU for inference without further compression techniques.
|
|
|
|
Also, they are less capable of making up reasonable sounding answers that are actually hallucinations.
|
|
|
|
## Set Danswer to use Q&A Transformers
|
|
- INTERNAL_MODEL_VERSION=transformers
|
|
- GEN_AI_MODEL_VERSION=deepset/deberta-v3-large-squad2
|
|
|
|
Credits to [deepset.ai](https://huggingface.co/deepset/deberta-v3-large-squad2) for the `deberta-v3-large-squad2` model.
|
|
This model is provided under `cc-by-4` License and used in Danswer without alterations.
|