Question answering

What it is

Classic extractive QA is narrower than it sounds. The model does not compose an answer; it predicts a start position and an end position within a passage you give it, and returns that substring. Everything it says came verbatim from the source.

That constraint is a feature in a bank. The answer is always traceable to a location in a document, which is exactly what an auditor wants and exactly what a generative model cannot promise.

Model families 3

Sub-categories within this task. The distinction that matters is not which is best, but which fits your constraints.

Extractive span models

66M–350M.

Use when: The answer exists verbatim in a passage and you want provenance. Cheap, fast, CPU-servable.
Avoid when: The answer requires combining several places, or rewording.
Examples: distilbert-base-cased-distilled-squaddeepset/roberta-base-squad2

Models trained with unanswerable examples

125M–350M.

Use when: Always preferable in production. SQuAD 2.0-trained models can decline to answer; SQuAD 1.1 models always return something, however wrong.
Avoid when: Never really — prefer these as the default.
Examples: deepset/roberta-base-squad2deepset/deberta-v3-large-squad2

Retrieval-augmented generation

Retriever plus a 7B+ generator.

Use when: Answers span documents or need rewording, and the corpus is too large to pass in whole.
Avoid when: You need the answer to be a literal quote, or you cannot afford the generator.
Examples: Embedding model + reranker + instruct LLM

In a bank 3

The third line in each is the part that makes it harder than the textbook version.

Contact centre agent assist

An agent types a customer question and gets the answer with the policy clause it came from.

The wrinkleThe citation is as important as the answer. Agents need to read the source before repeating it, and the retrieved span is what makes that possible.

Product terms lookup

Answer questions against terms and conditions across a product catalogue.

The wrinkleTerms vary by product variant, region and effective date. Retrieving from the wrong variant produces a confidently wrong answer — filtering metadata matters more than the model.

Regulatory obligation queries

Find what a given rule requires of a specific business line.

The wrinkleExtractive answers are safer here precisely because they cannot paraphrase a legal obligation into something subtly different.

Code 2

Named functions and the arguments that actually change the outcome. Comments flag the traps.

Extractive QA with provenance

handle_impossible_answer is the argument that separates a demo from something you would deploy.

from transformers import pipeline

qa = pipeline("question-answering", model="deepset/roberta-base-squad2")

context = ("Disputed transactions must be raised within 60 days of the "
           "statement date. Overdraft fees are waived for balances under "
           "50 GBP for up to 3 days.")

r = qa(question="How long do I have to dispute a transaction?",
       context=context,
       handle_impossible_answer=True)   # allows an empty answer

print(r["answer"], r["score"])
print("source span:", context[r["start"]:r["end"]])   # the audit trail

# A question the context does not answer - a SQuAD 1.1 model would
# confidently return nonsense here; this one returns "" with a low score.
print(qa(question="What is the interest rate on savings?",
         context=context, handle_impossible_answer=True))

QA over a corpus: retrieve, then extract

Extractive models take one passage. Getting the right passage is a separate problem, and usually the harder one.

from sentence_transformers import SentenceTransformer, util
from transformers import pipeline

bi = SentenceTransformer("BAAI/bge-large-en-v1.5")
qa = pipeline("question-answering", model="deepset/roberta-base-squad2")

passage_emb = bi.encode(passages, normalize_embeddings=True,
                        convert_to_tensor=True)

def answer(question, top_k=3, floor=0.20):
    q = bi.encode(question, normalize_embeddings=True, convert_to_tensor=True)
    hits = util.semantic_search(q, passage_emb, top_k=top_k)[0]

    best = None
    for h in hits:
        p = passages[h["corpus_id"]]
        r = qa(question=question, context=p, handle_impossible_answer=True)
        r["passage_id"] = h["corpus_id"]
        if r["answer"] and (best is None or r["score"] > best["score"]):
            best = r

    # Declining is a valid outcome. Systems that always answer are worse.
    if not best or best["score"] < floor:
        return {"answer": None,
                "message": "Not answerable from available documents."}
    return best

What catches people out 3

Always use a SQuAD 2.0 model

Models trained only on SQuAD 1.1 assume every question is answerable and will return a plausible-looking span for anything. In production, 'I don't know' is a feature.
Scores are comparable within a passage, not across passages

A 0.7 on one document and a 0.7 on another do not mean the same thing. To pick between passages, use the retriever's ranking as the primary signal and the QA score as a floor.
Long contexts get windowed

The pipeline splits long passages into overlapping strides and takes the best span. Answers spanning a boundary can be lost. Chunk deliberately rather than relying on the default.