<aside> 📄

Paper Latent Retrieval for Weakly Supervised Open Domain Question Answering (ORQA)

Authors Kenton Lee, Ming-Wei Chang, Kristina Toutanova

Venue ACL 2019 · arXiv 2019.06

Links arXiv

Keywords Open-Retrieval QA Inverse Cloze Task Unsupervised Pre-training

</aside>

TL;DR

ORQA is the first system to jointly learn the retriever and reader from question–answer pairs only, treating the evidence passage from all of Wikipedia as a latent variable. To make this learnable, the retriever is first pre-trained with the unsupervised Inverse Cloze Task (ICT).

1. Problem Definition

Task: Open-retrieval question answering.

• Prior open-domain QA either assumed gold evidence supervision or relied on a black-box IR system (e.g., BM25) to fetch candidates.
• Both are suboptimal: gold evidence is not always available, and QA is fundamentally different from IR — term matching does not capture what a question is really asking for.

2. Contributions

• Show for the first time that the retriever and reader can be learned jointly from question–answer string pairs, without any IR system.
• Evidence retrieval over all of Wikipedia is treated as a latent variable.

3. Method

3.1 Inverse Cloze Task (ICT) pre-training

Learning latent retrieval from scratch is impractical, so the retriever is pre-trained with ICT:

• Given a sentence, predict its context — the sentence acts as a pseudo-question and its surrounding passage as the pseudo-evidence.
• This requires learning more than word-matching features.
• At the same time, the authors don't want to discourage word matching entirely, so the sentence is removed from its context in only 90% of examples (in 10% it stays, rewarding lexical overlap).

This enables:

• Pre-encoding all Wikipedia evidence blocks, making fast top-k retrieval possible during training.