<aside> 📄

Paper Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation (QGen)

Authors Ji Ma, Ivan Korotkov, Yinfei Yang, Keith Hall, Ryan McDonald

Venue EACL 2021 · arXiv 2020.04

Links arXiv

Keywords Zero-shot Retrieval Synthetic Question Generation Domain Adaptation

</aside>

TL;DR

Train a question generator on general-domain QA data, apply it to passages from the target domain, and use the resulting synthetic (question, passage) pairs to train a neural retriever. Combined with BM25 in a simple hybrid, this approaches supervised accuracy in some domains — with no target-domain labels.

1. Problem Definition

Task: Unsupervised (zero-shot) learning of a passage retriever for an out-of-domain corpus, using a question generator.

• Neural retrievers need large supervised training sets to beat term-based methods like BM25, which work directly on raw corpora.
• Such labeled data rarely exists for new domains.

2. Contributions

1. A data augmentation approach for domain adaptation: train a generative model on naturally occurring QA pairs to synthesize questions given a passage, then apply it to the target domain.
2. A simple hybrid model that interpolates BM25 with the zero-shot neural model.

3. Method

1. Train a seq2seq question generator on general-domain (question, passage) pairs.
2. Generate questions for passages in the target-domain corpus → arbitrarily large but noisy, domain-specific relevance pairs.
3. Train a dual-encoder retriever on the synthetic pairs.
4. At retrieval time, interpolate the neural score with BM25.

4. Findings

• Synthetic question generation is an effective way to build neural retrievers without large training corpora.