[2021.05] Few-Shot Conversational Dense Retrieval (ConvDR)

<aside> 📄

Paper Few-Shot Conversational Dense Retrieval (ConvDR)

Authors Shi Yu, Zhenghao Liu, Chenyan Xiong, Tao Feng, Zhiyuan Liu

Venue SIGIR 2021 · arXiv 2021.05

Links arXiv

Keywords Conversational Search Dense Retrieval Few-shot Teacher–Student

</aside>

TL;DR

ConvDR retrieves documents for multi-turn conversational queries purely with embedding dot products. To make it work with little conversational supervision, a student query encoder learns to map the raw conversational query to the same embedding that an ad hoc teacher retriever (ANCE) produces for the manually rewritten oracle query, while reusing the teacher's document embeddings.

1. Problem Definition

Task: Dense retrieval for conversational search.

Dense retrieval could address the query-understanding challenge in conversations (ellipsis, coreference) by matching in a learned embedding space.
But DR models need much more relevance-oriented supervision, and conversational search data is scarce and long-tailed.

2. Contribution

A teacher–student framework that gives ConvDR few-shot ability by learning from an existing ad hoc dense retriever.

3. Method

Teacher: ANCE, a state-of-the-art ad hoc dense retriever. Two assumptions:

The information need behind the manual oracle rewrite and the raw conversational query is the same, so their embeddings should be the same → train the student query encoder to mimic the teacher's embedding of the oracle query (MSE).
The meaningful information in a document is the same for ad hoc and conversational search, so document embeddings can be shared → inherit the teacher's document encoder and index unchanged.

Optionally, a ranking loss on the few available relevance labels is combined with the distillation loss.

4. Findings

On TREC CAsT and OR-QuAC, ConvDR is effective in both few-shot and fully-supervised settings.
Outperforms sparse-space conversational systems, matches the accuracy of oracle query rewrites, and is more efficient thanks to its simplicity.
Analysis: ConvDR captures informative context while ignoring unrelated context from earlier turns, so it holds up better as conversations grow longer.