Module 1 · AI
📖 4 min read · By ['[“HungHM15”]']

Assignment: Query Transformation#

Assignment Metadata#

Field

Description

Assignment Name

Query Transformation with HyDE and Decomposition

Course

RAG and Optimization

Project Name

query-transformation-rag

Estimated Time

90 minutes

Framework

Python 3.10+, LangChain, OpenAI API, Sentence-Transformers

Learning Objectives#

By completing this assignment, you will be able to:

  • Implement Hypothetical Document Embeddings (HyDE) for improved query-document matching

  • Build Query Decomposition pipelines to handle complex multi-part questions

  • Design effective prompts for LLM-based query transformation

  • Evaluate the impact of query transformation on retrieval quality

  • Apply these techniques to real-world RAG scenarios

Problem Description#

Users of your RAG system often submit queries that perform poorly in retrieval:

  1. Short, vague queries: “remote work” instead of “What are the company’s remote work policies?”

  2. Question-answer mismatch: Questions are interrogative while documents are declarative

  3. Complex multi-part queries: “Compare the battery life and camera quality of iPhone 15 and Samsung S24”

Your task is to implement HyDE and Query Decomposition to transform user queries before retrieval.

Technical Requirements#

Environment Setup#

  • Python 3.10 or higher

  • Required packages:

    • langchain >= 0.1.0

    • openai >= 1.0.0

    • sentence-transformers >= 2.2.0

    • chromadb >= 0.4.0

API Requirements#

  • OpenAI API key (or compatible LLM endpoint)

  • Embedding model (e.g., text-embedding-3-small or local all-MiniLM-L6-v2)

Tasks#

Task 1: Implement HyDE (35 points)#

  1. Build a HyDE pipeline with three stages:

    • Generate: Use LLM to create a hypothetical answer paragraph

    • Encode: Convert the hypothetical answer to an embedding

    • Retrieve: Search using the hypothetical answer embedding

  2. Design the generation prompt that:

    • Instructs the LLM to write in the style of your target documents

    • Includes domain-specific vocabulary guidance

    • Handles different query types (how-to, what-is, troubleshooting)

  3. Test with at least 5 query types:

    • Short queries (1-3 words)

    • Technical troubleshooting queries

    • Conceptual/definition queries

    • How-to procedure queries

    • Comparison queries

Task 2: Implement Query Decomposition (35 points)#

  1. Build a Query Decomposition pipeline:

    • Use LLM to analyze if a query contains multiple sub-questions

    • Generate independent sub-queries for parallel retrieval

    • Aggregate retrieved documents from all sub-queries

    • Synthesize a final answer using all gathered context

  2. Design the decomposition prompt that:

    • Identifies multiple intents within a single question

    • Generates standalone sub-queries (each understandable without context)

    • Preserves important constraints and filters from the original query

  3. Handle these decomposition scenarios:

    • Comparison queries (A vs B)

    • Aggregation queries (list all X with property Y)

    • Sequential queries (first do A, then what happens to B?)

Task 3: Comparative Evaluation (30 points)#

  1. Create a test set with 15 queries:

    • 5 queries suitable for HyDE (short/vague)

    • 5 queries suitable for Decomposition (complex/multi-part)

    • 5 baseline queries (clear, single-intent)

  2. Compare retrieval quality across methods:

Query ID

Query Type

Baseline Recall@5

HyDE Recall@5

Decomposition Recall@5

Q1

Short

Q2

Complex

…

…

  1. Analyze LLM outputs:

    • Document 3 example HyDE generations and their effectiveness

    • Document 3 example decompositions and sub-query quality

    • Identify failure cases and suggest improvements

Submission Requirements#

Required Deliverables#

  • Source code (Jupyter notebook or Python scripts)

  • README.md with setup and usage instructions

  • Prompt templates used for HyDE and Decomposition

  • Evaluation results table

  • Analysis document with examples and failure case analysis

Submission Checklist#

  • HyDE correctly generates hypothetical documents

  • Query Decomposition produces valid sub-queries

  • Both pipelines integrate with the retrieval system

  • Evaluation demonstrates improvement over baseline

  • Documentation includes prompt design rationale

Evaluation Criteria#

Criteria

Points

HyDE implementation correctness

20

HyDE prompt design quality

15

Decomposition implementation

20

Decomposition prompt design

15

Evaluation methodology

15

Analysis and examples quality

10

Code quality and documentation

5

Total

100

Hints#

  • For HyDE, the hypothetical answer doesn’t need to be factually correct—focus on style and vocabulary

  • Use few-shot examples in your prompts for more consistent LLM outputs

  • Consider caching LLM responses during development to save API costs

  • The companion notebook 03-langchain-hyde-demo.ipynb provides a starting point

  • For decomposition, test that each sub-query makes sense in isolation