---
title: "DistilBERT"
description: "This page explains how to use DistilBERT in LEAN trading algorithms."
canonical: https://www.quantconnect.com/docs/v2/writing-algorithms/machine-learning/hugging-face/popular-models/distilbert
markdown: https://www.quantconnect.com/docs/v2/writing-algorithms/machine-learning/hugging-face/popular-models/distilbert.md
language: python
source: https://github.com/QuantConnect/Documentation/tree/master/03%20Writing%20Algorithms/31%20Machine%20Learning/04%20Hugging%20Face/02%20Popular%20Models/04%20DistilBERT
modified: 2026-11-02
index: https://www.quantconnect.com/llms.txt
---

# DistilBERT

[Documentation](https://www.quantconnect.com/docs/v2.md) > [Writing Algorithms](https://www.quantconnect.com/docs/v2/writing-algorithms.md) > Machine Learning > Hugging Face > [Popular Models](https://www.quantconnect.com/docs/v2/writing-algorithms/machine-learning/hugging-face/popular-models.md) > DistilBERT

> Code examples are shown in Python. Append `?lang=csharp` to this URL for the C# version of this page.

## Introduction

This page explains how to use DistilBERT in LEAN trading algorithms. The [model repository](https://huggingface.co/distilbert/distilbert-base-cased-distilled-squad) provides the following description:

> The DistilBERT model was proposed in the blog post [Smaller, faster, cheaper, lighter: Introducing DistilBERT, adistilled version of BERT](https://medium.com/huggingface/distilbert-8cf3380435b5), and the paper [DistilBERT, adistilled version of BERT: smaller, faster, cheaper and lighter](https://arxiv.org/abs/1910.01108). DistilBERT is a small, fast, cheap and light Transformer model trained by distilling BERT base. It has 40% less parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performances as measured on the GLUE language understanding benchmark. This model is a fine-tune checkpoint of [DistilBERT-base-cased](https://huggingface.co/distilbert-base-cased), fine-tuned using (a second step of) knowledge distillation on [SQuAD v1.1](https://huggingface.co/datasets/squad).

## Use Cases

The DistilBERT model is a question answering model. The following use cases explain how you might utilize it in trading algorithms:

- Summarize the upcoming changes reported through [regulatory articles](https://www.quantconnect.com/docs/v2/writing-algorithms/datasets/regalytics/us-regulatory-alerts---financial-sector.md) in preparation of sentiment analysis.
- Extract relevant information from news feeds like [Tiingo](https://www.quantconnect.com/docs/v2/writing-algorithms/datasets/tiingo/tiingo-news-feed.md) and [Benzinga](https://www.quantconnect.com/docs/v2/writing-algorithms/datasets/benzinga/benzinga-news-feed.md) while reducing noise.
- Parse through [US SEC Filings](https://www.quantconnect.com/docs/v2/writing-algorithms/datasets/securities-and-exchange-commission/us-sec-filings.md) to find critical information about company performance, legal issues, or changes in management.

## Load Pre-Trained Model

Follow these steps to load the pre-trained DistilBERT model:

1. Import the pipeline.
   ```python
   from transformers import pipeline
   ```
2. Create a pipeline object with the DistilBERT model.
   In QuantConnect Cloud, the path is `distilbert / distilbert-base-cased-distilled-squad`.
   ```python
   question_answerer = pipeline(
       "question-answering",
       model='distilbert/distilbert-base-cased-distilled-squad',
       local_files_only=True
   )
   ```

## Answer Questions

Follow these steps to answer questions with DistilBERT:

1. [Load the model](https://www.quantconnect.com/docs/v2/writing-algorithms/machine-learning/hugging-face/popular-models/distilbert.md#03-Load-Pre-Trained-Model).
2. Define the context.
   ```python
   context = """
   Introduced on 2024-11-01.
   The legislation introduces a flat corporate income tax rate, reducing the current graduated system
   to 5.5% for taxable years starting from January 1, 2025, and further lowering it to 3.5% for years
   beginning January 1, 2026. It establishes a bonus depreciation deduction for qualified property and
   research expenditures, allowing immediate cost recovery. The bill also terminates various corporate
   tax credits, exemptions, and deductions, including those related to motion picture production,
   research and development, and historic structure rehabilitation, with most credits expiring after
   June 30, 2025. Additionally, it repeals provisions for certain tax programs and credits, such as the
   corporate tax apportionment program and various incentives for businesses. The changes aim to simplify
   corporate taxation and reduce the number of available tax credits, with the new provisions effective
   for income tax periods starting January 1, 2025, and franchise tax periods beginning January 1, 2026.
   The legislation will take effect upon the governor's signature or after the designated period for
   gubernatorial action.
   """
   ```
3. Create the question.
   ```python
   question = "What do these changes accomplish?"
   ```
4. Pass the question and context to the pipeline.
   ```python
   result = question_answerer(question=question, context=context)
   ```
   The following dictionary is an example result:
   ```python
   {
       'score': 0.19066356122493744,
       'start': 820,
       'end': 895,
       'answer': 'simplify corporate taxation and reduce the number of available tax credits'
   }
   ```