Help mia

# Question About Best Practices for Batch Rotation and Indicator Management

Hello,

I'm working on an intraday trading strategy on QuantConnect that uses a **Batch Rotation System** to efficiently manage a large number of stocks. I've encountered an issue related to the timing of unsubscribing from stocks and indicator readiness, and I need to understand the best practices to solve this problem.

---

## Current Strategy Description

### System Architecture

The strategy uses a **Two-Phase System**:

1. **Phase 1**: Lightweight indicators for initial screening
  - Number of indicators: 3 (RDV, CMF, OBV)
  - Purpose: Quick scan to detect early accumulation signals
  - Number of stocks: 100 stocks per batch

2. **Phase 2**: Full indicators for detailed analysis
  - Number of indicators: 6 (RDV, CMF, OBV, VWAP, BOP, Volume SMA)
  - Purpose: Detailed analysis of promising stocks
  - Promotion: Stocks showing strong signals are promoted from Phase 1 to Phase 2

### Batch Rotation System

```python
# Current settings
self.BATCH_SIZE = 100  # Number of stocks per batch
self.ROTATION_INTERVAL_MINUTES = 5  # Rotate every 5 minutes
self.PROMOTION_THRESHOLD = 2  # Number of indicators required for promotion
self.PROMOTION_BARS = 3  # Number of consecutive bars required
```

**How it works:**
1. Strategy starts with a batch of 100 stocks
2. Creates Phase 1 indicators for each stock (RDV, CMF, OBV)
3. Uses `warm_up_indicator()` to warm up the indicators
4. Every 5 minutes, it:
  - Unsubscribes from the current batch
  - Loads a new batch of 100 stocks
  - Keeps promoted stocks (from Phase 1 to Phase 2)

---

## The Problem Discovered

When reviewing the log messages, I noticed the following sequence:

### Time: 21:50 - Indicator Creation
```
[IndicatorCreation] MPB | RDV created (period=2), CMF created (period=3), OBV created | WarmUp completed
[IndicatorCreation] KFRC | RDV created (period=2), CMF created (period=3), OBV created | WarmUp completed
```

### Time: 21:51-22:00 - Indicators Still Not Ready
```
[IndicatorReady-Diag] MPB | InDict: RDV=True CMF=True OBV=True | 
                      Ready: RDV=False CMF=False OBV=False | 
                      Samples: RDV=0 CMF=0 OBV=0

[IndicatorReady-Diag] LEGH | InDict: RDV=True CMF=True OBV=True | 
                       Ready: RDV=False CMF=False OBV=False | 
                       Samples: RDV=0 CMF=0 OBV=0
```

### Time: 22:00 - Unsubscribe (After Only 5 Minutes!)
```
[DIAG] rotate_batch START
[Extended-Hours-Unsub] KFRC | Removed extended hours subscription
[Extended-Hours-Unsub] LPG | Removed extended hours subscription
[Extended-Hours-Unsub] HVT | Removed extended hours subscription
[Extended-Hours-Unsub] IGIC | Removed extended hours subscription
```

**Key Observation:**
- Indicators are created and `warm_up_indicator()` is called
- Despite this, indicators show `Samples: 0` even after several minutes
- Stocks are unsubscribed before indicators receive any data
- Result: We get no signals from these stocks

---

## Current Code

### Indicator Creation Function
```python
def create_phase1_indicators(self, symbol):
   """Create Phase 1 indicators (3 lightweight indicators)"""
   try:
       self.phase1_rdv_indicators[symbol] = self.rdv(symbol, self.RDV_PERIOD, Resolution.MINUTE)
       self.phase1_cmf_indicators[symbol] = self.cmf(symbol, self.CMF_PERIOD, Resolution.MINUTE)
       self.phase1_obv_indicators[symbol] = self.obv(symbol, Resolution.MINUTE)
       
       # Warm up indicators
       self.warm_up_indicator(symbol, self.phase1_rdv_indicators[symbol], Resolution.MINUTE)
       self.warm_up_indicator(symbol, self.phase1_cmf_indicators[symbol], Resolution.MINUTE)
       self.warm_up_indicator(symbol, self.phase1_obv_indicators[symbol], Resolution.MINUTE)
       
       if symbol not in self._indicator_creation_logged:
           self._indicator_creation_logged.add(symbol)
           self.debug(f"[IndicatorCreation] {symbol.value} | " +
                     f"RDV created (period={self.RDV_PERIOD}), " +
                     f"CMF created (period={self.CMF_PERIOD}), " +
                     f"OBV created | WarmUp completed")
   except Exception as e:
       self.error(f"[ERROR] Error in create_phase1_indicators for {symbol.value}: {str(e)}")
```

### Readiness Check Function
```python
def are_indicators_ready(self, symbol):
   """Check if all indicators are ready"""
   try:
       in_rdv = symbol in self.phase1_rdv_indicators
       in_cmf = symbol in self.phase1_cmf_indicators
       in_obv = symbol in self.phase1_obv_indicators
       
       rdv_ready = self.phase1_rdv_indicators[symbol].is_ready if in_rdv else False
       cmf_ready = self.phase1_cmf_indicators[symbol].is_ready if in_cmf else False
       obv_ready = self.phase1_obv_indicators[symbol].is_ready if in_obv else False
       
       # Diagnostic messages
       if self._phase1_log_counter.get(symbol, 0) % 100 == 1:
           rdv_samples = self.phase1_rdv_indicators[symbol].samples if in_rdv else 0
           cmf_samples = self.phase1_cmf_indicators[symbol].samples if in_cmf else 0
           obv_samples = self.phase1_obv_indicators[symbol].samples if in_obv else 0
           
           self.debug(f"[IndicatorReady-Diag] {symbol.value} | " +
                     f"InDict: RDV={in_rdv} CMF={in_cmf} OBV={in_obv} | " +
                     f"Ready: RDV={rdv_ready} CMF={cmf_ready} OBV={obv_ready} | " +
                     f"Samples: RDV={rdv_samples} CMF={cmf_samples} OBV={obv_samples}")
       
       return all([in_rdv and rdv_ready, in_cmf and cmf_ready, in_obv and obv_ready])
   except Exception as e:
       return False
```

### Rotation Function
```python
def rotate_batch(self):
   """Rotate the current batch"""
   try:
       self.debug("[DIAG] rotate_batch START")
       
       # Save promoted stocks
       promoted_symbols = list(self.promoted_symbols)
       
       # Remove old batch
       old_batch_symbols = [s for s in self.active_symbols if s not in promoted_symbols]
       
       for symbol in old_batch_symbols:
           self.cleanup_phase1_indicators(symbol)
           try:
               if self.securities.contains_key(symbol):
                   self.remove_security(symbol)
                   self.debug(f"[Extended-Hours-Unsub] {symbol.value} | Removed extended hours subscription")
           except Exception as unsub_ex:
               self.error(f"[ERROR] Failed to unsubscribe from {symbol.value}: {str(unsub_ex)}")
       
       # Load new batch
       start_idx = self.current_batch_index * self.BATCH_SIZE
       end_idx = min(start_idx + self.BATCH_SIZE, len(self.universe_stocks))
       new_batch = self.universe_stocks[start_idx:end_idx]
       
       # Add new stocks
       self.active_symbols = promoted_symbols + new_batch
       
       # Create indicators for new stocks
       for symbol in new_batch:
           if symbol not in self.phase1_rdv_indicators:
               self.add_equity(symbol, Resolution.MINUTE, extendedMarketHours=True)
               self.create_phase1_indicators(symbol)
       
       # Update batch
       self.current_batch_index = (self.current_batch_index + 1) % self.total_batches
       self.rotation_count += 1
       
       self.debug("[DIAG] rotate_batch completed successfully")
   except Exception as e:
       self.error(f"[ERROR] Error in rotate_batch: {str(e)}")
```

---

## Questions

### 1. Why aren't indicators receiving data despite using `warm_up_indicator()`?
- Does `warm_up_indicator()` only work with historical data?
- Does a stock need additional time after `add_equity()` before indicators start receiving data?
- Is there a delay between indicator creation and receiving first live data?

### 2. What are the best practices for managing indicator lifecycle in a batch rotation system?
- How do I ensure indicators are ready before unsubscribing from a stock?
- What is the appropriate time period between indicator creation and unsubscription?
- Should I wait until `is_ready == True` before rotating?

### 3. How should I handle low-liquidity stocks?
- Some stocks don't trade every minute, how do I ensure indicators get enough data?
- Should I use a lower resolution (like 5 minutes) for low-liquidity stocks?
- How do I distinguish between a low-liquidity stock and a stock whose data hasn't loaded yet?

### 4. What is the best way to implement a batch rotation system with indicators?
- Should I keep stocks whose indicators haven't completed when rotating?
- How do I balance between covering more stocks and signal quality?
- Is there a way to speed up the indicator warm-up process for live data?

### 5. Is there a better alternative to `warm_up_indicator()` in the current context?
- Should I use `set_warm_up()` at the algorithm level instead of individual indicators?
- Can I manually load historical data and update indicators with it before starting?
- What is the most efficient way to ensure indicator readiness as quickly as possible?

---

## Additional Information

- **Strategy Type**: Intraday trading
- **Universe Size**: 200 stocks
- **Batch Size**: 100 stocks
- **Rotation Period**: 5 minutes
- **Resolution**: Minute
- **Extended Market Hours**: Enabled
- **Indicators Used**:
 - RDV (period=2)
 - CMF (period=3)
 - OBV
 - VWAP (period=5)
 - BOP
 - Volume SMA (period=10)

---

## What I'm Looking For

I need to understand the **best practices** in QuantConnect for:
1. Ensuring indicator readiness before making trading decisions
2. Managing indicator lifecycle in a batch rotation system
3. Handling low-liquidity stocks
4. Optimizing the timing of adding and removing stocks from the strategy
5. Achieving a balance between stock coverage and signal quality

Thank you for your help!