# Question for QuantConnect Bot - Current Status

---

## 📋 Subject: Universe Selection Not Being Called in Live Mode

---

## 🔍 The Problem:

I have an algorithm that uses **Universe Selection** (coarse + fine) to load 900 stocks in Live Mode.

**The Issue:**
- `initialize()` succeeds ✅
- `add_universe()` is registered successfully ✅
- But `coarse_selection()` and `fine_selection()` are **NOT being called** ❌
- Result: **0 stocks** are loaded

---

## 📊 Current Logs:

### Latest Run (October 3, 2025, 00:41:18 UTC):

```
2025-10-03 00:41:18 : Paper Brokerage account base currency: USD
2025-10-03 00:41:18 : Accurate daily end-times now enabled by default.
2025-10-03 00:41:18 : [Initialize] Setting up Universe Selection...
2025-10-03 00:41:18 : [Initialize] Universe Selection registered successfully
2025-10-03 00:41:18 : [Initialize] Wait 5-20 minutes for fundamental data loading and Universe Selection execution...
2025-10-03 00:41:18 : [Initialize] [CoarseSelection] and [FineSelection] messages will appear when loading starts
2025-10-03 00:41:18 : [Initialize] Live Mode - Sending startup message...
2025-10-03 00:41:18 : [Initialize] Initialize completed successfully
2025-10-03 00:41:18 : [OnSecuritiesChanged] Time: 2025-10-02 20:41:12.765990
2025-10-03 00:41:18 : [OnSecuritiesChanged] Update #1 | Added: 1 | Removed: 0
2025-10-03 00:41:18 : [OnSecuritiesChanged] Total active stocks: 0
```

**Analysis:**
- ✅ `initialize()` succeeded
- ✅ `add_universe()` registered
- ✅ `on_securities_changed()` called (once - to add SPY)
- ❌ `coarse_selection()` **was NOT called**
- ❌ `fine_selection()` **was NOT called**
- ❌ Total stocks: **0** (SPY was skipped)

**Important Note:**
- Time: **00:41:18 UTC** = **20:41:18 EST** (after market close)
- Market closes at **16:00 EST**
- After-market ends at **20:00 EST**

---

## 💻 Current Code (V7.1.2):

### In `initialize()`:
```python
def initialize(self):
   """Initialize the algorithm"""
   try:
       # ... other settings ...
       
       # V7.1: Use Universe Selection for initial loading (Live Mode Compatible)
       self.log("[Initialize] Setting up Universe Selection...")
       self.universe_settings.resolution = Resolution.DAILY
       self.add_universe(self.coarse_selection, self.fine_selection)
       self.log("[Initialize] Universe Selection registered successfully")
       self.log("[Initialize] Wait 5-20 minutes for fundamental data loading and Universe Selection execution...")
       self.log("[Initialize] [CoarseSelection] and [FineSelection] messages will appear when loading starts")
       
       # Add SPY as benchmark
       self.spy_symbol = self.add_equity("SPY", Resolution.MINUTE).symbol
       
       # ... rest of code ...
       
   except Exception as e:
       self.error(f"[Initialize] Error: {e}")
```

---

### `coarse_selection()`:
```python
def coarse_selection(self, coarse):
   """
   V7.1: Coarse Universe Selection - should be called automatically in Live Mode
   
   Selects top UNIVERSE_SIZE stocks based on:
   - Price (MIN_PRICE - MAX_PRICE)
   - Has fundamental data
   - Dollar volume
   """
   try:
       coarse_list = list(coarse)
       
       self.log(f"[CoarseSelection] Starting Coarse Selection - Time: {self.time}")
       self.log(f"[CoarseSelection] Total available stocks: {len(coarse_list)}")
       
       # Filter by price and volume
       filtered = [
           x for x in coarse_list
           if config.MIN_PRICE <= x.price <= config.MAX_PRICE
           and x.has_fundamental_data
           and x.dollar_volume > 0
       ]
       
       self.log(f"[CoarseSelection] After filtering: {len(filtered)} stocks")
       
       # Sort by dollar volume
       sorted_by_dollar_volume = sorted(filtered, key=lambda x: x.dollar_volume, reverse=True)
       
       # Select top UNIVERSE_SIZE stocks
       selected = sorted_by_dollar_volume[:config.UNIVERSE_SIZE]
       
       self.log(f"[CoarseSelection] Selected {len(selected)} stocks for Fine Selection")
       
       return [x.symbol for x in selected]
       
   except Exception as e:
       self.error(f"[CoarseSelection] Error: {e}")
       return []
```

**Note:** This function **was NOT called** - no log messages from it in the logs!

---

### `fine_selection()`:
```python
def fine_selection(self, fine):
   """
   V7.1: Fine Universe Selection - should be called automatically after Coarse
   
   Applies additional filter based on:
   - Free float shares (MAX_FREE_FLOAT_SHARES)
   - Market cap
   """
   try:
       fine_list = list(fine)
       
       self.log(f"[FineSelection] Starting Fine Selection - Time: {self.time}")
       self.log(f"[FineSelection] Received {len(fine_list)} stocks from Coarse Selection")
       
       # Filter by free float shares
       filtered = []
       for x in fine_list:
           try:
               if x.equity_per_share_growth.three_months > 0:
                   if x.company_profile.shares_outstanding * x.company_profile.shares_float <= config.MAX_FREE_FLOAT_SHARES:
                       filtered.append(x)
           except:
               continue
       
       self.log(f"[FineSelection] After Free Float filter: {len(filtered)} stocks")
       self.log(f"[FineSelection] Fine Selection completed - Sending {len(filtered)} stocks to on_securities_changed")
       
       return [x.symbol for x in filtered]
       
   except Exception as e:
       self.error(f"[FineSelection] Error: {e}")
       return []
```

**Note:** This function **was NOT called** - no log messages from it in the logs!

---

### `on_securities_changed()`:
```python
def on_securities_changed(self, changes):
   """
   V7.1: Handle Universe changes - called automatically
   
   Called when:
   - Initial loading (after Fine Selection)
   - Weekly refresh
   - Manual SPY addition
   """
   try:
       added_count = len(changes.added_securities)
       removed_count = len(changes.removed_securities)
       
       self.universe_updates_count += 1
       
       self.log(f"[OnSecuritiesChanged] Time: {self.time}")
       self.log(
           f"[OnSecuritiesChanged] Update #{self.universe_updates_count} | "
           f"Added: {added_count} | Removed: {removed_count}"
       )
       
       # Add new stocks
       for added in changes.added_securities:
           symbol = added.symbol
           
           # Skip SPY (added manually in initialize)
           if symbol == self.spy_symbol:
               continue
           
           # Create SymbolData
           self.symbol_data_dict[symbol] = SymbolData(self, symbol)
           
           # Subscribe to Benzinga
           try:
               self.add_data(BenzingaNews, symbol)
               self.log(f"[OnSecuritiesChanged] Added {symbol.value} + Benzinga")
           except Exception as e:
               self.error(f"[OnSecuritiesChanged] Failed Benzinga subscription for {symbol}: {e}")
           
           # Calculate average volume
           self._calculate_average_volume(symbol)
       
       # Remove old stocks
       for removed in changes.removed_securities:
           symbol = removed.symbol
           
           # Never remove SPY
           if symbol == self.spy_symbol:
               continue
           
           if symbol in self.symbol_data_dict:
               # Clean up data
               self.symbol_data_dict[symbol].dispose()
               del self.symbol_data_dict[symbol]
               self.log(f"[OnSecuritiesChanged] Removed {symbol.value}")
       
       self.log(f"[OnSecuritiesChanged] Total active stocks: {len(self.symbol_data_dict)}")
       
       # Mark initial loading as complete
       if not self.bootstrap_completed and len(self.symbol_data_dict) > 0:
           self.bootstrap_completed = True
           self.log(f"[OnSecuritiesChanged] Initial loading complete! Loaded {len(self.symbol_data_dict)} stocks successfully")
           self.log(f"[OnSecuritiesChanged] System is now ready to monitor news and send alerts")
           
           # Send Telegram alert when loading completes (Live Mode only)
           if self.live_mode:
               try:
                   # Additional protection: ensure reasonable number of stocks (more than 10)
                   if len(self.symbol_data_dict) >= 10:
                       # ... send Telegram alert
                       pass
               except Exception as e:
                   self.error(f"[OnSecuritiesChanged] Error sending Telegram alert: {e}")
       
   except Exception as e:
       self.error(f"[OnSecuritiesChanged] Error: {e}")
```

**Note:** Called **only once** to add SPY, then never called again!

---

### Weekly Refresh:
```python
def _refresh_weekly_universe(self):
   """
   V7.1: Refresh Universe weekly (every Monday 4:05 PM EST)
   
   Re-triggers Universe Selection to update stock list
   """
   try:
       # Guard - ensure not running more than once per day
       if self.time.date() == self.last_universe_update:
           return
       
       self.log(f"[RefreshUniverse] Weekly refresh - {self.time}")
       
       # Universe Selection will be called automatically
       # because we registered coarse_selection and fine_selection in initialize()
       # QuantConnect will call them automatically based on universe_settings.resolution
       
       # But to ensure immediate refresh, re-register Universe
       self.add_universe(self.coarse_selection, self.fine_selection)
       
       self.log(f"[RefreshUniverse] Universe Selection triggered")
       
       # Update state
       self.last_universe_update = self.time.date()
       
   except Exception as e:
       self.error(f"[RefreshUniverse] Error: {e}")
```

**Note:** This function **has not been called yet** (haven't reached Monday 4:05 PM)

---

## 🗂️ Existing But Unused Functions:

### Two functions exist in the code but are **NOT being called**:

```python
def _load_target_symbols_from_objectstore(self):
   """Load target symbol list from ObjectStore"""
   try:
       if self.object_store.contains_key(config.OBJECT_STORE_KEY):
           symbols_str = self.object_store.read(config.OBJECT_STORE_KEY)
           self.target_symbols = symbols_str.split(',')
           self.log(f"[ObjectStore] Loaded {len(self.target_symbols)} stocks from ObjectStore")
           return True
       else:
           self.log("[ObjectStore] No saved list in ObjectStore")
           return False
   except Exception as e:
       self.error(f"[ObjectStore] Loading error: {e}")
       return False

def _add_manual_universe(self):
   """Add stocks manually from saved list"""
   try:
       if not hasattr(self, 'target_symbols') or not self.target_symbols:
           self.log("[ManualUniverse] No stock list to add")
           return
       
       self.log(f"[ManualUniverse] Starting to add {len(self.target_symbols)} stocks manually...")
       
       for ticker in self.target_symbols:
           try:
               symbol = self.add_equity(ticker, Resolution.MINUTE).symbol
               self.symbol_data_dict[symbol] = SymbolData(self, symbol)
               self.add_data(BenzingaNews, symbol)
               self._calculate_average_volume(symbol)
               self.log(f"[ManualUniverse] Added {ticker}")
           except Exception as e:
               self.error(f"[ManualUniverse] Failed to add {ticker}: {e}")
       
       self.log(f"[ManualUniverse] Successfully added {len(self.symbol_data_dict)} stocks")
       
   except Exception as e:
       self.error(f"[ManualUniverse] Error: {e}")
```

**Question:** Should these functions be called in `initialize()` for immediate loading?

---

## ❓ Questions:

### 1️⃣ Root Cause:
**Why are `coarse_selection()` and `fine_selection()` NOT being called?**

**Possibilities:**
- ❓ Is it because the market is closed (20:41 EST = after After-market)?
- ❓ Does `Resolution.DAILY` cause a delay until the next day?
- ❓ Is there a silent error in the code preventing Universe Selection from being called?
- ❓ Should I wait for market open (9:30 AM EST) or start of next day (00:00 EST)?

---

### 2️⃣ Confirmation Messages:
**How can I add log messages to verify Universe Selection status?**

**I want to know:**
- Is QuantConnect waiting for fundamental data?
- When exactly will `coarse_selection()` be called?
- Is there a silent error in LEAN Engine?
- How can I check Universe Selection status in Live Mode?

---

### 3️⃣ Proposed Solution - Immediate Loading:
**Should I use ObjectStore for immediate loading?**

**Proposed mechanism:**

```python
# In initialize():
self.log("[Initialize] Setting up Universe Selection...")
self.universe_settings.resolution = Resolution.DAILY
self.add_universe(self.coarse_selection, self.fine_selection)
self.log("[Initialize] Universe Selection registered successfully")

# Try loading from ObjectStore for immediate loading
if config.USE_OBJECT_STORE:
   self.log("[Initialize] Attempting to load from ObjectStore...")
   if self._load_target_symbols_from_objectstore():
       self.log(f"[Initialize] Loaded {len(self.target_symbols)} stocks from ObjectStore")
       self._add_manual_universe()
       self.bootstrap_completed = True
       self.log("[Initialize] Immediate loading complete!")
   else:
       self.log("[Initialize] No saved list - waiting for Universe Selection...")
else:
   self.log("[Initialize] Wait 5-20 minutes for fundamental data loading...")
```

**Then in `on_securities_changed()` after Universe Selection:**
```python
# Save new list to ObjectStore for next time
if config.USE_OBJECT_STORE and self.universe_updates_count >= 1:
   try:
       symbols_list = [symbol.value for symbol in self.symbol_data_dict.keys()]
       symbols_str = ','.join(symbols_list)
       self.object_store.save(config.OBJECT_STORE_KEY, symbols_str)
       self.log(f"[ObjectStore] Saved {len(symbols_list)} stocks to ObjectStore")
   except Exception as e:
       self.error(f"[ObjectStore] Save error: {e}")
```

**Questions:**
- Is this solution correct?
- Will it cause conflicts between ObjectStore and Universe Selection?
- How do I save the list to ObjectStore correctly?
- Should I save the list after every weekly update?

---

### 4️⃣ Resolution:
**Should I change `Resolution.DAILY` to `Resolution.MINUTE`?**

**Questions:**
- Does `Resolution.DAILY` cause a delay until the next day in Live Mode?
- Will `Resolution.MINUTE` speed up Universe Selection invocation?
- What is the recommended Resolution for Universe Selection in Live Mode?

---

### 5️⃣ Timing:
**When exactly will Universe Selection be called in Live Mode?**

**Scenarios:**
- If I start at **20:41 EST** (after After-market): when will it be called?
- If I start at **09:00 EST** (before market open): when will it be called?
- If I start at **13:00 EST** (during market hours): when will it be called?

**Should I wait for:**
- ❓ Market open (9:30 AM EST)?
- ❓ Start of next day (00:00 EST)?
- ❓ Fundamental data update (when?)?

---

## 🎯 Request for Evaluation:

**Please evaluate:**

1. **Code:** Are `add_universe()`, `coarse_selection()`, and `fine_selection()` written correctly?
2. **Cause:** Why is Universe Selection not being called? Is it just timing?
3. **Solution:** Is the proposed solution (ObjectStore + Universe Selection) correct?
4. **Resolution:** Is `Resolution.DAILY` appropriate or should I change it?
5. **Timing:** When exactly will Universe Selection be called in Live Mode?
6. **Log Messages:** How can I add messages to verify Universe Selection status?

---

## 📌 Current Status Summary:

### ✅ What Works:
- `initialize()` succeeds
- `add_universe()` is registered
- `on_securities_changed()` is called (to add SPY)
- Clear log messages

### ❌ What Doesn't Work:
- `coarse_selection()` **is NOT called**
- `fine_selection()` **is NOT called**
- **0 stocks** are loaded
- System "waits" but nothing happens

### 🤔 Unclear Cause:
- Is it timing (20:41 EST = after market close)?
- Is it `Resolution.DAILY`?
- Is there a silent error?

### 💡 Proposed Solution:
- Use ObjectStore for immediate loading (1-2 minutes)
- Universe Selection for weekly updates
- Save updated list to ObjectStore

---

## 📂 Files:

- `main.py`: 995 lines
- `config.py`: 231 lines
- `UNIVERSE_SIZE`: 900 stocks
- `Resolution`: DAILY

---

**Thank you for your help!** 🙏

**I need to know:**
1. Why is Universe Selection not being called?
2. When will it be called?
3. Is the proposed solution correct?
4. How can I verify Universe Selection status