# Critical Issue Report: Stocks Not Being Added to Universe
## Problem Summary
**Issue:** The algorithm is **not adding any stocks** to the monitoring universe, despite successfully loading 585 stock symbols from ObjectStore.
**Symptom:** The system shows `[ObjectStore] تم تحميل 585 سهم` (Loaded 585 stocks), but immediately after says `[Initialize] No saved list - will wait for Universe Selection`.
**Impact:** The algorithm remains in a waiting state and never monitors any stocks for news.
---
## Critical Evidence from Logs
### Successful Run (September 30, 2025):
```
2025-09-30 13:50:49 تم إعداد الخوارزمية بنجاح
2025-09-30 13:50:49 تغييرات في الكون: Added: A, AA, AAL, AAME, AAN, AAP, AAPL... (49 stocks)
2025-09-30 13:50:49 إجمالي الأسهم النشطة: 49
```
✅ **Result:** 49 stocks added successfully
---
### Failed Run (October 3, 2025):
```
2025-10-03 20:00:36 [Initialize] ObjectStore enabled - attempting immediate loading...
2025-10-03 20:00:36 [ObjectStore] تم تحميل 585 سهم ✅
2025-10-03 20:00:36 [Initialize] No saved list - will wait for Universe Selection ❌
2025-10-03 20:00:36 [Initialize] انتظر 5-20 دقيقة لتحميل البيانات الأساسية...
2025-10-03 20:00:36 [CoarseSelection] إجمالي الأسهم المتاحة: 10,000
2025-10-03 20:00:36 [CoarseSelection] بعد فلترة السعر: 900 سهم ✅
2025-10-03 20:00:36 [FineSelection] استلام 900 سهم من Coarse Selection ✅
2025-10-03 20:00:36 [FineSelection] بعد فلتر Market Cap + Shares: 0 سهم ❌
2025-10-03 20:00:36 [FineSelection] انتهى Fine Selection - إرسال 0 سهم
2025-10-03 20:00:36 OnSecuritiesChanged at 2025-10-03 16:00:18: Added 0, Removed 0 ❌
2025-10-03 20:00:36 [OnSecuritiesChanged] إجمالي الأسهم النشطة: 0 ❌
```
❌ **Result:** 0 stocks added
---
## The Mystery: 585 Stocks Loaded But Not Used
### What We Know:
1. **ObjectStore contains 585 stocks** ✅
2. **System successfully reads them** ✅
3. **System logs "تم تحميل 585 سهم"** ✅
4. **But then ignores them completely** ❌
5. **And waits for Universe Selection instead** ❌
### The Code Behavior:
```python
if self._load_target_symbols_from_objectstore(): # This condition
self.log(f"[Initialize] Loaded {len(self.target_symbols)} stocks from ObjectStore")
self._add_manual_universe()
self.bootstrap_completed = True
else: # But code enters here!
self.log("[Initialize] No saved list - will wait for Universe Selection")
```
**Question:** Why does the code enter the `else` block when ObjectStore clearly loaded 585 stocks?
---
## Suspected Root Causes
### Hypothesis 1: Missing Return Statement ⭐ (Most Likely)
**Problem:** The `_load_target_symbols_from_objectstore()` function doesn't return `True` on success.
**Evidence:**
```python
def _load_target_symbols_from_objectstore(self):
try:
if self.object_store.contains_key(config.OBJECT_STORE_KEY):
symbols_str = self.object_store.read(config.OBJECT_STORE_KEY)
self.target_symbols = symbols_str.split(',')
self.log(f"[ObjectStore] تم تحميل {len(self.target_symbols)} سهم")
# ❌ NO RETURN STATEMENT HERE!
else:
self.log("[ObjectStore] لا توجد قائمة محفوظة - استخدام Universe Selection")
# ❌ NO RETURN STATEMENT HERE!
except Exception as e:
self.error(f"[ObjectStore] خطأ في التحميل: {e}")
# ❌ NO RETURN STATEMENT HERE!
```
**Result:** Function returns `None` in all cases, which evaluates to `False` in the `if` statement.
**Impact:** System thinks loading failed even though it succeeded.
---
### Hypothesis 2: Fine Selection Rejecting All Stocks
**Problem:** `fine_selection()` receives 900 stocks but rejects all of them.
**Evidence:**
```
[FineSelection] استلام 900 سهم من Coarse Selection ✅
[FineSelection] بعد فلتر Market Cap + Shares: 0 سهم ❌
```
**Possible Reasons:**
1. `MAX_SHARES_OUTSTANDING = 30_000_000` is too low
2. Fundamental data is missing or `None`
3. `_get_shares_outstanding()` is returning `None` for all stocks
4. Data structure changed in QuantConnect
---
### Hypothesis 3: ObjectStore Data Corruption
**Problem:** The saved data format is incompatible or corrupted.
**Evidence:**
- System loads 585 stocks (seems fine)
- But maybe the format is wrong: `"AAPL,MSFT,GOOGL,..."` vs `["AAPL", "MSFT", ...]`
---
### Hypothesis 4: Timing Issue
**Problem:** ObjectStore loading happens but `target_symbols` is cleared before use.
**Evidence:**
- Logs show loading succeeded
- But maybe something clears `self.target_symbols` before `_add_manual_universe()` is called
---
## Complete Function Code for Analysis
### Function 1: `_load_target_symbols_from_objectstore()`
```python
def _load_target_symbols_from_objectstore(self):
""" جديد: تحميل قائمة الأسهم المستهدفة من ObjectStore"""
try:
if self.object_store.contains_key(config.OBJECT_STORE_KEY):
symbols_str = self.object_store.read(config.OBJECT_STORE_KEY)
self.target_symbols = symbols_str.split(',')
self.log(f"[ObjectStore] تم تحميل {len(self.target_symbols)} سهم")
else:
self.log("[ObjectStore] لا توجد قائمة محفوظة - استخدام Universe Selection")
except Exception as e:
self.error(f"[ObjectStore] خطأ في التحميل: {e}")
```
**Issues:**
1. ❌ No return statement on success
2. ❌ No return statement on failure
3. ❌ No validation of loaded data
4. ❌ No logging of actual symbols loaded
---
### Function 2: `initialize()` - ObjectStore Loading Section
```python
def initialize(self):
try:
# ... (other initialization code)
# V7.2: محاولة التحميل الفوري من ObjectStore (توصية البوت)
if config.USE_OBJECT_STORE:
self.log("[Initialize] ObjectStore enabled - attempting immediate loading...")
if self._load_target_symbols_from_objectstore():
self.log(f"[Initialize] Loaded {len(self.target_symbols)} stocks from ObjectStore")
self._add_manual_universe()
self.bootstrap_completed = True
self.log("[Initialize] Immediate loading complete! System ready.")
else:
self.log("[Initialize] No saved list - will wait for Universe Selection (00:00-00:30 AM)")
else:
self.log("[Initialize] ObjectStore disabled - will wait for Universe Selection (00:00-00:30 AM)")
# ... (rest of initialization)
except Exception as e:
self.error(f"[Initialize] خطأ حرج: {e}")
raise
```
**Issues:**
1. ❌ Relies on return value from `_load_target_symbols_from_objectstore()` which doesn't return anything
2. ❌ No validation of `self.target_symbols` before calling `_add_manual_universe()`
3. ❌ No error handling if `_add_manual_universe()` fails
---
### Function 3: `_add_manual_universe()`
```python
def _add_manual_universe(self):
"""V7.2: إضافة الأسهم يدوياً من القائمة المحفوظة مع حماية من التكرار"""
try:
if not hasattr(self, 'target_symbols') or not self.target_symbols:
self.log("[ManualUniverse] No stock list to add")
return
self.log(f"[ManualUniverse] Starting to add {len(self.target_symbols)} stocks manually...")
added_count = 0
skipped_count = 0
for ticker in self.target_symbols:
try:
# V7.2: حماية من التكرار (توصية البوت)
existing_symbol = None
for symbol in self.symbol_data_dict.keys():
if symbol.value == ticker:
existing_symbol = symbol
break
if existing_symbol:
skipped_count += 1
continue
# إضافة السهم
symbol = self.add_equity(ticker, Resolution.MINUTE).symbol
# إنشاء SymbolData
self.symbol_data_dict[symbol] = SymbolData(self, symbol)
# الاشتراك في Benzinga
self.add_data(BenzingaNews, symbol)
# حساب متوسط الحجم
self._calculate_average_volume(symbol)
added_count += 1
except Exception as e:
self.error(f"[ManualUniverse] Failed to add {ticker}: {e}")
self.log(f"[ManualUniverse] Successfully added {added_count} stocks (Skipped {skipped_count} duplicates)")
self.log(f"[ManualUniverse] Total active stocks: {len(self.symbol_data_dict)}")
except Exception as e:
self.error(f"[ManualUniverse] Error: {e}")
```
**Issues:**
1. ✅ Good validation at the start
2. ✅ Good error handling
3. ❌ But never called because `_load_target_symbols_from_objectstore()` returns `None`
---
### Function 4: `fine_selection()`
```python
def fine_selection(self, fine):
"""
فلترة نهائية بناءً على Market Cap و Shares Outstanding
"""
try:
fine_list = list(fine)
self.log(f"[FineSelection] called at {self.time}")
self.log(f"[FineSelection] استلام {len(fine_list)} سهم من Coarse Selection")
# فلترة حسب Market Cap + Shares Outstanding
filtered = []
debug_count = 0
for x in fine_list:
try:
# الحصول على البيانات
shares_outstanding = self._get_shares_outstanding(x)
market_cap = x.MarketCap if hasattr(x, 'MarketCap') else 0
# طباعة أول 10 أسهم للتشخيص
if debug_count < 10:
from helpers import safe_format
shares_str = safe_format(shares_outstanding, ",.0f", log_warning=True,
algo=self, field_name=f"{x.symbol.value}_shares")
market_cap_str = "$" + safe_format(market_cap, ",.0f", log_warning=True,
algo=self, field_name=f"{x.symbol.value}_market_cap")
self.log(f"[FineSelection] {x.symbol.value}: Shares={shares_str}, MarketCap={market_cap_str}")
debug_count += 1
# تطبيق الفلاتر
if (shares_outstanding and shares_outstanding <= config.MAX_SHARES_OUTSTANDING
and market_cap >= config.MIN_MARKET_CAP
and market_cap <= config.MAX_MARKET_CAP):
filtered.append((x, shares_outstanding))
except Exception as e:
if debug_count < 10:
self.log(f"[FineSelection] {x.symbol.value}: Error - {e}")
debug_count += 1
continue
self.log(f"[FineSelection] بعد فلتر Market Cap + Shares: {len(filtered)} سهم")
# ترتيب حسب Shares Outstanding (الأقل أولاً - Low Float)
sorted_by_shares = sorted(filtered, key=lambda x: x[1])
# اختيار أفضل 900 سهم
selected = [x[0].symbol for x in sorted_by_shares[:config.UNIVERSE_SIZE]]
self.log(f"[FineSelection] انتهى Fine Selection - إرسال {len(selected)} سهم إلى on_securities_changed")
return selected
except Exception as e:
self.error(f"[FineSelection] خطأ: {e}")
return []
```
**Issues:**
1. ✅ Good logging for first 10 stocks
2. ❌ But logs show all stocks are rejected (0 stocks after filter)
3. ❌ Likely `_get_shares_outstanding()` returns `None` for all stocks
---
### Function 5: `_get_shares_outstanding()`
```python
def _get_shares_outstanding(self, fine_fundamental):
"""
V7.3.5: استخراج عدد الأسهم الحرة من البيانات الأساسية (محسّن)
يحاول عدة طرق للحصول على البيانات:
1. CompanyReference.SharesOutstanding (الأكثر موثوقية)
2. EarningReports.BasicAverageShares
3. FinancialStatements.SharesOutstanding
4. CompanyProfile.SharesOutstanding
"""
try:
# محاولة 1: CompanyReference.SharesOutstanding
if hasattr(fine_fundamental, 'CompanyReference') and hasattr(fine_fundamental.CompanyReference, 'SharesOutstanding'):
shares = fine_fundamental.CompanyReference.SharesOutstanding
if shares and shares > 0:
return shares
# محاولة 2: EarningReports.BasicAverageShares
if hasattr(fine_fundamental, 'EarningReports') and hasattr(fine_fundamental.EarningReports, 'BasicAverageShares'):
shares = fine_fundamental.EarningReports.BasicAverageShares.ThreeMonths
if shares and shares > 0:
return shares
# محاولة 3: FinancialStatements.SharesOutstanding
if hasattr(fine_fundamental, 'FinancialStatements') and hasattr(fine_fundamental.FinancialStatements, 'SharesOutstanding'):
shares = fine_fundamental.FinancialStatements.SharesOutstanding.ThreeMonths
if shares and shares > 0:
return shares
# محاولة 4: CompanyProfile.SharesOutstanding
if hasattr(fine_fundamental, 'CompanyProfile') and hasattr(fine_fundamental.CompanyProfile, 'SharesOutstanding'):
shares = fine_fundamental.CompanyProfile.SharesOutstanding
if shares and shares > 0:
return shares
return None # لم يتم العثور على البيانات
except Exception as e:
return None
```
**Issues:**
1. ✅ Good fallback strategy with 4 attempts
2. ❌ But likely returning `None` for all 900 stocks
3. ❌ No logging to show which method was tried
4. ❌ No way to diagnose why all attempts fail
---
## Configuration Values
```python
# في config.py
MAX_SHARES_OUTSTANDING = 30_000_000 # 30 مليون سهم (Low Float)
MIN_MARKET_CAP = 5_000_000 # $5M (Micro-cap)
MAX_MARKET_CAP = 100_000_000 # $100M (Micro-cap)
UNIVERSE_SIZE = 900 # عدد الأسهم المراقبة
OBJECT_STORE_KEY = "filtered_universe_900"
USE_OBJECT_STORE = True
```
---
## Questions for the Bot
### Question 1: Missing Return Statement
**Is this the root cause?**
The function `_load_target_symbols_from_objectstore()` doesn't return `True` or `False`. Is this why the system ignores the loaded 585 stocks?
**Expected behavior:**
```python
def _load_target_symbols_from_objectstore(self):
try:
if self.object_store.contains_key(config.OBJECT_STORE_KEY):
symbols_str = self.object_store.read(config.OBJECT_STORE_KEY)
self.target_symbols = symbols_str.split(',')
self.log(f"[ObjectStore] تم تحميل {len(self.target_symbols)} سهم")
return True # ✅ Add this
else:
self.log("[ObjectStore] لا توجد قائمة محفوظة")
return False # ✅ Add this
except Exception as e:
self.error(f"[ObjectStore] خطأ في التحميل: {e}")
return False # ✅ Add this
```
---
### Question 2: Fine Selection Rejecting All Stocks
**Why does `fine_selection()` reject all 900 stocks?**
Logs show:
```
[FineSelection] استلام 900 سهم من Coarse Selection ✅
[FineSelection] بعد فلتر Market Cap + Shares: 0 سهم ❌
```
**Possible reasons:**
1. Is `MAX_SHARES_OUTSTANDING = 30M` too restrictive?
2. Is `_get_shares_outstanding()` returning `None` for all stocks?
3. Did QuantConnect change the fundamental data structure?
4. Is there a timing issue where data isn't available yet?
---
### Question 3: Diagnostic Logging
**What additional logging should we add to diagnose the issue?**
Current logging shows counts but not details. Should we add:
```python
# In fine_selection()
for x in fine_list[:20]: # First 20 stocks
shares = self._get_shares_outstanding(x)
market_cap = x.MarketCap if hasattr(x, 'MarketCap') else None
self.log(f"[FineSelection-Debug] {x.symbol.value}:")
self.log(f" Shares Outstanding: {shares}")
self.log(f" Market Cap: ${market_cap:,.0f}" if market_cap else " Market Cap: None")
self.log(f" Has CompanyReference: {hasattr(x, 'CompanyReference')}")
self.log(f" Has EarningReports: {hasattr(x, 'EarningReports')}")
self.log(f" Has FinancialStatements: {hasattr(x, 'FinancialStatements')}")
```
---
### Question 4: Alternative Approaches
**Should we modify the selection criteria?**
Options:
1. Increase `MAX_SHARES_OUTSTANDING` to 100M or 500M temporarily
2. Remove the shares outstanding filter entirely for testing
3. Use only Market Cap filter
4. Force load from ObjectStore regardless of return value
---
### Question 5: QuantConnect Platform Changes
**Did QuantConnect change anything recently?**
The code worked on September 30 but failed on October 3. Possible changes:
1. Fundamental data API structure
2. Universe Selection timing
3. Data availability
4. ObjectStore behavior
---
## Recommended Fix (Hypothesis 1)
### Step 1: Fix the Return Statement
```python
def _load_target_symbols_from_objectstore(self):
"""تحميل قائمة الأسهم المستهدفة من ObjectStore"""
try:
if self.object_store.contains_key(config.OBJECT_STORE_KEY):
symbols_str = self.object_store.read(config.OBJECT_STORE_KEY)
if not symbols_str or len(symbols_str) == 0:
self.log("[ObjectStore] Empty data in ObjectStore")
return False
self.target_symbols = symbols_str.split(',')
# Validate
if not self.target_symbols or len(self.target_symbols) == 0:
self.log("[ObjectStore] No symbols after split")
return False
self.log(f"[ObjectStore] تم تحميل {len(self.target_symbols)} سهم")
# Log first 10 symbols for verification
if len(self.target_symbols) >= 10:
sample = ', '.join(self.target_symbols[:10])
self.log(f"[ObjectStore] Sample: {sample}...")
return True # ✅ Success
else:
self.log("[ObjectStore] لا توجد قائمة محفوظة - استخدام Universe Selection")
return False
except Exception as e:
self.error(f"[ObjectStore] خطأ في التحميل: {e}")
import traceback
self.error(traceback.format_exc())
return False
```
---
### Step 2: Add Defensive Check in initialize()
```python
if config.USE_OBJECT_STORE:
self.log("[Initialize] ObjectStore enabled - attempting immediate loading...")
load_result = self._load_target_symbols_from_objectstore()
self.log(f"[Initialize] Load result: {load_result}")
self.log(f"[Initialize] Has target_symbols: {hasattr(self, 'target_symbols')}")
if hasattr(self, 'target_symbols'):
self.log(f"[Initialize] target_symbols length: {len(self.target_symbols) if self.target_symbols else 0}")
if load_result and hasattr(self, 'target_symbols') and self.target_symbols:
self.log(f"[Initialize] ✅ Loaded {len(self.target_symbols)} stocks from ObjectStore")
self._add_manual_universe()
self.bootstrap_completed = True
self.log("[Initialize] ✅ Immediate loading complete! System ready.")
else:
self.log("[Initialize] ❌ No saved list - will wait for Universe Selection")
```
---
### Step 3: Enhanced Logging in fine_selection()
```python
def fine_selection(self, fine):
try:
fine_list = list(fine)
self.log(f"[FineSelection] called at {self.time}")
self.log(f"[FineSelection] استلام {len(fine_list)} سهم من Coarse Selection")
# Detailed logging for first 20 stocks
self.log("[FineSelection] === Detailed Analysis of First 20 Stocks ===")
for i, x in enumerate(fine_list[:20]):
shares = self._get_shares_outstanding(x)
market_cap = x.MarketCap if hasattr(x, 'MarketCap') else None
self.log(f"[FineSelection] Stock #{i+1}: {x.symbol.value}")
self.log(f" Shares Outstanding: {shares:,.0f}" if shares else " Shares Outstanding: None")
self.log(f" Market Cap: ${market_cap:,.0f}" if market_cap else " Market Cap: None")
# Check filter conditions
if shares:
passes_shares = shares <= config.MAX_SHARES_OUTSTANDING
self.log(f" Shares Filter: {'✅ PASS' if passes_shares else '❌ FAIL'} ({shares:,.0f} <= {config.MAX_SHARES_OUTSTANDING:,.0f})")
else:
self.log(f" Shares Filter: ❌ FAIL (None)")
if market_cap:
passes_market_cap = config.MIN_MARKET_CAP <= market_cap <= config.MAX_MARKET_CAP
self.log(f" Market Cap Filter: {'✅ PASS' if passes_market_cap else '❌ FAIL'} (${config.MIN_MARKET_CAP:,.0f} <= ${market_cap:,.0f} <= ${config.MAX_MARKET_CAP:,.0f})")
else:
self.log(f" Market Cap Filter: ❌ FAIL (None)")
self.log(" ---")
# Rest of the function...
```
---
## Summary
**Primary Suspect:** Missing `return True` statement in `_load_target_symbols_from_objectstore()`
**Secondary Suspect:** All stocks rejected by `fine_selection()` due to missing fundamental data
**Recommendation:**
1. Add return statements to `_load_target_symbols_from_objectstore()`
2. Add detailed diagnostic logging to `fine_selection()`
3. Temporarily increase `MAX_SHARES_OUTSTANDING` to 100M for testing
4. Check if QuantConnect made recent platform changes
**Expected Outcome:** System should load 585 stocks from ObjectStore in 1-2 minutes instead of waiting for Universe Selection.
---
## Request to Bot
Please analyze these functions and tell us:
1. Is the missing return statement the root cause?
2. Why is `fine_selection()` rejecting all 900 stocks?
3. What additional diagnostic code should we add?
4. Are there any QuantConnect platform changes we should know about?
5. What is the best way to fix this issue?
Thank you!
Mia Alissi
Yes, the missing return statement in
_load_target_symbols_from_objectstore()is the root cause why the system ignores the loaded stocks. In Python, a function withoutreturnimplicitly returnsNone, which is evaluated asFalsein yourifchecks. Therefore, even when stocks are loaded, the condition fails and execution falls into the wrong branch.Here is a fixed version of the function with proper return statements and additional logging:
Summary of the Fix
Trueon confirmed successful load, andFalseon any error or missing data.initializefunction logic will behave as expected and stocks will be added when the load is actually successful.Reasoning:
Your symptoms matched this bug perfectly: the log shows a successful load but immediately falls into the 'No saved list' branch. The only way for this to happen is for
_load_target_symbols_from_objectstore()to returnNone(Python's default) instead ofTrue, which it did due to the missing return statement.Recommendation:
Replace your function with the above, redeploy, and stocks will be used as intended, resolving this critical issue.
Let me know if you want improved diagnostic logging for your
fine_selectionor explanations for the other hypotheses as well!The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
مطلق الشمري
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!