Share New Research 
# Detailed Question for QuantConnect Expert Bot
---
## Problem Summary:
I have a QuantConnect trading algorithm that uses **Universe Selection** to select stocks and monitor Benzinga news. The problem is that the system **is not adding any stocks** after running `fine_selection()`, even though it successfully processes 900 stocks in `coarse_selection()`.
---
## Context:
### Old Version (Was Working):
- Added **49 stocks** immediately after `initialize()`
- **Did NOT use** `Universe Selection` (no `[CoarseSelection]` or `[FineSelection]` logs appeared)
- Loaded stocks from a **hardcoded list** or **ObjectStore**
- Stocks were added directly without going through the Universe Selection pipeline
### Current Version (Not Working):
- Uses dynamic `Universe Selection`
- `coarse_selection()` works successfully and sends **900 stocks** to `fine_selection()`
- `fine_selection()` **rejects ALL stocks** and returns **0 stocks**
- Result: `on_securities_changed()` receives **0 added stocks**
- No news monitoring happens because no stocks are in the universe
---
## Logs Evidence:
### Successful Run (Old Version - Sept 30):
```
2025-09-30 13:50:49 Algorithm initialized successfully.
2025-09-30 13:50:49 Universe changes: SecurityChanges: Added: A, AA, AAL, AAOI, AAON, AAP, AARD, AAT, AC, ABBV, ABCB, ABCL, ABG, ABM, ABNB, ABR, ABSI, ABT, ABVE, ACA, ACAD, ACB... (49 stocks total)
```
✅ **49 stocks added immediately**
✅ **No CoarseSelection or FineSelection logs** (bypassed Universe Selection)
---
### Failed Run (Current Version - Oct 3):
```
2025-10-03 00:41:18 [Initialize] Setting up Universe Selection...
2025-10-03 00:41:18 [Initialize] Universe Selection registered successfully
2025-10-03 00:41:18 [Initialize] ObjectStore enabled - attempting immediate loading...
2025-10-03 00:41:18 [ObjectStore] Loaded 585 stocks
2025-10-03 00:41:18 [Initialize] No saved list - will wait for Universe Selection (00:00-00:30 AM)
2025-10-03 00:41:18 [OnSecuritiesChanged] Update #1 | Added: 1 | Removed: 0
2025-10-03 00:41:18 [OnSecuritiesChanged] Total active stocks: 0
```
Then 5 hours later:
```
2025-10-03 06:00:09 [CoarseSelection] Starting Coarse Selection - Time: 2025-10-03 02:00:00
2025-10-03 06:00:09 [CoarseSelection] Total available stocks: 10334
2025-10-03 06:00:09 [CoarseSelection] After price filter: 3119 stocks
2025-10-03 06:00:09 [CoarseSelection] Selected 900 stocks for Fine Selection
2025-10-03 06:00:09 [FineSelection] Starting Fine Selection - Time: 2025-10-03 02:00:00
2025-10-03 06:00:09 [FineSelection] Received 900 stocks from Coarse Selection
2025-10-03 06:00:09 [FineSelection] After Market Cap + Shares filter: 0 stocks ❌
2025-10-03 06:00:09 [FineSelection] After sorting: Selected best 0 stocks (Low Float)
2025-10-03 06:00:09 [FineSelection] Fine Selection complete - Sending 0 stocks to on_securities_changed
```
❌ **0 stocks added**
❌ **All stocks rejected by fine_selection()**
---
## Complete Scenario Flow:
### **INTENDED FLOW (What Should Happen):**
```
1. INITIALIZATION PHASE
├─ initialize() called
├─ Set timezone to New York
├─ Register Universe Selection (coarse + fine)
├─ Try to load from ObjectStore
│ ├─ If found: Load 585 stocks → Add manually → System ready
│ └─ If not found: Wait for Universe Selection
└─ Schedule events (heartbeat, daily report, etc.)
2. UNIVERSE SELECTION PHASE (Runs at 00:00-00:30 AM EST daily)
├─ coarse_selection() called
│ ├─ Receive ~10,000 stocks from QuantConnect
│ ├─ Filter by price ($1-$50)
│ ├─ Filter by has_fundamental_data
│ └─ Return ~3,000 symbols to fine_selection()
│
├─ fine_selection() called
│ ├─ Receive ~3,000 stocks from coarse
│ ├─ For each stock:
│ │ ├─ Get shares_outstanding via _get_shares_outstanding()
│ │ ├─ Get market_cap from x.MarketCap
│ │ ├─ Filter: shares <= 30M AND 5M <= market_cap <= 100M
│ │ └─ Add to filtered list if passes
│ ├─ Sort by shares_outstanding (lowest first)
│ ├─ Take top 900 stocks
│ └─ Return 900 symbols to QuantConnect
│
└─ on_securities_changed() called
├─ Receive 900 stocks from QuantConnect
├─ For each stock:
│ ├─ Create SymbolData object
│ ├─ Subscribe to BenzingaNews
│ ├─ Calculate average volume (ADV)
│ └─ Add to symbol_data_dict
└─ Save list to ObjectStore for next run
3. DATA PROCESSING PHASE (Continuous during market hours)
├─ on_data() called every minute
│ ├─ Update SPY price
│ ├─ Process Telegram alert queue
│ ├─ Call news_analyzer.process_news()
│ │ ├─ Check for new Benzinga news in slice_data
│ │ ├─ For each news item:
│ │ │ ├─ Evaluate importance (critical/important/normal/low)
│ │ │ ├─ Open news window (5 minutes)
│ │ │ ├─ Store news in symbol_data.pending_news
│ │ │ └─ Mark symbol as in_news_window
│ │ └─ Close expired news windows
│ │
│ └─ For each stock in symbol_data_dict:
│ ├─ Update price from bar.close
│ ├─ Update volume from bar.volume
│ ├─ Calculate RVOL (relative volume)
│ └─ Call _check_for_signals()
4. SIGNAL CHECKING PHASE (For stocks with active news)
├─ _check_for_signals() called
│ ├─ Check if stock is in news window → If NO: return
│ ├─ Check cooldown period → If in cooldown: return
│ │
│ ├─ If news_alert_sent == False:
│ │ └─ Call _send_news_alert()
│ │ ├─ Extract headline and URL from news
│ │ ├─ Build news alert message
│ │ ├─ Queue alert with priority (critical/high)
│ │ └─ Mark news_alert_sent = True
│ │
│ └─ If reaction_alert_sent == False:
│ └─ Call _check_reaction_alert()
5. REACTION CHECKING PHASE (Check if stock reacted to news)
├─ _check_reaction_alert() called
│ ├─ Calculate metrics:
│ │ ├─ volume_spike_ratio = current_volume / avg_volume
│ │ ├─ rvol = session_volume / expected_volume
│ │ └─ price_change = (current_price - news_price) / news_price * 100
│ │
│ ├─ Get dynamic thresholds based on:
│ │ ├─ News importance (critical/important/normal/low)
│ │ └─ Time of day (pre_market/opening/lunch/power_hour/after_market)
│ │
│ ├─ Apply defensive filters:
│ │ ├─ Extreme volatility filter (>10% move)
│ │ ├─ Minimum move filter (<1% move)
│ │ └─ SPY correlation filter (market-wide move)
│ │
│ ├─ Check conditions:
│ │ ├─ Volume condition: spike_ratio >= threshold AND rvol >= threshold
│ │ └─ Price condition: abs(price_change) >= threshold
│ │
│ └─ If BOTH conditions met:
│ ├─ Build reaction alert message
│ ├─ Queue alert with priority (high/critical)
│ ├─ Mark reaction_alert_sent = True
│ └─ Update alert statistics
6. ALERT DELIVERY PHASE
├─ notifier.process_queue() called
│ ├─ Check rate limits (20 messages/minute)
│ ├─ Process alerts by priority:
│ │ ├─ Critical queue first
│ │ ├─ High queue second
│ │ ├─ Normal queue third
│ │ └─ Low queue last
│ │
│ └─ For each alert:
│ ├─ Send to Telegram via API
│ ├─ Wait 3 seconds between messages
│ └─ Update statistics
│
└─ User receives alert on Telegram
```
---
### **ACTUAL FLOW (What's Happening Now):**
```
1. INITIALIZATION PHASE ✅
├─ initialize() called ✅
├─ Universe Selection registered ✅
├─ ObjectStore loaded 585 stocks ✅
└─ BUT: Rejected ObjectStore stocks ❌
└─ Reason: Unknown condition prevents usage
2. UNIVERSE SELECTION PHASE ⚠️
├─ coarse_selection() called ✅
│ ├─ Received 10,334 stocks ✅
│ ├─ Filtered to 3,119 stocks ✅
│ └─ Sent 3,119 stocks to fine_selection ✅
│
├─ fine_selection() called ✅
│ ├─ Received 3,119 stocks ✅
│ ├─ Applied filters ✅
│ └─ Result: 0 stocks passed ❌
│ └─ Reason: ALL stocks rejected by filter
│
└─ on_securities_changed() called ⚠️
├─ Received 0 stocks ❌
└─ symbol_data_dict remains empty ❌
3. DATA PROCESSING PHASE ❌
├─ on_data() called ✅
└─ BUT: No stocks to process ❌
└─ symbol_data_dict is empty
4. SIGNAL CHECKING PHASE ❌
└─ Never reached (no stocks in universe)
5. REACTION CHECKING PHASE ❌
└─ Never reached (no stocks in universe)
6. ALERT DELIVERY PHASE ❌
└─ Never reached (no signals to check)
RESULT: System runs but does nothing ❌
```
---
## Code Implementation:
### 1. `initialize()` Function:
```python
def initialize(self):
"""Initialize the algorithm"""
try:
# Basic settings
self.set_start_date(2024, 1, 1)
self.set_cash(100000)
self.set_time_zone(TimeZones.NEW_YORK)
# Initialize dictionaries
self.symbol_data_dict = {}
# Subsystems
self.notifier = TelegramNotifier(self)
self.news_analyzer = NewsAnalyzer(self)
self.translator = TranslationService(self)
# System state
self.is_warmed_up = False
self.bootstrap_completed = False
self.universe_updates_count = 0
# Add SPY for market correlation filter
self.spy_symbol = self.add_equity("SPY", Resolution.MINUTE).symbol
self.spy_price_at_news = {}
self.spy_current_price = 0
# Register Universe Selection
self.log("[Initialize] Setting up Universe Selection...")
self.universe_settings.resolution = Resolution.DAILY
self.add_universe(self.coarse_selection, self.fine_selection)
self.log("[Initialize] Universe Selection registered successfully")
# Try to load from ObjectStore for immediate start
if config.USE_OBJECT_STORE:
self.log("[Initialize] ObjectStore enabled - attempting immediate loading...")
if self._load_target_symbols_from_objectstore():
self.log(f"[Initialize] Loaded {len(self.target_symbols)} stocks from ObjectStore")
self._add_manual_universe()
self.bootstrap_completed = True
self.log("[Initialize] Immediate loading complete! System ready.")
else:
self.log("[Initialize] No saved list - will wait for Universe Selection (00:00-00:30 AM)")
else:
self.log("[Initialize] ObjectStore disabled - will wait for Universe Selection")
self.log("[Initialize] Wait 5-20 minutes for fundamental data and Universe Selection...")
self.log("[Initialize] [CoarseSelection] and [FineSelection] logs will appear when loading starts")
# Schedule events
self._schedule_events()
# Send startup message (Live Mode only)
if self.live_mode:
self.log("[Initialize] Live Mode - sending startup message...")
self.notifier.send_startup_message_with_retry()
self.notifier.process_queue()
self.log("[Initialize] Initialize completed successfully")
except Exception as e:
self.error(f"[Initialize] Critical error: {e}")
raise
```
---
### 2. `_load_target_symbols_from_objectstore()` Function:
```python
def _load_target_symbols_from_objectstore(self):
"""Load target symbols list from ObjectStore"""
try:
if self.object_store.contains_key(config.OBJECT_STORE_KEY):
symbols_str = self.object_store.read(config.OBJECT_STORE_KEY)
self.target_symbols = symbols_str.split(',')
self.log(f"[ObjectStore] Loaded {len(self.target_symbols)} stocks")
return True # ✅ Successfully loaded
else:
self.log("[ObjectStore] No saved list found - using Universe Selection")
return False # ❌ No saved list
except Exception as e:
self.error(f"[ObjectStore] Error loading: {e}")
return False
```
**QUESTION 1:** The logs show `[ObjectStore] Loaded 585 stocks` but then `[Initialize] No saved list`. What condition prevents using the loaded stocks?
---
### 3. `_add_manual_universe()` Function:
```python
def _add_manual_universe(self):
"""Add stocks manually from saved list with duplicate protection"""
try:
if not hasattr(self, 'target_symbols') or not self.target_symbols:
self.log("[ManualUniverse] No stock list to add")
return
self.log(f"[ManualUniverse] Starting to add {len(self.target_symbols)} stocks manually...")
added_count = 0
skipped_count = 0
for ticker in self.target_symbols:
try:
# Check for duplicates
existing_symbol = None
for symbol in self.symbol_data_dict.keys():
if symbol.value == ticker:
existing_symbol = symbol
break
if existing_symbol:
skipped_count += 1
continue
# Add stock
symbol = self.add_equity(ticker, Resolution.MINUTE).symbol
self.symbol_data_dict[symbol] = SymbolData(self, symbol)
# Subscribe to Benzinga
self.add_data(BenzingaNews, symbol)
# Calculate ADV
self._calculate_average_volume(symbol)
added_count += 1
except Exception as e:
self.error(f"[ManualUniverse] Failed to add {ticker}: {e}")
self.log(f"[ManualUniverse] Successfully added {added_count} stocks (Skipped {skipped_count} duplicates)")
self.log(f"[ManualUniverse] Total active stocks: {len(self.symbol_data_dict)}")
except Exception as e:
self.error(f"[ManualUniverse] Error: {e}")
```
---
### 4. `coarse_selection()` Function:
```python
def coarse_selection(self, coarse):
"""
Coarse Universe Selection - called automatically in Live Mode
Selects best UNIVERSE_SIZE stocks based on:
- Price (MIN_PRICE - MAX_PRICE)
- Has fundamental data
"""
try:
coarse_list = list(coarse)
self.log(f"[CoarseSelection] called at {self.time}")
self.log(f"[CoarseSelection] Starting Coarse Selection - Time: {self.time}")
self.log(f"[CoarseSelection] Total available stocks: {len(coarse_list)}")
# Filter by price only (no volume filter)
filtered = [
x for x in coarse_list
if config.MIN_PRICE <= x.price <= config.MAX_PRICE
and x.has_fundamental_data
]
self.log(f"[CoarseSelection] After price filter: {len(filtered)} stocks")
self.log(f"[CoarseSelection] Sending all stocks to Fine Selection (no limit)")
# Send all stocks to Fine Selection (no 900 limit here)
return [x.symbol for x in filtered]
except Exception as e:
self.error(f"[CoarseSelection] Error: {e}")
return []
```
**Criteria:**
- `MIN_PRICE = 1.0`
- `MAX_PRICE = 50.0`
**Result:** ✅ Successfully filters ~3,000 stocks and sends to fine_selection
---
### 5. `fine_selection()` Function:
```python
def fine_selection(self, fine):
"""
Fine Universe Selection - called automatically after Coarse Selection
Applies additional filter based on:
- Shares Outstanding (MAX_SHARES_OUTSTANDING)
- Market Cap (MIN_MARKET_CAP - MAX_MARKET_CAP)
"""
try:
fine_list = list(fine)
self.log(f"[FineSelection] called at {self.time}")
self.log(f"[FineSelection] Starting Fine Selection - Time: {self.time}")
self.log(f"[FineSelection] Received {len(fine_list)} stocks from Coarse Selection")
# Filter by Market Cap + Shares Outstanding
filtered = []
debug_count = 0
for x in fine_list:
try:
shares_outstanding = self._get_shares_outstanding(x)
market_cap = x.MarketCap if hasattr(x, 'MarketCap') else 0
# Print first 10 stocks for debugging
if debug_count < 10:
from helpers import safe_format
shares_str = safe_format(shares_outstanding, ",.0f", log_warning=True,
algo=self, field_name=f"{x.symbol.value}_shares")
market_cap_str = "$" + safe_format(market_cap, ",.0f", log_warning=True,
algo=self, field_name=f"{x.symbol.value}_market_cap")
self.log(f"[FineSelection] {x.symbol.value}: Shares={shares_str}, MarketCap={market_cap_str}")
debug_count += 1
# Filter: Market Cap + Shares Outstanding
if (shares_outstanding and shares_outstanding <= config.MAX_SHARES_OUTSTANDING
and market_cap >= config.MIN_MARKET_CAP
and market_cap <= config.MAX_MARKET_CAP):
filtered.append((x, shares_outstanding))
except Exception as e:
if debug_count < 10:
self.log(f"[FineSelection] {x.symbol.value}: Error - {e}")
debug_count += 1
continue
self.log(f"[FineSelection] After Market Cap + Shares filter: {len(filtered)} stocks")
# Sort by Shares Outstanding (lowest first - Low Float)
sorted_by_shares = sorted(filtered, key=lambda x: x[1])
# Select best 900 stocks
selected = [x[0].symbol for x in sorted_by_shares[:config.UNIVERSE_SIZE]]
self.log(f"[FineSelection] After sorting: Selected best {len(selected)} stocks (Low Float)")
self.log(f"[FineSelection] Fine Selection complete - Sending {len(selected)} stocks to on_securities_changed")
return selected
except Exception as e:
self.error(f"[FineSelection] Error: {e}")
return []
```
**Criteria:**
- `MAX_SHARES_OUTSTANDING = 30_000_000` (30 million shares)
- `MIN_MARKET_CAP = 5_000_000` ($5M)
- `MAX_MARKET_CAP = 100_000_000` ($100M)
- `UNIVERSE_SIZE = 900`
**Result:** ❌ Returns 0 stocks (ALL rejected by filter)
---
### 6. `_get_shares_outstanding()` Function:
```python
def _get_shares_outstanding(self, fine_fundamental):
"""Extract shares outstanding (free float)"""
try:
# Attempt 1: CompanyReference.SharesOutstanding
if hasattr(fine_fundamental, 'CompanyReference') and \
hasattr(fine_fundamental.CompanyReference, 'SharesOutstanding'):
shares = fine_fundamental.CompanyReference.SharesOutstanding
if shares and shares > 0:
return shares
# Attempt 2: EarningReports.BasicAverageShares
if hasattr(fine_fundamental, 'EarningReports') and \
hasattr(fine_fundamental.EarningReports, 'BasicAverageShares'):
shares = fine_fundamental.EarningReports.BasicAverageShares.ThreeMonths
if shares and shares > 0:
return shares
# Attempt 3: FinancialStatements.SharesOutstanding
if hasattr(fine_fundamental, 'FinancialStatements') and \
hasattr(fine_fundamental.FinancialStatements, 'SharesOutstanding'):
shares = fine_fundamental.FinancialStatements.SharesOutstanding.ThreeMonths
if shares and shares > 0:
return shares
# Attempt 4: CompanyProfile.SharesOutstanding
if hasattr(fine_fundamental, 'CompanyProfile') and \
hasattr(fine_fundamental.CompanyProfile, 'SharesOutstanding'):
shares = fine_fundamental.CompanyProfile.SharesOutstanding
if shares and shares > 0:
return shares
return None
except Exception as e:
return None
```
**QUESTION 2:** Are these methods correct for accessing shares outstanding in QuantConnect? Is there a better/more reliable way?
---
### 7. `on_securities_changed()` Function:
```python
def on_securities_changed(self, changes):
"""Handle changes in monitored stocks"""
try:
self.universe_updates_count += 1
added_count = len(changes.added_securities)
removed_count = len(changes.removed_securities)
self.log(f"OnSecuritiesChanged at {self.time}: Added {added_count}, Removed {removed_count}")
self.log(f"[OnSecuritiesChanged] Time: {self.time}")
self.log(f"[OnSecuritiesChanged] Update #{self.universe_updates_count} | Added: {added_count} | Removed: {removed_count}")
# Add new stocks
for added in changes.added_securities:
symbol = added.symbol
# Skip SPY (added manually in initialize)
if symbol == self.spy_symbol:
continue
# Create SymbolData
self.symbol_data_dict[symbol] = SymbolData(self, symbol)
# Subscribe to Benzinga
try:
self.add_data(BenzingaNews, symbol)
self.log(f"[OnSecuritiesChanged] Added {symbol.value} + Benzinga")
except Exception as e:
self.error(f"[OnSecuritiesChanged] Failed Benzinga subscription for {symbol}: {e}")
# Calculate average volume
self._calculate_average_volume(symbol)
# Remove old stocks
for removed in changes.removed_securities:
symbol = removed.symbol
# Never remove SPY
if symbol == self.spy_symbol:
continue
if symbol in self.symbol_data_dict:
# Cleanup
self.symbol_data_dict[symbol].dispose()
del self.symbol_data_dict[symbol]
self.log(f"[OnSecuritiesChanged] Removed {symbol.value}")
self.log(f"[OnSecuritiesChanged] Total active stocks: {len(self.symbol_data_dict)}")
# Mark bootstrap complete
if not self.bootstrap_completed and len(self.symbol_data_dict) > 0:
self.bootstrap_completed = True
self.log(f"[OnSecuritiesChanged] Initial loading complete! Successfully loaded {len(self.symbol_data_dict)} stocks")
self.log(f"[OnSecuritiesChanged] System now ready to monitor news and send alerts")
# Send Telegram alert when loading complete (Live Mode only)
if self.live_mode and len(self.symbol_data_dict) >= 10:
msg = (
f"✅ Initial Loading Complete!\n\n"
f"📊 Stocks Loaded: {len(self.symbol_data_dict)}\n"
f"⏱ Time: {self.time.strftime('%Y-%m-%d %H:%M:%S')}\n"
f"🔔 System now ready to monitor news\n"
)
self.notifier.queue_alert(msg, priority="normal")
self.notifier.process_queue()
# Save new list to ObjectStore for next run
if config.USE_OBJECT_STORE and self.universe_updates_count >= 1:
try:
symbols_list = [symbol.value for symbol in self.symbol_data_dict.keys()
if symbol != self.spy_symbol]
if len(symbols_list) > 0:
symbols_str = ','.join(symbols_list)
self.object_store.save(config.OBJECT_STORE_KEY, symbols_str)
self.log(f"[ObjectStore] Saved {len(symbols_list)} stocks for next run")
except Exception as e:
self.error(f"[ObjectStore] Error saving: {e}")
except Exception as e:
self.error(f"[OnSecuritiesChanged] Error: {e}")
```
**Result:** Currently receives 0 stocks, so symbol_data_dict remains empty
---
### 8. `on_data()` Function:
```python
def on_data(self, slice_data):
"""Process incoming data"""
try:
self.data_updates_count += 1
# Diagnostic message on first call
if self.data_updates_count == 1:
self.log(f"[OnData] First call | Time: {self.time}")
self.log(f"[OnData] Active stocks: {len(self.symbol_data_dict)}")
# Update SPY price
if self.spy_symbol in slice_data.bars:
self.spy_current_price = slice_data.bars[self.spy_symbol].close
# Process alert queue
self.notifier.process_queue()
# Process news
self.news_analyzer.process_news(slice_data, self.symbol_data_dict)
# Clean expired news windows
self.news_analyzer.close_expired_news_windows(self.utc_time, self.symbol_data_dict)
# Process stocks
for symbol, symbol_data in self.symbol_data_dict.items():
if symbol not in slice_data.bars:
continue
bar = slice_data.bars[symbol]
# Update data
symbol_data.update_price(bar.close)
symbol_data.update_volume(bar.volume)
# Calculate RVOL
symbol_data.calculate_rvol(
self.utc_time,
config.SESSION_START_MIN,
config.SESSION_MINUTES
)
# Check for signals
self._check_for_signals(symbol, symbol_data)
except Exception as e:
self.error(f"[OnData] Error: {e}")
```
---
### 9. `news_analyzer.process_news()` Function:
```python
def process_news(self, slice_data, symbol_data_dict):
"""Process Benzinga news from slice data"""
try:
# Check if slice has news data
if not hasattr(slice_data, 'data') or not slice_data.data:
return
# Iterate through all data in slice
for symbol, data_list in slice_data.data.items():
# Skip if not in our universe
if symbol not in symbol_data_dict:
continue
symbol_data = symbol_data_dict[symbol]
# Process each data item
for data_item in data_list:
# Check if it's Benzinga news
if not isinstance(data_item, BenzingaNews):
continue
# Evaluate news importance
evaluation = self.evaluate_importance(data_item)
# Open news window
symbol_data.open_news_window(
self.algo.utc_time,
config.NEWS_WATCH_WINDOW
)
# Store news
symbol_data.pending_news.append({
"news_item": data_item,
"evaluation": evaluation,
"time": self.algo.utc_time
})
# Log
headline = self._get_headline(data_item)
self.algo.log(f"📰 [News] {symbol.value}: {evaluation['level_ar']} - {headline[:50]}...")
self.news_processed_today += 1
self.news_windows_opened_today += 1
except Exception as e:
self.algo.error(f"[ProcessNews] Error: {e}")
```
---
### 10. `_check_for_signals()` Function:
```python
def _check_for_signals(self, symbol, symbol_data):
"""Check for alert signals"""
try:
# If stock is in news window
if not symbol_data.is_in_news_window(self.utc_time):
return
# Check cooldown period
if not symbol_data.can_send_alert(self.utc_time):
return
# Send immediate alert when news breaks (once only)
if not symbol_data.news_alert_sent and symbol_data.pending_news:
self._send_news_alert(symbol, symbol_data)
# Check for stock reaction to news
if not symbol_data.reaction_alert_sent:
self._check_reaction_alert(symbol, symbol_data)
except Exception as e:
self.error(f"[CheckSignals] Error for {symbol}: {e}")
```
---
### 11. `_send_news_alert()` Function:
```python
def _send_news_alert(self, symbol, symbol_data):
"""Send immediate news alert"""
try:
news_data = symbol_data.pending_news[0]
news_item = news_data["news_item"]
evaluation = news_data.get("evaluation")
# Extract news details
headline = self._get_headline(news_item)
url = self._get_news_url(news_item)
# Save SPY price at news time
self.spy_price_at_news[symbol] = self.spy_current_price
# Build alert
msg = self.notifier.build_news_alert(
symbol, symbol_data, headline, url, evaluation
)
# Determine priority based on news importance
priority = "critical" if evaluation and evaluation['level'] == "critical" else "high"
self.notifier.queue_alert(msg, priority=priority)
symbol_data.news_alert_sent = True
symbol_data.mark_alert_sent(self.utc_time)
self.log(f"📱 [NewsAlert] {symbol.value} [{evaluation['level_ar'] if evaluation else 'News'}]")
except Exception as e:
self.error(f"[SendNewsAlert] Error for {symbol}: {e}")
```
---
### 12. `_check_reaction_alert()` Function:
```python
def _check_reaction_alert(self, symbol, symbol_data):
"""Check if stock reacted to news"""
try:
self.log(f"⚡ [CheckReaction] {symbol.value} | Price: ${symbol_data.current_price:.2f} | RVOL: {symbol_data.current_rvol:.2f}")
# Defensive protection - check for missing values
if symbol_data.current_price is None or symbol_data.current_price <= 0:
return
# Get news evaluation
evaluation = None
if symbol_data.pending_news:
evaluation = symbol_data.pending_news[0].get("evaluation")
# Get dynamic thresholds
importance_level = evaluation['importance_level'] if evaluation else 2
from helpers import get_time_of_day_period
time_period = get_time_of_day_period(self.time)
thresholds = config.get_dynamic_thresholds(importance_level, time_period)
volume_spike_threshold, rvol_threshold, price_change_threshold = thresholds
# Calculate metrics
has_spike, spike_ratio = symbol_data.check_volume_spike()
rvol = symbol_data.current_rvol
price_change = symbol_data.calculate_price_change_percent()
# Defensive protection - check for NaN
import math
if math.isnan(price_change):
return
# Get adaptive thresholds
adaptive_min_move = symbol_data.get_adaptive_min_move()
adaptive_extreme_vol = symbol_data.get_adaptive_extreme_volatility_threshold()
# Extreme volatility filter (adaptive)
if abs(price_change) > adaptive_extreme_vol:
self.rejected_alerts_stats["extreme_vol"] += 1
self.log(f"[Rejected-ExtremeVol] {symbol.value}: {price_change:.2f}% > {adaptive_extreme_vol:.2f}% (adaptive) - Extreme volatility, ignored")
return
# Minimum move filter (adaptive)
if abs(price_change) < adaptive_min_move:
self.rejected_alerts_stats["min_move"] += 1
return
# Check for Ultra-High-Confidence first (before SPY filter)
is_ultra_high = self._check_ultra_high_confidence(symbol, thresholds)
if is_ultra_high:
# Check cooldown before sending UHC alert
if not symbol_data.can_send_alert(self.utc_time, is_uhc=True):
self.rejected_alerts_stats["cooldown"] += 1
return
# Ultra-High-Confidence alert (all indicators confirmed)
headline, url = None, None
if symbol_data.pending_news:
news_item = symbol_data.pending_news[0]["news_item"]
headline = self._get_headline(news_item)
url = self._get_news_url(news_item)
msg = self.notifier.build_ultra_high_confidence_alert(
symbol, symbol_data, spike_ratio, rvol, price_change,
headline, url, evaluation
)
self.notifier.queue_alert(msg, priority="critical")
symbol_data.reaction_alert_sent = True
symbol_data.mark_alert_sent(self.utc_time)
self.alerts_sent_stats["uhc"] += 1
self.log(f"🚨 [UltraHighConfidence] {symbol.value}: Alert sent!")
return
# SPY filter (after UHC check)
spy_correlation = self._check_spy_correlation(symbol, price_change)
if spy_correlation:
self.rejected_alerts_stats["spy_correlation"] += 1
return
# Regular AND/OR logic (if not Ultra-High)
volume_condition = (has_spike and spike_ratio >= volume_spike_threshold and
rvol >= rvol_threshold)
price_condition = abs(price_change) >= price_change_threshold
if volume_condition and price_condition:
# Send reaction alert
headline, url = None, None
if symbol_data.pending_news:
news_item = symbol_data.pending_news[0]["news_item"]
headline = self._get_headline(news_item)
url = self._get_news_url(news_item)
msg = self.notifier.build_reaction_alert(
symbol, symbol_data, spike_ratio, rvol, price_change,
headline, url, evaluation
)
self.notifier.queue_alert(msg, priority="high")
symbol_data.reaction_alert_sent = True
symbol_data.mark_alert_sent(self.utc_time)
self.alerts_sent_stats["regular"] += 1
self.log(f"📈 [ReactionAlert] {symbol.value}: Alert sent!")
except Exception as e:
self.error(f"[CheckReaction] Error for {symbol}: {e}")
```
---
### 13. `notifier.build_reaction_alert()` and `process_queue()`:
```python
def build_reaction_alert(self, symbol, symbol_data, spike_ratio, rvol, price_change,
headline, url, evaluation):
"""Build reaction alert message"""
msg = (
f"📈 Stock Reaction Alert\n\n"
f"━━━━━━━━━━━━━━━━━━━━\n"
f"📌 Symbol: {symbol.value}\n"
f"💰 Price: ${symbol_data.current_price:.2f}\n"
f"📊 Change: {price_change:+.2f}%\n"
f"📈 Volume Spike: {spike_ratio:.1f}x\n"
f"🔥 RVOL: {rvol:.2f}\n"
f"━━━━━━━━━━━━━━━━━━━━\n"
)
if headline:
msg += f"📰 News: {headline[:100]}\n"
if url:
msg += f"🔗 Link: {url}\n"
msg += f"\n⏰ Time: {self.algo.time.strftime('%Y-%m-%d %H:%M:%S')}\n"
msg += f"📰 News Group"
return msg
def process_queue(self):
"""Process alert queue with priority and rate limiting"""
try:
# Check rate limits (20 messages/minute)
current_time = self.algo.time
# Remove old timestamps (older than 1 minute)
while self.messages_sent_last_minute and \
(current_time - self.messages_sent_last_minute[0]).total_seconds() > 60:
self.messages_sent_last_minute.popleft()
# Check if we can send more messages
if len(self.messages_sent_last_minute) >= self.max_messages_per_minute:
return # Rate limit reached
# Process queues by priority
queues = [
(self.critical_queue, "critical"),
(self.high_queue, "high"),
(self.normal_queue, "normal"),
(self.low_queue, "low")
]
for queue, priority in queues:
if len(queue) > 0:
message = queue.popleft()
# Send to Telegram
success = self._send_to_telegram(message)
if success:
self.alerts_sent_today += 1
self.messages_sent_last_minute.append(current_time)
self.last_message_sent = message
self.last_message_time = current_time
else:
self.alerts_failed_today += 1
# Wait 3 seconds between messages
time.sleep(3)
# Only send one message per call
break
except Exception as e:
self.algo.error(f"[ProcessQueue] Error: {e}")
```
---
## Specific Questions:
### **QUESTION 1: Why does `fine_selection()` reject ALL stocks?**
- Is the problem with `MAX_SHARES_OUTSTANDING = 30M` (too low)?
- Is fundamental data unavailable for most stocks?
- Is there a better way to get `shares_outstanding`?
- Should I use `FreeFloat` instead of `SharesOutstanding`?
### **QUESTION 2: Is there a timing issue with Universe Selection?**
- Logs show `coarse_selection()` is called at `02:00:00` (2 AM EST)
- Is this correct? Should it be called at `00:00-00:30 AM`?
- Why does it take 5 hours from initialize (00:41) to coarse_selection (06:00)?
### **QUESTION 3: Why doesn't ObjectStore work?**
- Logs show: `[ObjectStore] Loaded 585 stocks`
- But then: `[Initialize] No saved list - will wait for Universe Selection`
- What condition prevents using the loaded stocks?
- Is there a return value check missing?
### **QUESTION 4: Is there an error in `_get_shares_outstanding()`?**
- Are the methods used (`CompanyReference.SharesOutstanding`, `EarningReports.BasicAverageShares`, etc.) correct?
- Is there a more reliable way to access this data?
- Should I log which method succeeds for debugging?
### **QUESTION 5: What are reasonable values for `MAX_SHARES_OUTSTANDING`?**
- Current value: 30M shares
- Is this too restrictive for QuantConnect data?
- What percentage of stocks have shares_outstanding <= 30M?
- Should I increase to 100M or 500M?
### **QUESTION 6: How can I diagnose the problem more accurately?**
- What additional logs should I add?
- Should I print `shares_outstanding` and `market_cap` for first 50 stocks?
- Is there a way to check data availability before filtering?
- Should I add a fallback filter if fundamental data is missing?
---
## Ultimate Goal:
I want the system to work as follows:
1. **First Run:** System selects 900 stocks (Low Float) via Universe Selection
2. **Second Run:** System loads list from ObjectStore immediately (1-2 minutes)
3. **When News Breaks:** Send immediate alert via Telegram
4. **When Stock Reacts:** Send second alert based on RVOL and price movement
---
## Additional Notes:
- System runs in **Live Mode** on QuantConnect
- Using **Benzinga News** as news source
- Target: **Micro-cap stocks** (<$100M market cap) with **Low Float** (<30M shares outstanding)
- Old version worked, but after modifying code to use Universe Selection, it stopped adding stocks
- The complete flow from stock appearance to user alert is described above
---
## Please Help With:
1. Identify the **root cause** of why all stocks are rejected in `fine_selection()`
2. Suggest **practical solutions** to fix the problem
3. Explain **best practices** for using Universe Selection with Fundamental Data in QuantConnect
4. Clarify how to **diagnose** the problem more accurately (additional logs, values to print, etc.)
5. Confirm if the **complete scenario flow** I described is correct for QuantConnect
Thank you very much
مطلق الشمري
The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.
To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!