Help mia

# QuantConnect Bot Question - OnSecuritiesChanged Not Completing

## Problem Summary

**Issue:** `on_securities_changed()` appears to hang/freeze after adding stocks and never completes execution.

**Symptoms:**
- ✅ Initialize completes successfully
- ✅ 585 stocks loaded from ObjectStore
- ✅ 586 stocks added (585 + SPY)
- ✅ Telegram startup message sent successfully
- ✅ Benzinga subscription added for each stock
- ❌ **No completion message appears**
- ❌ **No final summary log**
- ❌ **System appears frozen**

**Even after disabling `_calculate_average_volume()`, the problem persists!**

---

## What We See in Logs

### Successful Steps:
```
2025-10-04 19:19:41 : [Initialize] ✅ Loaded 585 stocks from ObjectStore
2025-10-04 19:19:41 : [Initialize] ✅ Immediate loading complete! System ready.

2025-10-04 19:19:41 : [TelegramNotifier] Message sent successfully!

2025-10-04 19:19:41 : OnSecuritiesChanged at 2025-10-04 15:19:32: Added 586, Removed 0
2025-10-04 19:19:41 : [OnSecuritiesChanged] تحديث #1 | مضاف: 586 | محذوف: 0
2025-10-04 19:19:41 : [OnSecuritiesChanged] أضيف AAME + Benzinga
2025-10-04 19:19:41 : [OnSecuritiesChanged] أضيف ABTS + Benzinga
... (continues for all 586 stocks)
2025-10-04 19:19:41 : [OnSecuritiesChanged] أضيف APWC + Benzinga
```

### What's Missing:
```
❌ [OnSecuritiesChanged] ✅ تم الانتهاء من التحديث
❌ [OnSecuritiesChanged] إجمالي الأسهم النشطة: 586
❌ [OnSecuritiesChanged] ✅ النظام جاهز
❌ [OnSecuritiesChanged] 🎉 التحميل الأولي مكتمل
```

**The function never reaches the end!**

---

## Complete Code

### 1. `on_securities_changed()` - Full Function

```python
def on_securities_changed(self, changes):
"""
V7.2 محسّن: معالجة تغييرات Universe
يُستدعى عند:
- التحميل الأولي (بعد Fine Selection)
- التحديث الأسبوعي
"""
try:
added_count = len(changes.added_securities)
removed_count = len(changes.removed_securities)

self.universe_updates_count += 1

# V7.2: رسالة محسّنة (توصية البوت)
self.log(f"OnSecuritiesChanged at {self.time}: Added {added_count}, Removed {removed_count}")
self.log(f"[OnSecuritiesChanged] الوقت: {self.time}")
self.log(
f"[OnSecuritiesChanged] تحديث #{self.universe_updates_count} | "
f"مضاف: {added_count} | محذوف: {removed_count}"
)

# إضافة الأسهم الجديدة
for added in changes.added_securities:
symbol = added.symbol

# تخطي SPY (تمت إضافته يدوياً في initialize)
if symbol == self.spy_symbol:
continue

# إنشاء SymbolData
self.symbol_data_dict[symbol] = SymbolData(self, symbol)

# الاشتراك في Benzinga
try:
self.add_data(BenzingaNews, symbol)
self.log(f"[OnSecuritiesChanged] أضيف {symbol.value} + Benzinga")
except Exception as e:
self.error(f"[OnSecuritiesChanged] فشل اشتراك Benzinga لـ {symbol}: {e}")

# حساب متوسط الحجم
# self._calculate_average_volume(symbol) # ← DISABLED FOR TESTING

# إزالة الأسهم القديمة
for removed in changes.removed_securities:
symbol = removed.symbol

# عدم إزالة SPY أبداً
if symbol == self.spy_symbol:
continue

if symbol in self.symbol_data_dict:
# تنظيف البيانات
self.symbol_data_dict[symbol].dispose()
del self.symbol_data_dict[symbol]

self.log(f"[OnSecuritiesChanged] حُذف {symbol.value}")

# حفظ القائمة المحدثة في ObjectStore
if added_count > 0 and config.OBJECT_STORE_ENABLED:
try:
# استخراج الرموز (بدون SPY)
symbols_to_save = [
s.value for s in self.symbol_data_dict.keys()
if s != self.spy_symbol
]
symbols_str = ','.join(symbols_to_save)

# الحفظ
self.object_store.save(config.OBJECT_STORE_KEY, symbols_str)
self.log(f"[OnSecuritiesChanged] تم حفظ {len(symbols_to_save)} سهم في ObjectStore")
except Exception as e:
self.error(f"[OnSecuritiesChanged] فشل الحفظ في ObjectStore: {e}")

# V7.3.9: رسالة تأكيد نهائية
if added_count > 0:
self.log("============================================================")
self.log(f"[OnSecuritiesChanged] ✅ تم الانتهاء من التحديث #{self.universe_updates_count}")
self.log(f"[OnSecuritiesChanged] إجمالي الأسهم النشطة: {len(self.symbol_data_dict)}")
self.log(f"[OnSecuritiesChanged] الاشتراك في Benzinga: {added_count}")

# عينة من الأسهم المضافة
sample_symbols = [s.value for s in list(self.symbol_data_dict.keys())[:10]]
self.log(f"[OnSecuritiesChanged] عينة من الأسهم: {', '.join(sample_symbols)}...")

self.log(f"[OnSecuritiesChanged] ✅ النظام جاهز لاستقبال الأخبار ومراقبة الأسهم")
self.log("============================================================")

# إرسال رسالة Telegram عند اكتمال التحميل الأولي
if self.universe_updates_count == 1 and self.is_live_mode:
self.log("[OnSecuritiesChanged] 🎉 التحميل الأولي مكتمل!")
self.log(f"[OnSecuritiesChanged] تم تحميل {len(self.symbol_data_dict)} سهم بنجاح")
self.log("[OnSecuritiesChanged] النظام الآن جاهز لمراقبة الأخبار وإرسال التنبيهات")

self.log("[OnSecuritiesChanged] Preparing Telegram alert for completion...")

# إرسال تنبيه Telegram
completion_msg = (
f"✅ التحميل الأولي مكتمل!\n\n"
f"📊 عدد الأسهم: {len(self.symbol_data_dict)}\n"
f"🔔 النظام جاهز لمراقبة الأخبار"
)

self.log("[OnSecuritiesChanged] Queueing completion alert...")
self.notifier.queue_alert(completion_msg, priority="high")

self.log("[OnSecuritiesChanged] Processing queue...")
self.notifier.process_queue()

self.log("[OnSecuritiesChanged] ✅ Telegram alert queued and processed")

except Exception as e:
self.error(f"[OnSecuritiesChanged] خطأ: {e}")
import traceback
self.error(traceback.format_exc())
```

---

### 2. `_calculate_average_volume()` - Currently Disabled

```python
def _calculate_average_volume(self, symbol):
"""حساب متوسط الحجم اليومي"""
try:
history = self.history(symbol, config.RVOL_LOOKBACK, Resolution.DAILY)

if not history.empty and 'volume' in history.columns:
avg_volume = history['volume'].mean()
if symbol in self.symbol_data_dict:
self.symbol_data_dict[symbol].set_average_volume(avg_volume)
if config.DEBUG_MODE:
from helpers import safe_format
avg_vol_str = safe_format(avg_volume, ".0f", log_warning=True, algo=self, field_name=f"{symbol.value}_avg_volume")
self.debug(f"[CalculateAverageVolume] {symbol.value} | ADV: {avg_vol_str}")
except Exception as e:
self.error(f"[CalculateAverageVolume] خطأ لـ {symbol}: {e}")
```

---

### 3. `SymbolData.__init__()` - Constructor

```python
class SymbolData:
def __init__(self, algo, symbol):
self.algo = algo
self.symbol = symbol

# بيانات السعر
self.current_price = None
self.previous_close = None
self.percent_change = 0.0

# بيانات الحجم
self.current_volume = 0
self.average_volume = 0
self.current_rvol = 0.0

# بيانات الأخبار
self.pending_news = []
self.news_window_start = None
self.news_window_end = None

# حالة التنبيهات
self.news_alert_sent = False
self.reaction_alert_sent = False
self.last_alert_time = None

# بيانات إضافية
self.session_volume = 0
self.session_start_time = None
```

---

### 4. `add_data(BenzingaNews, symbol)` - QuantConnect API

This is a built-in QuantConnect method that subscribes to Benzinga news for a symbol.

```python
# Usage:
self.add_data(BenzingaNews, symbol)
```

**Potential Issues:**
- Does it block execution?
- Does it have a limit on number of subscriptions?
- Does it fail silently for some symbols?

---

## Questions for the Bot

### Question 1: Why does `on_securities_changed()` not complete?

**Evidence:**
- Logs show all 586 stocks being added
- Logs show Benzinga subscription for each stock
- But the function never reaches the final logging section
- Even with `_calculate_average_volume()` disabled!

**Possible causes:**
1. Does `add_data(BenzingaNews, symbol)` block for 586 symbols?
2. Is there a limit on Benzinga subscriptions?
3. Does QuantConnect have a timeout for `on_securities_changed()`?
4. Is there a memory issue with 586 SymbolData objects?

---

### Question 2: Does `add_data()` have performance implications?

**Our usage:**
```python
for added in changes.added_securities: # 586 iterations
self.add_data(BenzingaNews, symbol) # Called 586 times
```

**Questions:**
- Is this the correct approach?
- Should we batch the subscriptions?
- Is there a maximum number of data subscriptions?
- Does it cause the function to hang?

---

### Question 3: Are there any hidden exceptions?

**Our try-except:**
```python
try:
# ... all the code ...
except Exception as e:
self.error(f"[OnSecuritiesChanged] خطأ: {e}")
import traceback
self.error(traceback.format_exc())
```

**But no error appears in logs!**

**Questions:**
- Are there exceptions that don't get caught?
- Does QuantConnect have internal timeouts?
- Can `on_securities_changed()` be terminated silently?

---

### Question 4: Is there a log limit?

**Observation:**
- We log 586 times: `[OnSecuritiesChanged] أضيف {symbol} + Benzinga`
- Logs appear to show all 586 (or at least many of them)
- But then nothing after that

**Questions:**
- Does QuantConnect have a log message limit?
- If we exceed the limit, does execution stop?
- Should we reduce logging?

---

### Question 5: What is the recommended approach?

**Current approach:**
```python
for added in changes.added_securities:
# Create SymbolData
self.symbol_data_dict[symbol] = SymbolData(self, symbol)

# Subscribe to Benzinga
self.add_data(BenzingaNews, symbol)

# Log
self.log(f"Added {symbol}")
```

**Questions:**
- Is this correct for 586 stocks?
- Should we use a different approach?
- Are there performance best practices?
- Should we subscribe to Benzinga differently?

---

### Question 6: Market hours impact?

**Context:**
- Market is currently closed
- Does this affect `on_securities_changed()`?
- Does it affect `add_data(BenzingaNews, symbol)`?

**Questions:**
- Should we only run this during market hours?
- Does QuantConnect behave differently when market is closed?
- Could this cause the hang?

---

## Configuration

### From `config.py`:

```python
# Universe Selection
UNIVERSE_SIZE = 900
COARSE_UNIVERSE_SIZE = 4500

# Price Filters
MIN_PRICE = 1.0
MAX_PRICE = 50.0

# Market Cap Filters
MIN_MARKET_CAP = 5_000_000 # $5M
MAX_MARKET_CAP = 100_000_000 # $100M

# Float Filters
MAX_SHARES_OUTSTANDING = 30_000_000 # 30M shares

# Volume Filters
MIN_DOLLAR_VOLUME = 1_000_000 # $1M daily

# ObjectStore
OBJECT_STORE_ENABLED = True
OBJECT_STORE_KEY = "target_symbols_list"

# RVOL
RVOL_LOOKBACK = 20 # days
```

---

## System Info

- **Platform:** QuantConnect LEAN Engine v2.5.0.0.17309
- **Mode:** Live Trading (Paper)
- **Stocks:** 586 (585 + SPY)
- **Data Source:** Benzinga News
- **Issue:** `on_securities_changed()` does not complete

---

## What We Need

### Diagnostic Steps:

1. **Why does the function not complete?**
- Is there a hidden exception?
- Is there a timeout?
- Is there a resource limit?

2. **Is our approach correct?**
- Should we subscribe to Benzinga differently?
- Should we reduce logging?
- Should we batch operations?

3. **How can we debug this?**
- What additional logging should we add?
- Are there QuantConnect limits we should know about?
- What is the recommended pattern for 500+ stocks?

---

## Expected Behavior

### What should happen:

```
[OnSecuritiesChanged] Added 586, Removed 0
[OnSecuritiesChanged] أضيف AAME + Benzinga
... (586 times)
[OnSecuritiesChanged] تم حفظ 585 سهم في ObjectStore
============================================================
[OnSecuritiesChanged] ✅ تم الانتهاء من التحديث #1
[OnSecuritiesChanged] إجمالي الأسهم النشطة: 586
[OnSecuritiesChanged] ✅ النظام جاهز
============================================================
[OnSecuritiesChanged] 🎉 التحميل الأولي مكتمل!
[OnSecuritiesChanged] ✅ Telegram alert queued and processed
```

### What actually happens:

```
[OnSecuritiesChanged] Added 586, Removed 0
[OnSecuritiesChanged] أضيف AAME + Benzinga
... (586 times)
← STOPS HERE, NEVER COMPLETES
```

---

## Request

Please help us understand:
1. Why `on_securities_changed()` doesn't complete
2. If our approach is correct for 586 stocks
3. What diagnostic logging we should add
4. If there are QuantConnect limits or best practices we're missing

Thank you!

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

def on_securities_changed(self, changes): """ V7.2 محسّن: معالجة تغييرات Universe يُستدعى عند: - التحميل الأولي (بعد Fine Selection) - التحديث الأسبوعي """ import datetime import time try: started_at = time.time() self.log(f"[OnSecuritiesChanged] ⏰ Started at {datetime.datetime.now()}") added_count = len(changes.added_securities) removed_count = len(changes.removed_securities) self.universe_updates_count += 1 self.log(f"OnSecuritiesChanged at {self.time}: Added {added_count}, Removed {removed_count}") self.log(f"[OnSecuritiesChanged] الوقت: {self.time}") self.log(f"[OnSecuritiesChanged] تحديث #{self.universe_updates_count} | مضاف: {added_count} | محذوف: {removed_count}") batch_size = 50 # ⏱️ configurable batch size for progress logs # إضافة الأسهم الجديدة for i, added in enumerate(changes.added_securities): symbol = added.symbol if symbol == self.spy_symbol: continue # إنشاء SymbolData self.symbol_data_dict[symbol] = SymbolData(self, symbol) # الاشتراك في Benzinga try: self.add_data(BenzingaNews, symbol) # ⏳ progress log if (i + 1) % batch_size == 0: self.log(f"[OnSecuritiesChanged] ⏳ Added {i + 1}/{added_count} securities at {datetime.datetime.now()}") self.log(f"[OnSecuritiesChanged] أضيف {symbol.value} + Benzinga") except Exception as e: self.error(f"[OnSecuritiesChanged] فشل اشتراك Benzinga لـ {symbol}: {e}") self.log(f"[OnSecuritiesChanged] 🟢 Finished additions at {datetime.datetime.now()} (elapsed: {int(time.time() - started_at)}s)") # إزالة الأسهم القديمة for removed in changes.removed_securities: symbol = removed.symbol if symbol == self.spy_symbol: continue if symbol in self.symbol_data_dict: self.symbol_data_dict[symbol].dispose() del self.symbol_data_dict[symbol] self.log(f"[OnSecuritiesChanged] حُذف {symbol.value}") self.log(f"[OnSecuritiesChanged] 🔴 Finished removals at {datetime.datetime.now()} (elapsed: {int(time.time() - started_at)}s)") # حفظ القائمة المحدثة في ObjectStore if added_count > 0 and config.object_store_enabled: try: symbols_to_save = [s.value for s in self.symbol_data_dict.keys() if s != self.spy_symbol] symbols_str = ','.join(symbols_to_save) self.object_store.save(config.object_store_key, symbols_str) self.log(f"[OnSecuritiesChanged] تم حفظ {len(symbols_to_save)} سهم في ObjectStore") except Exception as e: self.error(f"[OnSecuritiesChanged] فشل الحفظ في ObjectStore: {e}") # ⌛ log right before summary / Telegram section self.log(f"[OnSecuritiesChanged] ⌛ Preparing summary at {datetime.datetime.now()} (total elapsed: {int(time.time() - started_at)}s)") # V7.3.9: رسالة تأكيد نهائية if added_count > 0: self.log("============================================================") self.log(f"[OnSecuritiesChanged] ✅ تم الانتهاء من التحديث #{self.universe_updates_count}") self.log(f"[OnSecuritiesChanged] إجمالي الأسهم النشطة: {len(self.symbol_data_dict)}") self.log(f"[OnSecuritiesChanged] الاشتراك في Benzinga: {added_count}") sample_symbols = [s.value for s in list(self.symbol_data_dict.keys())[:10]] self.log(f"[OnSecuritiesChanged] عينة من الأسهم: {', '.join(sample_symbols)}...") self.log(f"[OnSecuritiesChanged] ✅ النظام جاهز لاستقبال الأخبار ومراقبة الأسهم") self.log(f"[OnSecuritiesChanged] 🏁 Finished everything at {datetime.datetime.now()} (total elapsed: {int(time.time() - started_at)}s)") self.log("============================================================") if self.universe_updates_count == 1 and self.is_live_mode: self.log("[OnSecuritiesChanged] 🎉 التحميل الأولي مكتمل!") self.log(f"[OnSecuritiesChanged] تم تحميل {len(self.symbol_data_dict)} سهم بنجاح") self.log("[OnSecuritiesChanged] النظام الآن جاهز لمراقبة الأخبار وإرسال التنبيهات") self.log("[OnSecuritiesChanged] Preparing Telegram alert for completion...") completion_msg = ( f"✅ التحميل الأولي مكتمل!\n\n" f"📊 عدد الأسهم: {len(self.symbol_data_dict)}\n" f"🔔 النظام جاهز لمراقبة الأخبار" ) self.log("[OnSecuritiesChanged] Queueing completion alert...") self.notifier.queue_alert(completion_msg, priority="high") self.log("[OnSecuritiesChanged] Processing queue...") self.notifier.process_queue() self.log("[OnSecuritiesChanged] ✅ Telegram alert queued and processed") else: # 🏁 final log even when no additions happened self.log(f"[OnSecuritiesChanged] 🏁 Finished everything at {datetime.datetime.now()} (total elapsed: {int(time.time() - started_at)}s)") except Exception as e: self.error(f"[OnSecuritiesChanged] خطأ: {e}") import traceback self.error(traceback.format_exc())

Mia Alissi

STAFF ,

Here’s an enhanced diagnostic version of your on_securities_changed() function. This version adds precise timestamped logs at key checkpoints (start, progress during additions, after all additions, after all removals, before summary, and overall elapsed time). This will help pinpoint exactly where the method hangs or slows down, and will reveal if a timeout or silent halt occurs during bulk Benzinga subscriptions.

How to Use:

Deploy this as your diagnostic version.
Carefully watch the log timestamps to see:
- If logs suddenly stop (detect where—likely on specific subscription count or operation).
- If removals, summary, and Telegram finish.
- How long each phase takes.
You can change batch_size to 10 or 100 for finer control.
If execution breaks after a certain number, suspect resource limits, timeouts, or log quota issues.

If you still don’t reach the summary logs:

Next Diagnostic Step: Comment out the self.add_data(BenzingaNews, symbol) line and rerun. If the function then completes, this confirms the hang is due to the mass Benzinga subscriptions.

Summary of Documentation Findings:

add_data(BenzingaNews, symbol) is meant for individual symbols; no bulk/limitations listed in docs for universe-wide subscription.
QuantConnect notes per-symbol news feed usage in all docs/examples, not explicit about hard subscription limit.
Benzinga News claims coverage for up to 8000 equities.
There is no official documentation on a log message limit or on-slice event handler timeout, but community threads report silent terminations if too much logging or if a handler takes too long.

Best Practices:

Minimize log statements in high-frequency loops (e.g., only batch logs).
Batch or stagger news feed subscriptions if facing unexplained slowdowns/hangs.
Use try/except around all risky operations.
If you experience silent stalls or missing logs, reach out to QuantConnect support with a minimal reproducible example (especially if you confirm the problem is add_data scaling with 586+ symbols).

Let me know what your logs show with this diagnostic version!

Sources: docs writing algorithms datasets securities and exchange commission us sec filings, CNBC Trading - QuantConnect.com, Event Handlers - QuantConnect.com, Coarse Fine Universe Selection Benchmark

مطلق الشمري INVESTOR

Update Backtest

Notebook

person upvoted this people upvoted this

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

TOP 5 Research Publications

444,200 Quants.

VOTE FOR UPCOMING FEATURES

Help mia

Allocate to this Strategy

Organization

Team

Clone Strategy

Previous Ranking

IN THIS RESEARCH

PARTICIPANTS

Discussion Awards

Actions

Join QuantConnect for Free

SIGN IN

Radically Open-Source Algorithmic Trading Engine

Join Our Discord Channel

Draft Discussions

Bookmarked Discussions

SEARCH DISCUSSIONS

TOP 5 Research Publications

444,200 Quants.

VOTE FOR UPCOMING FEATURES

Help mia

Allocate to this Strategy

Organization

Team

Clone Strategy

Previous Ranking

IN THIS RESEARCH

PARTICIPANTS

Discussion Awards

SHARE RESEARCH

SHARE DISCUSSION

SHARE ARTICLE

SHARE

Actions

Join QuantConnect for Free