Question for Mia

Here is a clear explanation and solution for your issue, 'مطلق':

1. Why is fundamental_history empty in Live Mode?

• Timing of Data Availability: In Live Mode, when initialize() runs, no fundamental files are available yet. QuantConnect downloads and processes fundamental data only after you add a symbol or universe and the first data event actually happens.
• Meaning of self.time in initialize(): At this point, self.time is basically just the “deployment time” of your algorithm—not a market day, not a real-time data moment. There is no lookback data loaded.
• Reference times: reference_time = self.time - timedelta(days=1) is not going to return valid market data during initialize(), because the data pipeline isn’t loaded yet!

Result:
When you call self.history(...) in initialize() for Fundamentals (or even price data), it will return an empty DataFrame because no data has been loaded at all.

2. Does self.history(Fundamental, ...) work in Live Mode?

• Yes, but not in Initialize: After your universe is created and symbols subscribed, fundamental history will become available for those symbols. (Your QC organization must have the proper entitlements.)
• No Special API Required: The same History API is used.
• No Special Permissions Usually Required: Regular US Fundamental data is included with most subscriptions.
• But: You must wait until after data subscriptions are established, i.e., do not call it in initialize() for universe-wide fundamentals.

3. Recommended approach for immediate universe loading in Live Mode

• Don’t try in initialize(): Immediate bootstrapping of universe using history/fundamental data in initialize is not possible in Live mode.
• Alternatives:
  • Static List: If you need stocks right away, hardcode an S&P500 or similar static list until fundamentals are ready.
  • Universe Selection (Recommended): Use add_universe() with a coarse or fine selector. The fine selector receives FineFundamental objects with all fundamental fields, and is only called when the data is ready (typically after 7am ET in Live mode).
  • Debug in Selector: Add logging inside your selectors to confirm when data is available.
  • Scheduled Safety: Schedule a callback for shortly after market-open (with self.schedule.on()) to call history—by then, data is always loaded.
  • on_securities_changed: Use this method to know when your chosen symbols are officially in your universe.

4. How to debug and log availability?

• Log in Selector: Print self.time, and fundamental fields on a sample symbol in your fine selector—if it prints, universe data has arrived.
• Use on_securities_changed: Log each new security added/removed; you can then request price/fundamental history for those symbols (it will work now).
• Scheduled Callback: Use self.schedule.on() to request history after a delay (ex: 10 minutes after market open) and print length of result.

Sample Best-Practice Code Pattern

def initialize(self):
    self.set_start_date(2021, 1, 1)
    self.set_cash(100_000)
    self.universe_settings.resolution = Resolution.DAILY
    self.add_universe(self.coarse, self.fine)
    self.schedule.on(
        self.date_rules.every_day(),
        self.time_rules.after_market_open("SPY", 10),
        self._request_history_safely
    )

def coarse(self, coarse):
    # Pick top 200 liquid stocks with fundamental data
    return [x.symbol for x in sorted([c for c in coarse if c.has_fundamental_data], key=lambda c: c.dollar_volume, reverse=True)[:200]]

def fine(self, fine):
    qualified = [f for f in fine if f.valuation_ratios.pe_ratio and 0 < f.valuation_ratios.pe_ratio < 40]
    self.debug(f"{self.time} | First symbol PE: {qualified[0].symbol if qualified else 'None'}")
    return [f.symbol for f in sorted(qualified, key=lambda f: f.market_cap, reverse=True)[:20]]

def on_securities_changed(self, changes):
    for security in changes.added_securities:
        self.debug(f"{self.time} | Added {security.symbol} - now requesting history.")
        bars = self.history(security.symbol, 30, Resolution.DAILY)
        self.debug(f"{len(bars)} daily bars for {security.symbol}")

def _request_history_safely(self):
    spy = self.add_equity("SPY").symbol
    bars = self.history(spy, 10, Resolution.DAILY)
    self.debug(f"Safe history request returned {len(bars)} bars for {spy}")

Summary Table

Bottom Line:
In Live Mode, you must wait until after universe selection before asking for fundamental data or history. Use logging in your universe selectors and on_securities_changed to verify when data is ready.

If you need further clarification or code snippets, let me know!

Sources: Fundamental Regression Algorithm, Fundamental Universes - QuantConnect.com, US Fundamental Data - QuantConnect.com, Corporate Fundamentals - QuantConnect.com, US Equity Coarse Universe - QuantConnect.com