Help mia

Here’s diagnostic code you can adapt to your algorithm to precisely identify why your universe selection only returns ~152 instead of 900, and where your filter is most restrictive. This will help answer "Why only 152 stocks", and will inform how to adjust your criteria and fallback strategies:

# Place this diagnostic logic inside your own class/methods, or use as a new algorithm for testing.

class DiagnosticUniverseAlgorithm(QCAlgorithm):
    """QCAlgorithm with extensive diagnostic logging in universe selection."""

    def initialize(self):
        # Standard backtest setup
        self.set_start_date(2021, 1, 1)
        self.set_end_date(2022, 1, 1)
        self.set_cash(100_000)

        # --- Tune these to match your own ---
        self.min_price = 1.0
        self.max_price = 50.0
        self.max_shares_outstanding = 30_000_000
        self.min_market_cap = 5_000_000
        self.max_market_cap = 100_000_000
        self.universe_size = 900
        self._last_logged_week = -1
        self.add_universe(self.coarse_selection, self.fine_selection)

    def coarse_selection(self, coarse):
        coarse_list = list(coarse)
        if not coarse_list: return []
        init_count = len(coarse_list)
        liquid = [c for c in coarse_list if c.has_fundamental_data and self.min_price <= c.price <= self.max_price and c.dollar_volume > 10000]
        liquidity_count = len(liquid)
        sorted_by_dv = sorted(liquid, key=lambda c: c.dollar_volume, reverse=True)
        selected = sorted_by_dv[:10000]
        self.log(f"[CoarseSelection] {init_count}→{liquidity_count} (price/liquidity) → {len(selected)} sent to fine-selection.")
        return [c.symbol for c in selected]

    def fine_selection(self, fine):
        fine_list = list(fine)
        pre_count = len(fine_list)
        # Diagnostics
        n_price, n_shares, n_cap = 0, 0, 0
        passed = []
        for f in fine_list:
            price = getattr(f, 'price', None)
            shares = None
            try:
                shares = f.earning_reports.basic_average_shares.three_months
            except Exception:
                pass
            mcap = getattr(f, 'market_cap', None)
            if not (price and self.min_price <= price <= self.max_price):  n_price += 1; continue
            if not (shares and shares <= self.max_shares_outstanding):   n_shares += 1; continue
            if not (mcap and self.min_market_cap <= mcap <= self.max_market_cap): n_cap += 1; continue
            passed.append((f.symbol, price, shares, mcap))

        passing_count = len(passed)
        # For traceability:
        self.log(f"[FineSelection {str(self.time)[:10]}] FineIn:{pre_count}, n_price_fail:{n_price}, n_shares_fail:{n_shares}, n_cap_fail:{n_cap}, FinalPassing:{passing_count}")

        # Print a summary of the final filtered universe's market cap, price, shares
        if passing_count:
            prices = [x[1] for x in passed]
            shares = [x[2] for x in passed]
            mcaps = [x[3] for x in passed]
            self.log(f"[Summary] Price: min={min(prices):.2f}, max={max(prices):.2f}, mean={sum(prices)/len(prices):.2f}")
            self.log(f"[Summary] SharesOut: min={min(shares):.0f} max={max(shares):.0f} mean={sum(shares)/len(shares):.0f}")
            self.log(f"[Summary] MCap: min={min(mcaps):.0f} max={max(mcaps):.0f} mean={sum(mcaps)/len(mcaps):.0f}")
        else:
            self.log("[Summary] No stocks passed all fine filters!")

        # Take only required number
        top = passed[:self.universe_size]
        return [x[0] for x in top]

How this helps you:

• You'll see, at every weekly refresh, exactly how many stocks fail on price, shares, and market cap.
• You'll get summary statistics which can show if, for instance, almost all failing stocks are failing due to your low float cap or your market cap or liquidity restriction.
• You'll see the distribution in your survivors to decide whether your thresholds are realistic.

Next Steps To Diagnose & Fix:

1. Run this code in your environment for several weeks of data. Check logs to answer:
   • Are most stocks failing the price, shares, or market cap filter?
   • Is your 10,000 “coarse” pool (sent to fine) enough, or do you need to increase?
2. If a single constraint is limiting, consider relaxing it (e.g. set MAX_SHARES_OUTSTANDING = 40_000_000), rerun, and compare logs—a jump in counts will show it’s a constraint.
3. ObjectStore/refresh check: You’ll see logs every week or whenever the selection runs, so you’ll know the selection is actually being refreshed; you can also add a forced “self.log” inside whatever flag refreshes your selection process.

Fallback mechanism example (if fewer than 900 found):

if len(passed) < self.universe_size:
    self.log(f\"Warning: Only {len(passed)} stocks available after applying all filters, less than requested {self.universe_size}\")
    # Optionally: Relax the LEAST CRITICAL constraint here, e.g. increment MAX_SHARES_OUTSTANDING and retry.

Best Practices for News-Sensitive, Low-Float Stock Selection

• Always diagnose & tune using summaries as above.
• When very low prevalence, keep a fallback—e.g. relax “max float” in increments until enough candidates found.
• Overlay additional news/scanner universes to assure coverage when not enough microcaps.
• Track descriptive stats so you can prove your universe actually covers the universe you intend to alert on.

Summary:
Use explicit diagnostic logging + stats to see exactly why so few stocks pass and tune your logic. Always check constraint-by-constraint exclusion, and implement fallback logic for low-population periods. This will make your ObjectStore and weekly refreshes robust and transparent.

Let me know if you want ready-to-merge code with fallback/relaxation logic as well!

Sources: Coarse Fundamental Top3 Algorithm, Coarse Fine Fundamental Combo Algorithm, Fundamental Universes - QuantConnect.com, Key Concepts - QuantConnect.com, Universe Selection Definitions Algorithm