Input shape not accepted but its correct

Back

Hi everyone, Im trying to make a simple algorithm with keras work, so far it trains fine, but when I try to predict some value, it throws me a shape error, only that when I print to the debugger the shape it appears to be correct. Heres the code and the output.

for security in changes.AddedSecurities:
symbol = security.Symbol
new_feature = np.array(algorithm.History(security.Symbol,15, Resolution.Daily)["open"])
algorithm.Debug(f" Its shape is {new_feature.shape}")

self.mom[symbol] = self.model.predict(new_feature)
algorithm.Debug(f"Prediction for {symbol} is {self.mom[symbol]}")

The output in the debugger:

9 | 11:52:31:
Runtime Error: ValueError : Error when checking input: expected dense_1_input to have shape (15,) but got array with shape (1,)
at OnSecuritiesChanged in main.py:line 63
ValueError : Error when checking input: expected dense_1_input to have shape (15,) but got array with shape (1,) (Open Stacktrace)
10 | 11:52:41:
Algorithm Id:(200821177eb8c61af0f08a86be772a20) completed in 27.77 seconds at 0k data points per second. Processing total of 742 data points.
11 | 11:52:41:
Its shape is (15,)

Do you have any idea why this maight happen?

Update Backtest







 
0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


import clr
clr.AddReference("System")
clr.AddReference("QuantConnect.Algorithm")
clr.AddReference("QuantConnect.Common")

from System import *
from QuantConnect import *
from QuantConnect.Algorithm import *

import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.optimizers import SGD
from sklearn.preprocessing import MinMaxScaler
from datetime import timedelta

class MOMAlphaModel(AlphaModel):
# se puede hacer una knn aca adentro
def __init__(self):
self.mom = {}

self.model = Sequential()
self.model.add(Dense(10))
self.model.add(Activation('relu'))
self.model.add(Dense(1))


def OnSecuritiesChanged(self, algorithm, changes):
self.TrainNN(changes.AddedSecurities, algorithm)

for security in changes.AddedSecurities:
symbol = security.Symbol
new_feature = np.array([algorithm.History(security.Symbol,15, Resolution.Daily)["open"]])
algorithm.Debug(f"{new_feature}")
algorithm.Debug(f" its shape is {new_feature.shape}")
try:
self.mom[symbol] = self.model.predict(new_feature, batch_size=1)
algorithm.Debug(f"Prediction for {symbol} is {self.mom[symbol]}")
except Exception as e:
algorithm.Debug(f"{e}")
pass

def TrainNN(self, data, algorithm):
Xs = []
Ys = []

if len(data) == 0:
return None

for sec in data:
#price = [last 10 weeks prices]
if len(algorithm.History(sec.Symbol,16, Resolution.Daily)['open']) >= 16:
#Ys.append(algorithm.History(sec.Symbol,16, Resolution.Daily)["open"][0] / algorithm.History(sec.Symbol,16, Resolution.Daily)["open"][1] - 1)
Xs.append(algorithm.History(sec.Symbol,15, Resolution.Daily)["open"])

if len(Xs) >= 1:
for X in Xs:
Ys.append(X[0] / X[1] - 1)
Xs = np.array(Xs) #shape: (number of secs , 15 )
Ys = np.array(Ys) #shape: (number of secs, )

sgd = SGD(lr = 0.05) # learning rate = 0.01

self.model.compile(loss='mse', optimizer=sgd)

cost = self.model.fit(x=Xs, y=Ys, batch_size=None, epochs=10)


def Update(self, algorithm, data):

ups = []
downs = []

for i in self.mom:
if self.mom[i] > 0.01:
ups.append(Insight.Price(i, timedelta(days = 1), InsightDirection.Up))
elif self.mom[i] < -0.01:
downs.append(Insight.Price(i, timedelta(days = 1), InsightDirection.Down))
else:
pass

algorithm.Debug(f"Produced {len(ups)+len(downs)} insights")
return Insight.Group(ups + downs)


class FrameworkAlgorithm(QCAlgorithm):

def Initialize(self):
self.SetStartDate(2014, 10, 1)
self.SetEndDate(2014, 12, 1)
self.SetCash(100000)

#symbols = [Symbol.Create("SPY", SecurityType.Equity, Market.USA), Symbol.Create("BND", SecurityType.Equity, Market.USA)]
self.UniverseSettings.Resolution = Resolution.Daily
self.AddUniverse(self.MyCoarseFilterFunction, self.MyFineFundamentalFunction)

self.SetAlpha(MOMAlphaModel())

self.SetPortfolioConstruction(EqualWeightingPortfolioConstructionModel())
self.SetRiskManagement(MaximumDrawdownPercentPerSecurity(0.02))

#1. Set the Execution Model to an Immediate Execution Model
self.SetExecution(ImmediateExecutionModel())
#self.Debug(f"I hold these secs: {[sec.Symbol for sec in self.Securities]}")

def MyCoarseFilterFunction(self, coarse):
sortedByDollarVolume = sorted(coarse, key=lambda x: x.DollarVolume, reverse=True)
filtered = [ x.Symbol for x in sortedByDollarVolume if x.HasFundamentalData ]
return filtered[:80]

def MyFineFundamentalFunction(self, fine):
sortedByPeRatio = sorted(fine, key=lambda x: x.ValuationRatios.PERatio, reverse=False)
return [ x.Symbol for x in sortedByPeRatio[:20] ]

Full code

0

Hi Guillermo Bondonno 

Sorry about the wait.
In fact, one of the history requests from line 33 does not return 15 days because the security doesn't have 15 days of data. You are not catching it with the Debug statement because the debug messages are rate limited.
I would suggest adding a check for the size of the data frame to address the issue:

history = algorithm.History(security.Symbol,15, Resolution.Daily)
if len(history.index) < 15:
continue
new_feature = np.array([history.open])

 

0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Update Backtest





0

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.


Loading...

This discussion is closed