Loading custom CSV data by Sushil kumar

Sushil kumar

| Reward Discussion

How do I analyze custom data (lets say a csv file based on earning reports or some other events) using python (pandas/numpy) and generate buy/sell signals and backtest ?

A. Can I analyse the data in quantconnect so I don't have to uplaoda csv file and it could work in real time

B. If I analyse the data outside (lets say google colab), generate a CSV file with buy and sell indicator. Now can I upload to quantconnect and backtest it?

This is not the strategy I am considering but just to give an idea lets say I have earning report from some source which I downlaoded (or used api) and processed with python/pandas (on google colab or quantconnect) and created a csv like this.

Date Symbol EPS. Rating. Buy_indicator. Sell_indicator. Position

2020-1-1 APPL 23 Good. 1 0 1

2020-1-2 GOOG 12 Good 1 0 1

2020-04-1 APPL 30 Good 1 0 2

2020-04-2 GOOG 11 Bad 0 1 1

2020-07-1 APPL 25 Neutral 0 0 2

2020-07-2 GOOG 10 Bad 0 0 0

Now I want to the load the above csv data (if not created within quantconnect) in Quantconnect and backtest the strategy using buy/sell/positon indicators.

Update Backtest

Sushil kumar

Accepted Answer

Update Backtest

Notebook

The material on this website is provided for informational purposes only and does not constitute an offer to sell, a solicitation to buy, or a recommendation or endorsement for any security or strategy, nor does it constitute an offer to provide investment advisory services by QuantConnect. In addition, the material offers no opinion with respect to the suitability of any security or specific investment. QuantConnect makes no guarantees as to the accuracy or completeness of the views expressed in the website. The views are subject to change, and may have become unreliable for various reasons, including changes in market conditions or economic circumstances. All investments involve risk, including loss of principal. You should consult with an investment professional before making any investment decisions.

Dirk bothof

618 ,

easiest way is to upload your file to dropbox, in your algoritme you can read-in the file and do what is required to trade.

https://www.quantconnect.com/docs/algorithm-reference/importing-custom-data

Shile Wen

60.2k ,

Hi Sushil,

Following up on what Dirk said, the link he shared shows how to stream data real-time through Dropbox.As for using custom indicators from a csv, the same link also explains how to import a static file, which you can do with the following:

csv = self.Download("https://www.dropbox.com?....&amp;dl=1")

# read file (which needs to be a csv) to a pandas DataFrame. include following imports above

# from io import StringIO

# import pandas as pd

df = pd.read_csv(StringIO(file))

Best,
Shile Wen

AM H

552 ,

Hi Shile

I have successfuly ingested data from Dropbox using the methods in the NIFTY example. I have a problem in passing the new symbol/ticker data to a list. For example my script currently contains this code:

self.tickers = ["VTI", "TLT", "GLD"]

        for ticker in self.tickers:

            self.AddEquity(ticker, Resolution.Daily)

How can I make a list such as this with my AddData items? For example I initialise the data through:

self.AddData(GoldPhys, "IGLN.L", Resolution.Daily).Symbol

self.AddData(Treas20, "IDTL.L", Resolution.Daily).Symbol

self.AddData(VanSPY, "VDNR.L", Resolution.Daily).Symbol

GoldPhys = self.AddData(GoldPhys, "IGLN.L", Resolution.Daily).Symbol

Treas20 = self.AddData(Treas20, "IDTL.L", Resolution.Daily).Symbol

VanSPY = self.AddData(VanSPY, "VDNR.L", Resolution.Daily).Symbol

and my custom classes like this:

class GoldPhys(PythonData):

    '''IGLN.L Custom Data Class'''

    def GetSource(self, config, date, datafeed):

        return SubscriptionDataSource("https://www.dropbox.com/s/pqwv2psx3qeysl1/VDNR.csv?dl=0", SubscriptionTransportMedium.Rest)





    def Reader(self, config, line, date, datafeed):

        if not (line.strip() and line[0].isdigit()): return None



        # New GoldPhys object

        index = GoldPhys()

        index.Symbol = config.Symbol



        try:

            # Example File Format:

            # Date,       Open       High        Low       Close     Volume

            # 2011-09-13  7792.9    7799.9     7722.65    7748.7    116534670

            data = line.split(',')

            index.Time = datetime.strptime(data[0], "%Y-%m-%d")

            index.Value = data[4]

            index["open"] = float(data[1])

            index["high"] = float(data[2])

            index["low"] = float(data[3])

            index["close"] = float(data[4])







        except ValueError:

                # Do nothing

                return None



        return index

But I cannot successfuly pull this data into an list of symbols/tickers via the old code here. How can I do this?

self.tickers = ["VTI", "TLT", "GLD"]

        for ticker in self.tickers:

            self.AddEquity(ticker, Resolution.Daily)

Thanks!

Shile Wen

60.2k ,

Hi Am H,

This was answered in this thread.

Best,
Shile Wen

Sushil kumar

Update Backtest

Notebook

To unlock posting to the community forums please complete at least 30% of Boot Camp.
You can continue your Boot Camp training progress from the terminal. We hope to see you in the community soon!

Platform

QuantConnect Community

Alpha League Competition: $1,000 Weekly Prize Pool

Filter Discussion by Tags

Community Ambassadors

Earn free QuantConnect cloud credits by helping others, each week we airdrop credits to the most active community members.

188,129 Quants.

Online Now

Stay Connected