I've been trying to access files from Github via the SubscriptionDataSource. The idea behind this is to pipe some data like CSVs but that are hosted on Github rather than Dropbox. Because this specific data is private, I cannot make the repository that contains them public. For testing purposes I've created a dummy repo with dummy data.

I've attached a backtest and noticed a few things when attempting to pull the files from github.

  1. Including (uncommenting) the "headers" in the SubscriptionDataSource will make the code fail, on both public and private repositories.
  2. Fetching a private repository via the traditional Github API headers fails in SubscriptionDataSource, but succeeds when done via the self.Download() method with headers, for both public and private repositories. This leads me to believe that the SubscriptionDataSource headers are buggy.

I've put the file public at

so that you could test it out too with your own Github token.

Of course the github token in the backtest is disabled.

Is this intended behavior? If so, what would be other possibilities to pipe data into the OnData method for thousands of tickers daily?