Call for Design Input - Documentation V2

Hello i am new, it would be nice that when you use search bar in documentation you find the code and examples so you know how to use different functions/lines/code etc, that for a beginner will help a lot.

Hi Jared, what I noticed is that a lot of (new) people, me included, often post in the Community asking questions that have been answered numerous times already. I usually do this when, after looking through the documentation, I wasn't able to find a solution. Sometimes searching or googling my questions yields a useful post in the forums, but other times other people have described the problem differently, or worded them differently,  so I can't find them but the idea behind them is the same. Maybe if you could somehow incorporate Community posts with accepted answers into common topics in the documentation using the posts' tags to connect the post to the relevant topic? Or some other way. The idea sounds messy in my head, but maybe there's a neat way of doing it. 

Things to improve...

- as long as you want to do something fairly simple (one symbol, with one technique, etc.) the documentation is great. However, when different methods start to interact with each other----this is where I at least am at a loss. I would suggest a library of different examples of varying complexity that are clearly indexed by which methods they use and combine.  These examples should be listed on the relevant documentation pages. ---this is already the case, but there are too few examples and it's not very clear what is in each example, i.e. its just "name.py"

Good luck

It'd just be great if the documentation was accurate and upto date....

- More complex uses of the framework namespaces for each different segment. For instance, there is an example for Immediate Exeuction that imports QuantConnect.Orders but other than that one example I wouldn't know that existed or what it did.  I've tried using help() and dir() in the ipynotebook research but those descriptions aren't complete either.  Some attributes are intuitive but others not as much. I would love to know everything that's at my disposal. 

Thanks for asking Jared.

The existing documentation is very good for basic elements and functions, such as adding an indicator, OnSchedule, SetHoldings etc.

Since I consider myself a beginner programmer, I feel it is missing consistent strategy examples that I can reuse pieces of easily to construct something new with. For example if a sample strategy uses the newer Alpha Model, and another doesn't, I don't know how to utilize code snippets from the two to combine then into something I want.

Overall, organization of all reference and learning material is really needed.

For example BootCamp is only accessible in the Lab.

The Strategy Library I can only get to by going to the Community and doing a word search, then clicking on a link in the search results or by finding a forum post from someone at QC with a link for it. If I go to the Docs upper navigation link, then the left side navigation on the Docs Home page doesn't have Turotials or Strategy Libary included.

Also there is the GitHub link for example strategies, I think I got a link for that from someone in the forums too.

Overall this requires extra searching effort on the user to find a solution. To sum up, if I go to Docs/Help, I should have access to everything from there in the sub menu in the left nav.

1. I only started using Quantconnect in January 2020, so still really new to the platform. I knew a coding in VBA only before coming to use Python here, one thing I have struggled with is finding useful examples of the topics discussed in the documentation. It would be helpful to have backtests included or even just more examples of how certain features are coded. I struggled for a while to figure out how to use .of function (ended up having to post a question in the community, took two days for a response, glad I go one, but big delay for something that was so simple). 

2. Search bar (already mentioned above) would be so helpful

3. Q&A section for commonly asked questions with the correct solution posted. If a community post results in a solution (and the originator indicates that a solution has been posted) that filters to your staff to add to the Q&A section. reduces the clutter in the community, saves people time, more efficient site.

Thank you all for the comments. We'll try our best to incorporate them into the work we're doing now. Thank you Hector Barrio for joining the effort =)

I have tried to write algorithms in python. And there are a number of issues that have gotten in the way. Could you consider the following?

• Have a separate python-only section in the documentation. There's a lot about writing algorithms in python that simply would not apply to C#. For instance how to use pandas well, and how to optimize the algorithm's execution time.
• How is backtesting implemented? How is minute resolution work compared to daily resolution? This should be in at least enough detail so that we can figure out what would happen if a limit order was entered for an algorithm running on daily resolution and the target price was reached inter-day. And what would happen if a large order was entered such that it could not be filled given the observed volume for the security for the day.
• Document what happens in the service when an algorithm runs. This would be really useful for those of us that write software for a living and can make use of this information for optimizing algorithm efficiency.
• The API seems to have "accumulated" over time. There's the basic non-algorithm framework based approach. Then there's the algorithm framework based approach. And the two are not well separated, especially in the documentation. There are many aspects of the general algorithm API that don't appear to be available when using the algorithm framework.
• Document the behavior of the different algorithm framework elements.
• I've been using the framework, and quite a bit of the lower level API doesn't seem to be available. I'm not sure why I should be using the algorithm framework. The bootcamp uses the algorithm framework, so it appears you're encouraging us to use it. But so many examples in the documentation don't use the framework. This creates confusion.

You are welcome Jared,

I would like to chip in and comment on a topic that I perceive as recurring in the forum: the framework/non-framework approach. It seems to me that the non-framework approach serves to quickly set up a working algorithm from low-level components (the algorithm methods if you will) and the documentation illustrates this; how components work at a detailed level. 

The framework approach provides a higher-level API interface that abstracts many of the lower-level components; prevents repetition, allows for global modification of components, and keeps interfaces under control. The difference is very well illustrated by setting up a Rebalance/SetHoldings function combination vs. managing the emission timing for insights inside an alpha model and a portfolio construction model. The latter is much more verbose and complex, both are covered in the documentation.

The documentation shows both levels, low-level components, and high-level 'subsystems' assembled from these components that allow combination, adaptation, and configuration of a final 'product', a framework algorithm. The documentation, in my opinion, explains the components (methods) and the system (the framework) correctly and, apparently, fails to join the concepts properly, fails to "convince" the users to develop in the framework, fails to convince the user to take the extra work of defining additional components of the system today to (potentially) use them tomorrow, to reuse them, to locally modify them, to maintain them...

My personal experience is in systems engineering and the framework approach presented by the Quantconnect team is one of the best applied systems engineering examples I have seen, both OS and CS code. It also improves and benefits the development of the OS code, as development and user interfacing become more aligned. For sure it causes additional up-front work that will needs to be absorbed.

I would recommend users to learn both the parts and the framework and try to develop in-framework from day 1 even if they are not planning to enter the alpha market.

Thank you for taking the time to improve the docs.

I would like to see the QC API Reference held to at least the same standard as the Numpy reference. Take a random doc like hermfit. It's presented very well. It has

• A very good description
• All parameters are described completely. They don't leave cases up to interpretation. Contrast to QC's self.SetEndDate(end) which doesn't say what happens when end is missing. Just a simple, obvious example but others aren't. For additional information we usually need to search the tutorials or community posts.
• There are simple examples for almost everything, with some context. I've found that when teaching people or learning myself, even a simple usage example provides some confidence and lifts a small cognitive load.
• The language is excellent. The first sentence in any description (including arguments) is short and complete. It then progressively digs deeper. This addresses all levels of experience without slowing anyone down.
• The notes include usage tips, warnings and advice. These are invaluable.
• Versioning. Old posts and tutorials may not match the current docs so it would be great to have a version number to cross-reference.

I do feel the documentation overall is mostly adequate, and I especially like the various tutorials.

It's not necessarily documentation but having some kind of update notice or something when changes are made in the backend would be nice.  

Hi Jared and team, thanks for willing to improve the docs

1) annoys you the most

• It can be confusing and time-consuming to find the best practice about anything. It is even more confusing with the split between algo/non algo framework.

As an example with warming up and using indicators with alpha framework, there is multiple example showing different ways of doing it (using .Reset or history calls, consolidation, etc). But I have no clue which one would be the best and why. After searching or asking the forum for hours I can start getting a better idea of what to do.

• Does not seem always updated

2) what we're missing

• Clear examples of codes that are using consistent and rigorous ways of coding the most efficiently would be amazing.
• A proper library of strategies that would be updated regularly, a sort of benchmark for example about the best ways of doing things. With enough diversity to be able to build anything as a beginner. (with filtering features like the type of framework, markets, type of strategy, original author or book, etc)
• How to go live trading without making any mistake ? Kind of checklist or recommendation according to brokers, account type, etc

3) what you love about the current documentation

General organization of it, a good overview of the alpha, and how every part of the puzzle fit together.

Hi Jared,

I really appreciate that you want to improve the QC documentation. I think that the current situation is the biggest weakness of this project.

1) annoys you the most

• Parts of the documentation (or valuable information) are in different places. There is something on GitHub, something in the official documentation and other things in the forum.

2) what we're missing

• Lack of examples. The documentation describes the functions and classes only very marginally and completely lacks practical examples of use. I suggest you create a series of articles, always with one main topic. One topic and many examples.

You can be inspired by this site: Getting Started – QuantConnect

• FAQ - This should be extended to the QC framework and other parts.

3) what you love about the current documentation

• Clarity of documentation and division into sections.

I recommend Read the Docs:

https://readthedocs.org/

We built our documentation for AlphaPy with it.

https://alphapy.readthedocs.io/en/latest/

Hi Jared, 

I also want to take this opportunity and provide feedback. 

1) What annoys you most

• Information is scattered across different places like Docs, Forum, GitHub, Slack etc. It would be helpful to have a central place (-> Docs) from which we can get all the information needed. 

• Search also should list results for related terms. Example: search for "python modules" instead of "python libraries" shows the following output --> Link.

• Commonly asked questions in forum should be addressed in Docs. Either as a comprehensive FAQ or as a How-to-Guide broken down by topic or section. 

• Some chapters seems to be outdated or does not match with the associated instructions from other sources like GitHub. Example: Open Source > Debugging Python vs. GitHub > LEAN > Debugging Python

• Recent developments not always included in Docs. Example: self.Settings.RebalancePortfolioOnInsightChanges and self.Settings.RebalancePortfolioOnSecurityChanges (only found in forum)

2) What is missing

• Best Practices / Do's and Dont's. Example: Avoid to use History requests frequently, because of .... Use RollingWindow or deque instead. 

• Provide more guidance on the question "Framework vs. Non-Framework".  When to use what? I personally prefer Framework style. Hector already mentioned some good points here. However, I think there are scenarios where the Non-Framework approach could be the preferred way, especially for beginners since the initial hurdles are lower. 

• Separate chapter on the subject "Backtesting Biases and Pitfals and how to avoid them" (Overfitting, Lookahead-Bias, Survivorship-Bias etc.)

• more detailed explanations for LEAN Docs (George mentioned the Numpy API reference, which I think is a good example)

3) What I love about the current documentation

• The documentation is good as it is. We're not talking about making bad documentation good, but making a good documentation even better. ;-)

• The hurdles to get started are incredibly low. You can create an algo and run a backtest within a few minutes. This is especially helpful for beginners. 

• Every chapter starts with an overview, provides some context and then progressively digs deeper. 

• Wide range of examples (demo algos on GitHub, StrategyLibrary) and additional guidance in the form of bootcamp tutorials and videos. (There is a saying, "A picture is worth a thousand words." Well, a video is ~30 pictures per second. So for a 10 min video we have 10*60*30 = 18 thousand words ;-) )

Hi Jared, 

I am not sure that my message will help much, as previous posters already described most of what I thought, and explained the concepts better than I could!

Will just try to add my 2-cents:
 

1) What annoys you most

It is somewhat difficult to get all the needed information between the Docs (1), Github (2), the Forum (3), the Slack (4) and in email updates (5): my search is done in that order from my experience, but probably that this order depends on each person!

It can be even harder to be sure we know everything that is available when the subject has not been publicly detailled much, as that is currently the case of Future options for instance. 

Being able to search for everything at the same place would at time avoid wasting our time and wasting yours, as I am pretty sure that many messages sent to the QC's team could have been answered just by a thorough and efficient search on the same centralized place.

2) What is missing

1. As people noted above, the discrepancy between Framework and Non-Framework, when using it or not... has been a headeache for me: I learnt how to use the Framework model just because at one point it was required to send Alphas. Now, I keep using it when I belive it is worthwhile, but because I am not a computer scientist, I am still far from being sure that I really use it when it should be, or not. 

Arthur Asenheimer described very well in his message above what could help us (non CS experts) in being better QC programmers! 

2. I would have loved to be able to see all built-in methods and all properties described thoroughly somewhere: sometimes, I had to make some trials to be sure that the properties are the good ones and work as intented.


3. I also think that a much bigger "Common errors" would have been very useful when I began, like this example:

3) What I love about the current documentation

I think that the current documentation is much better than it was just a couple of years ago: you guys worked well, adding much more information, links to tutorials, videos, and so much more!

Keep doing the good work, by improving everything over time as you have done previously!

Hi, it would ideal if it was COMPREHENSIVE. Current documentation is a short book, but it should be at least 5-10 times its current size. This will enable us to help ourselves instead of asking questions every other minute. Also, would be ideal if the bootcamps were comprehensive. They are too short and should be at least 4 times their current length. :)

Eg, when trading futures, FuturesChain only can be used in OnData. Fortunately this was mentioned somewhere deep in the forums, but this kind of detail should be in the documentation or the bootcamps.

1. Pandas and Numpy are the bread-and-butter of Python analysis, follow their lead and the barrier to entry will be very low. Just by killing the context switching alone will make eveyone much more effecient in their workflows, much more effecient in their ability to absorb the documentation, i.e. they don't have to learn how to read your docs. Scikit-learn docs are ok, but not on-par with the previous 2 mentioned.
2. Make Numpy documentation style a standard for Lean Python, This will have other flow on effects...
3. Also, pick a standard for your Python code (such as PEP8), make this standard then apply to all example code. There are many linters avaiable that you can run as part of your devops processes to ensure items are passing. Note: Python devs are not .NET devs, Using CamelCase everywhere is a pain in the ass within Python code examples, it only gets used for ClassNames and everything else is snake_case, basically.
4. ... implement Numpy docstring standard in your Python code examples. Pandas also follow the Numpy docstring standard. Overhauling documentation should include the code itself (via docstrings, and having a standard to adhere to), as it is the source of truth for what gets executed.
5. Follow conventions like what Numpy and Pandas have done for imports in any code examples in your documentation: "import numpy as np", and "import pandas as pd". Then the namespacing in code examples is explicit (and therefore faster to learn from, and easier to maintain in the long run) => "import QuantConnect as qc". Explicit is good. It's fast to scan and udnerstand quickly (scan > reading > interpreting spaghetti). Make code examples scannable.
6. Further to API referencing, in any code examples don't do "from module.sub_module import *". I now have no sense of hierarchy when reading code and trying to understand things. This makes it especially hard when modularising the code (e.g. Alpha model is in its own file such as "Alpha.py", Universe model is in it's own file such as "Universe.py"). see my gripe below why this is important.
7. The API reference within the algo lab needs to be broken down by namespace. It's not explicit enough to just run thorugh the API namespace heirarchy and find things fast.
8. Gripe: Like I've seen mention further up, information is scattered. Blogs, forum, docs, code. Even the search functionality of QC is incosistent. I find the level of context switching extremelly high, and find myself spending upwards of 80% of my time tracking down disparate info sources, hacking code togther, then running a backtest just to debug. This is a major bottleneck, and even at times a deterrent from using the platform.
9. I like the ability to "Clone" from the forum, put this in the documentation! When you have worked examples, provide a "Clone" option! That would be awesome.
10. Search: Add last updated date, and posted date to the search results. Digging into a forum post and seeing it is old code is painful.
11. Functionality of the search itself needs improvement. Why can't I enter a search term in the search box at the top of this page, and just hit enter to go to the search results? Why can I only select what is in the shortened list (I never find what I'm looking for there either btw)? In-site search needs a major overhaul imo. It is the entrypoint to the docs, so to ignore overhauling it would be a mistake. We have all been google-fied for the past 20 years and expect to find things < 2 seconds.
12. Off-site search (e.g. Google), needs optimising, and can be done via the suggestions above. If I search for "pandas dataframe" in Google, the very first result takes me directly to the API docs I want/need immediately. The same thing happens for "pd.dataframe", "np.array" By having explicit namespacing in your docs, and example code, and adopting a convention such as "qc.Algorithim", for example, I should be able to find what I'm looking for within the first result (instead apparently I'm looking for genetic algorithims in this search result).

I love what Quantconnect has become, I've been watching it for many years now, playing around as new features have been released, but typically deterred by the level of context switching involved with the documentation practices. The Algorithm Framework is clean, I love what it is.

Well done taking on redesign of the docs, I appreciate it is a major undertaking, and any improvements will be well received by all I'm sure :) 

If I can add more context to my previous comment with my use case:

• I am trying to implement a custom indicator for a Clenow's momentum in an alpha model at the moment
• Found nothing obvious about this in the documentation
• I did a Google search "custom alpha indicator quantconnect"
• I found a few answers : 

  https://www.quantconnect.com/forum/discussion/10015/alpha-model-with-custom-indicator/p1

  https://www.quantconnect.com/forum/discussion/9976/custom-indicators-history-in-alpha-framework

  https://www.quantconnect.com/forum/discussion/3383/custom-indicator-in-python-algorithm/p1
  
  https://backtest-rookies.com/2019/01/25/quantconnect-create-an-indicator-awesome-oscillator/

• I need to spend time analyzing the answers, find which one is still working after lean updates, with the framework, etc
  
  -> Best doc would have been searching for custom indicator, finding a page about it in 1s, going to the alpha version of it, having everything explained with a perfect example that I know 100'% is the most updated and efficent way to do it