Hi All, 

Not sure if this is a bug or I missing a switch in the command:

lean data generate --start 20000101 --symbol-count 1 --security-type Equity --resolution Daily --include-coarse TRUE --market USA --update

this command generates extra columns in addition to Date+OHLC that is causing the “(String '1002900' was not recognized as a valid DateTime.)” error, feel free to correct me if I'm wrong.

here's the error:

20220511 16:29:20.521 ERROR:: CoarseUniverseGeneratorProgram.Run(): CoarseUniverseGeneratorProgram.Run(): FAILED! System.AggregateException: One or more errors occurred. (String '1004300'
was not recognized as a valid DateTime.) (String '1002900' was not recognized as a valid DateTime.)
 ---> System.FormatException: String '1004300' was not recognized as a valid DateTime.
   at QuantConnect.ToolBox.CoarseUniverseGenerator.CoarseUniverseGeneratorProgram.ParseDailyFile(FileInfo dailyFile) in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/ToolBox/CoarseUniverseGenerator/CoarseUniverseGeneratorProgram.cs:line 333
   at QuantConnect.ToolBox.CoarseUniverseGenerator.CoarseUniverseGeneratorProgram.<>c__DisplayClass9_0.<Run>b__0(SecurityIdentifierContext sidContext) in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/ToolBox/CoarseUniverseGenerator/CoarseUniverseGeneratorProgram.cs:line 164
   at System.Threading.Tasks.Parallel.<>c__DisplayClass44_0`2.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__DisplayClass44_0`2.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica`1.ExecuteAction(Boolean& yieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica.Execute()
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.TaskReplicator.Run[TState](ReplicatableUserAction`1 action, ParallelOptions options, Boolean stopOnFirstFailure)
   at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner`1 source, ParallelOptions parallelOptions, Action`1 simpleBody, Action`2 bodyWithState, Action`3
bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.ThrowSingleCancellationExceptionOrOtherException(ICollection exceptions, CancellationToken cancelToken, Exception otherException)
   at System.Threading.Tasks.Parallel.PartitionerForEachWorker[TSource,TLocal](Partitioner`1 source, ParallelOptions parallelOptions, Action`1 simpleBody, Action`2 bodyWithState, Action`3
bodyWithStateAndIndex, Func`4 bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body, Action`2 bodyWithState, Action`3 bodyWithStateAndIndex, Func`4bodyWithStateAndLocal, Func`5 bodyWithEverything, Func`1 localInit, Action`1 localFinally)
   at System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable`1 source, ParallelOptions parallelOptions, Action`1 body)
   at QuantConnect.ToolBox.CoarseUniverseGenerator.CoarseUniverseGeneratorProgram.Run() in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/ToolBox/CoarseUniverseGenerator/CoarseUniverseGeneratorProgram.cs:line 138
 ---> (Inner Exception #1) System.FormatException: String '1002900' was not recognized as a valid DateTime.
   at QuantConnect.ToolBox.CoarseUniverseGenerator.CoarseUniverseGeneratorProgram.ParseDailyFile(FileInfo dailyFile) in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/ToolBox/CoarseUniverseGenerator/CoarseUniverseGeneratorProgram.cs:line 333
   at QuantConnect.ToolBox.CoarseUniverseGenerator.CoarseUniverseGeneratorProgram.<>c__DisplayClass9_0.<Run>b__0(SecurityIdentifierContext sidContext) in
/LeanCloud/CI.Builder/bin/Debug/src/QuantConnect/Lean/ToolBox/CoarseUniverseGenerator/CoarseUniverseGeneratorProgram.cs:line 164
   at System.Threading.Tasks.Parallel.<>c__DisplayClass44_0`2.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Parallel.<>c__DisplayClass44_0`2.<PartitionerForEachWorker>b__1(IEnumerator& partitionState, Int32 timeout, Boolean& replicationDelegateYieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica`1.ExecuteAction(Boolean& yieldedBeforeCompletion)
   at System.Threading.Tasks.TaskReplicator.Replica.Execute()<---

20220511 16:29:20.523 TRACE:: RandomDataGeneratorProgram(): Press any key to exit...

 

this how the generated data looks like in the csv: (C:\LeanCLI\data\equity\usa\daily)

177660_1652287054.jpg

Please help…

Thanks