Trouble building

[dakr001]

Hi,

Hoping you can help please.

Have added the references are per the instructions. There are many dlls of the same name in different folders. Not sure if it matters which one I point to.

I'm getting the error below which suggests an issue with the reference?
source\repos\Lean-Batch-Launcher-master\Algorithm\BasicTemplateFrameworkAlgorithm.cs(3,7,3,19): error CS0246: The type or namespace name 'QuantConnect' could not be found (are you missing a using directive or an assembly reference?)

Also, it appears this code relies on .NET 4.5.2 but the the current Lean implementation is based on a later .NET version. Not sure if this matters. Got the following message:
warning MSB3274: The primary reference "QuantConnect.Algorithm" could not be resolved because it was built against the ".NETFramework,Version=v4.6.2" framework. This is a higher version than the currently targeted framework ".NETFramework,Version=v4.5.2".

Thank you.

[Doggie52]

Hi there. I'm unfortunately no longer actively working on this project.

Your first error suggests you haven't set the references right, but in light of the second error, it seems to me that the overall issue is indeed that the project is built against NET 4.5.2 and QC has moved onto 4.6.2.

The project is quite small and so I don't think there should be an issue to upgrade to 4.6.2 - you're welcome to submit a PR for this if you can get it to work!

[dakr001]

Hi @Doggie52 ,

I have zero experience coding in C# but if it's simple enough I'm happy to give it a try. QC looks to be a good platform so it's a shame there is no working version of a parameter search functionality. There Lean Optimizer here doesn't appear to work either, at least for Python.

Are you able to give pointers as to the first steps to upgrade to 4.6.2?

Thanks.

[Doggie52]

Fundamentally, the process could be as simple as loading up the project in VS and switching the target NET version in the solution. Or, alternatively, modifying the target version in the actual project file. That should probably be all you need to do - if any issues compiling come up, it's likely some incompatibility that needs to be addressed. But again, given the size of the project, I don't foresee any issues.

[dakr001]

Thanks @Doggie52 . Have progressed further and fixed up a couple of other issues after googling. Now have 3 succeeded, 1 failed. Stuck with the following errors and didn't get much from googling. Are there any further hints/help you can provide please?

error CS1747: Cannot embed interop types from assembly 'Algorithm, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' because it is missing the 'System.Runtime.InteropServices.GuidAttribute' attribute.

4>CSC : error CS1759: Cannot embed interop types from assembly 'Algorithm, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' because it is missing either the 'System.Runtime.InteropServices.ImportedFromTypeLibAttribute

A couple of warnings preceded these errors:

3>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2081,5): warning MSB3277: Found conflicts between different versions of "System.Net.Http" that could not be resolved. These reference conflicts are listed in the build log when log verbosity is set to detailed.

3>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(3198,5): warning MSB3327: Unable to find code signing certificate in the current user’s Windows certificate store. To correct this, either disable signing of the ClickOnce manifest or install the certificate into the certificate store.

[Doggie52]

First Google result I get is to clean and rebuild the solution. Try deleting everything in the bin and obj folders too, and rebuilding.

[dakr001]

Thanks @Doggie52

No luck unfortunately. Have included the full output below. I'll try to fiddle around over the weekend....

Also there are many dlls with the same name in the Lean directory. Does it matter which one I reference, in case this might have caused the isssue?

1>------ Build started: Project: Util, Configuration: Debug Any CPU ------
2>------ Build started: Project: Algorithm, Configuration: Debug Any CPU ------
1>C:\Users\username\source\repos\Lean-Batch-Launcher-master\Util\ThreadSafe.cs(49,29,49,30): warning CS0168: The variable 'e' is declared but never used
1>  Util -> C:\Users\username\source\repos\Lean-Batch-Launcher-master\Util\bin\Debug\Util.dll
2>  Algorithm -> C:\Users\username\source\repos\Lean-Batch-Launcher-master\Algorithm\bin\Debug\Algorithm.dll
3>------ Build started: Project: Launcher, Configuration: Debug Any CPU ------
3>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2081,5): warning MSB3277: Found conflicts between different versions of "System.Net.Http" that could not be resolved.  These reference conflicts are listed in the build log when log verbosity is set to detailed.
3>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(3198,5): warning MSB3327: Unable to find code signing certificate in the current user’s Windows certificate store. To correct this, either disable signing of the ClickOnce manifest or install the certificate into the certificate store.
3>  Launcher -> C:\Users\username\source\repos\Lean-Batch-Launcher-master\Launcher\bin\Debug\Launcher.exe
4>------ Build started: Project: Instance, Configuration: Debug Any CPU ------
4>C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Current\Bin\Microsoft.Common.CurrentVersion.targets(2081,5): warning MSB3277: Found conflicts between different versions of "System.Net.Http" that could not be resolved.  These reference conflicts are listed in the build log when log verbosity is set to detailed.
4>CSC : error CS1747: Cannot embed interop types from assembly 'Algorithm, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' because it is missing the 'System.Runtime.InteropServices.GuidAttribute' attribute.
4>CSC : error CS1759: Cannot embed interop types from assembly 'Algorithm, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null' because it is missing either the 'System.Runtime.InteropServices.ImportedFromTypeLibAttribute' attribute or the 'System.Runtime.InteropServices.PrimaryInteropAssemblyAttribute' attribute.
========== Build: 3 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========

[Doggie52]

Which .dll you reference should not matter, although I tend to go for the one that's in the directory with the same name.

Have you tried this answer, also on a Google search? https://stackoverflow.com/a/3320279/2449750

Thank you for looking into this, it's a shame it doesn't work out of the box.

[dakr001]

Hi @Doggie52

Thanks for the tip, that helped! The solution build ran successfully. However ran into a wierd error when I ran the launcher (F5). There seems to be an error getting the data:

20200913 22:49:32.111 Trace:: ApiDataProvider.Fetch(): Attempting to get data from QuantConnect.com's data library for symbol(EURUSD), resolution(Minute) and date(1/2/2018). 20200913 22:49:33.231 ERROR:: ApiConnect.TryRequest(): Content: 20200913 22:49:33.262 ERROR:: ApiConnect.TryRequest(): Content: 20200913 22:49:33.270 ERROR:: ZipDataCacheProvider.Fetch(): Inner try/catch System.NullReferenceException: Object reference not set to an instance of an object. at QuantConnect.Api.Api.DownloadData(Symbol symbol, Resolution resolution, DateTime date) in C:\Users\UserName\Source\Repos\Lean\Api\Api.cs:line 632 at QuantConnect.Lean.Engine.DataFeeds.ApiDataProvider.Fetch(String key) in C:\Users\UserName\Source\Repos\Lean\Engine\DataFeeds\ApiDataProvider.cs:line 71 at QuantConnect.Lean.Engine.DataFeeds.ZipDataCacheProvider.CacheAndCreateStream(String filename, String entryName) in C:\Users\UserName\Source\Repos\Lean\Engine\DataFeeds\ZipDataCacheProvider.cs:line 207 at QuantConnect.Lean.Engine.DataFeeds.ZipDataCacheProvider.Fetch(String key) in C:\Users\UserName\Source\Repos\Lean\Engine\DataFeeds\ZipDataCacheProvider.cs:line 81

The data in batch.config.json and data-start-date-by-symbol.json is in 2014 as per my Lean data.

Have included the debug output and relevant files. If you have any ideas please let me know...

debug_output.txt
batch_config_json.txt
data_start_date_by_symbol.txt

[Doggie52]

Does the algorithm you're trying to run work fine in standard Lean? I.e. outside of my launcher? Because this error looks like one that is unrelated to the launcher. If it's gotten as far as attempting to get data, it's gotten past the launcher stage and is now inside Lean.

[dakr001]

@Doggie52 sorry for the late reply. Lean broke and I got busy with my day job.

It doesn't appear the BasicTemplateFrameworkAlgorithm algorithm that came with the batch launcher works with Lean. I copied the algorithm into Leans' Algorithm.CSharp directory then hardcoded the EMA settings and it failed to build.

The original BasicTemplateFrameworkAlgorithm that came with Lean worked fine.

Lean's build log is below but not sure how difficult it is to fix this one up?

log_20200920.txt

[Doggie52]

If the algo that came with this batch launcher doesn't work but the LEAN one does, then just use that. From the algo that comes with this launcher you can pick up how the configuration entries are passed in and how to use them in the algo, which is the important part anyway. I suspect the reason the algo doesn't work on your machine is because you don't have the necessary subscriptions in your QuantConnect data feed.

[dakr001]

Thanks @Doggie52 . Still stuck but keen to keep going since we got this far...

1. If the data exists on the local drive, do I still need to worry about subscription?

2. Are all the parameters in batch.config.json mandatory, e.g. StartDate, Duration, MinuteResolutions; or can they be hardcoded in the alpha model and removed from batch.config.json?

3. There seems to be different parameter names in batch.config.json compared to BasicTemplateFrameworkAlgorithm that came with the batch launcher. Examples below:

In config file: batch.config.json has "StartDate", "Symbols", "AlphaModelNames"
BasicTemplateFrameworkAlgorithm calls Config.Get("LBL-start-date"), Config.Get("LBL-symbol"), Config.Get("LBL-alpha-model-name")

Are these typos and the parameters need to match?

Thank you so much.

[Doggie52]

Hey, thank you for sticking with it!

1. You shouldn't need to, no, but it's been a while since I played with Lean and can't quite recall. Your errors looked like they were related to download so I assumed it had to do with a subscription.
2. Can't quite say for sure whether they are mandatory but you can certainly hardcode them in the alpha, yes. I mean you don't need to use LBL-start-date as the start date for your strategy, you can use whatever you want and leave StartDate alone in the config. The only thing to note is that the LBL uses the inputs in the config to determine how many instances of Lean to spawn, so make sure you set these to something that won't spawn a bunch of instances you don't need! Happy to clarify this further if it's not clear.
3. Not typos, no. They get translated from one to the other.

[dakr001]

@Doggie52 Thanks for the feedback.

Just getting back into this. Been busy with day job. Managed to progress a bit more and now onto the next error.

Step 4 of the installation instructions says:
Edit Instance/Program.cs lines 75 and 76 to set the correct path to and name of the algorithm to be batched. By default, the BasicTemplateFrameworkAlgorithm is referenced. Whatever DLL you choose here must be referenced in Launcher and thus compiled at runtime.

My interpretation is the dll name in config.json and batch.config json must match (?). Also I would only compile from batch launcher solution, not Lean.

It appears to me the dll names and locations are different:

1. Lean generates the following: C:\Users\Username\Source\Repos\Lean\Launcher\bin\Debug\QuantConnect.Algorithm.CSharp.dll'

2. Batch launcher generates: 'C:\Users\Username\source\repos\Lean-Batch-Launcher-master\Launcher\bin\Debug\Algorithm.dll'.

Depending on whether I specify QuantConnect.Algorithm.CSharp.dll or Algorithm.dll in the config files, I'll get an error either in Lean or in the Batch launcher

Not sure if this question is clear. Let me know if any hints please. Thanks.

[Doggie52]

Hey,

There is no dll name in batch.config.json. The dll name is in Instance/Program.cs.

Indeed you will be compiling and running the batch launcher rather than Lean's standard launcher.

Fundamentally, the code is structured in such a way that the easiest way to get things to work is to build your algorithm inside the Algorithm folder and then reference the dll from there. There are ways around this, i.e. by referencing some external dll, but really if you want a working proof-of-concept you're better off building your algorithm in the Algorithm folder.

[Doggie52]

As I understand it, you've taken the repo as it stands, upgraded it to a new .NET version and disabled embedded interop. You then had the default algorithm running fine, as it was outputting errors relating to your QuantConnect data API.

When you got to that stage, were you successfully able to run "batches" of the configuration variables you passed in batch.config.json? I.e., did you launch the batch launcher and did it work?

If that's the case, then I'm not sure why you are now running into issues with the path to the dlls?

[Doggie52]

Thank you for the help on this, by the way. I've pushed some commits to master which should help you - perhaps you want to start afresh from a clean master clone?

[dakr001]

Thanks @Doggie52! Will give this a try over the next few days.

[dakr001]

Hi @Doggie52

Good news. Managed to get this working with a couple of minor mods. The batch launcher loops through periods until current date. If up to date data is not available on the local drive then Lean will attempt to connect to QuantConnect. Given Lean is deprecated according to QC, this may not be supported anymore. I had trouble with the connection. Modified code slightly to only use data available on the local drive.

Next step is to test Batch Launcher with python Algorithms. In Program.cs, I retained the following since the python algorithm is BasicTemplateFrameworkAlgorithm.py
Config.Set( "algorithm-type-name", "BasicTemplateFrameworkAlgorithm" );
Config.Set( "algorithm-location", "Algorithm.dll" );

Then I added BasicTemplateFrameworkAlgorithm.py to Algorithm in the Solution Explorer. However now getting the errors below. Any hints please how to get Batch Launcher to work with python algorithms.

20201209 23:11:02.526 ERROR:: API.GetExtendedTypeNames(): No types found in assembly.
20201209 23:11:02.527 ERROR:: Loader.TryCreateILAlgorithm(): Types array empty, no algorithm type found.
20201209 23:11:02.584 ERROR:: Engine.Run(): QuantConnect.Lean.Engine.Setup.AlgorithmSetupException: During the algorithm initialization, the following exception has occurred: Algorithm type was not found.
at QuantConnect.Lean.Engine.Setup.ConsoleSetupHandler.CreateAlgorithmInstance(AlgorithmNodePacket algorithmNodePacket, String assemblyPath) in C:\Users\UserName\Source\Repos\Lean\Engine\Setup\ConsoleSetupHandler.cs:line 105
at QuantConnect.Lean.Engine.Engine.Run(AlgorithmNodePacket job, AlgorithmManager manager, String assemblyPath, WorkerThread workerThread) in C:\Users\UserName\Source\Repos\Lean\Engine\Engine.cs:line 117
20201209 23:11:02.612 ERROR:: Algorithm.Initialize() Error: During the algorithm initialization, the following exception has occurred: Algorithm type was not found. Stack Trace: QuantConnect.Lean.Engine.Setup.AlgorithmSetupException: During the algorithm initialization, the following exception has occurred: Algorithm type was not found.
at QuantConnect.Lean.Engine.Setup.ConsoleSetupHandler.CreateAlgorithmInstance(AlgorithmNodePacket algorithmNodePacket, String assemblyPath) in C:\Users\UserName\Source\Repos\Lean\Engine\Setup\ConsoleSetupHandler.cs:line 105
at QuantConnect.Lean.Engine.Engine.Run(AlgorithmNodePacket job, AlgorithmManager manager, String assemblyPath, WorkerThread workerThread) in C:\Users\UserName\Source\Repos\Lean\Engine\Engine.cs:line 117
QuantConnect.Lean.Engine.Setup.AlgorithmSetupException: During the algorithm initialization, the following exception has occurred: Algorithm type was not found.

[Doggie52]

Hey - sorry for late reply.

I haven't tested this with python algorithms ever, so I'm not sure how that would work. Does the python algorithm you're testing work outside of the Launcher (i.e. when running in standard LEAN)?

Given Lean is deprecated according to QC, this may not be supported anymore.

What do you mean by Lean being deprecated? LEAN is the engine behind the backtesting functionality - I've not heard or seen QC deprecating this.

[dakr001]

Hi @Doggie52

On the QC forums, Jared mentioned Lean is 'deprecated' which appears to mean they are no longer providing support for the standalone Lean.

Yes the python algorithm works in standard Lean. Keen to get it working as the research environment on QC is also python based. I'll tinker around and see if I can get it to work.

Thanks.

[Doggie52]

I suspect getting Python algorithms to work will require a different way of accessing the algorithm. Most likely the launcher will need to import from Algorithm.Python instead of Algorithm.CSharp, as the dll you're trying to use is probably constructed from Algorithm.CSharp right now.

Not surprised they won't be supporting stand-alone LEAN anymore, but I am surprised they would refer to that as "deprecated".

[dakr001]

Hey,

This is well beyond my ability now. So won't progress from my end.

It looks there is a version of Lean from quantconnect that has parameter optimization but there are no instructions on how to run it:

QuantConnect/Lean#4923

[Doggie52]

Understood - your help so far with this repo is greatly appreciated! Let me know how you get along and if you're able to get around this. Thanks again!