Repurposing linkinvent prior to generate macrocycles guided by reinforcement learning

[cmwoodley]

Hi!

Firstly, thanks for making this repository open source!

I've been interested in using linkinvent to produce macrocycles guided by reinforcement leanring. I've put together a rough implementation which allows this. This is on a branch here: https://github.com/cmwoodley/REINVENT4/tree/feat/mac_linkinvent

Briefly, this adds two functionalities to the code:

• allows linkinent run mode to take an input in the form of a scaffold with two attachment points. This is converted to a linkinvent type input for use in the seq-to-seq model by splitting the scaffold into two warheads.
• After linkinvent links the warheads, the scaffold is reformed by RDKit reaction prior to scoring, enabling scoring based on the whole macrocycle.

Preliminary benchmarking of this method shows that we can use the linkinvent prior with curriculum learning to generate sensible macrocycles (benchmarking in pdf here). I think this would also be useful to facilitate scaffold hopping within a macrocyclic structure.

I thought I'd share this to check if this was something you would be interested in implementing? And if so, because I know my initial implementation is rough, what would be the preferable way to implement this method?

Cheers!

[halx]

Hi,

many thanks for this and your interested in REINVENT. Welcome to the community!

This does indeed look interesting and I would also expect interest in the wider community. I am not sure though if this is the right approach. The principal question is if the chemical space covered by Linkinvent is good enough for macrocycle design. I do not know. The technical concern is that you now need a SMILES pre-processor for scoring which, while fairly easy to do, would make the code more complicated. Have you thought about training a bespoke model like Libinvent instead, with 2 attachment points and 1 R-group, for instance?

Many thanks again,
Hannes.

[ismorphism]

@cmwoodley Hi! Do u have any configs about your scoring components? Because in linker for the macrocycles the Reinvent keeps adding rings even if I say so not to do it. Thanks 🙂

[cmwoodley]

Hi,

This was what I found too.

I found using curriculum learning was necessary to reduce the number of rings using the approach discussed here. So in the first stage set the lower transform of the reverse sigmoid higher than you want, then iteratively move the value towards your desired value for number of rings.

I don't have the scoring toml to hand now, but can share this later if you like!

Cheers!

[ismorphism]

@cmwoodley If you can share the scoring toml file that would be great! Thanks 🙂 I would go through the link you sent 👀

[cmwoodley]

See below scoring section from my config toml - I've used number of rings in the whole macrocycle (can't remember why, maybe because the other two scoring features I used are linker specific?)

I initially targeted the minimum number of rings in stage one, focussed on linker length in stage two, then reintroduced the number of rings component in stage three after noticing a drift in the number of rings in the linker.

Hope this help!

### Stage 1
[[stage]]

chkpt_file = '../models/test1_noether_stage1.chkpt'  # name of the checkpoint file, can be reused as agent

termination = "simple"  # termination criterion fot this stage
max_score = 0.6  # terminate if this total score is exceeded
min_steps = 25  # run for at least this number of steps
max_steps = 200  # terminate entire run when exceeded

[stage.scoring]
type = "geometric_mean"  # aggregation function

[[stage.scoring.component]]
[stage.scoring.component.NumRings]
[[stage.scoring.component.NumRings.endpoint]]
name = "num_rings"
weight = 0.8
transform.type = "reverse_sigmoid"
transform.high = 5
transform.low = 3
transform.k = 0.3

### Stage 2
[[stage]]

chkpt_file = '../models/test1_noether_stage2.chkpt'  # name of the checkpoint file, can be reused as agent

termination = "simple"  # termination criterion fot this stage
max_score = 0.8  # terminate if this total score is exceeded
min_steps = 25  # run for at least this number of steps
max_steps = 100  # terminate entire run when exceeded

[stage.scoring]
type = "geometric_mean"  # aggregation function

[[stage.scoring.component]]
[stage.scoring.component.FragmentGraphLength]
[[stage.scoring.component.FragmentGraphLength.endpoint]]
name = "Linker Length"
weight = 0.8
transform.type = "double_sigmoid"
transform.high = 8
transform.low = 4
transform.coef_div = 9.0
transform.coef_si = 20.0
transform.coef_se = 20.0

### Stage 3
[[stage]]

chkpt_file = '../models/test1_noether_stage3.chkpt'  # name of the checkpoint file, can be reused as agent

termination = "simple"  # termination criterion fot this stage
max_score = 0.8  # terminate if this total score is exceeded
min_steps = 25  # run for at least this number of steps
max_steps = 100  # terminate entire run when exceeded

[stage.scoring]
type = "geometric_mean"  # aggregation function

[[stage.scoring.component]]
[stage.scoring.component.FragmentHBondAcceptors]
[[stage.scoring.component.FragmentHBondAcceptors.endpoint]]
name = "Linker HBA"
weight = 0.45
transform.type = "double_sigmoid"
transform.high = 3
transform.low = 0
transform.coef_div = 4.0
transform.coef_si = 20.0
transform.coef_se = 20.0

[[stage.scoring.component]]
[stage.scoring.component.NumRings]
[[stage.scoring.component.NumRings.endpoint]]
name = "num_rings"
weight = 0.6
transform.type = "reverse_sigmoid"
transform.high = 5
transform.low = 3
transform.k = 0.3

[[stage.scoring.component]]
[stage.scoring.component.FragmentGraphLength]
[[stage.scoring.component.FragmentGraphLength.endpoint]]
name = "Linker Length"
weight = 0.4
transform.type = "double_sigmoid"
transform.high = 8
transform.low = 5
transform.coef_div = 9.0
transform.coef_si = 20.0
transform.coef_se = 20.0

[ismorphism]

@cmwoodley Thank you a lot! This .toml file really helps 🙂

[halx]

Hi again.

many thanks again for this and I think it is really useful. But the problem is that the released code here on GitHub is basically our production system which is being used in ongoing in-house drug discovery projects. Changes like this would require extensive testing, would probably be not that important for in-house work and ultimately we will only maintain a single code bases.

I hope that clears it up,
Hannes.

[ismorphism]

@halx Dear Hannes, speaking about the idea of training a bespoke model like Libinvent instead, with 2 attachment points and 1 R-group for macrocycle optimization - how should e.g. scaffolds.smi look like then?