Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PhACE architecture #434

Draft
wants to merge 197 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
197 commits
Select commit Hold shift + click to select a range
c54a4bf
Try to soft reset
frostedoyster Jun 4, 2024
fbf93a1
Adapt to new Trainer
frostedoyster Jun 4, 2024
f4a212c
Working PhACE with scripted training
frostedoyster Jun 4, 2024
74b336a
Merge branch 'main' into phace
frostedoyster Jun 19, 2024
467257c
Move PhACE to new folder
frostedoyster Jun 19, 2024
a7e05cc
Merge branch 'main' into phace
frostedoyster Jul 11, 2024
897d709
Finish merge
frostedoyster Jul 11, 2024
98219fb
Update to latest rascaline (#291)
frostedoyster Jul 11, 2024
d11c9e3
Remove consistency checks (except for tests)
frostedoyster Jul 12, 2024
166d15e
Merge branch 'main' into consistency-checks
frostedoyster Jul 12, 2024
7fc1743
NanoPET
frostedoyster Jul 17, 2024
65c1404
Use `check_consistency` in tests
frostedoyster Jul 17, 2024
98ac770
Implement `selected_atoms`
frostedoyster Jul 18, 2024
2e757fd
Merge branch 'consistency-checks' into nanopet
frostedoyster Jul 18, 2024
2ae3da2
Optimized training, there still seems to be a bug
frostedoyster Jul 18, 2024
836e3fc
Fixed bug
frostedoyster Jul 18, 2024
c0df552
Fix attention correctness
frostedoyster Jul 18, 2024
7931989
Add test for consistent predictions with different padding
frostedoyster Jul 18, 2024
aa7b478
Update defaults
frostedoyster Jul 18, 2024
67ffab6
Merge branch 'main' into nanopet
frostedoyster Jul 18, 2024
7223a95
Last-layer features for nanoPET
frostedoyster Jul 19, 2024
ec7b725
Merge branch 'main' into phace
frostedoyster Jul 22, 2024
57538cd
Some improvements
frostedoyster Jul 22, 2024
2cc92b8
Fix stuff
frostedoyster Jul 22, 2024
d25a340
More fixes
frostedoyster Jul 22, 2024
4d32bdb
More fixxxes
frostedoyster Jul 22, 2024
606b9e2
Fix restart
frostedoyster Jul 22, 2024
28cd708
Revert
frostedoyster Jul 22, 2024
90ade37
Merge branch 'main' into phace
frostedoyster Jul 23, 2024
10f08b7
Fix a few issues
frostedoyster Jul 23, 2024
fd91c75
Merge branch 'phace' of https://github.com/lab-cosmo/metatrain into p…
frostedoyster Jul 23, 2024
fec2e4a
Merge branch 'phace' of https://github.com/lab-cosmo/metatrain into p…
frostedoyster Jul 23, 2024
ff0b89f
Merge branch 'phace' of https://github.com/lab-cosmo/metatrain into p…
frostedoyster Jul 23, 2024
5539623
Fix bug in message passing
frostedoyster Jul 23, 2024
2269541
Fix bug
frostedoyster Jul 23, 2024
3012917
Merge branch 'main' into nanopet
frostedoyster Jul 25, 2024
df4aba6
Merge branch 'main' into nanopet
frostedoyster Jul 30, 2024
61ca9ce
Normalize residual connections
frostedoyster Jul 31, 2024
98aa9b4
Fix torchscript issue
frostedoyster Jul 31, 2024
72f0bcd
Try to change the position encoder
frostedoyster Aug 1, 2024
1cc2e1d
Remove torch hotfix
frostedoyster Aug 2, 2024
cdb1d3d
Upgrade to `metatensor-torch` 0.5.3
frostedoyster Aug 3, 2024
06f88fc
Merge branch 'metatensor-torch-0.5.3' into nanopet
frostedoyster Aug 3, 2024
2a3808c
Merge branch 'main' into phace
frostedoyster Aug 3, 2024
a77fb69
Cleaned everything up
frostedoyster Aug 7, 2024
b8f2d83
Messed up validation metric
frostedoyster Aug 7, 2024
8f66f8c
Version working very well for molecules, failing on HME21
frostedoyster Aug 10, 2024
ff898c7
Fix torchscripting
frostedoyster Aug 10, 2024
f430884
Fix bugg
frostedoyster Aug 10, 2024
1a400dc
Make sphericart and mops optional
frostedoyster Aug 15, 2024
e24b907
New files for physical basis
frostedoyster Aug 15, 2024
2a92f37
Less NN in radial basis
frostedoyster Aug 21, 2024
0e87e51
Integrate physical basis from PyPI
frostedoyster Sep 4, 2024
770bde6
Use covalent radii
frostedoyster Sep 4, 2024
293c7a6
Revert some changes to the encoder
frostedoyster Sep 5, 2024
94df82e
Remove random factor
frostedoyster Oct 7, 2024
9eff61f
Merge branch 'main' into nanopet
frostedoyster Oct 14, 2024
2fd9b8d
Speed up a tiny bit
frostedoyster Oct 14, 2024
6c8f1da
Merge branch 'main' into nanopet
Oct 15, 2024
713cfbb
Fair comparison to PET
frostedoyster Oct 15, 2024
33e2c70
Calculate displacements independently + format
frostedoyster Oct 16, 2024
4b8d8c1
Fix pyproject.toml
frostedoyster Oct 16, 2024
3377b2d
Dependencies
frostedoyster Oct 17, 2024
63bcfac
Merge branch 'mae-logging' into nanopet
frostedoyster Oct 17, 2024
f79a36d
Automatic continuation
frostedoyster Oct 17, 2024
3c136bf
Call processing function before new `outputs/` directory is created
frostedoyster Oct 18, 2024
c33bcf2
Make it distributed-proof
frostedoyster Oct 18, 2024
4b2a877
Merge branch 'auto-continue' into nanopet
frostedoyster Oct 18, 2024
129c9b9
Only save checkpoints from the main process
frostedoyster Oct 18, 2024
d88cfcf
Merge branch 'main' into nanopet
frostedoyster Oct 18, 2024
a675bfe
Get rid of nanopet-neighbors
frostedoyster Oct 18, 2024
72a6502
Speed up
frostedoyster Oct 18, 2024
fc33c70
Proper cutoff parameters
frostedoyster Oct 20, 2024
a951638
MLP for geometric embedding
frostedoyster Oct 20, 2024
92d2c80
Merge branch 'nanopet' of https://github.com/lab-cosmo/metatrain into…
frostedoyster Oct 20, 2024
45c2fc7
Fix bug
frostedoyster Oct 20, 2024
b8cd9ba
Add sample `TensorMap` to `TargetInfo`
frostedoyster Oct 26, 2024
60b6b0d
Change `TensorMap`s inside `TargetInfo` to `float64` to avoid seriali…
frostedoyster Oct 28, 2024
4c614f7
Better documentation
frostedoyster Oct 28, 2024
172af0b
Upgrade to `metatensor-torch` 0.6.0
frostedoyster Oct 29, 2024
0f3bfd7
Upgrade `metatensor-learn`
frostedoyster Oct 30, 2024
0e8e2f7
Update strict NL
frostedoyster Oct 30, 2024
d632dc2
Fix PBCs
frostedoyster Oct 30, 2024
b65536a
Draft
frostedoyster Nov 1, 2024
cf62ee7
Upgrade rascaline-torch
frostedoyster Nov 1, 2024
0ebcc9a
Fix `slice` argument name
frostedoyster Nov 2, 2024
f7355dd
Merge branch 'upgrade-metatensor' into targetinfo-tensormap
frostedoyster Nov 2, 2024
906d01a
Merge branch 'targetinfo-tensormap' into generic-readers
frostedoyster Nov 2, 2024
d850528
Merge branch 'adapt-models' into nanopet
frostedoyster Nov 4, 2024
5d6ccdb
FIx a few issues
frostedoyster Nov 4, 2024
f881951
Add dataset information overview to the dev docs
frostedoyster Nov 5, 2024
de9a28e
Fix docs
frostedoyster Nov 5, 2024
4f89649
Merge branch 'main' into targetinfo-tensormap
frostedoyster Nov 5, 2024
a9773f4
Merge branch 'targetinfo-tensormap' into generic-readers
frostedoyster Nov 5, 2024
5cc0806
Fix export issue
frostedoyster Nov 6, 2024
775be90
Small fixes
frostedoyster Nov 7, 2024
940bd3c
Better LR decreases
frostedoyster Nov 7, 2024
00f289e
Merge branch 'main' into generic-readers
frostedoyster Nov 10, 2024
758b9c5
Fix stuff
frostedoyster Nov 10, 2024
6d99827
Fix more stuff
frostedoyster Nov 10, 2024
012f141
Fix even more stuff
frostedoyster Nov 10, 2024
4146954
Revert changes to options file
frostedoyster Nov 10, 2024
e93ac4a
Fix tests
frostedoyster Nov 10, 2024
e86192d
Add warning when learning of `forces`, `stress`, `virial` not as grad…
frostedoyster Nov 10, 2024
a1c2146
Update developer docs
frostedoyster Nov 11, 2024
b8647f2
Add user docs for generic targets
frostedoyster Nov 11, 2024
00434c6
Overall and ASE reader tests
frostedoyster Nov 11, 2024
b743030
Move tests
frostedoyster Nov 11, 2024
b815435
Metatensor reader tests
frostedoyster Nov 11, 2024
21d6098
One more test
frostedoyster Nov 11, 2024
f3843d9
Eliminate useless checks that are already performed by the `DictConfi…
frostedoyster Nov 11, 2024
fd4a858
Test error with ASE and multiple spherical irreps
frostedoyster Nov 11, 2024
fb4a24d
One more test
frostedoyster Nov 11, 2024
239428d
Fix bug
frostedoyster Nov 11, 2024
15093de
Test more errors
frostedoyster Nov 11, 2024
2a52859
Allow SOAP-BPNN to fit Cartesian vectors
frostedoyster Nov 11, 2024
13da2b8
Fix composition test
frostedoyster Nov 12, 2024
078fbd6
Add checks and SOAP-BPNN vector test
frostedoyster Nov 12, 2024
77f275b
Bug fix in schema
frostedoyster Nov 12, 2024
af5c126
Merge branch 'generic-readers' into adapt-models
frostedoyster Nov 12, 2024
0370921
Add vector output training test
frostedoyster Nov 12, 2024
76b941f
Fix regression tests
frostedoyster Nov 12, 2024
13e7769
Check that ZBL gets correct targets
frostedoyster Nov 12, 2024
da544ff
Docs
frostedoyster Nov 12, 2024
0e6c46f
Fix bug
frostedoyster Nov 12, 2024
3ff9370
Fix bug
frostedoyster Nov 12, 2024
737c895
Extract function to add outputs for SOAP-BPNN
frostedoyster Nov 12, 2024
998b7ed
Fix some small issues
frostedoyster Nov 12, 2024
5da6e75
Merge branch 'main' into nanopet
frostedoyster Nov 12, 2024
4e1c38a
Merge branch 'adapt-models' into nanopet
frostedoyster Nov 12, 2024
04159f2
Fix leftover merge issues
frostedoyster Nov 12, 2024
dd23a63
Add tests
frostedoyster Nov 13, 2024
853ccf0
Fix `tests/resources` folder
frostedoyster Nov 13, 2024
a777fcb
Merge branch 'main' into generic-readers
frostedoyster Nov 13, 2024
27e6104
Merge branch 'generic-readers' into adapt-models
frostedoyster Nov 13, 2024
3be99cc
Remove commented torch hack
frostedoyster Nov 13, 2024
89bddcc
Merge branch 'adapt-models' into nanopet
frostedoyster Nov 13, 2024
160124f
Undo spurious changes
frostedoyster Nov 13, 2024
79e8b30
Codeowners, tox, GH workflows
frostedoyster Nov 13, 2024
8712d27
Add correct values for regression tests
frostedoyster Nov 13, 2024
afb9059
Add documentation
frostedoyster Nov 13, 2024
e665c7a
Regression test
frostedoyster Nov 13, 2024
aeff14f
Clean up
frostedoyster Nov 13, 2024
4aa2a1c
Fix restart issue
frostedoyster Nov 13, 2024
5cc35c0
Fix test
frostedoyster Nov 13, 2024
d0be159
Fix accidental deletion of test
frostedoyster Nov 13, 2024
a325942
Apply suggestions from code review
frostedoyster Nov 18, 2024
a69dfa7
Fix assert
frostedoyster Nov 18, 2024
025048d
Infer `quantity` in `get_energy_target_info`
frostedoyster Nov 18, 2024
03c6df3
Allow arbitrary properties with `metatensor` reader
frostedoyster Nov 18, 2024
b6f5cd5
Fix tests
frostedoyster Nov 18, 2024
7b4ae53
Merge branch 'main' into generic-readers
frostedoyster Nov 18, 2024
a5416be
Fix tests
frostedoyster Nov 18, 2024
a5a102c
Update docs/src/advanced-concepts/preparing-generic-targets.rst
frostedoyster Nov 19, 2024
155aac5
Suggestions from review
frostedoyster Nov 19, 2024
6fcc7f4
Remove private functions from docs
frostedoyster Nov 19, 2024
46223b5
Merge branch 'adapt-models' into phace
frostedoyster Nov 19, 2024
5328def
Update model according to new infrastructure
frostedoyster Nov 19, 2024
175264b
Fix buggggg
frostedoyster Nov 19, 2024
405c2cf
Change metatensor target format to a single `TensorMap`
frostedoyster Nov 19, 2024
0c207fc
Merge branch 'generic-readers' into adapt-models
frostedoyster Nov 20, 2024
e158719
Correctly inherit properties from targets in SOAP-BPNN
frostedoyster Nov 20, 2024
7d3bc7d
Merge branch 'adapt-models' into nanopet
frostedoyster Nov 20, 2024
2a590b0
Correctly inherit properties from targets in SOAP-BPNN
frostedoyster Nov 20, 2024
ab1446a
Fix LLPR test
frostedoyster Nov 20, 2024
a15ae29
Fix regressiont tests
frostedoyster Nov 20, 2024
364bd9b
Fix regression tests
frostedoyster Nov 20, 2024
dff9612
Merge branch 'adapt-models' into nanopet
frostedoyster Nov 20, 2024
7c73db6
Add head types to SOAP-BPNN and nanoPET
frostedoyster Nov 20, 2024
d28583e
Update for features and LLPR module
frostedoyster Nov 20, 2024
8296733
Update old tests
frostedoyster Nov 20, 2024
3a66aa9
Docs and new tests
frostedoyster Nov 20, 2024
f598aab
Merge branch 'main' into adapt-models
frostedoyster Nov 21, 2024
23a99fc
Fix docs
frostedoyster Nov 21, 2024
d35fd1b
Make composition restart-proof (continue, fine-tune, transfer learning)
frostedoyster Nov 21, 2024
7754e6d
Add scale to logger
frostedoyster Nov 21, 2024
59e3363
Turn old warning into error
frostedoyster Nov 21, 2024
e6993e4
Merge branch 'adapt-models' into nanopet
frostedoyster Nov 22, 2024
d4c59b6
Scaler class
frostedoyster Nov 22, 2024
7d2c940
Add to documentation
frostedoyster Nov 22, 2024
d46ca0f
num_properties -> num_subtargets
frostedoyster Nov 22, 2024
b137a4a
Merge branch 'nanopet' into heads-and-utils
frostedoyster Nov 22, 2024
5dbc9ed
Merge branch 'heads-and-utils' into scaler
frostedoyster Nov 22, 2024
40ef3ad
Tests
frostedoyster Nov 22, 2024
ae90f6f
Merge branch 'scaler' into phace
frostedoyster Nov 23, 2024
5c5f3f4
Scaler
frostedoyster Nov 23, 2024
9fb5a93
Actually use `log_mae`
frostedoyster Dec 15, 2024
a6f0dad
Fix final checkpoint bug
frostedoyster Dec 17, 2024
22014dd
Merge branch 'main' into phace
frostedoyster Dec 17, 2024
8fcb14e
Fix 0 epoch issue
frostedoyster Dec 17, 2024
7cf75d2
Merge branch 'checkpoint-bug' into phace
frostedoyster Dec 17, 2024
4a2f5ec
Add dependencies
frostedoyster Dec 17, 2024
6223234
Update trainer
frostedoyster Dec 17, 2024
8d19694
Merge branch 'main' into phace
frostedoyster Dec 17, 2024
596ff08
Add equivariant fitting
frostedoyster Dec 17, 2024
04a58a0
Fix regressiont test
frostedoyster Dec 17, 2024
497c538
Merge branch 'main' into phace
frostedoyster Dec 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/architecture-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ jobs:
- architecture-name: soap-bpnn
- architecture-name: pet
- architecture-name: nanopet
- architecture-name: phace

runs-on: ubuntu-22.04

Expand Down
4 changes: 4 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,10 @@ gap = [
"metatensor-learn",
"scipy",
]
phace = [
"physical_basis",
"wigners",
]

[tool.setuptools.packages.find]
where = ["src"]
Expand Down
26 changes: 14 additions & 12 deletions src/metatrain/cli/train.py
Original file line number Diff line number Diff line change
Expand Up @@ -395,18 +395,20 @@ def train_model(
# TRAIN MODEL #############
###########################

logger.info("Calling trainer")
try:
trainer.train(
model=model,
dtype=dtype,
devices=devices,
train_datasets=train_datasets,
val_datasets=val_datasets,
checkpoint_dir=str(checkpoint_dir),
)
except Exception as e:
raise ArchitectureError(e)
# logger.info("Calling trainer")
# from torch.profiler import profile, ProfilerActivity
# with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA]) as prof:
# if True:
trainer.train(
model=model,
dtype=dtype,
devices=devices,
train_datasets=train_datasets,
val_datasets=val_datasets,
checkpoint_dir=str(checkpoint_dir),
)
# print(prof.key_averages().table(sort_by="self_cpu_time_total", row_limit=20))
# exit()

if not is_main_process():
return # only save and evaluate on the main process
Expand Down
4 changes: 2 additions & 2 deletions src/metatrain/experimental/nanopet/trainer.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ def train(
if len(devices) > 1:
raise ValueError(
"Requested distributed training with the `multi-gpu` device. "
" If you want to run distributed training with SOAP-BPNN, please "
" If you want to run distributed training with nanoPET, please "
"set `device` to cuda."
)
# the calculation of the device number works both when GPUs on different
Expand Down Expand Up @@ -98,7 +98,7 @@ def train(

# Move the model to the device and dtype:
model.to(device=device, dtype=dtype)
# The additive models of the SOAP-BPNN are always in float64 (to avoid
# The additive models of nanoPET are always in float64 (to avoid
# numerical errors in the composition weights, which can be very large).
for additive_model in model.additive_models:
additive_model.to(dtype=torch.float64)
Expand Down
13 changes: 13 additions & 0 deletions src/metatrain/experimental/phace/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from .model import PhACE
from .trainer import Trainer

__model__ = PhACE
__trainer__ = Trainer

__authors__ = [
("Filippo Bigi <[email protected]>", "@frostedoyster"),
]

__maintainers__ = [
("Filippo Bigi <[email protected]>", "@frostedoyster"),
]
41 changes: 41 additions & 0 deletions src/metatrain/experimental/phace/default-hypers.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
architecture:
name: experimental.phace

model:
nu_max: 3
n_message_passing_layers: 2
cutoff: 5.0
n_element_channels: 8
radial_basis:
mlp: true
type: physical
E_max: 70.0
scale: 0.7
nu_scaling: 1.0
mp_scaling: 1.0
overall_scaling: 1.0
disable_nu_0: True
use_sphericart: False
use_mops: False
heads: {}
zbl: False

training:
distributed: False
distributed_port: 39591
batch_size: 8
num_epochs: 100
learning_rate: 0.001
early_stopping_patience: 100
scheduler_patience: 10
scheduler_factor: 0.8
log_interval: 10
checkpoint_interval: 25
scale_targets: true
fixed_composition_weights: {}
per_structure_targets: []
log_mae: False
loss:
type: mse
weights: {}
reduction: sum
Loading
Loading