Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gymnasium api #1

Open
wants to merge 76 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
147b61a
Custom Gym Env, Crazyflie simulator, sb3/tianshou progress, custom ke…
Jun 10, 2024
3169338
gymnasium API simulator
Jun 17, 2024
2ae4e38
Improved docstrings
Jun 17, 2024
c2056c6
support for actin history, numpy version of deque, slows down slightly
Jun 17, 2024
a700384
adapt reward function to target stabilizing controller rather than po…
Jun 17, 2024
2837bcf
Warn if action not in actionspace, might cause NaN values
Jun 17, 2024
909e07b
enable position term in reward, test and train scripts
Jun 18, 2024
d4d56c3
updates on n_episodes (for testing)
Jun 19, 2024
06fd50b
trained policy
korneelf1 Jun 19, 2024
0895da6
imu model, changed reward structure
Jun 19, 2024
70fdab0
changed reward str
Jun 19, 2024
e4c9bfd
Merge branch 'gymnasium_env' of https://github.com/korneelf1/fastPyDr…
Jun 19, 2024
3949c2d
matplotlib rendering
Jun 19, 2024
cf6bd60
mac policy
korneelf1 Jun 19, 2024
92e2f00
fix test envs + stabalization reward
Jun 19, 2024
e52ecd2
adapt train script
Jun 19, 2024
ddd1219
improved collector, progressed gpu kernels
korneelf1 Jun 24, 2024
cdd7482
stabalization ctrl
korneelf1 Jun 24, 2024
61da397
Prioritized buffer, plot origin point for rendering
korneelf1 Jun 24, 2024
6ce17a5
Cuda support for kernels
korneelf1 Jun 24, 2024
e15b93d
Recurrent training
korneelf1 Jun 25, 2024
4be9604
Trained policies
korneelf1 Jun 25, 2024
6b8fb11
position control
korneelf1 Jun 25, 2024
a6537ca
position targets added to observation space
korneelf1 Jun 26, 2024
2904f7e
Forgot to actually add the actions to the history........
korneelf1 Jun 26, 2024
ca06898
For training purposes, use stabilization reward
korneelf1 Jun 26, 2024
649590d
sim on cpu, train on gpu
korneelf1 Jun 26, 2024
71cd14f
changes ext
korneelf1 Jun 26, 2024
3d6577b
restore training characteristics to last_working branch?
korneelf1 Jun 27, 2024
fa52928
Spiking Actors, Updated Evotorch
korneelf1 Jul 25, 2024
e7854c9
wandb logging of evotorch
korneelf1 Jul 25, 2024
31e9e44
conda setup
korneelf1 Jul 26, 2024
1ae8af6
remove nullable slice in collector (unused)
korneelf1 Jul 26, 2024
8ec0218
make reqs file
korneelf1 Jul 26, 2024
fa19cc3
Merge remote-tracking branch 'refs/remotes/origin/gymnasium_env' into…
korneelf1 Jul 26, 2024
9863f5a
spiking actor implemented
korneelf1 Jul 29, 2024
0da6c0d
SMLP is now nn.Module, evotorch adapted to support SNN
korneelf1 Jul 30, 2024
43a7b19
save the evolutionary model
korneelf1 Jul 30, 2024
c30c968
debugging by adding option to specify task
korneelf1 Jul 31, 2024
18794db
swap around n_states
korneelf1 Jul 31, 2024
b63284b
more info on whats happening in terminal
korneelf1 Jul 31, 2024
e7863a8
debugging....
korneelf1 Aug 1, 2024
a87fd1b
Fix speed issue, add masked networks
korneelf1 Aug 2, 2024
e1ff0ab
!!! thrust constant and torque constant were swapped around (k is thr…
korneelf1 Aug 4, 2024
ba4f150
Trying out og drone
korneelf1 Aug 5, 2024
3fb2767
less strict termination conditions to speed up training
korneelf1 Aug 5, 2024
ec19a26
I wasnt logging the args...
korneelf1 Aug 5, 2024
54595d2
Cumulative reward if evaluation env
korneelf1 Aug 5, 2024
5761d52
specify observation space, implement safety feature for out of bound …
korneelf1 Aug 8, 2024
94919a9
clean evolearning file
korneelf1 Aug 8, 2024
9e523a5
implement standard collector compatibility
korneelf1 Aug 9, 2024
9d9e9b0
remove device arg in collector, change hyperparameters to ensure at l…
korneelf1 Aug 9, 2024
048aed6
batch size of 1 in PPO apperently causes it to crash :)
korneelf1 Aug 9, 2024
247c160
batch size of 1 does NOT work!
korneelf1 Aug 9, 2024
48a7226
ANN evo learning
korneelf1 Aug 9, 2024
6c2b343
Fix logging, fix PPO (action bounded)
korneelf1 Aug 9, 2024
374c6e1
Increase learning rate
korneelf1 Aug 12, 2024
04e5e89
Tianshou to evotorch wrapper
korneelf1 Aug 12, 2024
0939233
Normalize motor speeds
korneelf1 Aug 13, 2024
3481caa
Remove test statements for reward printing
korneelf1 Aug 13, 2024
1aafac8
Cant remove psets for now
korneelf1 Aug 13, 2024
32a1376
N=1 different return in step
korneelf1 Aug 13, 2024
795a669
reward function from git reward_squared_fast_learning
korneelf1 Aug 13, 2024
7ad36bd
dont normalize, 1e-4 learning rate with no scheduling
korneelf1 Aug 13, 2024
03926f8
allow tianshou policy with evotorch
korneelf1 Aug 13, 2024
b6ae688
Improved reset mechanism, initially completely random, but might rese…
korneelf1 Aug 14, 2024
9842ea6
Allwo multidim quaternion init
korneelf1 Aug 14, 2024
d4fb96e
allow multiple lr schedulers
korneelf1 Aug 14, 2024
9c58498
step size of 1k for lr schedulers
korneelf1 Aug 14, 2024
a17f5c5
format gym env
korneelf1 Aug 14, 2024
def3b11
CLEANUP: clean up gym_sim file, helpers file
korneelf1 Aug 14, 2024
9549371
Something of a test
korneelf1 Aug 14, 2024
ac0b222
Cleaned up file structure
korneelf1 Aug 14, 2024
dece0bf
Syncing with OG repo
korneelf1 Aug 14, 2024
67d0e6a
test dones and single steps
korneelf1 Aug 14, 2024
7174ac3
Fix of unittests
korneelf1 Aug 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
wandb logging of evotorch
  • Loading branch information
korneelf1 committed Jul 25, 2024
commit e7854c90382c6c3b19bc7834337fedca6ae8e6d9
5 changes: 3 additions & 2 deletions evolutionary_learn_nets.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from evotorch.algorithms import PGPE
from evotorch.logging import StdOutLogger
from evotorch.logging import StdOutLogger, WandbLogger
from evotorch.neuroevolution import GymNE
from gym_sim import Drone_Sim

Expand Down Expand Up @@ -30,7 +30,8 @@
num_interactions=150000,
popsize_max=3200,
)
logger = StdOutLogger(searcher)
# logger = StdOutLogger(searcher)
logger = WandbLogger(searcher, project="evotorch drone sim")
searcher.run(500)

population_center = searcher.status["center"]
Expand Down