Code

Gymnasium api#1

Code

Open

korneelf1 wants to merge 76 commits intotudelft:maintudelft/fastPyDroneSim:mainfrom korneelf1:Gymnasium-APIkorneelf1/SpikingCrazyflie:Gymnasium-APICopy head branch name to clipboard

+2,104-22

Commits

Commits on Jun 10, 2024

• Custom Gym Env, Crazyflie simulator, sb3/tianshou progress, custom kernels

  

  Korneel Van den Berghe

  committed
  147b61a

Commits on Jun 17, 2024

• gymnasium API simulator

  

  Korneel Van den Berghe

  committed
  3169338

• Improved docstrings

  

  Korneel Van den Berghe

  committed
  2ae4e38

• support for actin history, numpy version of deque, slows down slightly

  

  Korneel Van den Berghe

  committed
  c2056c6

• adapt reward function to target stabilizing controller rather than position controller

  

  Korneel Van den Berghe

  committed
  a700384

• Warn if action not in actionspace, might cause NaN values

  

  Korneel Van den Berghe

  committed
  2837bcf

Commits on Jun 18, 2024

• enable position term in reward, test and train scripts

  

  Korneel Van den Berghe

  committed
  909e07b

Commits on Jun 19, 2024

• updates on n_episodes (for testing)

  

  Korneel Van den Berghe

  committed
  d4d56c3

• trained policy

  korneelf1

  committed
  06fd50b

• imu model, changed reward structure

  

  Korneel Van den Berghe

  committed
  0895da6

• changed reward str

  

  Korneel Van den Berghe

  committed
  70fdab0

• Merge branch 'gymnasium_env' of https://github.com/korneelf1/fastPyDroneSim_gym into gymnasium_env

  

  Korneel Van den Berghe

  committed
  e4c9bfd

• matplotlib rendering

  

  Korneel Van den Berghe

  committed
  3949c2d

• mac policy

  korneelf1

  committed
  cf6bd60

• fix test envs + stabalization reward

  

  Korneel Van den Berghe

  committed
  92e2f00

• adapt train script

  

  Korneel Van den Berghe

  committed
  e52ecd2

Commits on Jun 24, 2024

• improved collector, progressed gpu kernels

  korneelf1

  committed
  ddd1219

• stabalization ctrl

  korneelf1

  committed
  cdd7482

• Prioritized buffer, plot origin point for rendering

  korneelf1

  committed
  61da397

• Cuda support for kernels

  korneelf1

  committed
  6ce17a5

Commits on Jun 25, 2024

• Recurrent training

  korneelf1

  committed
  e15b93d

• Trained policies

  korneelf1

  committed
  4be9604

• position control

  korneelf1

  committed
  6b8fb11

Commits on Jun 26, 2024

• position targets added to observation space

  korneelf1

  committed
  a6537ca

• Forgot to actually add the actions to the history........

  korneelf1

  committed
  2904f7e

• For training purposes, use stabilization reward

  korneelf1

  committed
  ca06898

• sim on cpu, train on gpu

  korneelf1

  committed
  649590d

• changes ext

  korneelf1

  committed
  71cd14f

Commits on Jun 27, 2024

• restore training characteristics to last_working branch?

  korneelf1

  committed
  3d6577b

Commits on Jul 25, 2024

• Spiking Actors, Updated Evotorch

  korneelf1

  committed
  fa52928

• wandb logging of evotorch

  korneelf1

  committed
  e7854c9

Commits on Jul 26, 2024

• conda setup

  korneelf1

  committed
  31e9e44

• remove nullable slice in collector (unused)

  korneelf1

  committed
  1ae8af6

• make reqs file

  korneelf1

  committed
  8ec0218

• Merge remote-tracking branch 'refs/remotes/origin/gymnasium_env' into gymnasium_env

  korneelf1

  committed
  fa19cc3

Commits on Jul 29, 2024

• spiking actor implemented

  korneelf1

  committed
  9863f5a

Commits on Jul 30, 2024

• SMLP is now nn.Module, evotorch adapted to support SNN

  korneelf1

  committed
  0da6c0d

• save the evolutionary model

  korneelf1

  committed
  43a7b19

Commits on Jul 31, 2024

• debugging by adding option to specify task

  korneelf1

  committed
  c30c968

• swap around n_states

  korneelf1

  committed
  18794db

• more info on whats happening in terminal

  korneelf1

  committed
  b63284b

Commits on Aug 1, 2024

• debugging....

  korneelf1

  committed
  e7863a8

Commits on Aug 2, 2024

• Fix speed issue, add masked networks

  korneelf1

  committed
  a87fd1b

Commits on Aug 4, 2024

• !!! thrust constant and torque constant were swapped around (k is thrust constant which is tiny)

  korneelf1

  committed
  e1ff0ab

Commits on Aug 5, 2024

• Trying out og drone

  korneelf1

  committed
  ba4f150

• less strict termination conditions to speed up training

  korneelf1

  committed
  3fb2767

• I wasnt logging the args...

  korneelf1

  committed
  ec19a26

• Cumulative reward if evaluation env

  korneelf1

  committed
  54595d2

Commits on Aug 8, 2024

• specify observation space, implement safety feature for out of bound actions, try different collector, skip action weight (not even sure what it should be as we use default drone)

  korneelf1

  committed
  5761d52

• clean evolearning file

  korneelf1

  committed
  94919a9

Commits on Aug 9, 2024

• implement standard collector compatibility

  korneelf1

  committed
  9e523a5

• remove device arg in collector, change hyperparameters to ensure at least 3M interactions (see learning to fly)

  korneelf1

  committed
  9d9e9b0

• batch size of 1 in PPO apperently causes it to crash :)

  korneelf1

  committed
  048aed6

• batch size of 1 does NOT work!

  korneelf1

  committed
  247c160

• ANN evo learning

  korneelf1

  committed
  48a7226

• Fix logging, fix PPO (action bounded)

  korneelf1

  committed
  6c2b343

Commits on Aug 12, 2024

• Increase learning rate

  korneelf1

  committed
  374c6e1

• Tianshou to evotorch wrapper

  korneelf1

  committed
  04e5e89

Commits on Aug 13, 2024

• Normalize motor speeds

  korneelf1

  committed
  0939233

• Remove test statements for reward printing

  korneelf1

  committed
  3481caa

• Cant remove psets for now

  korneelf1

  committed
  1aafac8

• N=1 different return in step

  korneelf1

  committed
  32a1376

• reward function from git reward_squared_fast_learning

  korneelf1

  committed
  795a669

• dont normalize, 1e-4 learning rate with no scheduling

  korneelf1

  committed
  7ad36bd

• allow tianshou policy with evotorch

  korneelf1

  committed
  03926f8

Commits on Aug 14, 2024

• Improved reset mechanism, initially completely random, but might reset to impossible states

  korneelf1

  committed
  b6ae688

• Allwo multidim quaternion init

  korneelf1

  committed
  9842ea6

• allow multiple lr schedulers

  korneelf1

  committed
  d4fb96e

• step size of 1k for lr schedulers

  korneelf1

  committed
  9c58498

• format gym env

  korneelf1

  committed
  a17f5c5

• CLEANUP: clean up gym_sim file, helpers file

  Show description for def3b11
  korneelf1

  committed
  def3b11

• Something of a test

  korneelf1

  committed
  9549371

• Cleaned up file structure

  korneelf1

  committed
  ac0b222

• Syncing with OG repo

  korneelf1

  committed
  dece0bf

• test dones and single steps

  korneelf1

  committed
  67d0e6a

Commits on Aug 15, 2024

• Fix of unittests

  korneelf1

  committed
  7174ac3