Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Puffer 0.4 #18

Merged
merged 431 commits into from
Sep 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
431 commits
Select commit Hold shift + click to select a range
ac4063e
cp
daveey Jul 2, 2023
c0832c1
fid pool
daveey Jul 2, 2023
fa75f6b
cp
daveey Jul 2, 2023
f72c0b6
cp
daveey Jul 2, 2023
4174ac5
fix import
daveey Jul 2, 2023
3cf55b8
fix import
daveey Jul 2, 2023
fa58a01
move model to device
daveey Jul 2, 2023
2ded322
move model to device
daveey Jul 2, 2023
fc19db6
new defaults
daveey Jul 3, 2023
873aa95
Setting Model to eval mode for more consistent embeddings for same input
nikhilpinnaparaju Jul 3, 2023
c0daa94
fix evaluate
daveey Jul 3, 2023
6288d8f
fix evaluate
daveey Jul 3, 2023
5b8011d
fix evaluate
daveey Jul 3, 2023
b643ece
fix evaluate
daveey Jul 3, 2023
6e00e99
fix evaluate
daveey Jul 4, 2023
dabdb19
cp
daveey Jul 4, 2023
ba1d978
cp
daveey Jul 4, 2023
b570996
cp
daveey Jul 4, 2023
2845dd5
cp
daveey Jul 4, 2023
5e42407
cp
daveey Jul 4, 2023
84159bd
Update for new sql pool branch
jsuarez5341 Jul 5, 2023
99a07b2
set ModelArchitecture.NUM_PLAYERS_PER_TEAM to 1
kywch Jul 5, 2023
3d2d3b7
cp
daveey Jul 5, 2023
9dc43c5
Merge branch 'pool' into pool-sql
daveey Jul 5, 2023
e86848a
cp
daveey Jul 5, 2023
38d0000
cp
daveey Jul 5, 2023
52a16b2
cp
daveey Jul 5, 2023
9e489bc
cp
daveey Jul 5, 2023
7e798e7
cp
daveey Jul 5, 2023
3d3fbf0
some fixes for realikun
kywch Jul 6, 2023
9f37f9a
cp
daveey Jul 6, 2023
253c4ed
cp
daveey Jul 6, 2023
65b4384
progress up to lstm_state nan error
kywch Jul 6, 2023
55a4c15
cp
daveey Jul 6, 2023
010c68d
cp
daveey Jul 6, 2023
2fe9505
cp
daveey Jul 6, 2023
1e982d1
cp
daveey Jul 7, 2023
d7376e2
made trainer.evaluate() run
kywch Jul 7, 2023
5e2bc01
cp
daveey Jul 9, 2023
6415b15
cp
daveey Jul 9, 2023
536d179
cp
daveey Jul 9, 2023
5551487
cp
daveey Jul 10, 2023
200bd22
cp
daveey Jul 10, 2023
cc95f06
cp
daveey Jul 10, 2023
2dafbda
cp
daveey Jul 10, 2023
bc39f52
cp
daveey Jul 10, 2023
38af005
Fix NaNs caused by bad attention masking
jsuarez5341 Jul 11, 2023
20c7691
cp
daveey Jul 12, 2023
5189909
cp
daveey Jul 12, 2023
52b3707
cp
daveey Jul 12, 2023
c6bfd0b
cp
daveey Jul 12, 2023
c499e24
cp
daveey Jul 12, 2023
616eb09
Temporary hack to fix action space
jsuarez5341 Jul 13, 2023
6689cc0
actions hack
daveey Jul 13, 2023
c2a583e
cp
daveey Jul 13, 2023
ce8b16f
cp
daveey Jul 14, 2023
f0e7e0d
Improved policy with decode. Run as model_type=decode. Does not work …
jsuarez5341 Jul 14, 2023
08bec75
Merge branch 'daveey-pool-realikun' of https://github.com/carperai/nm…
jsuarez5341 Jul 14, 2023
adb00b5
cp
daveey Jul 14, 2023
dad0ef0
Merge remote-tracking branch 'origin/daveey-pool-realikun' into davee…
daveey Jul 14, 2023
a9c5a5e
cp
daveey Jul 14, 2023
1043bcb
cp
daveey Jul 14, 2023
5a4dda1
cp
daveey Jul 14, 2023
9cdc89f
cp
daveey Jul 14, 2023
0d3fda6
cp
daveey Jul 14, 2023
6fb13b9
cp
daveey Jul 14, 2023
064ae4b
tweaked manual curriculum, use TaskSpec and removed team tasks
kywch Jul 14, 2023
c65eb0e
made elm_for_nmmo work with TaskSpec class
kywch Jul 14, 2023
be8281d
made task encoder work with TaskSpec class, refactored save to file
kywch Jul 14, 2023
1b996b4
modified the end-to-end script to sample tasks from curriculum file
kywch Jul 14, 2023
2b97388
Update submission_curriculum_track.py
kywch Jul 14, 2023
cd3c910
cp
daveey Jul 14, 2023
ae22cec
openskill
daveey Jul 14, 2023
9c493dd
fix ranker
daveey Jul 14, 2023
a9ed03a
cp
daveey Jul 14, 2023
c470d2f
openskill
daveey Jul 14, 2023
ad99b97
openskill
daveey Jul 14, 2023
950427c
openskill
daveey Jul 14, 2023
8bb2b54
merge task embed
daveey Jul 14, 2023
48055df
format
daveey Jul 14, 2023
04bc9cc
adds pickled tasks
daveey Jul 14, 2023
6e1c6f7
adds task embedding
daveey Jul 14, 2023
bc7a96a
Remove unused code
daveey Jul 14, 2023
9011f19
cp
daveey Jul 15, 2023
c94f62c
cp
daveey Jul 15, 2023
22eb8d2
cp
daveey Jul 15, 2023
086a10b
cp
daveey Jul 15, 2023
dee3e50
cp
daveey Jul 15, 2023
568a4d2
cp
daveey Jul 15, 2023
cdc3385
merged in 2.0, resolved conflicts
kywch Jul 15, 2023
9d04967
Merge pull request #39 from CarperAI/curriculum
kywch Jul 15, 2023
c3a7401
cp
daveey Jul 16, 2023
555d168
cp
daveey Jul 16, 2023
7b2366a
cp
daveey Jul 16, 2023
b67c2fe
cp
daveey Jul 16, 2023
1b430e7
policy args
daveey Jul 16, 2023
8aa18a3
policy args
daveey Jul 16, 2023
0e3bed0
policy args
daveey Jul 16, 2023
8b86864
cp
daveey Jul 16, 2023
9e5c8e4
cp
daveey Jul 16, 2023
33a8313
cleanup
daveey Jul 17, 2023
26f60e2
cleanup
daveey Jul 17, 2023
4922229
cleanup
daveey Jul 17, 2023
2ec196d
add arg for tasks
daveey Jul 17, 2023
f7f65ed
Mask, task encode, and recurrence fixes
jsuarez5341 Jul 18, 2023
376d051
Merge branch '2.0' into patch-1
daveey Jul 18, 2023
a73b20d
Merge pull request #38 from MarkHaoxiang/patch-1
daveey Jul 18, 2023
aebc5ce
cp
daveey Jul 18, 2023
5507f71
Merge remote-tracking branch 'origin/2.0' into daveey-cleanup
daveey Jul 18, 2023
4e14b75
cp
daveey Jul 18, 2023
eaaadba
cp
daveey Jul 18, 2023
5d4c6e3
cp
daveey Jul 18, 2023
a638bd4
added cod stats
kywch Jul 18, 2023
98397d7
removed task generator
kywch Jul 18, 2023
4aa6912
Merge pull request #42 from CarperAI/stat-cod
daveey Jul 19, 2023
bfcb037
add tasks with embeddings
daveey Jul 19, 2023
0ac984c
make lstm non default
daveey Jul 19, 2023
86c5a6e
made cod output between 0 - 1
kywch Jul 19, 2023
e48de89
Fix ablations + add some new ones
jsuarez5341 Jul 19, 2023
79583f1
merge
daveey Jul 19, 2023
15c3a76
merge
daveey Jul 19, 2023
63fd051
log cod on death tick only
kywch Jul 19, 2023
5479aad
Small tweak to masking logic
jsuarez5341 Jul 19, 2023
23fdd11
Merge branch 'decode-improvements' of https://github.com/carperai/nmm…
jsuarez5341 Jul 19, 2023
7353a23
fixed cod stats, added event-based stats
kywch Jul 20, 2023
7c96cc1
removed unnecessary lines
kywch Jul 20, 2023
676af33
refactored _reset_episode_stats, removed print()
kywch Jul 20, 2023
ce64673
Merge pull request #43 from CarperAI/fix-stats
kywch Jul 20, 2023
93bd3f0
Add mask key ablations: all, move, exclude-attack, none
jsuarez5341 Jul 20, 2023
1f12288
cp
daveey Jul 20, 2023
a5c927d
cp
daveey Jul 20, 2023
e7f56e1
change model defaults
daveey Jul 20, 2023
0a6ffff
Add Nikil and PyTorch task attention mechanisms
jsuarez5341 Jul 20, 2023
68f3b09
Fix merge conflict
jsuarez5341 Jul 20, 2023
76ca94d
fix masking
daveey Jul 21, 2023
a2bfd9f
fix masking
daveey Jul 21, 2023
4415264
fix masking
daveey Jul 21, 2023
d6bcbb8
skip connect attention
jsuarez5341 Jul 21, 2023
8670acd
Fix merge conflict
jsuarez5341 Jul 21, 2023
fdd1178
cp
daveey Jul 21, 2023
adba427
cp
daveey Jul 21, 2023
c4a245b
cp'
daveey Jul 21, 2023
f818993
cp
daveey Jul 22, 2023
d0c0b8b
cp
daveey Jul 23, 2023
bd68a0d
cp
daveey Jul 24, 2023
65ded2f
support noop actions
daveey Jul 24, 2023
78888eb
Initial cleanup for release
jsuarez5341 Jul 25, 2023
3c7b9a6
shuffle around directories
kywch Jul 25, 2023
bdab2f5
clean up tests, made these run
kywch Jul 25, 2023
e9e014e
renamed elm env
kywch Jul 25, 2023
bfdb9dc
Merge pull request #44 from CarperAI/curri-shuffle
jsuarez5341 Jul 25, 2023
60d9dc3
Big cleanup. ELM not working yet though
jsuarez5341 Jul 26, 2023
f81f0e9
made the nmmo elm test work
kywch Jul 26, 2023
245097e
Merge pull request #45 from CarperAI/fix-elm
jsuarez5341 Jul 26, 2023
c7bd816
LLM agent initial commit
jsuarez5341 Jul 27, 2023
39d5688
added player stat
kywch Jul 27, 2023
ed48d72
Visualize embeddings... relies on some moving of code around until pi…
jsuarez5341 Jul 27, 2023
dace634
Merge branch 'release' of https://github.com/carperai/nmmo-baselines …
jsuarez5341 Jul 27, 2023
f609bee
Cleanup and refactor all the things
jsuarez5341 Jul 29, 2023
cd4d3aa
fixed curriculm-related stuff
kywch Jul 29, 2023
b672158
select learnable tasks based on task progress metrics
kywch Jul 29, 2023
166252a
Merge pull request #47 from CarperAI/patch-rel
jsuarez5341 Jul 30, 2023
267e489
Merge pull request #48 from CarperAI/learnable
jsuarez5341 Jul 30, 2023
b88177c
Merge pull request #48 from CarperAI/learnable
kywch Jul 30, 2023
9070cad
added team result for leaderboard, added deps
kywch Aug 7, 2023
6fdec88
added names method to TeamResult
kywch Aug 7, 2023
7f6102d
Merge pull request #50 from CarperAI/leader-board
jsuarez5341 Aug 7, 2023
83d64e6
use codegen25 for task embedding and elm, few fixes
kywch Aug 9, 2023
68a2f12
fixed the broken curriculum-related var
kywch Aug 9, 2023
7abcdd7
Merge pull request #51 from CarperAI/codegen25
jsuarez5341 Aug 9, 2023
9914f8a
full config
daveey Aug 10, 2023
cbc740e
full config
daveey Aug 10, 2023
9251fff
full config
daveey Aug 10, 2023
493a5b3
full config
daveey Aug 10, 2023
161917a
full config
daveey Aug 10, 2023
d4e2ea6
full config
daveey Aug 10, 2023
1d4e92c
full config
daveey Aug 10, 2023
c9bbdfe
full config
daveey Aug 10, 2023
b7b55d5
full config
daveey Aug 10, 2023
944d899
full config
daveey Aug 10, 2023
00f94b5
Merge pull request #52 from CarperAI/daveey-release
jsuarez5341 Aug 10, 2023
a8662f1
added a tutorial for curriculum track
kywch Aug 11, 2023
36fba1e
few comment tweaks
kywch Aug 11, 2023
ed91604
Merge pull request #53 from CarperAI/curri-tutor
jsuarez5341 Aug 11, 2023
49427dc
separate curriculum files for RL and curriculum tracks
kywch Aug 11, 2023
384a3cb
Merge pull request #54 from CarperAI/sep-curri
jsuarez5341 Aug 11, 2023
7e85df7
cp
daveey Aug 11, 2023
4107ae3
merge
daveey Aug 11, 2023
6203564
Fix oom in local mode by reducing batch size
jsuarez5341 Aug 14, 2023
a761948
I got it to run but it gets killed? And .cuda hangs?
jsuarez5341 Aug 15, 2023
a04712e
Merge remote-tracking branch 'origin/release' into daveey-release
Aug 15, 2023
6d8ea4c
clearly specified curriculum (renamed from task_spec) in the curricul…
kywch Aug 15, 2023
96a1e3c
updated action targets keys to str
kywch Aug 16, 2023
842a129
Merge pull request #56 from CarperAI/str-mask
jsuarez5341 Aug 16, 2023
6beb996
Merge pull request #55 from CarperAI/rename-spec
jsuarez5341 Aug 16, 2023
703ed8c
made evalulate work, with some comments
kywch Aug 18, 2023
abe0d82
Merge pull request #57 from CarperAI/eval-py
jsuarez5341 Aug 18, 2023
5e194ec
removed debug lines, uncommented the actual code
kywch Aug 18, 2023
c979bea
Merge pull request #58 from CarperAI/eval-fix
kywch Aug 18, 2023
762b753
simplified environment.py Postprocessor by subclassing
kywch Aug 19, 2023
70dd770
fixed infos bug
kywch Aug 19, 2023
b37489d
added new wandb stats, etc
kywch Aug 22, 2023
0068944
added unique event-based reward
kywch Aug 22, 2023
9905965
Merge remote-tracking branch 'origin/release' into daveey-release
daveey Aug 22, 2023
d8134dc
merge
daveey Aug 22, 2023
bac045a
Merge pull request #59 from CarperAI/rev-postproc
jsuarez5341 Aug 23, 2023
90fd717
the setup for the current policy
kywch Aug 23, 2023
6d34bb6
Merge pull request #60 from CarperAI/for-sweep
daveey Aug 24, 2023
f7930fc
Merge remote-tracking branch 'origin/release' into daveey-release
daveey Aug 24, 2023
37ec148
sqrt rewards
daveey Aug 24, 2023
674d0c5
bonuses
daveey Aug 24, 2023
f869335
bonuses
daveey Aug 24, 2023
e861264
Merge pull request #61 from CarperAI/daveey-release
kywch Aug 24, 2023
c95b713
Begin puffer integrations
jsuarez5341 Aug 25, 2023
a5b0ccc
running
jsuarez5341 Aug 25, 2023
0a47a37
Remove unneeded horizon logic
jsuarez5341 Aug 26, 2023
7ff3a8f
migrated leader_board to new puffer
kywch Aug 26, 2023
6ed4141
fix infos
jsuarez5341 Aug 26, 2023
b669b2c
Merge branch 'puffer-integration' into kywch-puffer
jsuarez5341 Aug 26, 2023
5b16258
Merge pull request #63 from CarperAI/kywch-puffer
jsuarez5341 Aug 26, 2023
28fd736
added env early stop, removed task spam
kywch Aug 26, 2023
f9a921e
default val tweak
kywch Aug 26, 2023
90114ae
Merge pull request #64 from CarperAI/early-stop
jsuarez5341 Aug 26, 2023
9a7c242
add back task encoder to policy
kywch Aug 28, 2023
e7cfb36
Merge pull request #65 from CarperAI/task-back
jsuarez5341 Aug 28, 2023
6e39c59
enabled specify gpu device
kywch Aug 30, 2023
d117062
added arg for logging loss and actions
kywch Aug 30, 2023
4ac3ca4
fixed ranking feature
kywch Aug 31, 2023
2901817
support multi gpu
kywch Aug 31, 2023
f0a4b69
updated evaluate.py
kywch Aug 31, 2023
fbd3f04
quick bug fix on save replay
kywch Aug 31, 2023
2218f9c
upload trained checkpoints
kywch Aug 31, 2023
d74d05f
commented out record loss for debugging
kywch Aug 31, 2023
9536a6a
added seed, since puf 0.4 is deterministic
kywch Sep 1, 2023
2e0936d
Merge pull request #66 from CarperAI/multi-gpu
jsuarez5341 Sep 1, 2023
e030596
masked repeated move dir, price
kywch Sep 2, 2023
5b40818
count go-distance only once in unique event
kywch Sep 2, 2023
6b27d60
removed tasks encouraging one-way move
kywch Sep 2, 2023
7c1369c
fight entropy with entropy
kywch Sep 2, 2023
9b0bd18
allow neg meander bonus to create reward gradient
kywch Sep 2, 2023
2408e08
set default config for training
kywch Sep 3, 2023
7a8a9e4
tweaked lr, entity dim
kywch Sep 3, 2023
1d44a77
added clip_coef, tweaked ppo configs
kywch Sep 3, 2023
0b53e89
tweaked cfg
kywch Sep 4, 2023
f1a78e3
added arg for resilient agents
kywch Sep 4, 2023
b61a10a
included more profession tasks in curriculum
kywch Sep 4, 2023
35643cc
quick fix
kywch Sep 4, 2023
4d6d6ed
fixed eval script, new policy
kywch Sep 4, 2023
404aecc
Merge pull request #67 from CarperAI/mask-repeat
jsuarez5341 Sep 6, 2023
5b983bd
Merge pull request #68 from CarperAI/puffer-integration
jsuarez5341 Sep 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/pylint-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ jobs:
run: |
python -m pip install --upgrade pip setuptools wheel
pip install -r requirements.txt
pip install pytest==7.2.2
- name: Running unit tests
run: pytest
# NOTE: will put these back eventually
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ replay*/
checkpoints
wandb
maps
pool

# Byte-compiled / optimized / DLL files
__pycache__/
Expand Down
15 changes: 10 additions & 5 deletions .pylintrc
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
[MESSAGES CONTROL]

disable=W0511, # TODO/FIXME
W0105, # string is used as a statement
disable=C0103, # invalid name
C0114, # missing module docstring
C0115, # missing class docstring
C0116, # missing function docstring
W0221, # arguments differ from overridden method
C0301, # line too long
C0411, # wrong import order
C0415, # import outside toplevel
E0611, # no name in module
R0401, # cyclic import
R0901, # too many ancestors
R0902, # too many instance attributes
R0903, # too few public methods
Expand All @@ -17,7 +18,10 @@ disable=W0511, # TODO/FIXME
R0914, # too many local variables
R0914, # too many local variables
R0915, # too many statements
R0401, # cyclic import
W0105, # string is used as a statement
W1203, # use logging instead of print
W0221, # arguments differ from overridden method
W0511, # TODO/FIXME

[INDENTATION]
indent-string=' '
Expand All @@ -26,6 +30,7 @@ indent-string=' '
good-names-rgxs=^[_a-zA-Z][_a-z0-9]?$ # whitelist short variables
known-third-party=ordered_set,numpy,gym,pettingzoo,vec_noise,imageio,scipy,tqdm
load-plugins=pylint.extensions.bad_builtin
ignored-modules=torch

[BASIC]
bad-functions=print # checks if these functions are used
bad-functions=print # checks if these functions are used
Loading
Loading