Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extensibility to multi-agent settings #62

Open
drewstone opened this issue Jun 2, 2019 · 2 comments
Open

Extensibility to multi-agent settings #62

drewstone opened this issue Jun 2, 2019 · 2 comments

Comments

@drewstone
Copy link

Hello, I'm very new to this repo. Is this easily extensible to multi-agent settings? A bit about the problem I want to codify:

  • N agents uniformly distributed on the real line, with N odd.
  • Reward for agent $i$ at timestep $t$ is the magnitude of the intervals that are closest to $i$ and not $j$ forall $j\neq i$.
  • If two agents share the same interval, they each either half the reward or I'll flip a coin with probability 1/2 (or 1/k for k sharing) to select the agent to reward.

If you're familiar, it's a hotelling model with no Nash equilibrium. Gonna dive into the repo but maybe you can point me to a good place to start! Thanks :)

@tspooner
Copy link
Owner

tspooner commented Jun 3, 2019

Hey, good to hear from you. I realise there's not a lot of documentation or diverse use case examples yet so I apologise for that.

The problem sounds interesting but none of the files in "examples" will be relevant. What I suggest is to avoid the SerialExperiment-type handlers I've included. These are for single-agent problems only at the moment. Instead, you'll need to code up a Rust version of your model and get it working without any RL code. Then, once you can pass a Vec (for example) of actions and return a commensurate Vec of rewards, then start creating instances of the algos I've implemented. You will need to do all of the "for i in episode { for step in steps {"-type code yourself for now I'm afraid.

I'll keep this issue open so you can ask any further Qs and I'll do my best to help!

Regards,
Tom

N.B.
As I've mentioned on other threads, this is still under active development. In fact, I have some major changes that I am finishing off that I will push up to the repo soon. One of the things I will focus on next is "infrastructure" code. I.E. automating runs for both single- and multi-agent cases. The framework actually does support this, but it's all a bit hacky at the moment as mentioned - I recently completed some work using RSRL for a two-player zero-sum game project; simple but still multi-agent I guess!

@drewstone
Copy link
Author

Awesome, thanks for the info @tspooner !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants