Possible bugs?

[amneetb]

@enricoande

[1]

reinforcement_learning_examples/invertedpendulum/Sarsa/episodeFA.m

Line 35 in 95627db

phi = getPhi(x,rbf);

It appears that here phi corresponds to the old state x rather than the new state xp. Is this intentional?

[2]

reinforcement_learning_examples/invertedpendulum/invertedpendulum.m

Line 112 in 95627db

plotBF(statelist);

The function file plotBF.m seems to be missing.

[3]

reinforcement_learning_examples/invertedpendulum/Qlearning/episode.m

Line 35 in 95627db

Q(s,a) = Q(s,a) + alpha * ( r + gamma*Q(sp,ap) - Q(s,a) );

It should be max(Q(sp,:)) instead of Q(sp,ap) for Q-learning as done in the r = -1 case

reinforcement_learning_examples/invertedpendulum/Qlearning/episode.m

Line 43 in 95627db

Q(s,a) = Q(s,a) + alpha * ( r + gamma*max(Q(sp,:)) - Q(s,a) );

[enricoande]

Well spotted - I agree with all your comments and they are bugs to be solved. Feel free to make the changes on the branch and submit them to the master.
Thank you!