george@minster.york.ac.uk (02/01/91)
Has anyone implemented the pole balancing simulator whoses equations are
given in Barto and Sutton "Neuronlike Adaptive Elements That Can Solve
Difficult Learning Control Problems" 1983 IEEE Trans Sys Man & Cyb??? i.e.
the two equations given in the appendix.
I have just coded them (see below), but the behaviour of the simulator is not
reasonable, e.g. pole at angle 10 degrees say, small velocity to left, then
pole should just fall down increasing velocity of cart. However, actually
oscillates between the two - keeping the pole balanced! ;-)
Assumed that sgn() meant a function that returns +1 for +ve numbers (including
0), and -1 for -ve numbers.
Does anyone know what is wrong with the equations?? Otherwise, would be
glad to have alternative equations for cart acceleration and pole
acceleration (taking into account friction at cart/pole and cart/track
interfaces).
Many thanks - George Bolt
pole_acc = ( GRAVITY * sin(pole_angle) +
cos(pole_angle) *
( -1.0 * ((double) impulse_direction) * IMPULSE -
POLE_MASS * POLE_HLENGTH * pole_vel * pole_vel * sin(pole_angle) +
CART_FRIC * sgn(cart_vel)
) /
( CART_MASS + POLE_MASS ) -
POLE_FRIC * pole_vel / (POLE_MASS * POLE_HLENGTH)
) /
( POLE_HLENGTH *
( 4.0 / 3.0 -
POLE_MASS * cos(pole_angle) * cos(pole_angle) /
(CART_MASS + POLE_MASS)
)
)
cart_acc = ( ((double) impulse_direction) * IMPULSE +
POLE_MASS * POLE_HLENGTH *
( pole_vel * pole_vel * sin(pole_angle) -
pole_acc * cos(pole_angle)
) -
CART_FRIC * sgn(cart_vel)
) /
( CART_MASS + POLE_MASS );
____________________________________________________________
George Bolt, Advanced Computer Architecture Group,
Dept. of Computer Science, University of York, Heslington,
YORK. YO1 5DD. UK. Tel: [044] (0904) 432771
george@uk.ac.york.minster JANET
george%minster.york.ac.uk@nsfnet-relay.ac.uk ARPA
george!mcsun!ukc!minster!george UUCP
____________________________________________________________christopher@gacvx2.gac.edu (02/03/91)
Hmmmmm.... can't help you right off with the equations, but if you can get
at a NeXT computer, there is a neural network simulation on it called
Balancer. It simulates a seal learning to balance a pole on its nose,
illustrating unsupervised learning in a pain of hidden layered neural nets.
The specifics come from the 1986 dissertation of Charles Anderson at the
University of Massachusetts ("Learning and Problem Solving With Multilayer
Connectionist Systems"). The demo, however doesn't go into a lot of detail
on the implementation, but is fun to watch.
Assuming you can get at a NeXT, the program is located in the
/NextDeveloper/Demos directory.
Christopher Kane
kane@nic.gac.edu
Gustavus Adolphus College St. Peter, Minnesotattj10@eng.cam.ac.uk (T.T.Jervis) (02/04/91)
I have implemented Sutton and Barto's pole balancing work. I have
obtained results that agree with those found in their paper.
Without looking into your problem in detail, it would seem a sign may
be astray. Incidentally, your comment regarding the reasonable
behaviour of your simulation requires an assumption to be made as to
the sense of "... e.g. pole at angle 10 degrees say, ..." with respect
to your "... small velocity to left, ...".
I found that working through the simulation equations myself sorted
out any problems, especialy regarding the sign of angles and angular
velocities, and helped in the assessment of the relevance of the
simulation. Again, the equations I derived for myself were of the same
form as those in the paper.
I am incidentally running pole-balancing code on transputer hardware
linked to a real pole rig of the same form as the simultion in Sutton
and Barto's paper. Their controller needs to be modified for a plant
with differing masses and inertias, but preliminary results would seem
to indicate that an improvement in time to failure can be gained over
a period of some tens of minutes of operation. I would be interested
to hear of anyone else involved in this kind of control.
Good luck with the code!
--
_________ __ __ ___
/________/| /_/| /_ /\ /__/| Cambridge University
|__ ___|/ | || | \ \/ || Engineering Departmentttj10@eng.cam.ac.uk (T.T.Jervis) (02/04/91)
I have implemented Sutton and Barto's pole balancing work. I have obtained results that agree with those found in their paper. Without looking into your problem in detail, it would seem a sign may be astray. Incidentally, your comment regarding the reasonable behaviour of your simulation requires an assumption to be made as to the sense of "... e.g. pole at angle 10 degrees say, ..." with respect to your "... small velocity to left, ...". I found that working through the simulation equations myself sorted out any problems, especialy regarding the sign of angles and angular velocities, and helped in the assessment of the relevance of the simulation. Again, the equations I derived for myself were of the same form as those in the paper. I am incidentally running pole-balancing code on transputer hardware linked to a real pole rig of the same form as the simultion in Sutton and Barto's paper. Their controller needs to be modified for a plant with differing masses and inertias, but preliminary results would seem to indicate that an improvement in time to failure can be gained over a period of some tens of minutes of operation. I would be interested to hear of anyone else involved in this kind of control. Good luck with the code! _________ __ __ ___ /________/| /_/| /_ /\ /__/| Cambridge University |__ ___|/ | || | \ \/ || Engineering Department | || | || | |\ \/ /| || | || | || | ||\__/ | || email: ttj10@eng.cam.ac.uk |_|/ |_|/ |_|/ |_|/ phone: +44 223 440 921
ahlenius@motcid.UUCP (Mark Ahlenius) (02/04/91)
In reference to the notes on the broom balancing acts, also known as the inverted pendulum problem (right?) - I have seen a demo of a fuzzy logic approach to solve ths problem in a simulated fashion. It was solved for the 1, 2 and 3 stage inverted pendulm problem and the demo looked pretty good - though I did not get to see the code. Has anyone else seen or worked with this approach? 'mark -- =============== regards 'mark ============================================= Mark Ahlenius voice:(708)-632-5346 email: uunet!motcid!ahleniusm Motorola Inc. fax: (708)-632-2413 Arlington, Hts. IL, USA 60004
sontag@hilbert.rutgers.edu (Eduardo Sontag) (02/07/91)
Equations for this problem, as well as various results using standard control techniques, can be found in: Eduardo D. Sontag, __Mathematical Control Theory: Deterministic Finite Dimensional Systems__ Springer, New York, 1990. (396+xiii pages, ISBN 0-387-97366-4; 3-540-97366-4) (Series: Textbooks in Applied Mathematics, Number 6.) Look under "broom balancing" in the index... -eduardo PS: Of course, almost any other control book will do... -- Eduardo D. Sontag Department of Mathematics Rutgers Center for Systems and Control (SYCON) Rutgers University New Brunswick, NJ 08903, USA (Phone: (908)932-3072; dept.: (908)932-2390) sontag@hilbert.rutgers.edu ...!rutgers!hilbert.rutgers.edu!sontag sontag@pisces.bitnet
jalloul@elbereth.rutgers.edu (Louay M.A. Jalloul) (05/18/91)
Would you please Email me some information about the pole balancing problem and its NN modeling. Also, please Email me some information about Drive Reiforcement Theory (DRT). *** Please Email the above requested information to assaleh@caip.rutgers.edu. -------------------------