hartley@uvm-cs.UUCP (Stephen J. Hartley) (02/13/85)
I don't know if this is a bug in deqn, ditroff, or the DEC LN01 postprocessor
that we have (courtesy of the Unievrsity of Wisconsin). The following ditroff
input produces some bad subscripts. HOWEVER if one changes from 12 point to
10 point (remove .ps 12, .sz 12, .nr pp 12, gsize 12), the problem goes away.
In the line beginning "Thus, if I=...", the subscripts get closer and closer
to the variables as we go towards the right (in 12 point version). By the time
we get to "a sub M", the "M" overlaps the "a". Any ideas? Thanks.
---------------------(cut here and remove leading blank)------------------------
.\" ditroff -me
.ps 12
.sz 12
.nr pp 12
.po 1.0i
.ll 6.5i
.na
.ls 1
.EQ
delim @@
gsize 12
.EN
.hy 1
.pp
A multimodule Markov Decision Process (3MDP) is a specially
structured MDP with the following characteristics.
The system is composed of M modules, each of which has the dynamic structure
of an MDF, that is each module has its own state and action
spaces and transition probabilities.
The state of the entire system is therefore a vector
@( s sub 1 , s sub 2 , ... , s sub M )@ where @s sub m@ is the state
of module @m@, and likewise any action taken is a vector
@( a sub 1 , a sub 2 , ... , a sub M )@ where @a sub m@ describes
the action to be performed for module @m@.
The probability of module @m@ making a transition from its own state @i@
to @j@ when action @a@ is applied locally is given by @P sub ij sup m (a)@.
We assume that the modules make transitions independently.
.pp
Thus, if @I = ( i sub 1 , ... , i sub M )@, @J = ( j sub 1 , ... , j sub M )@,
and @A = ( a sub 1 , ... , a sub M )@, we see
.EQ
P sub IJ (A) = prod from m=1 to M P sub {{i sub m} {j sub m}} sup m ( a sub m ) .
.EN
Although the module dynamics are independent, applying a control to an
individual module causes costs to be incurred by the overall system
that depend on the states of the other modules.
Hence the cost structure can only be described globally.
Because of this dependence, a global optimal control rule cannot
be found by simply applying the optimal controls for each module.
It is necessary to solve the accompanying large scale MDP to find
this optimum.
Unfortunately, such a large scale problem is far too costly to
solve exactly.
--------------------------------------------------------------------------------
--
"If that's true, then I'm the Pope!" Stephen J. Hartley
USENET: decvax!dartvax!uvm-gen!uvm-cs!hartley The University of Vermont
CSNET: hartley%uvm@csnet-relay (802) 656-3330ed@mtxinu.UUCP (Ed Gould) (02/27/85)
> > I don't know if this is a bug in deqn, ditroff, or the DEC LN01 postprocessor > that we have (courtesy of the Unievrsity of Wisconsin). The following ditroff > input produces some bad subscripts. HOWEVER if one changes from 12 point to > 10 point (remove .ps 12, .sz 12, .nr pp 12, gsize 12), the problem goes away. > In the line beginning "Thus, if I=...", the subscripts get closer and closer > to the variables as we go towards the right (in 12 point version). By the time > we get to "a sub M", the "M" overlaps the "a". Any ideas? Thanks. The text ran fine on my QMS, using the postprocessor I wrote for QMS. The only thought I have is that eqn needs to know the resolution of the output device. Does it correctly know 300/inch? There's also a comment in the eqn manual about it being tuned for Roman, Italic, and Bold. If your fonts are of significantly different widths and/or aspect rations, that could be the problem, too. -- Ed Gould mt Xinu, 739 Allston Way, Berkeley, CA 94710 USA {ucbvax,decvax}!mtxinu!ed +1 415 644 0146