BRENT@uwovax.UWO.CDN (Brent Sterner) (03/18/87)
Hmmm, another "interaction" with management. I have proposed
that our site implement a "background batch" queue, and charge
certain users of that queue at a discounted rate. This sort of
queue would run with a priority less than interactive. In fact
I suggested a "long" and "fast" version of this queue type, with
priorities of 2 and 3 (relative to 4 for interactive users).
There is a fear that if these queues are allowed to continue
to run while the system is busy with interactive users, that the
interactive users will be adversely affected. I have argued that
any such effect will be minor to the magnitude of being not
detectable to 99% of interactive users. Obviously, if I'm right
and can "prove" myself, (a) I face no batch queue management
hastles, and (b) we can effectivel utilize "NULL" job cycles
(and generate income from them) during the day.
My question to the wizards: "Is our management's fear in
any way justified?" What can I do to alleviate the fear if in
fact it IS justified? I'm aware of the ways that queues can be
controlled. But we have *other* systems here whose interactive
performance IS (apparently) affected by batch (CYBER/NOS to be
specific). Is VMS being tarred with the same brush unfairly?
I know. Another very general question requiring a specific
answer (I try not to ask "easy" questions). Thanks to all who
I'm sure will reply. Brent.
--
Brent Sterner ***********************
Lord Protector, d i g i t a l Systems * *
Computing & Communications Services * ...this space *
Natural Sciences Building * for rent... *
The University of Western Ontario * Apply within. *
London, Ontario, Canada N6A 5B7 * *
Telephone (519)661-2151 x6036 ***********************
Network <BRENT@uwovax.UWO.CDN> ! VAX 8600
<A105@UWOCC1.BITNET> ! IBM 4341TLI%Sargas@USC-OBERON.ARPA (Tony Li) (03/19/87)
Brent, Your management does have a legitamate concern. It is possible to notice the effects of low priority batch jobs under VMS, especially if the memory consumption of those processes is significant. Regardless of the priority of the process, if it starts paging or hogging memory, you will notice the effects. I think that the best way to demonstrate the effects to your management is to get permission to set up a limited test. Create a priority-2 queue with only 1 active job and a limited (but reasonable) amount of memory. And maybe a CPU time limit. Let it run and see if anyone notices any degradation.... As long as the batch job is reasonably behaved and you're not memory poor, the effects should not be noticeable. Cheers, Tony ;-) -------
sasaki@HARVARD.HARVARD.EDU (Marty Sasaki) (03/19/87)
Adding low priority batch jobs may have a dramatic impact on the overall performance of a VMS system, so it isn't as easy as saying yes or no to Brent's questions. As an example, suppose that the system with regular interactive loads is using 80% of the physical memory available (working sets have been adjusted and all of that). Now imagine adding a few batch jobs. Depending on what these added processes are doing, their individual quotas, working sets, and such, you might suddenly be using all of real memory (and needing more), and paging like crazy. If you have set your sysgen parameters right (or wrong) you might even be swapping. The scenario that I just presented is more likely than not, especially in a university environment. When researchers see the cheap rates, they will submit big programs. At Harvard many of the jobs were SPSS and SAS jobs with huge datasets, or simulations with lots of array manipulation (imagine a two dimensional fourier transfer on a 1024 by 1024 array :-) ). We had four batch queues, quick$batch, sys$batch, slow$batch and back$batch. Each with declining priority with quick$batch running at priority 5, but with a 5 minute CPU limit. (Actually, we had another batch queue sysonly$batch which we used for system things like accounting and special backup routines. There is also another phenomena which might not be an issue on version 4 of VMS, but was on version 3. It was possible for processes to get in a state where batch jobs, with the priority boost after doing disk i/o, would steal CPU cycles from interactive programs. This was most noticeable with programs like EDT and Emacs which would momentarily go compute bound while calculating screen updates. This would cause their priority to fall below batch jobs. If there were enough batch jobs doing disk i/o followed by a little calculation, but not enough to bump their priority down, the "interactive jobs" might have to wait a long time to get the CPU. At Harvard, my solution to this problem was to move the base priority of interactive jobs up to 7. I ran the version 4 systems this way too, but never tested to see if this problem occured on version 4. Having a slightly higher load might mean tweaking sysgen parameters a little. When I increased the job limit on the queues, I played with the paging/swapping parameters, QUANTUM, and IOTA to squeeze a little bit more out of the system. So, the answer is, "it depends". ---------------- Marty Sasaki uucp: harvard!sasaki Ziff Davis Technical Information Co. arpa: sasaki@harvard.harvard.edu 80 Blanchard Road bitnet: sasaki@harvunxh Burlington, MA 01803 phone: 617-273-5500