emmo@moncam.co.uk (Dave Emmerson) (08/06/89)
Continuing the discussion prompted by jbww@ukc.ac.uk (J.B.W.Webber) Quote: (for those who missed this wish) thus the user is up and running, without having to wait for the whole network to boot, ramdisks/caches to be filled. By the time one is ready to, say, C compile, the compiler parts, libraries etc. are sitting in memory, on the relevant processors. i.e. the first part of a pipeline of processes : by the time the data pops out of the first part of the pipeline, the program can be in place for the next. Endquote Well I won't trash the idea, it doesn't deserve it, but if you think on it a bit further, you'll realise that parallel PROCESSORS aren't enough. Your programmes, libraries, data etc.. are stored as SERIAL data, so you can only load one item (part item if you multi-task and thrash the drive) at a time. To realise your wish, you'll have to store executables on one drive, and data (sources) on another. Then multi processors with both private (during upload) and shared (during e.g. compile) memory areas can both access their resources simultaneously. It'll get a bit messy when you have to decide exactly what to keep on each drive, and no doubt you'll end up with some duplication, with all the associated problems maintaining 2 copies.. Maybe someone'll do it one day. If it weren't for the dreamers among us we'd still be living in trees. Dave E.
jbww@ukc.ac.uk (J.B.W.Webber) (08/08/89)
In article <259@marvin.moncam.co.uk> emmo@moncam.co.uk (Dave Emmerson) writes: > >Continuing the discussion prompted by jbww@ukc.ac.uk (J.B.W.Webber) > >> thus the user is up and running, without having to wait for >> the whole network to boot, ramdisks/caches to be filled. >> >> by the time the data pops out of the first part of the pipeline, >> the program can be in place for the next. > >... you'll have to store executables on one drive, data (sources) on another. >Maybe someone'll do it one day. > >Dave E. Yes, sure, if your problem is I/O limited, more disks will help; our main problem (molecular dynamics simulations of liquids) is not : a few numbers in, a few numbers out (except for any tempory files, to allow for recovery from power outs, etc) - on a Vax 750, nearly single user, it was about one new point on the graph per week. for moderate amounts of data, the problem can be managed; consider : We usually boot a 2Mbyte MWC ramdisk into a Mega4's memory, at power on; this gives us the Helios OS, resources, libraries, etc on ramdisk. The transputer network boots from this; any parts or the network that need to be booted/rebooted later can come up fast from this. The rest of this ramdisk (and the ST4's menory) can be used for data, and each processor has at least 1M of memory, some of which can be used to squirrel data away on (preferably where it will be needed). If there are multiple STs, these drives can be used. The local drive is assumed, unless otherwise specified. Yes, sure, it is possible to get confused as to what is where (ever done a recursive directory listing on a network of Suns?), but, as always, placing the data where it is needed is the best strategy. Oh, and yes, the fastest way of processing a small amount of data is to give each of the processors that is to work on it, its own copy. We actually use something called a systollic loop - there are two copies of the data, and one circulates round and round a `race-track' of processors, getting modified as it goes. P.S. anyone up to writing an eternal ramdisk bigger than 2M. (preferably that can be resized without rebooting) Ah, well, one can always hope ! I should be on holiday, I am writing too much. cheers, beau webber ukc.ac.uk