[comp.sys.mac.programmer] Distributed Data Collection

mkb@rover.ri.cmu.edu (Mike Blackwell) (06/09/89)

I'm building a system which must collect data from a number of Macs, and
save it in a central database. The Macs are interconnected with AppleTalk.
The distrbuted Macs (perhaps 10 - 15 of them) only need to generate data (on
the order of 2 - 3 100 byte packets per minute -- slow), and the database
machine only needs to receive the data - information flow can be one
direction. The database file will need to be off-loaded daily, preferably
without having to shut the system down.

I've thought of two methods to acheive this. One is to have the database
machine simply be an AppleShare server. The clients then open, append, and
close the database file as needed. Simple, but I wonder how reliable this
will be - it seems that failure of one client can wedge the system, or
worse, corrupt the database. The second is to write a simple ATP
server/client setup where the server can receive packets and append them to
the database. This is more work to implement, but might be more reliable.

So my question - pros and cons to these two methods? Have I overlooked
anything? Your comments greatly appreciated.

		cheers, -m-

		Mike Blackwell		(mkb@rover.ri.cmu.edu)
-- 

minow@mountn.dec.com (Martin Minow) (06/09/89)

In article <5167@pt.cs.cmu.edu> mkb@rover.ri.cmu.edu (Mike Blackwell)
describes a system he's building that will collect data from a number
of Macs which will be saved in a central database.

I did exactly that for program that was used in the pressroom of this
year's Boston Marathon: two computers collected intermediate timings
and transmitted the results to two other computers that managed large
screen displays.

I programmed this using AppleTalk: the display managers were servers
to the data collection clients, and think this was a good choice.  I
actually wrote both the server and client modules in the same application
with a startup dialog that let the user choose between "server," "client",
or "both" -- the latter let me debug the entire application on a single
computer without a network connection.

The overall event loop had (very roughly) the following organization:

	for (;;) {
	    ... normal Macintosh event processing ...
	    for (all server packets) {
		if (server_request[i]->atp.ioRequest <= 0)
		    process_incoming_server_packet();
	    }
	    for (all client packets) {
		if (client_request[i]->atp.ioRequest <= 0) {
		    check for errors, then free the packet;
		}
	    }
	}

The client (data entry side) sends a message by getting a free packet,
loading it up, setting the atp.ioRequest to +1 (can't be a result code)
and sending it by PSendRequest(&client_request[i]->atp, ASYNC).  When
the request completes, process_client_server_packet() checks for errors
or server response status and frees the packet.

The server (display side) queues up all of its free packets for asychronous
reception (calling PGetRequest(..., ASYNC)).  Incoming server packets
are checked for errors, then handled by message-type specific handlers and
eventually requeued for another request.

Note that all I/O is asychronous.  I/O completion is detected in the event
loop polling sequence.

Hope this helps.

Martin Minow
minow%thundr.dec@decwrl.dec.com

Raines.Cohen@f444.n161.z1.FIDONET.ORG (Raines Cohen) (06/10/89)

 Mike -
 
I'm facing a similar problem, with different constraints, in a
project I'm working on.
 
This one, I'm doing in SuperCard, and using the HyperAppleTalk
XCMDs from Gava systems to handle the ATP side - I just give it
a message to pass, and it sends it.
 
-- Raines Cohen
 SYSOP, BMUG BBS
 
P.S. I'm putting hard returns on this message - are they
making it out to the net?

--  
-------------------------------------------------------------
FidoNet:  1:161/445      UUCP: sun!apple!bmug!<User.Name>
INTERNET:  bmug!<User.Name>@apple.COM or <User.Name>@bmug.fidonet.org
USNAIL:   BMUG, 1442A Walnut St. #62, Berkeley, CA 94709-1496
-------------------------------------------------------------
BMUG Newsletter articles due June 15!  Authors get free membership.
Send articles to:  pub@bmug.fidonet.org