[comp.sys.apollo] reducing the delay between edrgy and getpwuid

cameron@spectrum.cs.unsw.oz.au (Cameron Simpson) (02/18/91)

I have some home-grown software here we use for maintaining accounts. We have
several hundred students all requiring account each session, and thus the
interactive make-an-account-by-hand-with-edrgy approach is out, thus this
code. It basicly opens a few pipes and talks to edrgy and provides a function
call interface.

My problem is currently that after I've called the make-an-account call, and
edrgy's replied with a prompt indicating all is happiness, I often won't be
able to successfully do a getpwuid() or getpwnam() on said account for quite
a while (typically about a minute). Currently I sit in a loop polling
getpwuid() and sleeping for 10 seconds until it works, so that the program
can continue on its way and make home directories and set ownership, etc.

What I'd like is to learn of some way to poke the registry daemons (or the
mechanism getpwuid() uses) to reduce this delay as much as possible. Saying
`synch' to the local registry via edrgy has no obvious effect. Any ideas?
	- Cameron Simpson
	  cameron@spectrum.cs.unsw.oz.au

P.S.	Currently I'm contemplating mad schemes involving yet-another-child
	process which hadles a queue of pending new accounts which does the
	propagation waiting for me, and follows up with the chown stuff when
	it can, or inline queues which get polled at convenient spots to see
	if the new uid has propagated, etc. None of them are very nice.
--
"I guess this is what you get for being on the leading edge,
 you get cut sometimes."	- Douglas Rand

mmuegel@camdev.comm.mot.com (Michael Muegel) (02/20/91)

In article <1093@usage.csd.unsw.oz.au> cameron@spectrum.cs.unsw.oz.au writes:
>My problem is currently that after I've called the make-an-account call, and
>edrgy's replied with a prompt indicating all is happiness, I often won't be
>able to successfully do a getpwuid() or getpwnam() on said account for quite
>a while (typically about a minute). Currently I sit in a loop polling
>getpwuid() and sleeping for 10 seconds until it works, so that the program
>can continue on its way and make home directories and set ownership, etc.

In my Perl scripts I wrote for our admins I simply parse edrgy -v -a login
output. I know better than to even try to trust the /etc/passwd file ;-).
We have a fairly large network (~130) and it seems to take an unberably long
time for the passwd file to reflect the true state of the registry.

Thats my two and a half cents worth,
-Mike

-- 
+-----------------------------------------------------------------------------+
| Mike Muegel                              | Internet: mmuegel@mot.com        |
| Software Tools Group                     | UUCP:     uunet!motcid!muegel    |
| Fort Worth Research & Development Center | Voice:    (817) 232-6129         |
| Cellular Infrastructure Group            | Fax:      (817) 232-6081         |
| Radio Telephone and Systems Group        | Mail:     5555 North Beach St.   |
| Motorola, Inc.                           |           Fort Worth,  TX 76137  |
+-----------------------------------------------------------------------------+

system@alchemy.chem.utoronto.ca (System Admin (Mike Peterson)) (02/21/91)

In article <384@camdev.comm.mot.com?> mmuegel@camdev.comm.mot.com	 (Michael Muegel) writes:
>In article <1093@usage.csd.unsw.oz.au> cameron@spectrum.cs.unsw.oz.au writes:
>>My problem is currently that after I've called the make-an-account call, and
>>edrgy's replied with a prompt indicating all is happiness, I often won't be
>>able to successfully do a getpwuid() or getpwnam() on said account for quite
>>a while (typically about a minute). Currently I sit in a loop polling
>>getpwuid() and sleeping for 10 seconds until it works, so that the program
>>can continue on its way and make home directories and set ownership, etc.
>
>In my Perl scripts I wrote for our admins I simply parse edrgy -v -a login
>output. I know better than to even try to trust the /etc/passwd file ;-).
>We have a fairly large network (~130) and it seems to take an unberably long
>time for the passwd file to reflect the true state of the registry.

I can believe it. I added a 'sleep 15' after the account creation 
and before creating the new users directory in our Addusers script
(we have only 5 nodes, with 2 rgyd nodes, yet they still can't keep in
synch it seems). For some "reason", our DN10000 master registry node does ALL
its registry lookups on the DN4500 slave rgyd node (a great waste of time -
all such "network" requests should be satisfied locally if possible). 
Basing a critical service like authentication on a daemon is crazy; it
should be file based (a daemon system could be used to update the file,
retaining the "old" copy in case of a crash during the update).
This would avoid most of the problems of screwed registries, since the
master file could be editted by hand (from backups if necessary).
Unfortunately, I gather OSF has adopted the rgyd approach.
-- 
Mike Peterson, System Administrator, U/Toronto Department of Chemistry
E-mail: system@alchemy.chem.utoronto.ca
Tel: (416) 978-7094                  Fax: (416) 978-8775

tonyf@bnr.ca (Tony Farrow) (02/21/91)

We have experienced this problem, but have had to use greater delays in
the timing. 

#define         RegistryRetry   10
#define         RegistrySleep   20


 /* Loop til user has been added to the Registry or the RetryLimit is reached 

       for (i = 0; ((!add_usr_rgy_ok(dir)) && (i < RegistryRetry)); i++) {
         call_registry(batchfile); /* system call with a shell script */
         fprintf(logfd,"[SYS_ADMIN] add_usr_rgy: Sleeping waiting for ...
         sleep(RegistrySleep * i);
        }


There has been an application written by the Specials Group (sp?) at
APOLLO to provide a  NIS(YP)/Registry interface where updates from
flat NIS files are updated into the Registry. I assume that this application
is using a registry API.

Our scheme here seems to work with ~180 node networks with about 10 registry
slaves.

regards,
tony
-- 
*--------------------------+----------------------------------------*
| Tony Farrow              | NetNorth/BITNET: TONYF@BNR.CA          |
| Internet Systems         |                                        | 
| Bell-Northern Research   | UUCP: ...uunet!bnrgate!bwdla54!tonyf   |
| P.O. Box 3511, Station C | Phone: (613)763-4903  ESN: 393-4903    |
| Ottawa, Canada   K1Y 4H7 | FAX:   (613)763-3283  ESN: 393-3283    |
*--------------------------+----------------------------------------*

goldfish@CONCOUR.CS.CONCORDIA.CA (02/21/91)

I have a small program which interrogates the registry with the
"getpwent" system call and formats the output in a variety of ways.
If anyone is interested send me a message (By E-Mail, not by posting)
I will shar it and send it to you.  If there is sufficient interest, I
will post the shar file.  (What is the policy on posting complete
sources here anyway?)

The program syntax is soemthing like:

	userinfo [-all] [-version] [-pw alt.passwd.file]
		[-f tokenized%format%string] [-c] user1 - user2 ...

	-all print all users
	-pw  select alternate password file
	-version  display version number
	-f   select alt format (uses "\"and "%" style tokens)
	-c   supress warning messages
	-    read user names from "stdin"
	user1 ... list of names to print

useage:
	userinfo root
will print:

root:f7y7jJ9AUCl8:0:10:Apollo System,Account:/.:/bin/ksh

Useful for accessing the registry as a password file, and virtually
eliminates unreadable invocations of AWK, SED, GREP in my admin
scripts.

--	  Paul Goldsmith
<goldfish@concour.cs.concordia.ca>				 (514) 848-3031
	(Shirley Maclaine told me there would be LIFETIMES like this)

pato@apollo.HP.COM (Joe Pato) (02/21/91)

In general it is a bad idea to use the getpwent call when getpwnam
or getpwuid would be better suited (getpwent will iterate the entire
database while getpwnam and getpwuid use keyed access to find the
appropriate entry).  If you find that you need to use getpwent
you will observe much better performance if you first call:

	setpwfile("/etc//passwd");

The "//" in this pathname is significant - it will force the 
library functions to actually read the ascii file instead of using
RPC calls to the server.

You should also note that as of sr10.3 the passwd (and group and
org) type managers will obtain a current view of the file and cache
it locally.  Before sr10.3 the passwd (and others) file is checkpointed
every two hours.

                    -- Joe Pato
                       Cooperative Computing Division
                       Hewlett-Packard Company
                       pato@apollo.hp.com

e07@nikhefh.nikhef.nl (Eric Wassenaar) (02/24/91)

In article <4ff2e1af.20b6d@apollo.HP.COM>, pato@apollo.HP.COM (Joe Pato) writes:
> appropriate entry).  If you find that you need to use getpwent
> you will observe much better performance if you first call:
> 	setpwfile("/etc//passwd");

This indeed gives a VERY substantial performance increase.
You certainly want to include this line in your favourite version
of the finger program and the like. However:
a. As Joe Pato outlined, in SR10.2 and before this way of reading
   the passwd file may not reflect the most up-to-date status of
   the registry database.
b. You may want to filter out entries for names that appear in the
   'person' database, but for which there is not an actual account.
   Such entries show up with a password field of '*NOACCT*', a home
   directory of '/homeless', and a gid of '2' (sic!).

Eric Wassenaar
-- 
Organization: NIKHEF-H, National Institute for Nuclear and High-Energy Physics
Address: Kruislaan 409, P.O. Box 41882, 1009 DB Amsterdam, the Netherlands
Phone: +31 20 592 0412, Home: +31 20 909449, Telefax: +31 20 592 5155
Internet: e07@nikhef.nl

tonyf@bwdla54.bnr.ca (Tony Farrow) (02/26/91)

In article <1157@nikhefh.nikhef.nl>, e07@nikhefh.nikhef.nl (Eric Wassenaar) writes:
|> In article <4ff2e1af.20b6d@apollo.HP.COM>, pato@apollo.HP.COM (Joe Pato) writes:
|> > appropriate entry).  If you find that you need to use getpwent
|> > you will observe much better performance if you first call:
|> > 	setpwfile("/etc//passwd");
|> 

If you are using getpwent the above will help you. However applications such as
cron use initgroups/getgrent to initialize your group entries. If you
have a large group file then you will see a fair amount of data 
transfer during the initgroups call. This is improved in 10.3 (thx for
the info Joe) but if you are running at 10.1 and 10.2 applications such
as cron will pull the entire groups file everytime they have an entry
to be executed.

regards.

-- 
*--------------------------+----------------------------------------*
| Tony Farrow              | NetNorth/BITNET: TONYF@BNR.CA          |
| Internet Systems         |                                        | 
| Bell-Northern Research   | UUCP: ...uunet!bnrgate!bwdla54!tonyf   |
| P.O. Box 3511, Station C | Phone: (613)763-4903  ESN: 393-4903    |
| Ottawa, Canada   K1Y 4H7 | FAX:   (613)763-3283  ESN: 393-3283    |
*--------------------------+----------------------------------------*