[comp.unix.aix] Making A request to IBM

cmo@softpro.stgt.sub.org (Christian Motz) (03/11/91)

In article <1991Mar6.211740.25556@ibmpa.awdpa.ibm.com> jsalter@slo.awdpa.ibm.com (Jim Salter) writes:
>
> [...]
>
>Please feel free to call up IBM with requests
>for it, though.  If enough people want it and *communicate* this to IBM,
>it might get in there.

The big problem I (and I suppose quite a lot of people) have with this is
that I don't have the slightest idea where and how I should make these
kinds of requests. I doubt that I should bother the software support
people with this ...

-- 
SOFTPRO doesn't speak for me, and I do not speak for SOFTPRO. So what?

                              Christian Motz, cmo@softpro.stgt.sub.org

bengsig@dk.oracle.com (Bjorn Engsig) (03/13/91)

Article <96@softpro.stgt.sub.org> by cmo@softpro.stgt.sub.org (Christian Motz) says:
|The big problem I (and I suppose quite a lot of people) have with this is
|that I don't have the slightest idea where and how I should make these
|kinds of requests. I doubt that I should bother the software support
|people with this ...
They _are_ the people to tell about software problems.
-- 
Bjorn Engsig, ORACLE Corporation, E-mail: bengsig@oracle.com, bengsig@oracle.nl

jsalter@ibmpa.awdpa.ibm.com (03/14/91)

In article <96@softpro.stgt.sub.org> cmo@softpro.stgt.sub.org (Christian Motz) writes:
>In article <1991Mar6.211740.25556@ibmpa.awdpa.ibm.com> jsalter@slo.awdpa.ibm.com (Jim Salter) writes:
>>Please feel free to call up IBM with requests
>>for it, though.  If enough people want it and *communicate* this to IBM,
>>it might get in there.
>The big problem I (and I suppose quite a lot of people) have with this is
>that I don't have the slightest idea where and how I should make these
>kinds of requests.

Well, there is supposed to be an 800 phone number that everyone gets.  And
you should have an ID number to identify yourself.  Your SE/CE should know
what this number is.  Look, IBM has gone to A LOT of trouble to put together
a support structure to satisfy you, the customer.  In fact, from comments
I've seen in the trade mags, IBM's support is better than of older, more
established *IX companies.

>I doubt that I should bother the software support people with this ...

Ask them about it.  I don't speak for the support folks, but I would guess
that valid problem reports include:

DOCUMENTATION: If you're confused by how to do something you're probably not
the only one, call it in;

USABILITY: if you can't use something, or it doesn't work as documented,
call it in;

ERROR MESSAGES: if the error message is more confusing than the error,
call it in;

PERFORMANCE: if the same thing works 5 times faster on a Sun 3/50,
CALL IT IN! :-)

DESIGN.: If something is designed wrong (like the malloc() controversy),
and you want to see it changed, call it in; (this probably entails a few more
levels of beuracracy, though, and there has to be enough customer support
to justify changing it...)

>                              Christian Motz, cmo@softpro.stgt.sub.org

jim/jsalter  IBM PSP, Palo Alto  T465/(415)855-4427  VNET: JSALTER at AUSVMQ
Internet: jsalter@slo.awdpa.ibm.com         UUCP: ..!uunet!ibmsupt!jsalter 
  PS/2 it, or DIE!  :-)  The ramblings above have nothing to do with Big Blue.

pa@appmag.com (Pierre Asselin) (03/14/91)

jsalter@ibmpa.awdpa.ibm.com writes:

>In article <96@softpro.stgt.sub.org> cmo@softpro.stgt.sub.org (Christian Motz) writes:
>>In article <1991Mar6.211740.25556@ibmpa.awdpa.ibm.com> jsalter@slo.awdpa.ibm.com (Jim Salter) writes:
>>>Please feel free to call up IBM with requests
>>>for it, though.  If enough people want it and *communicate* this to IBM,
>>>it might get in there.
>>The big problem I (and I suppose quite a lot of people) have with this is
>>that I don't have the slightest idea where and how I should make these
>>kinds of requests.

>Well, there is supposed to be an 800 phone number that everyone gets.  And
>you should have an ID number to identify yourself.  Your SE/CE should know
>what this number is.  Look, IBM has gone to A LOT of trouble to put together
>a support structure to satisfy you, the customer.  In fact, from comments
>I've seen in the trade mags, IBM's support is better than of older, more
>established *IX companies.
>[...]

Don't believe everything you read [ 1/2 :-) ].  The two numbers I have are
  800-237-5511  IBM Software Defect Support
  800-426-7378  IBM Hardware Defect Support
The first is for bug reports, the second is for replacement parts.
Neither is a tech hot-line.  It's a good idea to check with Software,
in case your bug is already known and has a fix, but that's the exception,
not the rule.  Unless the origin of the problem is obvious, call your CE.
Mine is phenomenally competent, but he has enough work for ten.  The system
doesn't work too well here.

>DOCUMENTATION: If you're confused by how to do something you're probably not
>the only one, call it in;
>USABILITY: if you can't use something, or it doesn't work as documented,
>call it in;
>ERROR MESSAGES: if the error message is more confusing than the error,
>call it in;
>PERFORMANCE: if the same thing works 5 times faster on a Sun 3/50,
>CALL IT IN! :-)

Software Defect Support is *not* the place to call.  They handle specific
bugs, not across-the-board disasters like these.
(Yeah, the number-crunching is good.  And we don't need super graphics.)

>DESIGN.: If something is designed wrong (like the malloc() controversy),
>and you want to see it changed, call it in; (this probably entails a few more
>levels of beuracracy, though, and there has to be enough customer support
>to justify changing it...)

I believe the magic words here are ``DESIGN APAR''.  Then they have to
listen to you.  I submitted a design apar for this very same malloc;
they want an example of broken code(!)  OK, we'll see, stay tuned...
NB: This malloc madness is not an IBM exclusivity.  Claims that this
    bright idea comes from SYSV may be true, denials notwithstanding.

>>                              Christian Motz, cmo@softpro.stgt.sub.org

>jim/jsalter  IBM PSP, Palo Alto  T465/(415)855-4427  VNET: JSALTER at AUSVMQ
>Internet: jsalter@slo.awdpa.ibm.com         UUCP: ..!uunet!ibmsupt!jsalter 
>  PS/2 it, or DIE!  :-)  The ramblings above have nothing to do with Big Blue.

Pierre Asselin, R&D, Applied Magnetics Corp.  I speak for me.
3003jalp@ucsbuxa.ucsb.edu     (soon pa@appmag.com)

wohler@sapwdf.UUCP (Bill Wohler) (03/14/91)

jsalter@ibmpa.awdpa.ibm.com writes:
>CALL IT IN! :-)

jim, et al,

  i'd be much more likely to report bugs if i could mail them in (via
  the Internet, not the IBM network).  this takes less time than
  calling it in as you merely have to cut and paste the error output
  rather than spelling it out with possible errors in the process.
  in my case, i have to talk to the IBM folks in german which makes it
  worse.  
  
  yes, an e-mail bug address would be very nice.  thanks for listening.

						--bw
						wohler@sap-ag.de

karish@mindcraft.com (Chuck Karish) (03/15/91)

In article <1991Mar14.031806.3002@appmag.com> pa@appmag.com
(Pierre Asselin) writes:

>jsalter@ibmpa.awdpa.ibm.com writes:
>>Well, there is supposed to be an 800 phone number that everyone gets.  And
>>you should have an ID number to identify yourself.

>Neither [ 800 number ] is a tech hot-line.  It's a good idea to check with
>Software,
>in case your bug is already known and has a fix, but that's the exception,
>not the rule.  Unless the origin of the problem is obvious, call your CE.

Your CE is the hardware support person.  The SE is the one to
go to.  She's the one whose responsibility it is to help you
put all the pieces of hardware and software together to make
a working system, and she works out of IBM's regional sales
office.  That's the place to go to provide customer input
about desired changes/new products.

	Chuck Karish		karish@mindcraft.com
	Mindcraft, Inc.		(415) 323-9000

jsalter@ibmpa.awdpa.ibm.com (03/15/91)

In article <1991Mar14.031806.3002@appmag.com> pa@appmag.com (Pierre Asselin) writes:
>Don't believe everything you read [ 1/2 :-) ].  The two numbers I have are
>  800-237-5511  IBM Software Defect Support
>  800-426-7378  IBM Hardware Defect Support
>The first is for bug reports, the second is for replacement parts.

I believe you still need your IBM support id number; these are not
general info-type phone numbers.  Probably Rudy C. or one of the
support folks can explain this more fully...

>>DOCUMENTATION: If you're confused by how to do something you're probably not
>>the only one, call it in;
>>USABILITY: if you can't use something, or it doesn't work as documented,
>>call it in;
>>ERROR MESSAGES: if the error message is more confusing than the error,
>>call it in;
>>PERFORMANCE: if the same thing works 5 times faster on a Sun 3/50,
>>CALL IT IN! :-)
>Software Defect Support is *not* the place to call.  They handle specific
>bugs, not across-the-board disasters like these.

Well, if "across-the-board disasters" aren't "bugs" then I don't know
what is?

>Pierre Asselin, R&D, Applied Magnetics Corp.  I speak for me.
>3003jalp@ucsbuxa.ucsb.edu     (soon pa@appmag.com)

jim/jsalter  IBM PSP, Palo Alto  T465/(415)855-4427  VNET: JSALTER at AUSVMQ
Internet: jsalter@slo.awdpa.ibm.com         UUCP: ..!uunet!ibmsupt!jsalter 
  PS/2 it, or DIE!  :-)  The ramblings above have nothing to do with Big Blue.

benson@odi.com (Benson I. Margulies) (03/15/91)

From my experience, many of these messages are not correct.

Defect support is for defects. Bugs. When you submit a defect, the
person from defect support creates an APAR and sends it to the
developer.

If the developer decided that it is a design issue, and not a bug,
they will tell defect support to tell you that "The software is
working as designed." The APAR is rejected, and defect support will
tell you politely but firmly that your only recourse is a DCR (Design
Change Request). Defect support cannot and will not create such
things. Your SE/marketing rep can do this, via a form called a PASR.
If there is such a thing as a Design APAR, I've never had someone from
defect support admit it or be willing to initiate it.

Further, the developer can decide that your problem, while a bug, is a
"permanent restriction," (i.e., too hard to fix) and decline to fix
it, ever. This is what happened to me when I reported that AIX dbx,
unlike any other, can't trace the stack below a sigaction-established
SIGSEGV handler.

There appears to be no way to instigate a management review of the
designation "permanent restriction" via defect support. The
immediately responsible developer calls the shot. All you can do is
submit a DCR.


-- 
Benson I. Margulies

pa@appmag.com (Pierre Asselin) (03/16/91)

karish@mindcraft.com (Chuck Karish) writes:

>Your CE is the hardware support person.  The SE is the one to
>go to.  She's the one whose responsibility it is to help you
>put all the pieces of hardware and software together to make
>a working system, and she works out of IBM's regional sales
>office.  That's the place to go to provide customer input
>about desired changes/new products.

I stand corrected.  My *SE* is extremely competent and grossly overworked.
Is there an anon ftp site with a list of IBM acronyms?  How many megs?

  --PA
    3003jalp@ucsbuxa.ucsb.edu       appmag.com isn't up yet.

d5peteg@dtek.chalmers.se (Peter Gustafsson) (03/16/91)

>>CALL IT IN! :-)

>jim, et al,
>
>  i'd be much more likely to report bugs if i could mail them in (via
>  the Internet, not the IBM network).  this takes less time than
>  calling it in as you merely have to cut and paste the error output
>  rather than spelling it out with possible errors in the process.
>  in my case, i have to talk to the IBM folks in german which makes it
>  worse.  
>  
>  yes, an e-mail bug address would be very nice.  thanks for listening.
>
>						--bw
						wohler@sap-ag.de

Agree!
I try to manage a E-mail "hotline" with all the persons who is running
RS/6000 here. It would be much more efficient if IBM here could be reached
on Internet. 

After some time I convinced some of the persons at IBM that I
deal with daily to get acces to the internet-world... Via some gateway
on the other side of the world (and it works) But the AIX Competence 
Guys can't understand why, and some of them never heard of this net!

Hope that someone at IBM are listening...

      /Peter Gustafsson
       Chalmers Workstation Centre
       Gothenburg Universities' Computing Centre

julie@levell.austin.ibm.com (Julie A. Levell) (03/16/91)

In article <1991Mar15.123532.8036@odi.com> benson@odi.com (Benson I. Margulies) writes:
>From my experience, many of these messages are not correct.
>
>Defect support is for defects. Bugs. When you submit a defect, the
>person from defect support creates an APAR and sends it to the
>developer.

Sends it to the Change Team (or Level 3)

>If the developer decided that it is a design issue, and not a bug,
>they will tell defect support to tell you that "The software is
>working as designed." The APAR is rejected, and defect support will
>tell you politely but firmly that your only recourse is a DCR (Design
>Change Request). Defect support cannot and will not create such
>things. Your SE/marketing rep can do this, via a form called a PASR.
>If there is such a thing as a Design APAR, I've never had someone from
>defect support admit it or be willing to initiate it.

Not true.  Each apar is handled on a case by case basis, but when we
see an apar that is a design change, that just isn't do-able as a "code fix",
we close the apar and open a "Design" for development.  Then the architects 
look at it.  I've opened a few for security and lvm.

>Further, the developer can decide that your problem, while a bug, is a
>"permanent restriction," (i.e., too hard to fix) and decline to fix
>it, ever. This is what happened to me when I reported that AIX dbx,
>unlike any other, can't trace the stack below a sigaction-established
>SIGSEGV handler.

Can't comment on this one, I don't know the story behind it.

>There appears to be no way to instigate a management review of the
>designation "permanent restriction" via defect support. The
>immediately responsible developer calls the shot. All you can do is
>submit a DCR.

Again, not true.  Every apar that we close as "permanent restriction" is
brought to upper level management review.  I can't close PRS without
going thru managment.  We don't like telling customers
"that's just the way it is", but sometimes it does have to happen.

It's not a perfect system, but we're trying.

>Benson I. Margulies

Your humble Change Team servant,
Julie Levell 
-- 
*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=
Julie A. Levell        IBM Advanced Workstation Division         Austin, Texas
Internet: julie@aixwiz.austin.ibm.com    IBM VMNET:  JULIEL at AUSVMQ
DeskNet:      4C-29/994                      SpeakNet:  823-5178 (Tie 793-5178)

schafer@devils.rice.edu (Richard Alan Schafer) (03/17/91)

And if you don't like the way IBM closed an APAR (e.g., they decided
the problem was a user error, you think still think it's a bug), you can
always appeal the closing of the APAR.  You write up	
why you think they're wrong, and the closing gets submitted for 
management review.  Like any appeal process, you win some, you lose some,
but at least you have a chance to convince them that they made the wrong
decision the first time around.  If you still don't like the decision,
*then* get your SE to open a PASR.

Another hint: don't *ever* let the support center person convince you
that you don't have the right to demand an APAR be opened.  If you really
think that your problem is a bug, then you have the right to insist that
an APAR be opened, regardless of what the Level 1 person thinks.  Again,
there's no guarantee that the problem will be fixed to your satisfaction,
but you *never* have to accept the judgement of the Level 1 person that
your problem isn't worth opening an APAR about if you disagree with him or
her.  

If you ever call up the support center and believe you're getting 
inappropriate responses from the person you talking to, then demand to
speak to the Duty Manager.  

There are a bunch of tricks on making the support center work for you, 
some of which are documented in GA21-9824 You and the Support Center,
an IBM manual you can get your SE to order for you.  Also, if someone
at your site attends SHARE (and there are growing numbers of UNIX and 
RS/6000 sessions at SHARE), there are occasional sessions given on 
Making the Support Center Work for You.

Richard Schafer
Manager, Network & Systems Support
Networking & Computing Systems
Rice University

andrew@ramona.Cary.NC.US (Andrew Ernest) (03/17/91)

jona@zonker.iscp.bellcore.com (Jon Alperin) writes:
>     Yes, but how do you ask technical questions? Going through your CE/SE
>to post to IBMLink is long and torturous, and bedsides, its hard to ask
>a complex question without having an ongoing conversation.

I'd like to add to the above.  Customers with SE support are *LUCKY*.
You don't know how bad support is until you've experienced what it is
like to buy through an ordinary "IBM Authorized Advanced Products Dealer".
You don't *get* an 800 number when you buy this way.  You don't *get* a
customer number.  Your only "support" is from the dealer and, as everyone
knows, computer store chains don't pay enough to staff technically competent
people.  I *stupidly* bought AIX PS/2 this way because I didn't know in
advance that I wouldn't get direct support from IBM.  Live and learn.
-- 
Andrew Ernest <andrew@ramona.Cary.NC.US>

ng@cfd.di.nrc.ca (Kai Ng) (03/18/91)

In article <2656@sapwdf.UUCP>, wohler@sapwdf.UUCP (Bill Wohler) writes:
|> 
|>   yes, an e-mail bug address would be very nice.  thanks for listening.
|> 

I was very supprised that the IBM tech support (defect support, whatever)
does not have an internet email channel such that customers could send
test case of bug reports. We have to do it in a very, very inefficient,
tedious and expensive way: tar to a diskette and send by mail.

In a number of instances, I just felt 'who cares' and 'never mind'ed.
I simply didn't not have the time. That's IBM's problems, why my employer
has to pay for it?

-- 
-----------------------------------------------------------------------------
Kai S. Ng                     Informatics, National Research Council Canada
INTERNET ng@cfd.di.nrc.ca     M-60 Montreal Road, Ottawa, Canada    K1A 0R6
BITNET   kain@nrcvm01.bitnet  VOICE (613) 993-0240       FAX (613) 954-2561

jsalter@ibmpa.awdpa.ibm.com (03/19/91)

In article <1991Mar15.195902.9588@mathrt0.math.chalmers.se> d5peteg@dtek.chalmers.se (Peter Gustafsson) writes:
>>  i'd be much more likely to report bugs if i could mail them in (via
	[...]  
>>  yes, an e-mail bug address would be very nice.  thanks for listening.
>						wohler@sap-ag.de
	[...]
>After some time I convinced some of the persons at IBM that I
>deal with daily to get acces to the internet-world... Via some gateway
>on the other side of the world (and it works) But the AIX Competence 
>Guys can't understand why, and some of them never heard of this net!

I believe this whole concept of system support over the Internet has come
up a couple of times before.  The pro's being:

	You're hooked up, and IBM's hooked up, so why not?

	Speed.

	If you find a problem, you're much more likely to type up a quick
	message and include the problem file, than call some 800-number and
	be expected to accurately describe it over the phone.

There are cons, also:

	There is some *written* rule that the Internet is not supposed to
	do business using it's lines.  (Don't know the official statement
	on this, I think one of the news.admin groups has more information.)

	Not *everyone* is hooked up to the Internet, so you deal with a
	form of favoritism (minor, but it does exist).

	There are others, I'm sure.....

If someone could post the *official* restraints on using the Internet for
business, that might help.  And it could then go in the FAQ, right??? :-)

>Hope that someone at IBM are listening...

There's not much that can be done.

>      /Peter Gustafsson
>       Chalmers Workstation Centre
>       Gothenburg Universities' Computing Centre

Note: this is my *opinion* only.  Thoughts of me representing IBM in this
matter should not be inferred, referred, incurred, submerged, or expurged.  :-)

jim/jsalter  IBM PSP, Palo Alto  T465/(415)855-4427  VNET: JSALTER at AUSVMQ
Internet: jsalter@slo.awdpa.ibm.com         UUCP: ..!uunet!ibmsupt!jsalter 
  PS/2 it, or DIE!  :-)  The ramblings above have nothing to do with Big Blue.

pmartin@athena.mit.edu (Pete Martin) (03/19/91)

If you require support, for fixes, broken code, ect.  call 1- 800-237-5511, 
you will be asked your "Customer Number"  this is a seven digit number I 
believe identifying your "Establishment" you will be asked for a problem number
and if no number, your operating system.Then you get patched through to level one
support and from there you should be in good hands.

robin@pensoft.UUCP (Robin Wilson) (03/20/91)

In article <1991Mar15.123532.8036@odi.com> benson@odi.com (Benson I. Margulies) writes:
>Further, the developer can decide that your problem, while a bug, is a
>"permanent restriction," (i.e., too hard to fix) and decline to fix
                                 ^^^^^^^^^^^^^^^
>it, ever. This is what happened to me when I reported that AIX dbx,
>unlike any other, can't trace the stack below a sigaction-established
>SIGSEGV handler.

Well, here is the REAL story from someone who works inside of the system.
I used to work at IBM Level 2 Defect support for AIX version 3.  I now 
work at IBM Level 3 (also called, "the change team") RT Defect support
(AIX version 2.2.1).  To start with, a "permanent restriction" (PRS) isn't 
really just a problem that is "too hard to fix".  Actually, a permanent
restriction closing means 1 (or more) of several criteria were met:
1) the problem is isolated to a specific type of application, that is 
   not widely used, AND the problem can be worked around through some
   other method (other than a code change), AND the fix for this problem
   would take a significant code re-write to fix.  (The reason a 
   "significant code re-write" is required to make a PRS closing is to
   prevent a developer from just saying, "that's too hard, I don't want to
   fix it".  The reason this is justified is because a "significant code
   re-write" may cause other "un-forseen" problems, and could seriously
   impact other customers who depend on that program.)
2) Making the requested fix would put AIX in a position to be "out-of-
   specification" from either our published specifications, or the published
   standard for a given program.
3) The requested fix is in reality a "design change" but the design was clearly
   flawed in the first place.  (This is the "greyest" area of a PRS, but it
   serves a definite purpose.  The PRS closing code indicates that IBM has
   evaulated the code AND found it to be defective, but chose not to drop a
   fix at this time.  So calling something a PRS really means that IBM has
   acknowledged a "bug" in the code.)
4) The code is already being re-written for the next release, and spending
   significant "man-hours" making a major change would consitute a 
   serious duplication of effort.

In all cases, the programmer that decides to make a PRS closing is 
required to get 4th line manager approval before the closing is accepted.
This assures that IBM management has reviewed the problem and is aware of
the impact to customers.  BTW, the Level 2 representative is your (the 
customer) advocate in these proceedings, but once the "change-team" has 
decided to close the problem, Level 2 has no say in the matter anymore.

I will provide a complete description of the IBM software support process
in another post...  Coming soon to a news reader near you....

robin@pensoft.UUCP (Robin Wilson) (03/21/91)

OK here it is... the complete (almost) description of how IBM software 
support works.

The customer is supposed to call the Systems Engineer (SE) for ANY problems
with their system.  The SE is responsible for Problem Determination (PD)
and Problem Source Identification (PSI).  Often times the SE is experienced
in systems other than the IBM AIX systems, so this PD/PSI is sometimes not
very complete when done by a "less expereinced" SE.  This is one area where
IBM sometimes has trouble.  If the SE is either not experienced enough, or
is unable to resolve your problem, he is supposed to determine whether the 
problem is a "DEFECT" or "HOW-TO".  Unfortunately, someone with little 
experience, usually doesn't know enough to determine if it is a "DEFECT" or
"HOW-TO".  So they call the channel that provides the fastest response:
AIX Software DEFECT Support Level 2.  For a second, lets assume they really
did know that the problem was "HOW-TO" and they followed the proper channel
to resolve the problem.  HOW-TO problems will go through the following chain
of people.  The SE contacts the Area-Specialist.  If the AS cannot solve the
problem, he directs the SE to the National Technical Support Center.  The
SE can contact this center through electronic mail ONLY, so this method
usually takes several days to resolve a problem.  (IBM is working on the
possiblity of making the Tech Support group accessable by phone, but right
now that is not a possibility.)  If the NTSC cannot resolve the problem,
they will contact Level 2 DEFECT support (to find out if "Anybody has 
heard of this problem"), and if Level 2 can't help, NTSC will contact
Level 3 (change team), and finally NTSC will contact development.  NOTE:
there is a distinct difference between Level 3 (CT) and Development.
Development writes the NEW code for the "next" or "future" releases, and
CT maintains the existing release(s).

Now here's what happens when the problem goes from SE to Level 2 DEFECT
support.  (This does not assume that the problem is a HOW-TO, but either
way it is started the same at Level 2.)  IBM Level 2 Software Defect 
Support (L2) is just what the name implies; DEFECT support.  They are 
contacted by calling the 1-800-237-5511 number.  The people answering the
phone are at one of several regional support centers.  They are Level 1
support.  And they receive calls for ALL IBM systems and OS's.  They ask
the called a few questions: "customer number", "type of machine", "operating
system", etc.  Then the level 1 person routes the information (entered into
a database system that is distributed across the nation) to the proper
L2 group for the product (well, ususally they do... sometimes they 
accidentally route calls to the wrong group, but usually a callback solves
that).  If your product happens to be the RS/6000 and AIX V.3, they route
you to Austin, Texas, and live transfer your call to a Level 2 representative
here in Austin.  (RT and AIX 370, and AIX PS/2 also get routed to Austin, but
their calls are not live transfered, they Level 2 person must call the
customer back.)  The person answering the call is a FULL Level 2 support
person, who has been assigned to answer incoming calls at that particular
time (or who just saw the light blinking on the wall -- that indicates an
incoming call has not yet been answered -- and decided to grab the call).
NOTE: Just for clarification L2 for the RS/6000 takes over 250 calls every
day so sometimes it takes several minutes to get to a specific call during a 
busy time.  IBM is committed to provide instant response to all calls, but
sometimes the system they use hits a glitch (when this happens they usually
are quick to make adjustments).  Anyway, when the L2 person takes the live
call, they have no way of knowing what the person who is calling is 
ahving trouble with, so they start by asking questions.  Sometimes, you get
lucky and happen to have your call answered by someone knowledgable in
your specific problem, but more often the first person you talk to will 
take down basic information and queue your problem over to someone who works
in the group that best understands your problem.  These people will review
the basic information provided by the call taker, and then proceed to resolve
the problem.  This ususally requires contacting the customer for more 
information, running testcases, attempting to re-create your problem locally
etc.  If the problem is a "HOW-TO" question, Level 2 is required to send the
customer back to the SE.  When I left L2 (about 2 months ago) we averaged
about a 60-40 HOW-TO to DEFECT ratio.  So you can see, that often times 
DEFECT support spends significant amounts of time determining if a problem 
is HOW-TO, and then contacting the customer to have them call the SE back.

When L2 has sufficiently tested a problem to determine that "there appears
to be a defect", they will pass the problem on to the Change Team (CT).
The CT (also called Level 3 (L3)) will then read through the problem record
and evaluate the problem as a possible code defect.  Try to remember that 
by-and-large the L2 person is less knowledgable (although not in all cases)
that the CT person, so some of the problems are rejected by CT as "User
Errors" (which is functionally the same as "HOW-TO").  Basically the CT
can close a problem in the following ways:

	USE: User error.  The customer is not using the program as it was 
	     intended/documented.

	IDD: Documentation error (IDD is IBM document group).  This is sometimes
	     used instead of a USE when the documentation is unclear, but the
	     code was not intended to be used like the customer was attempting
	     to use it.

	PER: Programming Error (in the IBM supplied code).  This indicates that
	     a software DEFECT was found, and corrected.

	PRS: Permanent Restriction.  There is a software DEFECT (or code error)
	     but it is not reasonable to fix it at this time.  (It may be fixed
	     in a later release.)

	SUG: Suggestion.  The code is working as designed, but the requested 
	     change is being evaluated for a possible future enhancement.

	MCH: Machine Error.  The provided debug information indicates a hardware
	     error.  This can either be a hardware design defect, or a specific
	     defective piece of hardware.

	UR5: Unreproducable at the described level.  Basically this is only used
	     when the problem is clear (ie. this program should do this but 
	     instead it does this...)

There may be a few more, but these are the most widely used closing codes.

When a DEFECT is corrected, CT reviews the code change, and then builds the
code change into an update.  The update is then tested by the Regression Test
Lab, to see if the update has any defects.  Of course the regression test is
not perfect, so problems sometimes slip through... Then the update is tested
by the CT people who made code fixes.  Each person tests their own fixes.  
Some programs require special equipment, and cannot be tested in the lab 
environment at IBM, so they are sent off to the customer to test before the
Regression Tests begin (just the specific program that was failing).  The 
customer then tests the new version, and the CT person merely verifies that
the code the customer tested is the same as the "REAL" update.

Some other proceedural thingys... 

When Level 1 takes your call, they create a PMR, and then queue that PMR to
L2.  Level 2 is responsible for the PMR until it is resolved.  When Level 2
decides that the problem is a possible DEFECT, they create an APAR.  The 
APAR is sent to CT, and a copy of the PMR is sent to the CT person who will
work the APAR.  CT is responsible for the resolution of the APAR.

When a PMR is created, it is given a PRIORITY.  This indicates the desired
responsiveness that the customer requires on the PMR FROM LEVEL 2.  This 
priority is set by the customer.  It is a number from 1-4, and indicates the
following:

	1) - 1 hour contact required... 

	2) - 2 hour contact required...

	3) - 1 day contact required...

	4) - 1 week contact required...

NOTE: there is no requirement that the problem be serious for the customer,
only that the customer be contacted within the specified time period.  On 
the live transfers, this number is irrelavent until the problem is queued up
to the Subject Matter group for the problem.  Then this number is used to 
determine the priority the call takes in getting a Level 2 response.

When an APAR is created, it is given a Severity.  This is also a number from
1-4, but it is not determined by the customer.  This number is determined by
the level 2 person who creates the APAR, and is based on several criteria:

	1) - The customer's machine is not operational.  This requires 24 hours-
	     a-day response.  CT must work round-the-clock to provide a
	     workaround solution to get the customer minimally operational.

	2) - The customer's operations are seriously impacted.  The CT must
	     provide a "code fix" (for DEFECT problems) within 10 days.

	3) - The customer is not seriously impacted, but the problem is 
	     affecting customer operations minimally.  The CT must provide
	     a "code fix" within 26 days.

	4) - Reserved for DOCUMENTATION errors.  The IDD group must accept the
	     problem and agree to a documentation change within 40 days.

The L2 person will attempt to work with the customer to get the proper 
severity set for a problem, but usually these are the criteria that he/she
must meet in order for CT management to work the problem.  NOTE: the SEV1
24 hour response is intended to be for a "workaround" to the problem.  That
means that CT will spend their efforts trying to find the FASTEST method 
to get the customer operational (minimally).  Sometimes this will involve 
the actual code fix, but often it does not.  Once the customer is operating
again, the proble should be lowered to Sev2.

Sorry for the length of this posting, but I hope this clears up some of the
mystery behind IBM software support.

+-----------------------------------------------------------------------------+
|The views expressed herein, are the sole responsibility of the typist at hand|
+-----------------------------------------------------------------------------+
|UUCP:     pensoft!robin                                                      |
|USNail:   701 Canyon Bend Dr.                                                |
|          Pflugerville, TX  78660                                            |
|          Home: (512)251-6889      Work: (512)343-1111                       |
+-----------------------------------------------------------------------------+

robin@pensoft.UUCP (Robin Wilson) (03/23/91)

In article <1991Mar17.160744.19508@nrcnet0.nrc.ca> ng@cfd.di.nrc.ca writes:
>In article <2656@sapwdf.UUCP>, wohler@sapwdf.UUCP (Bill Wohler) writes:
>|>   yes, an e-mail bug address would be very nice.  thanks for listening.
>I was very supprised that the IBM tech support (defect support, whatever)
>does not have an internet email channel such that customers could send
>test case of bug reports. We have to do it in a very, very inefficient,

This is not true.  IBM Defect support does have an email route in via 
uunet.  The following address will work, but you will have to talk to 
the IBM Defect Support Person handling your problem to get his/her ID:

			<ACCOUNT>%aixserv@uunet.uu.net

I myself used to receive email from customers via this route.  (Now I have
moved on, and this route won't work for me any more.)  The person in Defect
Support may or may not know about this setup (try to remember that most of
these people have worked at IBM their entire careers, and they have not 
been exposed to USENET or the Internet).  If the support person doesn't 
know about it, ask them to talk to one of the local "gurus" about it.  It 
does work.

There are several problems though... 

	IBM will not accept test cases more than a few hundred bytes in 
length, because they download all of this stuff from uunet over a very
slow modem link.

	Many of the support personnel have never used the facility, so many
are not aware that it is available.  (If you are having trouble with 
someone in particular, send me a note (at the email address pensoft!robin)
and I will contact the person and inform them how to use the utility of
email.)

	If you send binary testcases they will have to be uuencoded.  This
highlights another deficiency in the support personnel... Nobody knows
about uudecode.  Make sure you tell them "EXPLICTLY" how to recover the
files you sent.  Assume that they person is new to unix, and unaware of
the various commands and utilities.  If they act like they know what 
their doing, adjust your instructions accordingly.  NOT ALL OF THE LEVEL
2 PEOPLE HAVE BEEN THERE SINCE DAY ONE... MANY ARE NEW EMPLOYEES STRAIGHT
OUT OF COLLEGE.  They are learning just like most of you people had to, 
so give them a chance.


+-----------------------------------------------------------------------------+
|The views expressed herein, are the sole responsibility of the typist at hand|
+-----------------------------------------------------------------------------+
|UUCP:     pensoft!robin                                                      |
|USNail:   701 Canyon Bend Dr.                                                |
|          Pflugerville, TX  78660                                            |
|          Home: (512)251-6889      Work: (512)343-1111                       |
+-----------------------------------------------------------------------------+

robin@pensoft.UUCP (Robin Wilson) (03/23/91)

In article <1991Mar19.152638.19508@athena.mit.edu> pmartin@athena.mit.edu (Pete Martin) writes:
>If you require support, for fixes, broken code, ect.  call 1- 800-237-5511, 
                                                               ^^^^^^^^^^^^
												This is Level 1.  They answer
												this phone number ONLY!

>you will be asked your "Customer Number"  this is a seven digit number I 
>believe identifying your "Establishment" you will be asked for a problem number
                           ^^^^^^^^^^^^^
                           Company

>and if no number, your operating system.Then you get patched through to level one
                                                                         ^^^^^^^^^
                                                                         level 2

>support and from there you should be in good hands.
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
               Definitely these people are good.  for the most part...
               (After all, nobody's perfect.)


+-----------------------------------------------------------------------------+
|The views expressed herein, are the sole responsibility of the typist at hand|
+-----------------------------------------------------------------------------+
|UUCP:     pensoft!robin                                                      |
|USNail:   701 Canyon Bend Dr.                                                |
|          Pflugerville, TX  78660                                            |
|          Home: (512)251-6889      Work: (512)343-1111                       |
+-----------------------------------------------------------------------------+

resnick@cogsci.uiuc.edu (Pete Resnick) (03/23/91)

I first want to thank Robin for his last two postings. It did clarify
some stuff. I have dealt with Robin for a couple of defects I have
reported, and he knows what he is talking about. I think, however,
this method of defect support for Unix workstations is just bad policy
on the part of IBM. This is exactly the kind of support you want for
VM machines, where the customer is usually totally dependent on IBM
for it's system software and is usually running a limited number of
different system level programs. But in a Unix workstation environment
when you have 15 programs off the network and system administrators
like me who need to get software working without having to call IBM
at the drop of a hat, this is really unacceptable.

Officially, I have to deal with my SE first. I don't. Chances are
my SE doesn't know any better what is going wrong than I do, and
can not just drop by in the next hour all the time to help. No doubt,
once when I was really hung using IBM's updatep program (more to say
about this later), my SE was great; but for 99 percent of the problems
he does not have the resources to be effective.

So I call defect support when I have a problem that is a defect. I
have had both good and bad experiences here. The problems that I have
had stem basically from the fact that there is a policy to have the
customer talk to level 2, and have level 2 talk to level 3. Half,
and only half, the time this works. The problem is that there is a
high probability that I know more about the problem and where to
look for an error than any given level 2 person on any given problem.
If I was on the phone with level 3, as finally happens after weeks
of back and forth conversations sometimes, diagnosis would speed
up incredibly. As it stands now, I have to try and convey all of the
information I can to a person at level 2 (inevitably losing some in
the translation from me to level 2 and level 2 to level 3) and wait
for someone to figure out something that would be found out in 10
minutes if the level 3 person would walk me through a diagnosis on
the phone. I *know* that there are cases where this would bog down
level 3 incredibly. I am claiming that a good deal of the time, this
would speed up the process.

Sometimes, level 2's diagnosing techniques are annoyingly rote:
if you have a TCP/IP problem, someone is going to ask your for
your /etc/hosts file. Not that it has anything to do with the
problem, but they ask for it anyway. I have heard the same questions
20 times for a lot of defects because some, and only some, level 2
people are not asking logical questions; it sounds like they read out
of a manual. I have had good experiences with people at level 2,
but the bad ones stick out more.

Then, once a problem is addressed, I get an update. I get a disk
that says "Use updatep command", no clue of what is ever being
changed, no bug fix list except the APAR list, nothing. Not only
that, I only get updates when I ask for them, and the production
to install them borders on the unbelievable. Again, this method
is great for a VM machine, but not for a Unix machine that I am
taking care of. I want to know what's happening, and be able
to back off if I need to. (*No, applying updates without comitting
them does not always work, and you can not do more than one
change, test them together and then reject.*)

IBM's defect support is not oriented for Unix workstations. This
is a problem of philosophy. It takes alot of change to get me
to talk live to a "level 3" type person quickly, but that is what
is necessary in most cases. I think the savings of time and money
would be incredible.

End of pontification. Thanks again to Robin, one of the level 3
people I have actually talked to, and the bunch of level 2's that
have helped me in the past. This is certainly not a flame of the
people in defect support; most of them are rather smart people.
This is a flame of a system that is just poorly operated for a
product that it was never designed for.

pr
--
Pete Resnick             (...so what is a mojo, and why would one be rising?)
Graduate assistant - Philosophy Department, Gregory Hall, UIUC
System manager - Cognitive Science Group, Beckman Institute, UIUC
Internet/ARPAnet/EDUnet  : resnick@cogsci.uiuc.edu
BITNET (if no other way) : FREE0285@UIUCVMD

geoff@edm.uucp (Geoff Coleman) (03/23/91)

In article <3299@pensoft.UUCP> robin@pensoft.UUCP (Robin Wilson) writes:
>In article <1991Mar15.123532.8036@odi.com> benson@odi.com (Benson I. Margulies) writes:
>>Further, the developer can decide that your problem, while a bug, is a
>>"permanent restriction," (i.e., too hard to fix) and decline to fix
>                                 ^^^^^^^^^^^^^^^
>>it, ever. This is what happened to me when I reported that AIX dbx,
>>unlike any other, can't trace the stack below a sigaction-established
>>SIGSEGV handler.
>
>Well, here is the REAL story from someone who works inside of the system.

	Funny how the real story from the inside is not the real story 
from the outside. My normal sequence is phone IBM software (please note
this is Canada so your mileage may vary :-) ) support hotline. I'm then 
told that because it's an RS/6000 I have to phone the hardware support
line even though it is a software problem. 
	I then get a call back from someone at IBM's national support 
center.  They argue with me if UNIX is or is not supposed to act this
way and then say they will take it up with higher powers. 
	A week to two weeks later I phone them and requeue the call.
They finally phone back and say well we have decided it is a problem
and we'll get back to you.

Geoff Coleman

P.s. This is not meant as a slight at anyone at IBM Canada support.
     I believe they are doing the best job they can given a lack of
     background UNIX knowledge, head count to wade through all of
     the problems, etc.

raj@pollux.geog.ucsb.edu (Richard A Johnson) (03/23/91)

karish@mindcraft.com (Chuck Karish) writes:

>In article <1991Mar14.031806.3002@appmag.com> pa@appmag.com
>(Pierre Asselin) writes:

>Your CE is the hardware support person.  The SE is the one to
>go to.  She's the one whose responsibility it is to help you
>put all the pieces of hardware and software together to make
>a working system, and she works out of IBM's regional sales
>office.

I find that *I* usually know a WHOLE LOT more about what's going on with these
RS/6000s than either my SE *or* most of the people I talk to in Austin!  I've
been trying to get a technical phone support number for YEARS and it's simply
impossible.  IBM doesn't work that way.  If you have a software bug, you call
software support, but unless you can formulate your question as though it were
a bug report you're pretty much on your own.

>That's the place to go to provide customer input
>about desired changes/new products.

I have tried to turn in software change requests MANY times and they get
NOWHERE.  My SE says she has tried turning in a "Design Change Request" form
(the form a person in Austin told me to turn in) and it got nowhere.  She
says apparently Austin isn't taking change requests in that form.  Supposedly
they're using some "different route for requesting changes" which she can't
find anything about.  The latest thing the people in Austin said on this was:
"You should be able to check on the availability of [the program you're
interested in] via 'Hone access'."  My SE said she knew what this meant but
couldn't find anything useful.  Thus, I ask:

How do you go about turning in a software change request to Austin?  I would
love to suggest (somewhat strongly!) that they get a REAL xterm program
(i.e. if they want to say they support X11 Rev. 3, they should have an X11
Rev. 3 xterm, not an X10 version which has been ported to X11 as aixterm really
is.)  Maybe by the time I get a straight answer on this they will have gone
to X11 Rev. 4?  Will they port aixterm to Rev. 4 and ignore the X11R4 xterm
still?

Oh well, someone a while back posted a X11R3 and X11R4 xterm so I guess I can
use those but (1) this nonsense doesn't help IBM's position w.r.t. universities
or anyone else in the know w.r.t. X11 and (2) the newly posted X11R3 xterm
gets its bottom line cut off when you're using MWM (now who's at fault THERE?).

The struggle for consistency goes on...

slamont@network.ucsd.edu (Steve Lamont) (03/23/91)

In article <3325@pensoft.UUCP> robin@pensoft.UUCP (Robin Wilson) writes:
>This is not true.  IBM Defect support does have an email route in via 
>uunet.  The following address will work, but you will have to talk to 
>the IBM Defect Support Person handling your problem to get his/her ID:
>
>			<ACCOUNT>%aixserv@uunet.uu.net

According to the person I spoke to in Austin today, this does not work
reliably.  If it does, would *somebody* at IBM/Austin *please* pass the word!
It gets really annoying when the users have to instruct the tech support
people on how to use their own system. :-<

I'm tired of spending $$$ on Federal Excess!       

							spl (the p stands for
							pretty peeved at the
							level of "support"
							I've been getting)
-- 
Steve Lamont, SciViGuy -- (408) 646-2752 -- a guest at network.ucsd.edu --
NPS Confuser Center / Code 51 / Naval Postgraduate School / Monterey, CA 93943
"The only way to deal with exploiters is to terrorize the bastards."
				- The late Congressmember Phillip Burton

jona@iscp.Bellcore.COM (Jon Alperin) (03/23/91)

Ok....so now that we know what IBM does, and who does what at IBM let me 
post the following questions:

1. What recourse do I have if I am trying to develop code for the RS/6000/AIX
system, and am running into problems with cc,xlc,ld, etc. Most of the time,
I cannot ship source code to IBM (I.E. proprietary information), but I am at
a loss as how to get IBM to look at a particular problem. For example, I am working with a 2MB archive file (see previous postings..) and I am having
problems with the linker. I can't very well send this archive file to IBM,
and it would seem to me that level 1 and 2 support would not be able to create
a 2MB archive file just to test out what happens during the link....

2. When I call IBM and report a bug, I sometimes get a patch number. HOWEVER,
it has been my experience that the patch often "fixes" more items than the
one I was looking for. This is "great, until something that worked breaks",
and I don't know where to look....Does IBM distribute just a single fix?
How does one request just a single fix?

To be fair, SE's which I deal with usually attempt to handle problems, but 
they deal more with the operating system itself (and its utilities), that
with the low level programming parts. I am lucky enough to have access to
people who know AIX fairly well, but more often than not I call Defect Support
simply because I know that someone will be at the end of the phone, rather than
an answering machine.


-- 
Jon Alperin
Bell Communications Research

---> Internet: jona@iscp.bellcore.com
---> Voicenet: (908) 699-8674
---> UUNET: uunet!bcr!jona

* All opinions and stupid questions are my own *

wohler@sapwdf.UUCP (Bill Wohler) (03/25/91)

robin@pensoft.UUCP (Robin Wilson) writes:
>OK here it is... the complete (almost) description of how IBM software 
>support works.

  there will be an exam on the material next week!

bware@slate.mines.colorado.edu (Bob Ware) (03/25/91)

In article <5026@network.ucsd.edu> slamont@network.ucsd.edu (Steve Lamont) writes:
>>			<ACCOUNT>%aixserv@uunet.uu.net
>
>According to the person I spoke to in Austin today, this does not work
>reliably.  If it does, would *somebody* at IBM/Austin *please* pass
the word!

I have tried for over a week to get a file to austin via this route and
had no luck (using several "account names" that IBM/Austin gave me to
try.)

I had one of the "System Xtra" (level 2) people indicate a strong
interest in learning how he might use electronic communication with the
outside world, but he was transfered (promoted) to a level 3 position
before we managed to get a test message through.

-- 
Bob Ware, Colorado School of Mines, Golden, Co 80401, USA
(303) 273-3987
bware@mines.colorado.edu bware@slate.mines.colorado.edu bware@mines.bitnet

robin@pensoft.UUCP (Robin Wilson) (03/27/91)

In article <1991Mar23.144908.9009@bellcore.bellcore.com> jona@iscp.Bellcore.COM (Jon Alperin) writes:
>1. What recourse do I have if I am trying to develop code for the RS/6000/AIX
>system, and am running into problems with cc,xlc,ld, etc. Most of the time,
>I cannot ship source code to IBM (I.E. proprietary information), but I am at
>a loss as how to get IBM to look at a particular problem. For example, I am
>working with a 2MB archive file (see previous postings..) and I am having
>problems with the linker. I can't very well send this archive file to IBM,
>and it would seem to me that level 1 and 2 support would not be able to create
>a 2MB archive file just to test out what happens during the link....

IBM has a real problem with proprietary source code.  Think about it from 
their perspective: "We are the world's largest computer company.  Everybody
wants to get us.  If we take your source code "to debug a problem", and at
some point in the future, we come out with a similar product....  Who do
you suppose will be accusing us of "stealing" their ideas?"

In most cases this will not happen; but if IBM "as a policy" accepts 
proprietary source code from other companies, it will show a precedence 
for the court cases that do occur, and leave IBM with its legal butt hanging
in the wind.

To get resolution on this problem you can take two approaches: 1) provide
defect support with a "non-proprietary" testcase that shows the problem.
2) Provide IBM a legal release from liability for using your proprietary
code to debug the problem.  The first method (for some reason) usually
is the prefered method to resolve the problem.  (Most customers just don't
like IBM having their source code.)


>2. When I call IBM and report a bug, I sometimes get a patch number. HOWEVER,
>it has been my experience that the patch often "fixes" more items than the
>one I was looking for. This is "great, until something that worked breaks",
>and I don't know where to look....Does IBM distribute just a single fix?
>How does one request just a single fix?

For right now, there is no way to get a 'single' fix; except for emergency
fixes.  (An emergency fix, is where there is no update available yet, but the
code has been fixed.)

Think about like this: IBM sends you a new xlc compiler.  2 months later, your
replacement (since you are really talented, and have moved up to management)
calls defect support and reports a bug in xlc.  How does IBM remember that you
got a special fix to xlc 2 months ago.  Now multiply the time by a factor of
years, and the number of fixes by a factor of thousands.  That would be a 
problem management nightmare.  Not to mention the people who get fixes from
"unsupported" routes.  (ie. my friend at "x" company got this fix, and gave it
to me, and I gave it to "y", "z", etc.)  Soon ther would be no way for the 
defect support people to know "what level" your code is at.  


>To be fair, SE's which I deal with usually attempt to handle problems, but 
>they deal more with the operating system itself (and its utilities), that
>with the low level programming parts. I am lucky enough to have access to
>people who know AIX fairly well, but more often than not I call Defect Support
>simply because I know that someone will be at the end of the phone, rather than
>an answering machine.

IBM markets classes, and provides assistance to companies porting code to the
RS/6000.  Here in Austin, they have a porting center for all companies trying
to move code from other platforms.  They also offer the "System Xtra" support
route, which provides a "single point of contact" for all RS/6000, AIX V3
questions/problems.  The System Xtra offering provides how-to, config, defect,
hardware, updates, everything support through a single 800 number.  The 
System Xtra people dispatch CE's, SE's, and provide on the phone assistance
for any type problems you might run across.


+-----------------------------------------------------------------------------+
|The views expressed herein, are the sole responsibility of the typist at hand|
+-----------------------------------------------------------------------------+
|UUCP:     pensoft!robin                                                      |
|USNail:   701 Canyon Bend Dr.                                                |
|          Pflugerville, TX  78660                                            |
|          Home: (512)251-6889      Work: (512)343-1111                       |
+-----------------------------------------------------------------------------+

robin@pensoft.UUCP (Robin Wilson) (03/27/91)

In article <1991Mar22.181533.21699@ux1.cso.uiuc.edu> resnick@cogsci.uiuc.edu (Pete Resnick) writes:
>I first want to thank Robin for his last two postings. It did clarify
>some stuff. I have dealt with Robin for a couple of defects I have
>reported, and he knows what he is talking about. I think, however,

thanks Pete, but I did have to reply to this article... 

>this method of defect support for Unix workstations is just bad policy
>on the part of IBM. This is exactly the kind of support you want for
>VM machines, where the customer is usually totally dependent on IBM
>for it's system software and is usually running a limited number of
>different system level programs. But in a Unix workstation environment
>when you have 15 programs off the network and system administrators
>like me who need to get software working without having to call IBM
>at the drop of a hat, this is really unacceptable.

What you really want is bug-less code... right?   If the code always 
worked, you would never have to call IBM for anything...  then we
wouldn't be having this discussion.

>Officially, I have to deal with my SE first. I don't. Chances are
>my SE doesn't know any better what is going wrong than I do, and
>can not just drop by in the next hour all the time to help. No doubt,
>once when I was really hung using IBM's updatep program (more to say
>about this later), my SE was great; but for 99 percent of the problems
>he does not have the resources to be effective.

OK, I forfiet this one.  SEs aren't trained well enough.  If you; as the
customer, can figure out the difference between a defect and a "how-to"
then by all means, call defect support (if it is a defect).  But if they
disagree with you; remember that you will have to get your SE out to do
the PD/PSI (Problem Determination/Problem Source Identification).

>So I call defect support when I have a problem that is a defect. I
>have had both good and bad experiences here. The problems that I have
>had stem basically from the fact that there is a policy to have the
>customer talk to level 2, and have level 2 talk to level 3. Half,

I think you probably thought you got Level 3 when in fact you only got 
the local Level 2 expert... (As an example; I worked in Level "2", and
later in this article, you refer to me as a Level 3 person...)

When I worked at level 2, they had people that they called the "Response
Team".  This was a select group of "experts" in most areas of AIX.  I
was a member of the response team the last 3 months I was in Level 2.
(I am told that this system was disbanded after I left.)  Basically the
response team was used to provide a point-of-contact for the less experienced
Level 2 personnel to go to for help working problems.  Now they just have to 
go to their manager, who will appoint an expert to assist.  There are a 
limited number of "experts" at level 2, so their time is very valuable.  I
often spent 9 hrs (of my normal 10-11 hour day) on the phone with customers
helping out on other Level 2 people's problems.  The other 'experts' had
similar schedules.  Just try to imagine how hard it is to hold a phone to
your ear for several hours at a time... 

Granted, there are more "experts" in Level 3, and for most problems there
would be a significant speed increase if Level 2 just moved you straight 
to the Level 2 expert, or the Level 3 expert... But this just isn't
feasible.  Would you rather have level 3 talking on the phone, or coding
the fixes?  Basically the problem is a lack of resources.  If IBM could
hire 100 more "experts" to man the phones, then there would be no 
problem.  But IBM can't afford to hire 100 MORE people, much less experts
to man the phones.  The more people they hire, the more the box costs...
The people they have will take time to become 'experts'.  Then when they
do, they will eventually move on; meaning there is a natural attrition 
from phone support; so they will be replaced with NEW people, who will 
have to learn all over again.

Part of the problem is the level of acceptance of the product.  IBM never
expected the RS/6000 to take off like it did.  They expected a slow 
methodical build up of sales.  Instead, sales skyrocketed right out of the
chute, causing a huge overhead expense for staffing support; and that
expense was not expected, so staffing took a little long.  Now, I believe
they are nearly adequately staffed for the number of problems that they 
have, but it will take several months before the existing staff is 
adequately trained to handle the most complex problems.  Give them a 
little more time, and see if things don't improve.

>Sometimes, level 2's diagnosing techniques are annoyingly rote:
>if you have a TCP/IP problem, someone is going to ask your for
>your /etc/hosts file. Not that it has anything to do with the
>problem, but they ask for it anyway. I have heard the same questions
>20 times for a lot of defects because some, and only some, level 2
>people are not asking logical questions; it sounds like they read out
>of a manual. I have had good experiences with people at level 2,
>but the bad ones stick out more.

Well, I would expect the bad ones to stick out more.  And they should too.
As for "reading out of a manual"; funny you should mention that... For 
network problems of ALL KINDS, it is virtually impossible to find experts
who are willing to talk on the phone 8 hrs a day.   They make too much 
money working elsewhere, doing other more meaningful jobs.  That means 
that most of the Level 2 network support (with 3 or 4 exceptions only) are
home grown.  This area will take a long time to bring up to speed.  
You know that networks are one of the hardest things to understand within
the 'computing arena'.  So learning the nuances of networking will take
time.

>Then, once a problem is addressed, I get an update. I get a disk
>that says "Use updatep command", no clue of what is ever being
>changed, no bug fix list except the APAR list, nothing. Not only
>that, I only get updates when I ask for them, and the production
>to install them borders on the unbelievable. Again, this method
>is great for a VM machine, but not for a Unix machine that I am
>taking care of. I want to know what's happening, and be able
>to back off if I need to. (*No, applying updates without comitting
>them does not always work, and you can not do more than one
>change, test them together and then reject.*)

I can't really address this other than to say; thats the way it works.
That's basically why IBM suggests (strongly) that you make a backup 
before doing the update.  You can restore whatever you want.  Also,
the System Xtra offering provides a "bug report" for each interested 
customer (I think...).

>IBM's defect support is not oriented for Unix workstations. This
>is a problem of philosophy. It takes alot of change to get me
>to talk live to a "level 3" type person quickly, but that is what
>is necessary in most cases. I think the savings of time and money
>would be incredible.

On the contrary, the costs would be enormous.  It would save time
until IBM tried to actually start fixing the problems that were reported.
Then it would be a nightmare to find someone who wasn't on the phone to
a customer.  Some people have this image of RS/6000 people in the 
thousands down here in Austin... But in reality there are less than 300
people in Level 2 and Level 3 combined for the RS/6000.  Take the number
of calls (around 300 per day), extend the problems that take several 
conversations (callbacks) and you have about 10 people left to actually
code fixes.

On the RT, the numbers are even more meager.  There are 5 Level 2 people
and 7 Level 3 people.  The RT calls come in much slower (thank god) but
there just aren't that many people available.

Its not a perfect system.. no doubt.  But they are still working on it.
The live call support is New for the RS/6000.  The organization is based 
on the lessons they learned from RT support (they have corrected many
mistakes) and will take time to completely iron out.  After all the product
is not even officially a year old.

>End of pontification. Thanks again to Robin, one of the level 3
>people I have actually talked to, and the bunch of level 2's that
>have helped me in the past. This is certainly not a flame of the
>people in defect support; most of them are rather smart people.

Again... I was Level 2.  

>This is a flame of a system that is just poorly operated for a
>product that it was never designed for.

The system was designed completely from scratch for the RS/6000.  They
still have a lot of tweaking to do, but I think in the end they will
have a 'good' (if not great) system.


+-----------------------------------------------------------------------------+
|The views expressed herein, are the sole responsibility of the typist at hand|
+-----------------------------------------------------------------------------+
|UUCP:     pensoft!robin                                                      |
|USNail:   701 Canyon Bend Dr.                                                |
|          Pflugerville, TX  78660                                            |
|          Home: (512)251-6889      Work: (512)343-1111                       |
+-----------------------------------------------------------------------------+

robin@pensoft.UUCP (Robin Wilson) (03/27/91)

In article <1991Mar22.183550.30160@edm.uucp> geoff@edm.uucp (Geoff Coleman) writes:
>In article <3299@pensoft.UUCP> robin@pensoft.UUCP (Robin Wilson) writes:
>>Well, here is the REAL story from someone who works inside of the system.
>	Funny how the real story from the inside is not the real story 
>from the outside. My normal sequence is phone IBM software (please note
>this is Canada so your mileage may vary :-) ) support hotline. I'm then 

Sorry, I guess I should have limited distribution.  The previously described
support structure is only good for the USA.  (I often forget that USENET is
read around the world.  Sorry.)  

Each country, (or in Europe - Economic Community) has its own level 2 support.
Part of this is physical.. ie. they are located far away.  And part of it is
political.  Due to export restrictions, IBM employees are not allowed to 
transmit certain technical information to "non-IBM" employees across 
national boundaries.  This means that I cannot get on the phone in Austin
and provide technical assistance to someone in Canada who doesn't work for
IBM.  I cannot send an update to another country, and I cannot send or 
suggest any code fix over the phone lines.

Since each country has its own (read different) restrictions on how to
perform support, each country has its own support structure.  All problems
(that turn out to be defects) are still resolved in Austin, and all code
changes are done through the existing level 3; but all contacts are made
electronically from IBMer to IBMer.  This is not IBMs idea... It is the 
LAW.


+-----------------------------------------------------------------------------+
|The views expressed herein, are the sole responsibility of the typist at hand|
+-----------------------------------------------------------------------------+
|UUCP:     pensoft!robin                                                      |
|USNail:   701 Canyon Bend Dr.                                                |
|          Pflugerville, TX  78660                                            |
|          Home: (512)251-6889      Work: (512)343-1111                       |
+-----------------------------------------------------------------------------+

karish@mindcraft.com (Chuck Karish) (03/31/91)

In article <3351@pensoft.UUCP> robin@pensoft.UUCP (Robin Wilson) writes:
>In article <1991Mar23.144908.9009@bellcore.bellcore.com>
jona@iscp.Bellcore.COM (Jon Alperin) writes:

>>2. When I call IBM and report a bug, I sometimes get a patch number. HOWEVER,
>>it has been my experience that the patch often "fixes" more items than the
>>one I was looking for. This is "great, until something that worked breaks",
>>and I don't know where to look....Does IBM distribute just a single fix?
>>How does one request just a single fix?
>
>For right now, there is no way to get a 'single' fix; except for emergency
>fixes.  (An emergency fix, is where there is no update available yet, but the
>code has been fixed.)
>
>Think about like this: IBM sends you a new xlc compiler.  2 months later, your
>replacement (since you are really talented, and have moved up to management)
>calls defect support and reports a bug in xlc.  How does IBM remember that you
>got a special fix to xlc 2 months ago.

This last is a valid problem, but the main reason is more closely
related to the concerns stated in the question:  It's difficult and
expensive to test fixes in isolation so they can be shipped
individually.  Such testing is needed to keep even a single "fix" from
breaking something else.

Watching the way IBM provides fixes, it looks to me as if, for AIX 3,
fixes are added to a reference copy of the OS which is tested more or
less continuously.  Patches aren't shipped until the build of the OS to
which the fixes have been made proves itself by running without
newly-created or newly-exposed bugs for a while.  Doing adequate
testing individually for each fixed problem would be prohibitively
expensive.  More important, it wouldn't work; fixes for different
problems often interact to produce new problems.

Getting back to Robin's answer, try to imagine the poor support
peoples' responsibilities under the suggested scheme:  How would you
characterize the state of your system well enough to let them try to
duplicate the behavior you see?  Would you expect them to apply the
particular set of fixes you've used, to a system freshly loaded with a
down-rev version of the OS?

	Chuck Karish		karish@mindcraft.com
	Mindcraft, Inc.		(415) 323-9000

jona@iscp.Bellcore.COM (Jon Alperin) (04/01/91)

Ok....I understand the position of not tracking each single fix, as
well as distributing multiple fixes. However.....

1. When a person is designing a software product that executes under a
specific OS level, it is often unacceptable to tell a client to upgrade
an entire system  to fix an AIX bug which is underlying your code. It is
muchg simpler to convince them to "simply apply patch XXXX".

2. When supporting multiple users doing software developement, it is
often necessary to apply a patch to solve a single users problem. However,
when multiple fixes are provided in a patch, its hard to tell users what has been
fixed or upgraded.

3. If IBM is going to provide multiple fixes, then they should be just that...
fixes. No new code should be in those non-mandatory releases. I am not sure
if IBM is doing this, but I figured that I would stay up here on the soapbox
just a little longer.


-- 
Jon Alperin
Bell Communications Research

---> Internet: jona@iscp.bellcore.com
---> Voicenet: (908) 699-8674
---> UUNET: uunet!bcr!jona

* All opinions and stupid questions are my own *

robin@pensoft.uucp (Robin Wilson) (04/03/91)

In article <1991Apr1.135433.26762@bellcore.bellcore.com> jona@iscp.Bellcore.COM (Jon Alperin) writes:
>1. When a person is designing a software product that executes under a
>specific OS level, it is often unacceptable to tell a client to upgrade
>an entire system  to fix an AIX bug which is underlying your code. It is
>much simpler to convince them to "simply apply patch XXXX".

Clearly the best answer is to design software to work with the entire system.
You will always be expected to "update" your machine to the latest level when
you receive a patch.  This is due to the fact that software is dynamic, not
static.  It is constantly evolving, and you certainly don't want to limit
your application to working on out-of-date system software.

>2. When supporting multiple users doing software developement, it is
>often necessary to apply a patch to solve a single users problem. However,
>when multiple fixes are provided in a patch, its hard to tell users what has
>been fixed or upgraded.

Actually, you can select individual lpps to update.  So the only really "big"
lpp is the "BOS" (Base Operating System).  The update procedure provides you
with some level of control over the update, but most importantly, if your
code doesn't work with the "latest and greatest" how are you going to hold
your customers back to the "old" level that you used to used?  You really 
need to make your software work with the current version of th system 
software.

>3. If IBM is going to provide multiple fixes, then they should be just that...
>fixes. No new code should be in those non-mandatory releases. I am not sure
>if IBM is doing this, but I figured that I would stay up here on the soapbox
>just a little longer.

Most of the time, IBM gets beat up for just the opposite argument:

"The 'design' was flawed, so my design change should be implemented right now!"

Your very argument is why IBM trys so hard to avoid putting 'design' fixes into
the APAR process.  They don't want to break things that aren't already broken.
Sometimes, it is just plain required... times change.. people expect new things.But by-and-large IBM avoids adding "enhancements" to the current release.


+-----------------------------------------------------------------------------+
|The views expressed herein, are the sole responsibility of the typist at hand|
+-----------------------------------------------------------------------------+
|UUCP:     pensoft!robin                                                      |
|USNail:   701 Canyon Bend Dr.                                                |
|          Pflugerville, TX  78660                                            |
|          Home: (512)251-6889      Work: (512)343-1111                       |
+-----------------------------------------------------------------------------+

moody@snap.austin.ibm.com (04/04/91)

In article <1991Mar23.144908.9009@bellcore.bellcore.com> jona@iscp.Bellcore.COM (Jon Alperin) writes:
>Ok....so now that we know what IBM does, and who does what at IBM let me 
>post the following questions:
[I can't answer the first question]
[stuff deleted]
>2. When I call IBM and report a bug, I sometimes get a patch number. ....,
>....Does IBM distribute just a single fix?

Yes.  Level 3 support develops hundreds of single fix requests.  We sometimes
get double fix requests.  We store these things on a big server that is 
available to level 2.  The fixes are stored by problem number.  If the level 2
support person can associate your problem with a problem number, then they can
send you a fix immediately for that problem.  If the problem is a kernel
panic, you can send a dump and level 2 or level 3 should be able to determine
if this is a known problem and you can request a fix for it (whether it's
known or not, if we can fix it).  The most critical thing is that the problem
is described and understood well enough to be found in the data base or
debugged.

>How does one request just a single fix?

Just ask for it.  If you know the problem number, it can be a big plus.  If
you don't know the problem number, describe it to the support folks.  You
might even query this group to see if it is known.  

>Jon Alperin
-- 
James Moody
Personal Systems Programming Austin	VNET:MOODY@AUSVMQ
Level 3 Support				internet:moody@aixwiz.austin.ibm.com

robin@pensoft.uucp (Robin Wilson) (04/04/91)

In article <6378@awdprime.UUCP> moody@snap.austin.ibm.com writes:
>In article <1991Mar23.144908.9009@bellcore.bellcore.com> jona@iscp.Bellcore.COM (Jon Alperin) writes:
>>2. When I call IBM and report a bug, I sometimes get a patch number. ....,
>>....Does IBM distribute just a single fix?
>
>Yes.  Level 3 support develops hundreds of single fix requests.  We sometimes
>get double fix requests.  We store these things on a big server that is 
>available to level 2.  The fixes are stored by problem number.  If the level 2
>support person can associate your problem with a problem number, then they can
>send you a fix immediately for that problem.  If the problem is a kernel
>panic, you can send a dump and level 2 or level 3 should be able to determine
>if this is a known problem and you can request a fix for it (whether it's
>known or not, if we can fix it).  The most critical thing is that the problem
>is described and understood well enough to be found in the data base or
>debugged.

Actually, this needs clarification.  The "single" fixes to which James refers
are what Level 2 calls "Emergency Fixes".  These are only available for a 
specific problem until the fix is "built" into an update, and then Level 2 
will only ship the update to fix the problem.  You cannot get an "Emergency
Fix" for a problem that is "fixed" in an available update.


+-----------------------------------------------------------------------------+
|The views expressed herein, are the sole responsibility of the typist at hand|
+-----------------------------------------------------------------------------+
|UUCP:     pensoft!robin                                                      |
|USNail:   701 Canyon Bend Dr.                                                |
|          Pflugerville, TX  78660                                            |
|          Home: (512)251-6889      Work: (512)343-1111                       |
+-----------------------------------------------------------------------------+