[comp.dcom.lans] Cabletron Repeater Problems

sjs@eros.network.com (Steve Senum) (06/08/91)

In article <1991May31.015914.7054@zia.aoc.nrao.edu> rmilner@zia.aoc.nrao.edu (Ruth Milner) writes:
>About a month ago I posted a query about experience with Auspex NFS servers,
>and promised a summary. Apologies for the delay, here it is.

[Much text deleted]

>From: jfd@octelb.octel.com (John F. Detke)
>Organization: Octel Communications Inc., Milpitas Ca.

[More text deleted]

>It handles the routing fine (heavy traffic on the backbone, and bursts from 
>the 3/80 network to the backbone during compiles). The only major "problem" we
>have had is that it is too fast. We have Cabletron 10baseT equipment, and have
>3 repeaters in the loop (MT8000, MMAC's) and the Auspex complains sometimes
>about "late collisions" and no carrier. Several Cabletron upgrades later and 
>this only happens a few time a week. Auspex real follows the 9.6 msec packet 
>spacing (not 9.61, 6 packets bursts at 9.6).

[New text follows]

Our site has a number of Cabletron MMAC-8s, IRMs (and IRM2s), and THIN-MIMs.
We also have a Auxpex NS/5000 server, and have had a *lot* of problems with
the Cabletron repeaters dropping packets.  We noticed this problem last
October, and have been trying to get a fix for the problem since then.
We are getting closer to a solution, but I can't believe how really awful
Cabletron's tech support department is.  All they seem capable of doing
is telling us their equipment can't be the problem (it is).  We have also
seen this problem with Cabletron FR3000 fiber repeaters, and their 8 port
thinwire repeaters.  If you have a fast machine (the Auspex NS/5000 is not
the only machine that can send frames with minimum spacing), and Cabletron
equipment, you are likey to have this problem.

I would be interested to hear from other sites that have seen a problem
like this with Cabletron equipment.

Steve Senum, sjs@network.com

gdelong@ctron.com (Gary Delong) (06/10/91)

In article <1991Jun7.200603.9378@ns.network.com> sjs@eros.network.com (Steve Senum) writes:

>Our site has a number of Cabletron MMAC-8s, IRMs (and IRM2s), and THIN-MIMs.
>We also have a Auxpex NS/5000 server, and have had a *lot* of problems with
>the Cabletron repeaters dropping packets.  We noticed this problem last
>October, and have been trying to get a fix for the problem since then.
>We are getting closer to a solution, but I can't believe how really awful
>Cabletron's tech support department is.  All they seem capable of doing
>is telling us their equipment can't be the problem (it is).  We have also
>seen this problem with Cabletron FR3000 fiber repeaters, and their 8 port
>thinwire repeaters.  If you have a fast machine (the Auspex NS/5000 is not
>the only machine that can send frames with minimum spacing), and Cabletron
>equipment, you are likey to have this problem.

>I would be interested to hear from other sites that have seen a problem
>like this with Cabletron equipment.

>Steve Senum, sjs@network.com

Subject: Re: Cabletron Repeater Problems
To: Steve Senum <sjs@network.com>
Cc: jfd@octelb.octel.com (John F. Detke)
    rmilner@zia.aoc.nrao.edu (Ruth Milner)
    alt.sys.sun,comp.dcom.lans

Mailed/Posted for: Graig R. Benson

Today at 1:00 PM your time Bob Levine, Cabletron's President, is meeting with
your company to resolve all problems. His trip is just to visit Network Systems
Corp. which is 3.5 hours by plane, one-way. We are committed to supporting you.
I know you have had problems, and I don't know whether it is our products or
not; I do know that we will fix the problems regardless of whose product or
products is causing it. I am very sorry for what ever we have done to make
your job a more difficult one, and I will make sure we use your experience to
help improve our company.

Craig R. Benson
Chairman/COO
Cabletrons Systems, Inc.

antonio@qualcom.qualcomm.com (Franklin Antonio) (06/20/91)

In article <1991Jun7.200603.9378@ns.network.com> sjs@eros.network.com (Steve Senum) writes:
>Our site has a number of Cabletron MMAC-8s, IRMs (and IRM2s), and THIN-MIMs.
>... have had a *lot* of problems with
>the Cabletron repeaters dropping packets. ...
>...
>I would be interested to hear from other sites that have seen a problem
>like this with Cabletron equipment.

Let me tell you about MY adventures with Cabletron repeaters!

We have a bunch of Cabletron thinwire repeaters: Several MMAC-8s filled with
THIN-MIM modules, IRM2s, and also about a dozen MR-9000C's. 

The first problem is that while the Cabletron repeaters have collision
detectors on each port, and gather collision statistics by port, these
circuits DO NOT succeed at informing me which Ethernet segment contains
the problem causing the collisions.  Many times I have walked into one
of our comm closets and seen EVERY collision light blinking (on MR-9000's)
or seen collision statistics accumulating for every port (on MMACs).
My response has always been to disconnect Ethernet segments from the
repeater(s) one by one, until the collisions stop.  What is the purpose
of these collision detectors on each port if they don't help you 
isolate problems to a particular port?  We've recently implemented
Remote-Lanview, so we can see all these statistics for all the repeaters
remotely.  This SOUNDS great, but since all the collision statistics are
unreliable, all this provides is a very fancy Windows program to display
random numbers.  I expect these statistics should be a TOOL i can use to
diagnose my network.  Do i ask too much?

Typically in each building we have an MMAC filled with THIN-MIM modules and
an IRM2.  In the same comm closet, we often have several Shiva FastPath
Ethernet-to-Appletalk gateway boxes which create the appletalk segments
within that building.  There are several remarkable things that occur
when a Shiva FastPath is connected to a Cabletron ethernet repeater.
Long ago, we used to run an ethernet cable from one port on the Cabletron
repeater to all the FastPaths.  (Nothing else on this cable, except, of
course, a terminator.)  The first problem is that the Cabletron repeater
reports a lot of collisions on this segment.  The really amazing symtom
is that we've seen the Cabletron repeater report lots of collisions on
the NEXT repeater port too!  In other words, say port 6 on the repeater
goes to the FastPaths.  Port 7 will report collisions.  This can happen
even if port 7 is connected to no other ethernet devices!  You can put
a terminator right on the BNC connector on the front of the box and it
will still report collisions on the port.  This provides even more evidence
that the Cabletron collision detection/reporting mechanism somewhat bogus
information.

We've had better luck when we use one repeater port for EACH Shiva
FastPath.  That's the way we've been running for a long time.  We still
get lots of collisions on these ports, sometimes 10% or 15%, but the
network still seems to work.  Just a few days ago we made yet another
discovery.  We can reduce the number of collisions by choosing WHICH
Cabletron ports we wire up to the Shiva FastPaths.  We used to take the
first THN-MIM card in the MMAC, and dedicate it to the FastPaths.
Now we find that we get many fewer collisions if we spread out the
ports which connect to FastPaths, putting no more than one FastPath
on each THN-MIM card, then using the other 7 ports on each THN-MIM for
anything else.  I'm not sure exactly what to conclude from this.  
Sure looks like this has uncovered some kind of timing problem in the
MMAC design, but that's just one possibility.  

Shiva and Cabletron have each indicated that they haven't seen this
before.  It can't be a simple piece of broken hardware, as all our
Shiva and Cabletron boxes do the same thing.  Must be a design flaw.

Anybody else use Cabletron and Shiva boxes together?
Anybody else got Cabletron collision bogosity?
Anybody out there using Cabletron's collision statistics AND finding
them meaningful?

Thanks.
Franklin Antonio

wrt@kepler.unh.edu (Walter R Trachim) (06/21/91)

In article <1991Jun20.075328.6540@qualcomm.com> antonio@qualcom.qualcomm.com (Franklin Antonio) writes:
>
>isolate problems to a particular port?  We've recently implemented
>Remote-Lanview, so we can see all these statistics for all the repeaters
>remotely.  This SOUNDS great, but since all the collision statistics are
>unreliable, all this provides is a very fancy Windows program to display
>random numbers.  I expect these statistics should be a TOOL i can use to
>diagnose my network.  Do i ask too much?

No, you're not asking for too much. However, there is something you might
not have done: If you have the most recent version of Remote LANview (2.00.
11 or 12, I don't remember) you should upgrade the firmware on your IRM boards.
Apparently the re-write of the f/w is specifically to correct some timing-
related issues between LANview and hardware on the IRM. This does seem, at
least to some degree, to affect statistics. With the new firmware, we've 
had seemingly very accurate counts of all the vitals. I think the upgrade for
us was free of charge, but I'm not certain. 

>We've had better luck when we use one repeater port for EACH Shiva
>FastPath.  That's the way we've been running for a long time.  We still
>get lots of collisions on these ports, sometimes 10% or 15%, but the
>network still seems to work.  Just a few days ago we made yet another
>discovery.  We can reduce the number of collisions by choosing WHICH
>Cabletron ports we wire up to the Shiva FastPaths.  We used to take the
>first THN-MIM card in the MMAC, and dedicate it to the FastPaths.
>Now we find that we get many fewer collisions if we spread out the
>ports which connect to FastPaths, putting no more than one FastPath
>on each THN-MIM card, then using the other 7 ports on each THN-MIM for
>anything else.  I'm not sure exactly what to conclude from this.  
>Sure looks like this has uncovered some kind of timing problem in the
>MMAC design, but that's just one possibility.  

I can't speak for the problems with Shiva that you've had, but the collision
trouble you're having with your THN-MIM cards sounds like something bizarre
that happens to us here with an MT8-MIM and the PC we use to run LANview. We
have certain MMACs that, if the PC is connected to a port on the MT8-MIM, will
frequently (whenever the MMAC is polled) drop the connection. LANview then thr-
ows out an alarm (gee, I can't understand why :-). We then moved the PC's drop
to a 10BT-MIM on another MMAC we have in the same building. The problem ceased 
as soon as we did this. An interesting side to this is that when we had an
older release of LANview (the non-Windows 3.0 release), we had the exact same
problem. Makes me think the MT8-MIM itself is exhibiting a timing problem of
some sort. It could be related to the collision problem you're describing...

>Anybody else got Cabletron collision bogosity?
>Anybody out there using Cabletron's collision statistics AND finding
>them meaningful?

At one time we did - it was before we made the upgrades (see above). Since then
the worst trouble I've had to deal with has been getting the proper video
drivers for Windows and LANview to work correctly together. Otherwise the
issue with collisions and statistics has seemed to clear up. Unfortunately
we don't have any Cabletron repeaters on campus anywhere (sorry I didn't
reference), so I can't comment on the problem you're experiencing.

BTW: I don't represent Cabletron in any way, shape, or form. I'm just a 
reasonably satisfied customer.

>Thanks.
>Franklin Antonio

No problem. I hope my comments are worth at least 2 cents.

Walt

        ---------------------------------------------------------------
	   Walter R. Trachim - UNH Network Services, Durham, NH 03824
	   Phone: 603-862-4742/4773        Fax: 603-862-2030
	   E-Mail: walt@unhsst.unh.edu / "My home away from home."
        --------------------------------------------------------------- 

richa@gnarly.WV.TEK.COM (Rich Ahrendt) (06/21/91)

I have been following the last few articles about problems with Cabletron
repeaters.  We over the years have only a few Cabletron products but are
now looking at the 10baseT Cabletron product. We have used TCL and
over the last five years out of 150+ MPU's only had 1 failure.  When new
systems have been added to the net we have not had to upgrade any TCL
products due to their strict adhearance to spec.'s   We are now having to
qualify the networks for compute servers that claim extreamly fast network
throughput.

Has anyone else had good or bad experiences with these repeaters.  The 
problem with dropped packets is a concern for us here at Tektronix as the 
state of the art in computer systems gets faster and more intense.  The 
issues with increased utilization is enough of a problem without having the 
MPU's and repeaters adding potential retries on the nets.


  *****************************************************************
       Rich Ahrendt *  (503)685-2605 *  richa@orca.wv.tek.com 
       ComputerGraphicsGroup *** Tektronix Wilsonville,Oregon 
  ************************  NETWorks Support  ************************
       Rich Ahrendt *  (503)685-2605 *  richa@orca.wv.tek.com 
       ComputerGraphicsGroup *** Tektronix Wilsonville,Oregon 

lim@slc6.INS.CWRU.Edu (Hock Koon Lim) (06/22/91)

>>In article <1991Jun20.075328.6540@qualcomm.com> antonio@qualcom.qualcomm.com (Franklin Antonio) writes:
>>
>>isolate problems to a particular port?  We've recently implemented
>>Remote-Lanview, so we can see all these statistics for all the repeaters
>>remotely.  This SOUNDS great, but since all the collision statistics are
>>unreliable, all this provides is a very fancy Windows program to display
>>random numbers.  I expect these statistics should be a TOOL i can use to
>>diagnose my network.  Do i ask too much?

  You are not asking too much.  Remote-Lanview(PCOV) has helped me solve many problems
on our network.  It is great to have thoese statistics on every single ports
of the multiport repeater.  We have more that 50 MMAC-8 concentrators,  30
NB25E and many many Cabletron Ethernet cards ( > 2000+ )on our
network and the network statistics report by the Remote-Lanview did show a pretty
accurate result.  For example, one of the port show high number of the CRC Errors 
counts, and base on the information given by the PCOV, we manage to track down the 
offender.  We used the cabletron IRM and IRBM on the MMAC-8.   

>>We've had better luck when we use one repeater port for EACH Shiva
>>FastPath.  That's the way we've been running for a long time.  We still
>>get lots of collisions on these ports, sometimes 10% or 15%, but the
>>network still seems to work.  Just a few days ago we made yet another

  What do you mean by 10% to 15 %?  PCOV give the collision percentage as
%port/Board and %Port/MMAC on the IRM.  The defination of the %Port/MMAC
is the percentage of collisions of which the port is responsible, based on 
total MMAC activity.  i.e.  total port collisions count/total MMAC collision count * 100%.
It is not = total port collision count/total packet that the MMAC is transmit * 100 %.
So if PCOV report that 60% collision on Port 1/ Board 1 of you Thin-MIM and you only
have two ports active on you MMAC, then it mean that 60% of the collision the MMAC
has is from port 1.  It does not mean 60 percent of the packets transmit result
in collision.  
  

>>discovery.  We can reduce the number of collisions by choosing WHICH
>>Cabletron ports we wire up to the Shiva FastPaths.  We used to take the
>>first THN-MIM card in the MMAC, and dedicate it to the FastPaths.
>>Now we find that we get many fewer collisions if we spread out the
>>ports which connect to FastPaths, putting no more than one FastPath
>>on each THN-MIM card, then using the other 7 ports on each THN-MIM for
>>anything else.  I'm not sure exactly what to conclude from this.  
>>Sure looks like this has uncovered some kind of timing problem in the
>>MMAC design, but that's just one possibility.  
  
  It make perfect sense here.  If you put less devices on the THN-MIM port
the less collision percentage it will display on the PCOV.  You you have done
is using more ports on you THN-MIM at the same time, you have just distribute
the collision packet transmission per port.  The total collision percentage
of the THN-MIM bord still remain the same but the percentage per port will be
much smaller now - see the defination.


 
>
>
>>Anybody else got Cabletron collision bogosity?
>>Anybody out there using Cabletron's collision statistics AND finding
>>them meaningful?
>

   Yes, it is meaningful only if you read them correctly.  I have based on the 
PCOV to help me to manage our network for the last two years - it has save my 
life many time since.    I also have start using the new Cabletron Spectrum 
network management software here and it is powerful.



-- 
Hock-Koon Lim, Information Network services
Case Western Reserve University; Cleveland, Ohio, USA  44106   
(216) 368-2982        lim@ins.cwru.edu

antonio@qualcom.qualcomm.com (Franklin Antonio) (06/22/91)

In article <1991Jun20.203445.12130@unhd.unh.edu> wrt@kepler.unh.edu (Walter R Trachim) writes:
>No, you're not asking for too much. However, there is something you might
>not have done: If you have the most recent version of Remote LANview (2.00.
>11 or 12, I don't remember) you should upgrade the firmware on your IRM boards.

We're running Remote LANview 2.00.11, and we have upgraded the firmware in
all the IRM2 cards.  Thanks for mentioning it tho, and thanks for your other
comments about your sort-of-similar experiences.  We need all the help we
can get.  Anybody else god ideas about bogus collision reporting from
Cabletron repeaters?

antonio@qualcom.qualcomm.com (Franklin Antonio) (06/24/91)

In article <1991Jun21.212938.29822@usenet.ins.cwru.edu> lim@po.CWRU.Edu writes:
>>>In article <1991Jun20.075328.6540@qualcomm.com> antonio@qualcom.qualcomm.com (Franklin Antonio) writes:
>>>
>>>isolate problems to a particular port?  We've recently implemented
>>>Remote-Lanview ...
>>>unreliable, all this provides is a very fancy Windows program to display
>>>random numbers.  I expect these statistics should be a TOOL i can use to
>
>  You are not asking too much.  Remote-Lanview(PCOV) has helped me solve
> many problems on our network.  It is great to have those statistics on
> every single port...

Thank you for this report.  Now that we know that the Cabletron statistics
CAN work, we just have to figure out how to make them work for us.


>>>... we use one repeater port for EACH Shiva
>>>FastPath.  ...  We still
>>>get lots of collisions on these ports, sometimes 10% or 15%...
>
>  What do you mean by 10% to 15 %?  PCOV give the collision percentage as
>%port/Board and %Port/MMAC on the IRM.  The defination of the %Port/MMAC
>is the percentage of collisions of which the port is responsible, based on 
>total MMAC activity.

I just look at the total number of collisions reported for a port, and
divide by the number of packets on that port.  We often have these numbers
in the 10% to 15% range on Cabletron repeater ports which have nothing
but a single Shiva FastPath on them.   


>>>discovery.  We can reduce the number of collisions by choosing WHICH
>>>ports we use for the Shiva FastPaths...

>  It make perfect sense here.  If you put less devices on the THN-MIM port
>the less collision percentage it will display on the PCOV.  You you have done
>is using more ports on you THN-MIM at the same time, you have just distribute
>the collision packet transmission per port.  The total collision percentage
>of the THN-MIM bord still remain the same but the percentage per port will be
>much smaller now - see the defination.

I guess i didn't make myself clear enough.  We dedicate one Cabletron
repeater port to each Shiva FastPath, even tho the FastPath's are only
about 2ft away from the repeater.  These particular Cabletron repeater
ports have one each Shiva FastPath, and _nothing_ _else_ on them.

Now, given that, which ports on the MMAC should i use?  Are all repeater
ports created equal?  If i pull some cables from the front of the THN-MIMs
and plug them in different ports -- this is _just_ a _permutation_ of ports!
-- i would expect all the collision statistics to remain the same.
(of course now reported on different port numbers, because i have moved
the "interesting" traffic to different repeater ports)
What we seem to have discovered is that all repeater ports are _not_ 
created equal.  The worst choice seems to be to plug those cables from
the FastPaths into ports all on one THN-MIM card.  Still one repeater
port per FastPath, and nothing else on those ports.  The best choice seems
to be to plug the cables from the FastPaths into ports chosen one per
each THN-MIM card.  Still one repeater port per FastPath, and nothing else
on those ports.  This doesn't make sense at all for any reasonable 
functioning of the repeaters.

No way will i believe that permuting the cables at the front panel of
the repeater should change the performance of my network.

Thanks again for your thoughts tho.  I'm pleased to know that someone
is haveing good luck with Cabletron's repeaters.  I was beginning to 
think we just made a really bad decision when we decided to go with them.

I still have my network problems... and i'm still interested to hear
from anyone who has Shiva FastPath boxes successfully working with
Cabletron repeaters.

jal@acc.flint.umich.edu (John Lauro) (06/24/91)

In article <1991Jun24.000210.4759@qualcomm.com> antonio@qualcom.qualcomm.com (Franklin Antonio) writes:
>I just look at the total number of collisions reported for a port, and
>divide by the number of packets on that port.  We often have these numbers
>in the 10% to 15% range on Cabletron repeater ports which have nothing
>but a single Shiva FastPath on them.   
>
>
>>>>discovery.  We can reduce the number of collisions by choosing WHICH
>>>>ports we use for the Shiva FastPaths...
>
>>  It make perfect sense here.  If you put less devices on the THN-MIM port
>>the less collision percentage it will display on the PCOV.  You you have done
>>is using more ports on you THN-MIM at the same time, you have just distribute
>>the collision packet transmission per port.  The total collision percentage
>>of the THN-MIM bord still remain the same but the percentage per port will be
>>much smaller now - see the defination.
>
>Now, given that, which ports on the MMAC should i use?  Are all repeater
>ports created equal?  If i pull some cables from the front of the THN-MIMs
>and plug them in different ports -- this is _just_ a _permutation_ of ports!
>-- i would expect all the collision statistics to remain the same.
>(of course now reported on different port numbers, because i have moved
>the "interesting" traffic to different repeater ports)
>What we seem to have discovered is that all repeater ports are _not_ 
>created equal.  The worst choice seems to be to plug those cables from
>the FastPaths into ports all on one THN-MIM card.  Still one repeater
>port per FastPath, and nothing else on those ports.  The best choice seems
>to be to plug the cables from the FastPaths into ports chosen one per
>each THN-MIM card.  Still one repeater port per FastPath, and nothing else
>on those ports.  This doesn't make sense at all for any reasonable 
>functioning of the repeaters.
>
>No way will i believe that permuting the cables at the front panel of
>the repeater should change the performance of my network.

This is all speculation, and I probabbly don't know what I am talking
about...

I don't know how the fastpaths work, but if they advertise information
based on responding to broadcast then 15% collisions of an idle network
sounds reasonable.  Basically they all receive a packet, and then all
respond at the same time.  By changing the cable arragement just slightly,
you minutely alter the timing from port to port just enough to allow the
collision avoidance built into ethernet controllers to operate better.
I would expect (but do not know for sure) that the timing from port to
port on the same card is slightly smaller than from port of one card to
port of a second card.

   - John_Lauro@ub.cc.umich.edu

lim@slc6.INS.CWRU.Edu (Hock Koon Lim) (06/25/91)

In article <1991Jun24.000210.4759@qualcomm.com> antonio@qualcom.qualcomm.com (Franklin Antonio) writes:
>
>Thank you for this report.  Now that we know that the Cabletron statistics
>CAN work, we just have to figure out how to make them work for us.
>
>
>I just look at the total number of collisions reported for a port, and
>divide by the number of packets on that port.  We often have these numbers
>in the 10% to 15% range on Cabletron repeater ports which have nothing
>but a single Shiva FastPath on them.   
>
  Did you try to connect other devices beside Shiva FastPath?  I pick up a few ports
on my network which connect to the Sun4/490 files servers, the packet collision is 
about 3%.  These servers provide many services to our network and the network utilization
on this MMAC8/IRM is about 10-30% in the day time.  I do not have a Shiva FastPath on 
hand to verify the number.

>
>I guess i didn't make myself clear enough.  We dedicate one Cabletron
>repeater port to each Shiva FastPath, even tho the FastPath's are only
>about 2ft away from the repeater.  These particular Cabletron repeater
>ports have one each Shiva FastPath, and _nothing_ _else_ on them.
>

   How is you thinwire terminated?  You might want to make sure that the cable
is terminated only at one end but not on the THIN-MIM side.   

>Now, given that, which ports on the MMAC should i use?  Are all repeater
>ports created equal?  If i pull some cables from the front of the THN-MIMs
>and plug them in different ports -- this is _just_ a _permutation_ of ports!
>-- i would expect all the collision statistics to remain the same.
>(of course now reported on different port numbers, because i have moved
>the "interesting" traffic to different repeater ports)
>What we seem to have discovered is that all repeater ports are _not_ 
>created equal.  The worst choice seems to be to plug those cables from
>the FastPaths into ports all on one THN-MIM card.  Still one repeater
>port per FastPath, and nothing else on those ports.  The best choice seems
>to be to plug the cables from the FastPaths into ports chosen one per
>each THN-MIM card.  Still one repeater port per FastPath, and nothing else
>on those ports.  This doesn't make sense at all for any reasonable 
>functioning of the repeaters.
>

   I also check the number with the IRBM(similar to IRM-2), the collision rate is
about 0.3% to 1 % on a particular building.  All repeater ports on the THIN-MIM is
created equal.  I would suggest you try to connect other devices beside the
Shiva FastPath and see what happan.

>No way will i believe that permuting the cables at the front panel of
>the repeater should change the performance of my network.
>
>Thanks again for your thoughts tho.  I'm pleased to know that someone
>is haveing good luck with Cabletron's repeaters.  I was beginning to 
>think we just made a really bad decision when we decided to go with them.
>
>I still have my network problems... and i'm still interested to hear
>from anyone who has Shiva FastPath boxes successfully working with
>Cabletron repeaters.

-- 
Hock-Koon Lim, Information Network services
Case Western Reserve University; Cleveland, Ohio, USA  44106   
(216) 368-2982        lim@ins.cwru.edu