kenkel@mcnc.org (Stephen Kenkel) (11/22/89)
It has struck me recently that there does not seem to
exist a good, standard, publicly available collection of
benchmark circuits for circuit and timing level simulators.
At ICCAD, for instance, I noticed a number of talks giving the
performance of simulators, but on circuits
that had been selected by the authors of the programs.
This makes it difficult to qualify claims of "100X Spice"
and "always converges".
In contrast, the place and route community and the logic
synthesis community each have standard benchmarks which
allow comparisons.
I would like to propose that a publicly available collection be
established of circuits. The circuits would be selected to
include tough circuits for traditional simulators (bipolar,
opamps, feedback, bi-stable, etc), very large circuits, circuits
which are suitable for waveform relaxation, circuits which
give rise to unknown states with timing simulators, etc.
MCNC would be willing to maintain an make the circuits available
by anonymous ftp, if no one else wants to take on this responsbility.
At a minimum, the circuits would have to be:
1. Freely distributable. This should not be too much of a problem,
since it is difficult to reverse engineer a circuit from the flattened
netlist.
2. In SPICE compatabile netlist form. (For lack of a more universal
standard)
3. Use a standard device model: SPICE Level I, II, III, IV or BJT.
Many industrial circuits use custom device models, which again
confuses comparisons.
4. MOST important: include an output file containing what is
believed to be a correct simulation at the circuit level, for
comparison purposes. If possible, enought information should be
given so that SPICE, HSPICE, PSPICE, or whatever can be re-run to
verify the correctness of the output.
So, is there interest in such a collection? Anyone want to contribute
circuits?
DISCLAIMER:
MCNC distributes a circuit level simulator (CAzM), and
has a vested interest in the benchmarking of such programs.mark@mips.COM (Mark G. Johnson) (11/22/89)
In article <5780@alvin.mcnc.org> kenkel@mcnc.org (Stephen Kenkel) writes: > >I would like to propose that a publicly available collection be >established of circuits. The circuits would be selected to >include tough circuits for traditional simulators (bipolar, >opamps, feedback, bi-stable, etc), very large circuits, circuits >which are suitable for waveform relaxation, circuits which >give rise to unknown states with timing simulators, etc. > Here is a "shar" archive containing one such circuit, and also its output when run on MetaSoft's program HSPICE. CPU time on an M2000 computer (20 VUPS hardware) was 5.80 seconds of which 4.01 seconds was actual transient analysis. The input file is a SPICE compatible netlist, which only produces output by doing ".PRINT" statements (for ease of comparison ... plots are notoriously tough to compare algorithmically). Semiconductor technology is 2.0 micron CMOS, modelled with SPICE LEVEL=2 equations. I hereby release the input file into the public domain; anybody can use it freely without reservation. There are several more input files that are easily distributed (in fact MIPS uses these others as benchmarks; see "MIPS Performance Brief" for details). However, I will delay posting these additional files for a while, pending confirmation that CaZM and the other non-SPICE simulators can indeed run the first one and get similar answers. Note that the shar archive contains 2 files; following the directions below will ensure that you get them both. #--------------------------Cut Here-------------------------- #! /bin/sh # This is a shell archive. Remove anything before the "#! /bin/sh" line, # then unpack it by saving it in a file and typing "sh file." # # Wrapped by Mark G. Johnson,,,,obiwan,open (mark) at hal on Tue Nov 21 14:00:16 1989 # # unpacks with default permissions # # Contents : gbump gbump.hspice # if `test ! -s gbump` then echo "x - gbump" cat > gbump << '@\Rogue\Monster\' * FILE GBUMP: GROUND BOUNCE OF CMOS 74ACT244 OCTAL DRIVER * * * .OPTIONS ACCT ITL1=300 GMIN=1E-11 ABSTOL=1E-9 + LIMPTS=205 NOMOD .WIDTH OUT=80 * * .TRAN 0.1NS 13NS * ********************************************************************** * * POWER SUPPLIES * * VDD 1 0 DC 5.0 VSS 2 0 DC 0.0 * * * THE OUTPUT ENABLES ARE PERMANENTLY ASSERTED IN THIS SIMULATION VOD 11 0 DC 0.0 VOE 12 0 DC 5.0 * * * RDD 1 2 1E6 ROD 11 12 1E6 * ********************************************************************** * * INPUT WIGGLES (PIECEWISE LINEAR WAVEFORMS) * * * VIN 5 0 PWL( 0NS 5.0 1NS 5.0 2NS 0V 10NS 0V ) * RA1A 5 0 100K * * * * ********************************************************************** ********************************************************************** ********************************************************************** * * CIRCUIT HOOKUP (DEFINITION AND INTERCONNECTION) * * * FROM TABLE 1.1-1 OF THE TEXAS INSTRUMENTS "ADVANCED CMOS * LOGIC DESIGNERS HANDBOOK" (SCAA001A) WE SEE THAT A 20 PIN * 300-MIL PLASTIC DIP WITH CORNER POWER AND GROUND HAS 13.7 * NANOHENRIES OF INDUCTANCE IN THE SUPPLY LEADS. * * RATHER THAN SIMULTANEOUSLY SWITCHING EIGHT BUFFERS INTO 13.7 NH, * THIS SIMULATION SWITCHES ONE BUFFER INTO (8 * 13.7) = 109 * NANOHENRIES. * * * * * * HERE ARE THE PARASITIC INDUCTORS (PIN + PACKAGE + BONDWIRE) * AND RESISTORS IN SERIES WITH VDD AND VSS. THERE IS ALSO SOME * AMOUNT OF ON-CHIP CAPACITANCE BETWEEN VDD AND VSS * L44 1 43 109.0E-9 R44 43 44 0.050 * L77 2 76 109.0E-9 R77 76 77 0.050 * C4477 44 77 50.0E-12 * * * * * * * * THE PREDRIVER STAGES * M900 100 5 44 44 P W=22U L=2U AD=176P PD=37U M901 100 5 77 77 N W=11U L=2U AD=88P PD=26U * * M902 201 11 44 44 P W=40U L=2U AD=320P PD=55U M903 200 100 201 44 P W=40U L=2U AD=320P PD=55U M904 200 100 77 77 N W=25U L=2U AD=200P PD=40U M905 200 11 77 77 N W=25U L=2U AD=200P PD=40U * M906 300 12 44 44 P W=24U L=2U AD=192P PD=39U M907 300 100 44 44 P W=24U L=2U AD=192P PD=39U M908 300 100 301 77 N W=11U L=2U AD=88P PD=26U M909 301 12 77 77 N W=11U L=2U AD=88P PD=26U * * M910 400 200 44 44 P W=130U L=2U AD=1040P PD=145U M911 400 200 77 77 N W=65U L=2U AD=520P PD=80U * M912 500 300 44 44 P W=66U L=2U AD=528P PD=81U M913 500 300 77 77 N W=33U L=2U AD=264P PD=48U * * * THE PAD DRIVER TRANSISTORS * M914 600 400 44 44 P W=1200U L=2U AD=4800P PD=600U M915 600 500 77 77 N W=600U L=2U AD=2400P PD=300U * * * * THE OUTPUT LOAD NETWORK * R916 1 600 1.0K R917 2 600 0.5K C918 2 600 75.0P * * * * ********************************************************************** * * OUTPUTS (RAW OUTPUT FROM SPICE) * * * * .PRINT TRAN V(77) V(44) V(500) V(600) * * * ********************************************************************** * * MOS MODEL PARAMETERS FOR 2 MICRON CMOS * .MODEL N NMOS + LEVEL=2 VTO=0.7 TOX=400E-10 + NSUB=9E15 XJ=0.15U LD=0.20U + UO=666 UCRIT=.65E5 UEXP=0.123 + VMAX=5E4 NEFF=4.0 DELTA=1.4 + RSH=36 CGSO=200P CGDO=200P + CJ=200U CJSW=500P MJ=0.75 + MJSW=0.30 PB=0.80 NFS=1E11 * * .MODEL P PMOS + LEVEL=2 VTO=-0.70 TOX=400E-10 + NSUB=7E15 XJ=0.06U LD=0.20U + UO=250 UCRIT=.85E5 UEXP=0.3 + VMAX=3E4 NEFF=2.65 DELTA=1.0 + RSH=100 CGSO=190P CGDO=190P + CJ=250U CJSW=350P MJ=.55 + MJSW=0.34 PB=0.80 NFS=1E11 * ********************************************************************** * * * * .END @\Rogue\Monster\ else echo "shar: Will not over write gbump" fi if `test ! -s gbump.hspice` then echo "x - gbump.hspice" cat > gbump.hspice << '@\Rogue\Monster\' 1****** h s p i c e 8807b 14: 0:38 21-nov89 u n i x ****** copyright 1988 meta-software,inc. *****site:mips sunnyvale ***** * file gbump: ground bounce of cmos 74act244 octal driver ****** input listing evaluation expires 9008 ****** * * * .options acct itl1=300 gmin=1e-11 abstol=1e-9 + limpts=205 nomod .width out=80 * * .tran 0.1ns 13ns * ********************************************************************** * * power supplies * * vdd 1 0 dc 5.0 vss 2 0 dc 0.0 * * * the output enables are permanently asserted in this simulation vod 11 0 dc 0.0 voe 12 0 dc 5.0 * * * rdd 1 2 1e6 rod 11 12 1e6 * ********************************************************************** * * input wiggles (piecewise linear waveforms) * * * vin 5 0 pwl( 0ns 5.0 1ns 5.0 2ns 0v 10ns 0v ) * ra1a 5 0 100k * * * * ********************************************************************** ********************************************************************** ********************************************************************** * * circuit hookup (definition and interconnection) * * * from table 1.1-1 of the texas instruments advanced cmos * logic designers handbook (scaa001a) we see that a 20 pin * 300-mil plastic dip with corner power and ground has 13.7 * nanohenries of inductance in the supply leads. * * rather than simultaneously switching eight buffers into 13.7 nh, * this simulation switches one buffer into (8 * 13.7) = 109 * nanohenries. * * * * * * here are the parasitic inductors (pin + package + bondwire) * and resistors in series with vdd and vss. there is also some * amount of on-chip capacitance between vdd and vss * l44 1 43 109.0e-9 r44 43 44 0.050 * l77 2 76 109.0e-9 r77 76 77 0.050 * c4477 44 77 50.0e-12 * * * * * * * * the predriver stages * m900 100 5 44 44 p w=22u l=2u ad=176p pd=37u m901 100 5 77 77 n w=11u l=2u ad=88p pd=26u * * m902 201 11 44 44 p w=40u l=2u ad=320p pd=55u m903 200 100 201 44 p w=40u l=2u ad=320p pd=55u m904 200 100 77 77 n w=25u l=2u ad=200p pd=40u m905 200 11 77 77 n w=25u l=2u ad=200p pd=40u * m906 300 12 44 44 p w=24u l=2u ad=192p pd=39u m907 300 100 44 44 p w=24u l=2u ad=192p pd=39u m908 300 100 301 77 n w=11u l=2u ad=88p pd=26u m909 301 12 77 77 n w=11u l=2u ad=88p pd=26u * * m910 400 200 44 44 p w=130u l=2u ad=1040p pd=145u m911 400 200 77 77 n w=65u l=2u ad=520p pd=80u * m912 500 300 44 44 p w=66u l=2u ad=528p pd=81u m913 500 300 77 77 n w=33u l=2u ad=264p pd=48u * * * the pad driver transistors * m914 600 400 44 44 p w=1200u l=2u ad=4800p pd=600u m915 600 500 77 77 n w=600u l=2u ad=2400p pd=300u * * * * the output load network * r916 1 600 1.0k r917 2 600 0.5k c918 2 600 75.0p * * * * ********************************************************************** * * outputs (raw output from spice) * * * * .print tran v(77) v(44) v(500) v(600) * * * ********************************************************************** * * mos model parameters for 2 micron cmos * .model n nmos + level=2 vto=0.7 tox=400e-10 + nsub=9e15 xj=0.15u ld=0.20u + uo=666 ucrit=.65e5 uexp=0.123 + vmax=5e4 neff=4.0 delta=1.4 + rsh=36 cgso=200p cgdo=200p + cj=200u cjsw=500p mj=0.75 + mjsw=0.30 pb=0.80 nfs=1e11 * * .model p pmos + level=2 vto=-0.70 tox=400e-10 + nsub=7e15 xj=0.06u ld=0.20u + uo=250 ucrit=.85e5 uexp=0.3 + vmax=3e4 neff=2.65 delta=1.0 + rsh=100 cgso=190p cgdo=190p + cj=250u cjsw=350p mj=.55 + mjsw=0.34 pb=0.80 nfs=1e11 * ********************************************************************** * * * * .end 1****** h s p i c e 8807b 14: 0:38 21-nov89 u n i x ****** copyright 1988 meta-software,inc. *****site:mips sunnyvale ***** * file gbump: ground bounce of cmos 74act244 octal driver ****** operating point information tnom= 25.000 temp= 25.000 ****** ***** operating point status is voltage simulation time is 0. node =voltage node =voltage node =voltage +0:1 = 5.0000 0:2 = 0. 0:5 = 5.0000 +0:11 = 0. 0:12 = 5.0000 0:43 = 5.0000 +0:44 = 4.9995 0:76 = 0. 0:77 = 34.8429p +0:100 = 71.7641n 0:200 = 4.9995 0:201 = 4.9995 +0:300 = 4.9995 0:301 = 35.8888n 0:400 = 12.1605n +0:500 = 23.9224n 0:600 = 4.7878 1****** h s p i c e 8807b 14: 0:38 21-nov89 u n i x ****** copyright 1988 meta-software,inc. *****site:mips sunnyvale ***** * file gbump: ground bounce of cmos 74act244 octal driver ****** transient analysis tnom= 25.000 temp= 25.000 ****** time voltage voltage voltage voltage 0:77 0:44 0:500 0:600 0. 0. 4.9995 0. 4.7878 100.00000p 0. 4.9995 0. 4.7878 200.00000p 0. 4.9995 0. 4.7878 300.00000p 0. 4.9995 0. 4.7878 400.00000p 0. 4.9995 0. 4.7878 500.00000p 0. 4.9995 0. 4.7878 600.00000p 0. 4.9995 0. 4.7878 700.00000p 0. 4.9995 0. 4.7878 800.00000p 0. 4.9995 0. 4.7878 900.00000p 0. 4.9995 0. 4.7878 1.00000n 0. 4.9995 0. 4.7878 1.10000n -3.7585m 4.9959 -3.4256m 4.7875 1.20000n -5.7944m 4.9940 -5.3921m 4.7873 1.30000n -6.3895m 4.9934 -6.0209m 4.7870 1.40000n -6.5644m 4.9932 -6.2022m 4.7867 1.50000n -6.6304m 4.9929 -6.2071m 4.7864 1.60000n -6.4940m 4.9926 -5.9209m 4.7861 1.70000n -6.0875m 4.9925 -5.3024m 4.7858 1.80000n -5.1198m 4.9927 -4.1490m 4.7856 1.90000n -3.9268m 4.9928 -3.5274m 4.7853 2.00000n -2.4118m 4.9925 -3.7128m 4.7851 2.10000n 4.8552m 4.9969 -64.2519u 4.7851 2.20000n 9.9400m 4.9985 996.9036u 4.7852 2.30000n 13.3635m 4.9976 -1.1587m 4.7852 2.40000n 16.4834m 4.9963 -3.9596m 4.7853 2.50000n 16.8181m 4.9913 -8.3046m 4.7854 2.60000n 15.3034m 4.9836 -3.4711m 4.7855 2.70000n 12.9268m 4.9745 17.1126m 4.7857 2.80000n 8.1996m 4.9627 64.0100m 4.7858 2.90000n 2.6955m 4.9510 140.8365m 4.7861 3.00000n 5.3863m 4.9481 260.2752m 4.7863 3.10000n 16.8809m 4.9541 440.5934m 4.7865 3.20000n 34.6304m 4.9662 662.6986m 4.7868 3.30000n 68.1812m 4.9933 936.2604m 4.7870 3.40000n 156.4917m 5.0714 1.2945 4.7870 3.50000n 287.7428m 5.1891 1.7131 4.7866 3.60000n 562.4576m 5.4412 2.2819 4.7849 3.70000n 945.1114m 5.7957 2.9473 4.7819 3.80000n 1.6399 6.4570 3.8832 4.7754 3.90000n 2.5605 7.3427 5.0040 4.7649 4.00000n 3.3872 8.1390 5.9303 4.7471 4.10000n 3.9673 8.6932 6.5744 4.7244 4.20000n 4.0807 8.7876 6.8102 4.6915 4.30000n 3.9769 8.6669 6.8709 4.6516 4.40000n 3.7797 8.4494 6.8063 4.6013 4.50000n 3.5337 8.1812 6.6762 4.5455 4.60000n 3.2015 7.8229 6.4386 4.4809 4.70000n 2.7994 7.3921 6.1204 4.4103 4.80000n 2.2683 6.8297 5.6760 4.3320 4.90000n 1.7198 6.2491 5.2034 4.2515 5.00000n 1.1447 5.6397 4.6715 4.1661 5.10000n 624.0104m 5.0831 4.2039 4.0787 5.20000n 237.5489m 4.6582 3.8852 3.9902 5.30000n 5.3356m 4.3849 3.7223 3.9018 5.40000n -88.2877m 4.2477 3.6903 3.8140 5.50000n -63.2096m 4.2276 3.7624 3.7274 5.60000n 26.5661m 4.2718 3.8869 3.6417 5.70000n 168.8664m 4.3694 4.0474 3.5573 5.80000n 313.6076m 4.4700 4.2047 3.4734 5.90000n 450.3763m 4.5643 4.3421 3.3902 6.00000n 516.6785m 4.5915 4.4016 3.3077 6.10000n 577.2181m 4.6131 4.4548 3.2253 6.20000n 568.2798m 4.5686 4.4412 3.1434 6.30000n 553.6599m 4.5186 4.4221 3.0615 6.40000n 508.4516m 4.4406 4.3729 2.9803 6.50000n 460.7419m 4.3603 4.3213 2.8991 6.60000n 407.1149m 4.2767 4.2592 2.8190 6.70000n 353.0039m 4.1927 4.1962 2.7390 6.80000n 303.8419m 4.1165 4.1329 2.6606 6.90000n 255.0846m 4.0408 4.0696 2.5823 7.00000n 217.7782m 3.9796 4.0132 2.5061 7.10000n 181.4082m 3.9195 3.9573 2.4301 7.20000n 159.6902m 3.8774 3.9130 2.3566 7.30000n 139.1703m 3.8367 3.8696 2.2833 7.40000n 133.6212m 3.8145 3.8395 2.2128 7.50000n 129.2964m 3.7938 3.8106 2.1425 7.60000n 137.9897m 3.7897 3.7946 2.0751 7.70000n 147.7475m 3.7870 3.7796 2.0079 7.80000n 167.1518m 3.7977 3.7757 1.9436 7.90000n 187.3449m 3.8095 3.7727 1.8795 8.00000n 213.2601m 3.8307 3.7782 1.8181 8.10000n 239.6431m 3.8527 3.7844 1.7569 8.20000n 267.7777m 3.8802 3.7961 1.6981 8.30000n 296.0554m 3.9081 3.8084 1.6396 8.40000n 321.8643m 3.9371 3.8231 1.5833 8.50000n 347.4714m 3.9662 3.8380 1.5272 8.60000n 364.3462m 3.9902 3.8509 1.4730 8.70000n 380.5069m 4.0137 3.8637 1.4190 8.80000n 372.1641m 4.0165 3.8633 1.3669 8.90000n 361.8174m 4.0176 3.8618 1.3150 9.00000n 336.3708m 4.0072 3.8520 1.2648 9.10000n 309.6894m 3.9958 3.8415 1.2149 9.20000n 286.5441m 3.9912 3.8344 1.1668 9.30000n 263.6879m 3.9872 3.8276 1.1189 9.40000n 242.6576m 3.9883 3.8240 1.0730 9.50000n 221.7766m 3.9898 3.8206 1.0273 9.60000n 204.6850m 3.9983 3.8224 983.6733m 9.70000n 187.9033m 4.0074 3.8245 940.2202m 9.80000n 171.9300m 4.0206 3.8304 898.8939m 9.90000n 156.0227m 4.0340 3.8365 857.7415m 10.00000n 141.9410m 4.0516 3.8462 818.1376m 10.10000n 127.9582m 4.0715 3.8585 780.0395m 10.20000n 114.5722m 4.0936 3.8729 743.0569m 10.30000n 101.6392m 4.1174 3.8891 706.9079m 10.40000n 89.1974m 4.1431 3.9075 671.8417m 10.50000n 77.2469m 4.1709 3.9280 637.8582m 10.60000n 65.5567m 4.2004 3.9506 604.9115m 10.70000n 54.1269m 4.2315 3.9751 573.0014m 10.80000n 42.8960m 4.2642 4.0015 542.0721m 10.90000n 31.8641m 4.2984 4.0299 512.1236m 11.00000n 20.8904m 4.3338 4.0599 483.0925m 11.10000n 9.9749m 4.3706 4.0916 454.9790m 11.20000n -918.2743u 4.4084 4.1249 427.7142m 11.30000n -11.7891m 4.4473 4.1597 401.2981m 11.40000n -22.5285m 4.4872 4.1960 375.6555m 11.50000n -33.1364m 4.5283 4.2338 350.7864m 11.60000n -44.0325m 4.5698 4.2727 326.6143m 11.70000n -55.2168m 4.6119 4.3127 303.1394m 11.80000n -66.5155m 4.6545 4.3537 280.2810m 11.90000n -77.9284m 4.6976 4.3957 258.0390m 12.00000n -89.4443m 4.7412 4.4385 236.3309m 12.10000n -101.0633m 4.7851 4.4822 215.1567m 12.20000n -112.9299m 4.8291 4.5264 194.4335m 12.30000n -125.0440m 4.8732 4.5711 174.1612m 12.40000n -137.4449m 4.9172 4.6162 154.2588m 12.50000n -150.1326m 4.9612 4.6617 134.7261m 12.60000n -163.0989m 5.0048 4.7073 115.4851m 12.70000n -176.3438m 5.0483 4.7530 96.5358m 12.80000n -189.8371m 5.0913 4.7987 77.8020m 12.90000n -203.5789m 5.1341 4.8444 59.2836m 13.00000n -217.5504m 5.1764 4.8899 40.8709m y ***** job concluded 1****** h s p i c e 8807b 14: 0:38 21-nov89 u n i x ****** copyright 1988 meta-software,inc. *****site:mips sunnyvale ***** * file gbump: ground bounce of cmos 74act244 octal driver ****** job statistics summary tnom= 25.000 temp= 25.000 ****** # nodes = 18 # elements= 32 # real*8 mem avail/used= 250000/ 9320 # diodes= 0 # bjts = 0 # jfets = 0 # mosfets = 16 analysis time # points tot. iter conv.iter op point 0.22 1 14 transient 4.01 131 246 110 rev= 0 input+setup 1.29 pass1 0.86 readin 0.30 errchk 0.08 setup 0.05 output 0.20 total cpu time 5.80 seconds job started at 14: 0:38 21-nov89 job ended at 14: 0:45 21-nov89 @\Rogue\Monster\ else echo "shar: Will not over write gbump.hspice" fi # to concatenate archives, remove anything after this line exit 0 -- -- Mark Johnson MIPS Computer Systems, 930 E. Arques, Sunnyvale, CA 94086 (408) 991-0208 mark@mips.com {or ...!decwrl!mips!mark}
waters@darla.sps.mot.com (Strawberry Jammer) (11/22/89)
In article <5780@alvin.mcnc.org> kenkel@mcnc.org (Stephen Kenkel) writes: }I would like to propose that a publicly available collection be }established of circuits. The circuits would be selected to }include tough circuits for traditional simulators (bipolar, }opamps, feedback, bi-stable, etc), very large circuits, circuits }which are suitable for waveform relaxation, circuits which }give rise to unknown states with timing simulators, etc. An excellent idea! With the usual caveats as to generality and applicability of the test cases. As an industrial CAD type I would also urge the inclusion of some realisticly sized models. For example a portion of a microprocessor with analog I/O - maybe 1000+ (10K+ ?) devices. Far too often I see papers talking about "large" behavioral simulators or routers which have been tested on 100 or 200 gates and are totally useless with 500K+ which is where such things are useful today. }1. Freely distributable. This should not be too much of a problem, }since it is difficult to reverse engineer a circuit from the flattened }netlist. I think this implies some "diddling" of models and process parameters. I know we (Motorola) would not give out any accurate process models and I'm sure the other manufacturers feel the same way. }2. In SPICE compatabile netlist form. (For lack of a more universal }standard) I would suggest EDIF V 2 0 0 which is used for netlists throughout the ASIC industry. There are even examples in some of the older documentation about translating SPICE netists to/from EDIF, it is a very straightforward mapping. }3. Use a standard device model: SPICE Level I, II, III, IV or BJT. }Many industrial circuits use custom device models, which again }confuses comparisons. Yes (see 1 above). It would however have to be of "equivalent" complexity to models in actual use. The best way of achieving this might be by having a number of device engineers at IC companies comment on and validate the model being used. }4. MOST important: include an output file containing what is }believed to be a correct simulation at the circuit level, for }comparison purposes. If possible, enought information should be }given so that SPICE, HSPICE, PSPICE, or whatever can be re-run to }verify the correctness of the output. Again I think EDIF V 2 0 0 might be useful here, possibly one of the EDIF Test proposals for handling test vectors might be worth testing too. It is definitly worth some trouble to use a common standard for both stimuli and output analysis. *Mike Waters AA4MW/7 waters@dover.sps.mot.com * Justice is incidental to law and order. -- J. Edgar Hoover
rxet30@toto.oakhill.uucp (Steve Hamm) (11/23/89)
In article <5780@alvin.mcnc.org> kenkel@mcnc.org (Stephen Kenkel) writes: > It has struck me recently that there does not seem to > exist a good, standard, publicly available collection of > benchmark circuits for circuit and timing level simulators. Nice idea. I hope there's some response. I put a request out for problem circuits some time ago and got, I believe, exactly six circuit files. (Mark Johnson, from MIPS, sent me four of the six circuits. Thanks Mark!) I also got a larger number of requests for my "library of problem circuits" -- but most of the circuits weren't really "problem" circuits, and six isn't a library. Anyway, I suspect there may be more interest in having such a library than in helping to build one. One place to start may be Berkeley. In the SPICE 3C1 distribution, Tom Quarles' writeup (actually an appendix to his dissertation) includes results from a number of "benchmark" circuits (with a fairly descriptive naming convention) that are said to be available from the Berkeley Industrial Liaison office. You might check to see if they wish to contribute these (and thereby reduce their distribution load). Anyone from UC Berkeley care to comment? As a footnote, if you're going to set something up, I'd personally prefer a mail server (ala NETLIB), since I don't have FTP available from my site. > At ICCAD, for instance, I noticed a number of talks giving the > performance of simulators, but on circuits > that had been selected by the authors of the programs. > This makes it difficult to qualify claims of "100X Spice" > and "always converges". Ah, the inevitable ICCAD shell game. I've found it interesting that in industry, at least, in spite of the many papers presented on simulators that all are 100X SPICE and "always converge" nothing seems to have displaced the direct method circuit simulators (e.g. SPICE). TANSTAAFL. Even with UCB standard SPICE input using 2G6 or 3C1 models, there will be games that can be played. Commercial simulators typically relax the SPICE 2G6 default convergence tolerances by factors of 20 to 500000. Some define convergence differently. And different versions of, for instance, the level 3 mosfet model (say, using nice smooth splines to remove nastyness in the I-V and Q-V characteristics :-)) aren't necessarily the "same" model as SPICE for all applications, but this massively changes the overall efficiency and convergence characteristics of a simulator. So an answer is not necessarily the "same" even when it (perhaps by fortunate coincidence) is close to correct. Still, having a collection of standard circuits would be far better than not. > DISCLAIMER: > MCNC distributes a circuit level simulator (CAzM), and > has a vested interest in the benchmarking of such programs. That beats a number of the commercial simulator folks, who may be more interested in making sure that such benchmarking doesn't occur ;-) Oh, yeah, my own disclaimer: I make my living hacking on a circuit simulator for in-house use. --Steve -- Steve Hamm Motorola Inc. Semiconductor Sector CAD, Austin TX (512) 928-6612 oakhill!monarch!rxet30@cs.utexas.edu
tquarles@mentor.com (Thomas Quarles) (11/23/89)
In article <RXET30.89Nov22150818@toto.oakhill.uucp> rxet30@toto.oakhill.uucp (Steve Hamm) writes: > >In article <5780@alvin.mcnc.org> kenkel@mcnc.org (Stephen Kenkel) writes: >> It has struck me recently that there does not seem to >> exist a good, standard, publicly available collection of >> benchmark circuits for circuit and timing level simulators. > >Nice idea. I hope there's some response. I put a request out for >problem circuits some time ago and got, I believe, exactly six >circuit files. (Mark Johnson, from MIPS, sent me four of the >six circuits. Thanks Mark!) I also got a larger number of requests >for my "library of problem circuits" -- but most of the circuits >weren't really "problem" circuits, and six isn't a library. Anyway, >I suspect there may be more interest in having such a library than >in helping to build one. > >One place to start may be Berkeley. In the SPICE 3C1 distribution, >Tom Quarles' writeup (actually an appendix to his dissertation) >includes results from a number of "benchmark" circuits (with a fairly >descriptive naming convention) that are said to be available from the >Berkeley Industrial Liaison office. You might check to see if they wish >to contribute these (and thereby reduce their distribution load). > >Anyone from UC Berkeley care to comment? > Well, since my name has been brought up, I guess I'd better stick in a few words here. I'm still working on a release tape of the benchmark circuits used in the appendix to my dissertation. The problem is that it takes a LONG time to track down the origin of 150+ circuits and make sure that they are all redistributable, and contact the originators of those that aren't clearly marked to ascertain whether they are redistributable. Many of these circuits came to Berkeley in bug reports, and thus their origins are buried in my archive of 2000+ old SPICE3 messages. When I can get through all of these, or at least most of them, Berkeley plans to release a tape of collected benchmark circuits. I hope to get this done before long, and will post a message here when such a tape is available. I no longer have any direct input on the matter of ftp access or just mag tape by mail, that decision is up to U.C. Berkeley. Tom Quarles tquarles@mntgfx.mentor.com
whitaker@manse.cad.cs.man.ac.uk (Nigel Whitaker) (11/24/89)
Hello folks, I saw Stephens request for circuit simulation benchmarks, and thought I would add my 1/2 pence (:-) worth, in my hunt for gate level (and above) simulation benchmarks. In article <5780@alvin.mcnc.org>, kenkel@mcnc.org (Stephen Kenkel) writes: > It has struck me recently that there does not seem to > exist a good, standard, publicly available collection of > benchmark circuits for circuit and timing level simulators. The same situation is also true for simulators at the gate level and above. Indeed I, others (I have seen requests from "Jack V. Briner" <jvb@duke.cs.duke.edu> and gld@cunixd.cc.columbia.edu) have made requests for benchmarks to the net. I assume others working on simulation algorithms, or parallel simulation algorithms, would also benefit from a set of standard benchmarks. The only `standard' simulation benchmarks have been those from ISCAS 85 & 89. These were not intended as simulation benchmarks and to my mind are very poor examples for anything but very simplistic gate level simulations. > I would like to propose that a publicly available collection be > established of circuits. The circuits would be selected to > include tough circuits for traditional simulators (bipolar, > opamps, feedback, bi-stable, etc), very large circuits, circuits > which are suitable for waveform relaxation, circuits which > give rise to unknown states with timing simulators, etc. My wish list would include: gate, functional, behavioural descriptions of large and small, ASICs, PCB designs and possibly microprocessors, or even complete systems?.... > 2. In SPICE compatabile netlist form. (For lack of a more universal > standard) > > 3. Use a standard device model: SPICE Level I, II, III, IV or BJT. > Many industrial circuits use custom device models, which again > confuses comparisons. There seems to be an overlap of standards at the gate level and above. The `standards' in effect in this country include: VHDL, ELLA and EDIF (logicmodel and netlist are probably applicable here) Any others????? I would like to conduct an informal email survey to determine which of these languages are in use, and which could provide the greatest number of benchmark circuits. This would help me, and hopefully others, when trying to find/use benchmarks. Do you use one, or more, of these languages? Do you intend to switch to using one of these? Do you have any opinions as to which language a set of simulation benchmarks should use? Do you have anything at all to contribute on the subject? Please email responses to: whitaker@cs.man.ac.uk, I will summarise any responses back to the net (please state if you require any information supplied to be treated in confidence). > 4. MOST important: include an output file containing what is > believed to be a correct simulation at the circuit level, for > comparison purposes. If possible, enought information should be > given so that SPICE, HSPICE, PSPICE, or whatever can be re-run to > verify the correctness of the output. I agree with this part, there needs to be a set of input waveforms, and output ones, in order to test the correctness of the simulation. What standards could be used in this area? > DISCLAIMER: > MCNC distributes a circuit level simulator (CAzM), and > has a vested interest in the benchmarking of such programs. The same applies here, I am working on workload distribution algorithms for parallel simulation (*currently* using both ELLA and EDIF for various forms of input), and would dearly like some large benchmark circuits. Anything I receive in the way of benchmarks, with the suppliers permission, I would be quite happy to make available to anyone on the net. Thnaks for your time, Nigel -- -------------------------------------------------------------------------------- Nigel Whitaker, Room IT202, Department of Computer Science, University of Manchester, Oxford Road, Manchester, M13 9PL, U.K. Tel: (061) 275 6163 Fax: (061) 275 6280 EMAIL: whitaker@cs.man.ac.uk or ...!uunet!mcsun!ukc!mucs!nigelw
waters@darla.sps.mot.com (Strawberry Jammer) (11/29/89)
In article <61.256c4a76@manse.cad.cs.man.ac.uk> whitaker@cs.man.ac.uk writes: }> 2. In SPICE compatabile netlist form. (For lack of a more universal }> standard) }> }> 3. Use a standard device model: SPICE Level I, II, III, IV or BJT. }> Many industrial circuits use custom device models, which again }> confuses comparisons. } }There seems to be an overlap of standards at the gate }level and above. The `standards' in effect in this country }include: } } VHDL, } ELLA and } EDIF (logicmodel and netlist are probably applicable here) } Any others????? } }I would like to conduct an informal email survey to determine }which of these languages are in use, and which could provide the }greatest number of benchmark circuits. This would }help me, and hopefully others, when trying to find/use benchmarks. } }Do you use one, or more, of these languages? }Do you intend to switch to using one of these? }Do you have any opinions as to which language a set of simulation }benchmarks should use? At Motorola we use both VHDL and EDIF in various groups for various purposes. Our intent is to track the ANSI/ISO standards scene as far as coordination between the two is concerned. We use EDIF netlists in our ASIC work as the only input for our HD CMOS gate array line and as at least an acceptable input for every other ASIC product. As for the VHDL/ELLA tradeoff we take no position, other than to observe that VHDL is more available in the US. Again our intent is to track the formal standards activities coordinating the "overlap" between these standards. For the basic models either ELLA or VHDL would seem to be good candidates, but as far as I know the models beingdone today are in actually C code or an equivalent which is not the intent of either ELLA or VHDL as far as I am aware. For the connectivity between models the EDIF netlist view is clearly (IMHO) the choice since it can handle the entire connectivity problem in a manner consistent with normal circuit level design. I would suggest making contact with the IEEE/EIA committees working on the VHDL/EDIF interaction schemes as well as the ELLA equivalent before making a selection. *Mike Waters AA4MW/7 waters@dover.sps.mot.com * You might have mail