SPEC Group Releases Unix-Based CPU Benchmarks
 
Copyright (c) 1989, McGraw-Hill, Inc.
NEW YORK (Microbytes Daily News Service) --- SPEC yesterday
released its suite of Unix-based applications-oriented benchmark
tests. The benchmarks are designed to objectively evaluate the
performance of differing workstations.
 
SPEC, which stands foir Systems Performance Evaluation Coopera-
tive, is a nonprofit group formed by major workstation manufac-
turers. Apollo, Hewlett-Packard, MIPS Computer Systems, Sun
Microsystems, and Electronic Engineering Times were the original
founders; they have since been joined by AT&T, Control Data, DEC,
Data General, IBM, Intergraph, Motorola, Multiflow Computers, and
Stardent.
 
The SPEC Benchmark Suite Release 1.0 consists of 10 engineering
and scientific programs written in Fortran and C that measure CPU
and floating-point performance. The tests are proposed as a new
standard for comparing systems from a wide variety of
manufacturers; they are available for $450 to anybody who wants
to purchase the source code.
 
SPEC was established in September 1988 with the goal of creating
a set of open and portable benchmarks that would help normalize
comparisons between competing systems. The group says existing
measures such as MIPS are often manipulated by manufacturers and
misinterpreted by the press and users. "Benchmarks have become
essentially useless for comparing systems," said SPEC president
Stan Baker. SPEC says traditional tests such as Dhrystones,
Whetstones, or Linpack fail to adequately exercise or account for
advanced features such as the interleaved memory, pipelining, and
large memory caches found in many of today's systems, while
falling easy victim to optimizing compilers that can distort
their results.
 
In addition to choosing tests, SPEC developed a detailed
specification for how benchmark results must be reported by
member companies. The benchmarks are normalized to a VAX 11/780,
and results for individual worksations must be provided in both
absolute terms and relative to the reference value. To obtain a
single figure that summarizes the 10 tests, called the SPECmark,
SPEC uses the geometric, not arithmetic, mean of the results; the
geometric mean compensates for the different execution times of
the benchmark tests (which vary by more than an order of
magnitude) while weighing each equally.
 
In future releases, SPEC plans to address different subsystems,
such as video graphics, mass storage, I/O, and communications
(including performance of networked systems), different operating
environments, multiuser and multiprocessor systems, and
applications such as CAD, text processing, and business software.
 
 
First Workstation Benchmark Results
SPEC also released the first test results for worksations sold by
some of its member companies. SPEC urges that all benchmark
results be carefully weighed. What follows are some of the
results of the group's testing; for more complete results, please
write to SPEC.
 
 
System                  SPECmark
------                  --------
 
MIPS M/2000             16.5
Motorola Delta 8864DP   15.1
Apollo DN10010          14.5
Sun SPARCstation 330    11.3
MIPS M/120-5            11.2
DECstation 3100         10.1
MIPS RC2030              9.3
Sun SPARCstation 1       8.3
Motorola Delta 8864SP    8.2
Motorola Delta 8608      7.8
DECstation 2100          7.5
HP9000 Model 834         9.5
HP9000 Model 370         3.9
HP9000 Model 340         1.6
 
 
Contact: Systems Performance Evaluation Cooperative, 39510
Paseo Padre Parkway, Suite 350, Fremont, CA 94538; (415)
792-2901; fax: (415) 792-4748.
 
                              --- Andy Reinhardt
 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
