|
|
|
Benchmarks
Used |
|
Todays hardware supplied to the
community is becoming faster and more
powerful than most of us would ever
imagine. Therefore to ensure that
you the reader have no unequivocal doubt
in your mind to what the optimal
performance of the hardware is; we use
some of the best known professional
benchmarks around which truly stretch
these new hardware parts to their
limits.
In order to get the most out of all
reviews we will endeavour to complete
tests in both Microsoft Windows 32 Bit
and 64 Bit modes were and when ever
possible (and time scales permitting).
Some benchmarks are designed fully to
run on 64 bit platforms and their
results are completely different to that
of the 32 bit platform. Please
take these points into careful
consideration when comparing results. |
|
SPECviewperf® 10 |
What is This Thing Called
"SPECviewperf®"?
SPECviewperf® is a portable OpenGL
performance benchmark program written in
C. It was developed by IBM. Later
updates and significant contributions
were made by SGI, Digital (Compaq, HP),
3Dlabs (Creative Labs) and other
SPECopcSM project group members.
SPECviewperf provides a vast amount of
flexibility in benchmarking OpenGL
performance. Currently, the program runs
on most implementations of UNIX, Windows
XP, Windows 2000, and Linux.
The OpenGL Performance Characterization
(SPECopc) project group endorsed
SPECviewperf as its first OpenGL
benchmark. Performance numbers based on
SPECviewperf were first published in the
Q4 1994 issue of The GPC Quarterly.
SPECopc group member companies have
ported the SPECviewperf code to their
operating systems and window
environments. The SPECopc project group
maintains a single source code version
of the SPECviewperf code that is
available to the public.
Benchmarking with SPECviewperf
SPECviewperf parses command lines and
data files, sets the rendering state,
and converts data sets to a format that
can be traversed using OpenGL rendering
calls. It renders the data set for a
pre-specified amount of time or number
of frames with animation between frames.
Finally, it outputs the results.
SPECviewperf reports performance in
frames per second. Other information
about the system under test -- all the
rendering states, the time to build
display lists (if applicable), and the
data set used -- are also output in a
standardized report.
A "benchmark" using SPECviewperf is
really a single invocation of
SPECviewperf with command-line options
telling the SPECviewperf program which
data set to read in, which texture file
to use, what OpenGL primitive to use to
render the data set, which attributes to
apply and how frequently, whether or not
to use display lists, and so on. One
quickly realizes that there are infinite
numbers of SPECviewperf "benchmarks" (an
infinite number of data sets multiplied
by an almost infinite number of
command-line states).
Real-World Benchmarking
SPECopc project group members recognize
the importance of real-world benchmarks.
From the beginning, the group has sought
benchmarks representative of the OpenGL
rendering portion of independent
software vendor (ISV) applications.
Along these lines, the project group has
come up with what it calls a viewset. A
viewset is a group of individual runs of
SPECviewperf that attempt to
characterize the graphics rendering
portion of an ISV's application.
Viewsets differ from SPECapc benchmarks
in that they exercise only the graphics
functionality of the application. This
enables direct performance comparisons
of graphics hardware. Since SPECviewperf
does not require an application software
license to run, it is accessible to a
wider range of users than SPECapc
benchmarks. It is also easier to use and
runs faster than SPECapc benchmarks.
Viewsets are generally not developed by
the SPECopc project group; they come
from the ISVs themselves. Members of the
SPECopc project group often "sponsor"
the ISV. Sponsorship entails helping the
ISV in several areas, including how to
obtain the SPECviewperf code, how to
convert data sets to a SPECviewperf
format, how to use SPECviewperf, how to
create SPECviewperf tests to
characterize the application, how to
determine weights for each of the
individual SPECviewperf tests based on
application usage, and finally to help
offer the viewset to the SPECopc project
group for consideration as a standard
SPECopc viewset. Any ISV wishing to
develop a viewset should contact gpcopc-info@spec.org. |
Fundamental Upgrades
SPECviewperf 10 is available initially
for Windows XP and Vista, with versions
for Linux and Mac OS expected in
upcoming months.
Major new features in SPECviewperf 10
focus on performance evaluation in areas
that are of increasing interest to the
workstation market: higher-quality
imaging and multi-core systems.
SPECviewperf now provides the ability to
compare performance of systems running
in higher-quality graphics modes that
use full-scene anti-aliasing. It also
measures how effectively graphics
subsystems scale when running
multithreaded graphics content. |
Full Screen Anti-Aliasing and How it
is Tested in SPECviewperf 10 - by Allen
Bourgoyne
Below are a few brief snippets from the
full article.
The Full article may be found here.
We strongly advise that you the reader
actually take time in reading this full
and interesting article to gain better
understanding of what is actually being
achieved.
SPECviewperf 10 includes new
functionality to test performance for
full-scene anti-aliasing (FSAA), a
feature of most modern graphics cards.
This document gives a brief description
of FSAA, its benefits and limitations,
and describes how to best interpret the
results provided by SPECviewperf 10.
Interpreting SPECviewperf FSAA
Test Results
SPECviewperf tests FSAA performance by
running its standard test suite several
times, setting the FSAA feature of the
graphics card being tested to all
possible FSAA sample values. If a
graphics card supports up to 16 samples
– supporting 16, 8, 4, and 2 samples –
the test would run five times, running
tests for 16, 8, 4, 2, and no samples.
The test is run with no FSAA enabled so
that a baseline test result can be
produced. The goal of the test is to
determine what performance penalty, if
any, enabling FSAA with a given sample
rate incurs. During the individual
tests, screen shots are captured. This
allows the tester to review the effects
that FSAA is having on the rendered
image. It is important to note that as
FSAA is altering the image, there is no
way to automatically validate pixel
accuracy of the images rendered. It is
up to the tester to evaluate the
individual image captures to determine
if the FSAA-produced images for a given
sample size are visually pleasing to the
user. As mentioned earlier, a subjective
evaluation is necessary as individual
preference for the FSAA results plays a
part with respect to the aesthetic
quality of the images produced.
If the FSAA test produces a score that
is within 10 percent of the non-FSAA
score, under SPECviewperf 10 rules that
score will be considered valid for the
specific sample rate. If a SPECviewperf
test score for a particular test is
20.0, for example, and the same test
score with FSAA enabled with a sample of
8 produces 19.5, the official results
will be listed as 20.0 with FSAA up to a
sample of 8 enabled. Sample rates may
affect individual tests differently with
respect to performance, so each test
will include the best FSAA sample rate
score within the 10-percent threshold.
If no sample rate falls within the
10-percent threshold, the test score
will indicate that no FSAA sample rate
achieved the performance threshold for
this test.
In summary...
Full-scene anti-aliasing can be
implemented in many ways, with varying
results, even within a single graphics
card vendor’s product line. Quality of
images created with FSAA enabled is in
the eyes of the beholder.
With its new testing for FSAA
performance, SPECviewperf 10 relies on
the tester to judge the visual quality
of the image. FSAA performance is
measured by the highest level of
sampling achieved within 10 percent of
the non-FSAA score.
Allen Bourgoyne is the Technical
Marketing Manager for Workstation
Graphics at AMD, and vice chair of the
SPECopc project group.
|
Understanding the Impact of Windows
Vista on SPECviewperf Performance
Measurement - by Ian Willaims
Below are
a few brief snippets from the full
article.
The Full article may be found here.
We strongly advise that you the reader
actually take time in reading this full
and interesting article to gain better
understanding of what is actually being
achieved.
Windows Vista
introduces many major new features that
greatly improve user experience compared
to Windows XP. Some of these features,
however, affect performance
benchmarking. This document aims to help
SPECviewperf users better understand
these features and how they impact
performance measurement.
Inside
Aero and DWM
A key feature of
Windows Vista is the Aero user
interface. Aero includes features such
as transparent and blended windows,
animated icons, and application preview.
Because Aero is tightly integrated with
the GPU, it is essentially a complete 3D
application in itself, using much of the
GPU’s capabilities and horsepower.
Windows Vista’s Desktop Window Manager (DWM)
is the underlying architecture and
mechanism that enables Aero’s
capabilities.
Fundamental Differences
Windows Vista
architecture is fundamentally different
than that for Windows XP, where the
content of all 3D graphics windows is
managed almost exclusively by the ICD
graphics driver and no Desktop Window
Manager exists. In XP, a 3D window
represents a hole in the desktop as far
as the operating system is concerned.
The change in the DWM architecture
between XP and Vista has two
implications when benchmarking
performance using SPECviewperf.
In
Summary...
Vista delivers to
Windows users a radical change in
features and quality, which is good for
the workstation industry in general.
Inherent in the new features is an
architectural re-design that has two
main implications when benchmarking on
Vista compared with Windows XP:
the impact of the Aero interface running
concurrently with a benchmarked
application; and
the impact of the architectural design
of the Desktop Window Manager in
situations when an application window
updates faster than the display refresh
rate.
In order to provide true
apples-to-apples performance comparison
between a specific system configuration
running under Windows XP and the same
configuration running under Windows
Vista, both the DWM and Aero should be
disabled. Unfortunately, this does not
represent the default nor desired user
experience of professional workstation
users running Windows Vista applications
– most users will want the DWM and Aero
turned on. Benchmarking in this case is
perfectly valid, but these results
should not be compared to those
generated while running Windows XP, even
if testing is done with the same
hardware configuration.
Ian
Williams is Senior Applied Engineer at
NVIDIA and chair of the SPECopc project
group, developer of SPECviewperf. |
|
SPECviewperf® 10
Viewset Descriptions |
|
Currently, there are eight standard
SPECopc application viewsets:
|
 3ds
max Viewset (3dsmax-04)
The 3dsmax-04 viewset was created from
traces of the graphics workload
generated by 3ds max 3.1. To ensure a
common comparison point, the OpenGL
plug-in driver from Discreet was used
during tracing.
The models for this viewset came from
the SPECapc 3ds max 3.1 benchmark. Each
model was measured with two different
lighting models to reflect a range of
potential 3ds max users. The
high-complexity model uses five to seven
positional lights as defined by the
SPECapc benchmark and reflects how a
high-end user would work with 3ds max.
The medium-complexity lighting models
uses two positional lights, a more
common lighting environment.
The viewset is based on a trace of the
running application and includes all the
state changes found during normal 3ds
max operation. Immediate-mode OpenGL
calls are used to transfer data to the
graphics subsystem.
More detailed information
Click Here
http://www.spec.org/gpc/opc.static/3dsmax04.html |
 CATIA
Viewset (catia-02) The
catia-02 viewset was created from traces
of the graphics workload generated by
the CATIA™ V5R12 application from
Dassault Systemes.
Three models are measured using various
modes in CATIA. Phil Harris of LionHeart
Solutions, developer of CATBench2003,
supplied SPEC/GPC with the models used
to measure the CATIA application. The
models are courtesy of CATBench2003 and
CATIA Community.
The car model contains more than two
million points. SPECviewperf replicates
the geometry represented by the smaller
engine block and submarine models to
increase complexity and decrease frame
rates. After replication, these models
contain 1.2 million vertices (engine
block) and 1.8 million vertices
(submarine).
State changes as made by the application
are included throughout the rendering of
the model, including matrix, material,
light and line-stipple changes. All
state changes are derived from a trace
of the running application. The state
changes put considerably more stress on
graphics subsystems than the simple
geometry dumps found in older
SPECviewperf viewsets.
More detailed information
Click Here
http://www.spec.org/gpc/opc.static/catia02.html |
 EnSight
Viewset (ensight-03) The
ensight-03 viewset provides the ability
to compare display list and immediate
mode paths, and additional quality
checks for display list results. It
represents engineering and scientific
visualization workloads created from
traces of CEI's EnSight application.
CEI contributed the models and suggested
workloads. Various modes of the EnSight
application are tested using both
display-list and immediate-mode paths
through the OpenGL API. The model data
is replicated by SPECviewperf 9 to
generate 3.2 million vertices per frame.
State changes as made by the application
are included throughout the rendering of
the model, including matrix, material,
light and line-stipple changes. All
state changes are derived from a trace
of the running application. The state
changes put considerably more stress on
graphics subsystems than the simple
geometry dumps found in older viewsets.
More detailed information
Click Here
http://www.spec.org/gpc/opc.static/ensight03.html |
 Maya
Viewset (maya-02)
The maya-02 viewset was created from
traces of the graphics workload
generated by the Maya 6.5 application
from Alias.
The models used in the tests were
contributed by artists at NVIDIA.
Various modes in the Maya application
are measured. Models used in this
version of the Maya viewset contain many
more vertices than those used in
maya-01, better reflecting models used
by animators in the real world.
State changes such as those executed by
the application -- including matrix,
material, light and line-stipple changes
-- are included throughout the rendering
of the model. All state changes are
derived from a trace of the running
application. The state changes put
considerably more stress on graphics
subsystems than the simple geometry
dumps found in older viewsets.
More detailed information
Click Here
http://www.spec.org/gpc/opc.static/maya02.html |
 Pro/ENGINEER
Viewset (proe-04)
The proe-04 viewset was created from
traces of the graphics workload
generated by the Pro/ENGINEER 2001™
application from PTC.
Two models and three rendering modes are
measured during the test. PTC
contributed the models to SPEC for use
in measurement of the Pro/ENGINEER
application. The first of the models,
the PTC World Car, represents a
large-model workload composed of 3.9 to
5.9 million vertices. This model is
measured in shaded, hidden-line removal,
and wireframe modes. The wireframe
workloads are measured both in normal
and antialiased mode. The second model
is a copier. It is a medium-sized model
made up of 485,000 to 1.6 million
vertices. Shaded and hidden-line-removal
modes were measured for this model.
This viewset includes state changes as
made by the application throughout the
rendering of the model, including
matrix, material, light and line-stipple
changes. The PTC World Car shaded frames
include more than 100MB of state and
vertex information per frame. All state
changes are derived from a trace of the
running application. The state changes
put considerably more stress on graphics
subsystems than the simple geometry
dumps found in older viewsets.
More detailed information
Click Here
http://www.spec.org/gpc/opc.static/proe04.html |
 SolidWorks
Viewset (sw-02) The
sw-02 viewset was created from traces of
the graphics workload generated by the
Solidworks 2004 application from
Dassault Systemes.
The model and workloads used were
contributed by Solidworks as part of the
SPECapc for SolidWorks 2004 benchmark.
State changes as made by the application
are included throughout the rendering of
the model, including matrix, material,
light and line-stipple changes. All
state changes are derived from a trace
of the running application. The state
changes put considerably more stress on
graphics subsystems than the simple
geometry dumps found in older viewsets.
More detailed information
Click Here
http://www.spec.org/gpc/opc.static/sw02.html |
 UGS
Teamcenter Visualization Mockup Viewset
(tcvis-01) The
tcvis-01 viewset is based on traces of
the UGS Teamcenter Visualization Mockup
application (also known as VisMockup)
used for visual simulation.
State changes such as those executed by
the application -- including matrix,
material, light and line-stipple changes
-- are included throughout the rendering
of the model. All state changes are
derived from a trace of the running
application. The state changes put
considerably more stress on graphics
subsystems than the simple geometry
dumps found in older viewsets.
More detailed information
Click Here
http://www.spec.org/gpc/opc.static/tcvis01.html
|
 UGS
NX 3 (ugnx-01)
The ugnx-01 viewset is based on traces
of the UGS NX 3 application. The traces
represent very large models, some
approaching 800MB of data per frame.
The new benchmark resulted from a
collaborative effort among the
automotive industry, UGS and SPEC/GPC.
State changes such as those executed by
the application -- including matrix,
material, light and line-stipple changes
-- are included throughout the rendering
of the model. All state changes are
derived from a trace of the running
application. The state changes put
considerably more stress on graphics
subsystems than the simple geometry
dumps found in older viewsets.
More detailed information
Click Here
http://www.spec.org/gpc/opc.static/ugnx01.html |
|
|
|
SPECapc for 3ds Max™ 9 |
|
 |
 |
|
Measures
performance based on the workload of a
typical user, including functions such
as wireframe modeling, shading,
texturing, lighting, blending, inverse
kinematics, object creation and
manipulation, editing, scene creation,
particle tracing, animation and
rendering. The benchmark runs under both
OpenGL and DX implementations of 3ds max
9, and tests all the components that
come into play when running the
application.
Test
models within SPECapc for 3ds Max 9
range from simple primitives to complex
objects and large scenes. Multiple
repetitions are used to test the limits
of graphics cards. Separate results are
reported for CPU and graphics scores.
The
latest benchmark incorporates upgrades
made in SPECapc for 3ds Max 8, including
new models and functionality that
reflect the content and workflow of
today’s application users. The upgrades
resulted from the cooperative efforts of
Autodesk, independent 3ds Max animators,
and SPECapc members. Major new features
carried over from SPECapc for 3ds Max 8
include:
-
Larger
models with greater complexity that
provide more rigorous workloads for
system components and better represent
content created by 3ds Max
professionals.
-
DirectX
(DX) shaders that exercise advanced
shader functionality within graphics
systems.
-
Shorter
run times achieved by streamlining
code and eliminating tests that don’t
contribute to a better understanding
of system performance.
-
New and
improved tests for radiosity, inverse
kinematics and other advanced
features.
The total number of
seconds to run each test is normalized
based on a reference machine, in this
case a system with a 2.4-GHz Intel Xeon,
Intel 860 chipset running Windows XP SP2
with 2GB of PC800 ECC RDRAM, an NVIDIA
QuadroFX 1000 graphics card, and a 40GB
ATA/100 hard drive. The normalization
process ensures a scoring system where a
bigger score is better. Composite scores
are reported for interactive graphics
and CPU rendering |
|
|
|
|
MAXON CINEBENCH 10
It's a real-world test suite that
assesses your computer's performance
capabilities. MAXON CINEBENCH is based
on MAXON's award winning animation
software CINEMA 4D, which is used
extensively by studios and production
houses worldwide for 3D content
creation. MAXON software has been used
in blockbuster movies such and
Spiderman, Star Wars, The Chronicles of
Narnia and many more.
MAXON CINEBENCH runs several tests on
your computer to measure the performance
of the main processor and the graphics
card under real world circumstances. The
benchmark application makes use of up to
16 CPUs or CPU cores and is available
for Windows (32- and 64-Bit) and
Macintosh (PPC- and Intel-based).
The resulting values among different
Operating Systems are 100% comparable
and therefore
very useful with regard to purchasing decision-making. It can also
be used as a marketing tool for
hardware vendors or simply to compare hardware among colleagues or
friends.
How does MAXON CINEBENCH work?
The test procedure consists of two main
components: The first test sequence is
dedicated to the computer's main
processor. A 3D scene file is used to
render a photo realistic image. The
scene makes use of various CPU-intensive
features such as reflection, ambient
occlusion, area lights and procedural
shaders. In the first run, the benchmark
only uses one CPU (or CPU core), to
ascertain a reference value. On machines
that have multiple CPUs or CPU cores,
and also on those who simulate multiple
CPUs (via HyperThreading or similar
technologies), MAXON CINEBENCH will run
a second test using all available CPU
power.
The second test measures the performance
of the graphics card and is run inside
the 3D editor window. The project file
used can test all graphics cards that
support the OpenGL standard. In this
scene, only the camera was animated.
This scene places medium to low demands
on graphics cards and tests the maximum
speed with which the scene can be
properly displayed.
Who should use MAXON CINEBENCH?
There is a wide range of applications
for a real world benchmarking tool like
MAXON CINEBENCH. Anyone who needs to
compare the performance of computer
hardware should put MAXON CINEBENCH into
his or her tool box. Contrary to
abstract benchmarks, which only test
specific functions of CPUs or GPUs, a
real world benchmark applies a user's
common tasks to measure a system's
performance.
You can download MAXON CINEBENCH R10
here |
|
|
|
SiSoftware Sandra (the System
ANalyser, Diagnostic and Reporting
Assistant) is an
information & diagnostic utility. It should provide most of the
information (including
undocumented) you need to know about your hardware, software and
other devices
whether hardware or software.
It works along the lines of other
Windows utilities, however it tries to
go beyond them and show you more of
what's really going on. Giving the user
the ability to draw comparisons at both
a high and low-level. You can get
information about the CPU, chipset,
video adapter, ports, printers, sound
card, memory, network, Windows
internals, AGP, PCI, PCIe, ODBC
Connections, USB2, 1394/Firewire, etc.
Using the latest version
SiSoftware Sandra 2007 SP1, listed
below are just some of the Benchmark
Modules that we use.
-
CPU
Arithmetic Benchmark (MP/MT support)
-
CPU
Multi-Media Benchmark (including MMX,
MMX Enh, 3DNow!, 3DNow! Enh, SSE(2))
(MP/MT support)
-
File
System (Removable, Hard Disks,
Network, RamDrives) Benchmark
-
Memory
Bandwidth Benchmark (MP/MT support)
-
Cache &
Memory Bandwidth Benchmark (MP/MT
support)
|
|
|
PCMark®05 is everything you need to
reliably and easily measure the
performance of
your PC and determine its strengths and weaknesses. With
PCMark05, you will be
able to select the optimal upgrades for your existing PC, or choose
the right new PC that fits your specific
needs. This easy-to-use product gives
you the same tools and knowledge that
virtually every professional tester in
the industry uses. |
|
|
3DMark®06 is
the worldwide standard in advanced 3D
game performance
benchmarking. A fundamental tool for every company in the PC
industry as well as
PC users and gamers,
3DMark06 uses advanced real-time 3D
game workloads to measure PC performance
using a suite of DirectX 9 3D graphics
tests, CPU tests, and 3D feature tests.
3DMark06 tests include all new
HDR/SM3.0 graphics tests, SM2.0 graphics
tests, AI and physics driven single and
multiple cores or processor CPU tests
and a collection of comprehensive
feature tests to reliably measure next
generation gaming performance today.
Futuremark's exclusive Online
ResultBrowser web service tracks and
compares
3DMark06 scores. |
|
|
|
|
