3DNews Vendor Reference English Resource -
All you need to know about your products!
Biostar And ECS CPU Boundedness Foxconn 9800GTX
About Us | Advertise  
Digital-Daily.com
Digital-Daily

Motherboard
CPU & Memory
Video
Mobile
Cooling
Editorial
Digital
Links

Google
Web
www.digital-daily.com
www.3dnews.ru








Digital-Daily : Video : vga_testing_2007

Methodology for testing video cards: 2007. Use of FRAPS

Methodology for testing video cards: 2007. Use of FRAPS
Author: Dmitry Sofronov
Date: 06.12.2006

Tests in NFS Most Wanted using FRAPS

The test setup and the video card are the same. We chose the circular track "Ironwood", drove two rounds, with three rivals and the road traffic set to the maximum. That is, the game conditions are quite real and absolutely not predictable in terms of repeatability. It rains, gets dark or light, and the rivals behave like real hooligans. Now let's take the results file " frametimes.csv" and build first a graph of instant FPS values, and then a distribution diagram.

nfs-graph-detailed.gif

As you see, the FPS values are quite uniformly distributed around the 60 FPS point. To be more precise, the average value is 59.2 FPS.

nfs-full-time.gif

The shape of the distribution diagram resembles a "bell" even more than in the F.E.A.R. test, even without discarding "redundant harmonics". The vertical red line, as before, marks the average FPS value and is almost in the very middle of the imaginary "bell". Therefore, we can state that the resultant average FPS value does reflect the real performance of the video card in this game.

Let's turn to the FPS graph and its division into four parts. We did that to clarify the following point how long should we ride along the track to produce the normal "bell"? In other words How does the form of the diagram and the average FPS value deduced with FRAPS depend on the time of tests.

In our conditions of tests, we drove two rounds along the circular track. That is, each quarter from the division of the FPS graphs is equivalent to approximately half the round. Let's build a diagram of FPS distribution for the first quarter (or first half of the first round).

nfs-first-quarter.gif

First quarter. The time of test is 42 s, with the average value being 56.5 FPS. The distribution diagram does not look like a "bell" at all.

nfs-first-half.gif

Two quarters (first round). The time of test is 84 s, with the average value being 59.9 FPS. On the diagram, we gradually get the sought for "bell".

nfs-three-quarter.gif

Three quarters. The time of test is 126 s, with the average value being 58.8 FPS. There are no essential differences from the previous diagram. The "bell" becomes more oblong, but its shape almost doesn't change.

nfs-full-time.gif

Four quarters (two full rounds). The time of test is 169 s, with the average value being 59.2 FPS.

It follows from the above that the criterion of minimum time required for tests a duration of tests - should be such that further increase in the time of tests not result in the change of the shape of the "main bell" of the distribution diagram.

The resultant series of average FPS values for each sector looks like this 56.5, 59.9, 58.8, 59.2. As you can see, only the value for the first quarter makes an essential difference from that of the other three. If we discard it (as per the formulated criterion) and find the average of them for each sector, we get 59.3 FPS. At the same time, the deviation amounts to merely 0.6 FPS both sides, that is, no more than 1% !

In our case, when testing in NFS Most Wanted it suffices to "drive" a round (84 sec of the testing time) or one and a half rounds (126 sec of testing time). Knowing the time of tests and the average FPS value, we can approximately estimate the overall counts using the following formula "multiply the testing time by the average FPS value". Therefore, the required number of counts when the statistical method starts working is 5000-7000. We intentionally draw your attention to this fact. If we take a rather weak video card, the average FPS will be much lower, therefore, the overall number of counts will go down if the testing time remains unchanged. Therefore, when testing "weak" video cards it is advisable that the time of tests be longer.

Final Words

FRAPS is a powerful and precise tool for measuring performance of video cards. To produce true data,

  • choose a demo scene most adequate to the typical gameplay
  • run the test for quite a sufficient time
  • use the FRAPS results file of the type " frametimes.csv"
  • upon transformation of source data of the file " frametimes.csv", calculate the sought-for average FPS values and, if needed, the minimum and maximum FPS.

Conclusion

The presented method of tests using FRAPS and further data processing, it is possible to produce precise and adequate enough values of video cards performance in games that have no integrated benchmark. Even more than that. The specified method allows to analyze the "quality" of demo scenes in benchmarks integrated into games as well as get the idea of the nature of the game engine.

Unfortunately, in this review we had to omit one rather vital point - what should be regarded the minimum FPS value? As you saw on the example of test integrated into F.E.A.R., the minimum FPS values shown by the integrated benchmark may substantially differ from reality. Believe me, that is merely a tip of the iceberg. This issue requires an in-depth investigation. And we'll tell about that next time.

Discuss the review in a forum

Content:

  • Page 1 - Adequacy of FRAPS as a testing tool
  • Page 2 - Mystery behind FRAPS
  • Page 3 - Use of FRAPS for games without integrated benchmark




  • Top Stories:
    MoBo:


    ECS X58B-A (Intel X58)
    ASUS Rampage II Extreme (Intel X58)
    MSI DKA790GX and ECS A780GM-A Ultra
    MSI P7NGM (NVIDIA GeForce 9300)
    Intel X58 and ASUS P6T Deluxe
    MSI P45 Neo2 (Intel P45)
    Foxconn A7GMX-K (AMD 780G)
    VGA Card:


    NVIDIA GeForce GTX 295 – a new leader in 3D graphics!
    ECS HYDRA GeForce 9800GTX+. Water-cooled and SLI "all-in-one"
    Radeon HD 4830 CrossFire - better than Radeon HD 4870!
    XFX GeForce GTX 260 Black Edition in the SLI mode
    Leadtek WinFast PX9500 GT DDR2 – better than GeForce 9500GT DDR-3
    Palit Radeon HD 4870 Sonic: exclusive, with unusual features
    Palit HD 4850 Sonic: almost Radeon HD 4870, priced as HD 4850
    CPU & Memory:

    GSkill high-capacity memory modules
    CPU Intel Core i7-920 (Bloomfield)
    DDR3 memory: late 2008
    CPU AMD Phenom X3 8750 (Toliman)
    AMD Phenom X4 9850 – a top-end CPU at affordable price
    CPU Intel Atom 230 (Diamondville)
    Chaintech Apogee GT DDR3 1600


      Management by AK
      Design VisualPharm.com

    Copyright © 2002-2014 3DNews.Ru All Rights Reserved.
    contact - info@digital-daily.com
    Digital-Daily - English-language version of the popular Russian web-project 3DNews