Frame Capture and Analysis Tools Reviewed

[Bosco]

Tonight ccokeman takes a look at Frame Capture and Analysis Tools.

 

This could change the whole landscape on how Video Cards are tested.

 

Read Franks review here:

 

http://www.overclockersclub.com/reviews/frame_capture_and__analysis_tools/

 

Expect more testing from him as we go forward.

 

And for those of you that really want to understand every little piece of this software, Read Ryans in depth article over at PCPER http://www.pcper.com/reviews/Graphics-Cards/Frame-Rating-Dissected-Full-Details-Capture-based-Graphics-Performance-Testin/

 

Ryan has been working on this from the beginning and is probably the most knowledgeable editor when it comes too this software, and he had a very good write up on it.

[VaporX]

I am torn by the material I am seeing coming out on this. What I mean is, and PcPer actually said this, a lot of this data gives an impression of a certain result but in the real world holds little meaning. The "stutter" that seems to be the root of this data is something that is highly subjective to the viewer when it comes to how the game plays. The question I keep asking people is, if you did not have this data would you have been worried, concerned or upset about your quality of game play and the majority say no. The reason is, a stutter that cannot be perceived is not meaningful.

 

Also I would take any testing methodology provided by one of the two GPU makers with a grain of salt. Both have a history of fudging data in a way that makes the other look worse than they are. When we see an independent company produce a testing methodology then I am more willing to consider the results.

 

Now as I have said I am torn on this and the reason why is I have been a huge advocate over the years that the so called average frame rate was not as meaningful as massive dips in the frame rate during play, that consistency is much more important than high numbers. This material, if taken at face value does show that AMD has an issue with Crossfire, something BTW that most gamers already knew. However it is being attributed by way to many people to way to many other issues.

 

My concern is that at the end of the day, without a third dis-interested party actually providing the methodology, we could very well see a big witch hunt going on right now. And just as happened in Salem the accused have no counter because how do you counter evidence that has no other way to be shown but the word (and methodology) offered by the accuser.

[Waco]

TechReport was the first to uncover this well over a year ago. They recently published an update using FCAT as well: http://techreport.com/review/24553/inside-the-second-with-nvidia-frame-capture-tools

 

Extremely in depth for anyone who has a good bit of time to sit down and digest it.

[Guest_Jim_*]

DATA!

Can you tell I have a degree in math? (The weird part though is I don't like statistics.)

 

Oh and good job Frank

[Bosco]

I am torn by the material I am seeing coming out on this. What I mean is, and PcPer actually said this, a lot of this data gives an impression of a certain result but in the real world holds little meaning. The "stutter" that seems to be the root of this data is something that is highly subjective to the viewer when it comes to how the game plays. The question I keep asking people is, if you did not have this data would you have been worried, concerned or upset about your quality of game play and the majority say no. The reason is, a stutter that cannot be perceived is not meaningful.

 

Also I would take any testing methodology provided by one of the two GPU makers with a grain of salt. Both have a history of fudging data in a way that makes the other look worse than they are. When we see an independent company produce a testing methodology then I am more willing to consider the results.

 

Now as I have said I am torn on this and the reason why is I have been a huge advocate over the years that the so called average frame rate was not as meaningful as massive dips in the frame rate during play, that consistency is much more important than high numbers. This material, if taken at face value does show that AMD has an issue with Crossfire, something BTW that most gamers already knew. However it is being attributed by way to many people to way to many other issues.

 

My concern is that at the end of the day, without a third dis-interested party actually providing the methodology, we could very well see a big witch hunt going on right now. And just as happened in Salem the accused have no counter because how do you counter evidence that has no other way to be shown but the word (and methodology) offered by the accuser.

 

Maybe something like this was needed for AMD to get off their asses and fix their drivers

[ccokeman]

I am torn by the material I am seeing coming out on this. What I mean is, and PcPer actually said this, a lot of this data gives an impression of a certain result but in the real world holds little meaning. The "stutter" that seems to be the root of this data is something that is highly subjective to the viewer when it comes to how the game plays. The question I keep asking people is, if you did not have this data would you have been worried, concerned or upset about your quality of game play and the majority say no. The reason is, a stutter that cannot be perceived is not meaningful.

 

Also I would take any testing methodology provided by one of the two GPU makers with a grain of salt. Both have a history of fudging data in a way that makes the other look worse than they are. When we see an independent company produce a testing methodology then I am more willing to consider the results.

 

Now as I have said I am torn on this and the reason why is I have been a huge advocate over the years that the so called average frame rate was not as meaningful as massive dips in the frame rate during play, that consistency is much more important than high numbers. This material, if taken at face value does show that AMD has an issue with Crossfire, something BTW that most gamers already knew. However it is being attributed by way to many people to way to many other issues.

 

My concern is that at the end of the day, without a third dis-interested party actually providing the methodology, we could very well see a big witch hunt going on right now. And just as happened in Salem the accused have no counter because how do you counter evidence that has no other way to be shown but the word (and methodology) offered by the accuser.

The idea behind this is not only to show what has already been widely surmised for years but show the comparison. As I am sure you have experienced with AMD cards in Crossfire over the years something was not quite right. Call it microstutter or something else.  My first Multi GPU setup was a pair of 1900XT cards back when you had a master & slave card ( Yes I know its politically incorrect but the terminology was used back then for the combo). I had never ending driver issues back then with the game I played the most IL2 Sturmovik.

Discussing the issues and pointing out the opportunities will give AMD the chance to clean up the latency concerns with its multi GPU cards. If you look at the percentile charts with single card tests you can see that AMD does fairly well by comparison with an almost flat line down to the 95th and 99th percentile. Frame times for each frame observed and reported by FCAT with single GPU cards show that AMD is right on target. Its getting that multi GPU configuration timing down correctly. Something I hear they are working on with something similar to NVIDIA's frame metering system in the works.

 

As Waco said Scott at Tech report has been looking at this and has a great article on it that was updated to incorporate the data using the FCAT tools. Ryan at PCPer has had the tools for over a year and has an in depth article  NVIDIA has released everything but the proprietary overlay software to the public so that standalone solutions from non vested parties can take over the system they employ to get a truly independent viewpoint. Its another tool that like I said generates a lot of data that needs to be interpreted.