The Round Table, Beta Testing

I must say that Beta Testing can become a dangerous option for Add-on developers! Why? Because it’s a two-sided resource. Pick the team well and you’ll get fine release candidates, pick them wrong and you’ll get hundreds of patches for product correction instead of updates for the product’s features. I’ll give an example:

I beta tested several products in the past. I’m not saying the products I tested were any better than the others, but the assembled team tended to gather hardcore users and thorough explorers. We ended up, most of the times, with fine products which needed few updates, in many cases just small corrections, sometimes not even noticeable at first glance (for example the aeroSOFT Coast Guard, which, let me tell you, came up with some weird problems prior to launch, but we all decided to make the Release Version top notch. Did aeroSOFT pick their Beta testers carefully? Yes they did!).

On the other hand I’ve seen products being released with absurd (sometimes rude) errors. For example a recent released and expensive product from a known developer came up with a ridiculous error: When shutdown every power source, including batteries, you still get some lights on and warning sounds. My question came up immediately: Didn’t their Beta Team see this? Come on this is perhaps the first or second thing we check: Electrics and power supply!

But then another question came up: Did they have a Beta Team at all? If so, how experienced are they? Many developers have different criteria when picking the beta teams. But most of the times, picking someone with flight experience or specific aircraft knowledge, doesn’t exactly fit the needs. I know several cases of guys who teamed up with beta testing teams who knew nothing about the aircraft type, but where acknowledged for their criteria analysis and thorough critics.

I, for instance, tend to look out for a handful of reviewers of products who I know are methodical and honest in their evaluations. Once again, that doesn’t mean they are former pilots or flight engineers. So why does Beta Teams tend to be a bunch of friends of friends? We are a band of brothers for sure, but a team shouldn’t get picked randomly or with poor criteria with the danger of having a buggy final product with rude errors like the example I described above. Pick your knights carefully for the round table!

 

Joao P

Joao P is one of our favorite customers (you can find his own blog at helifreak.blogspot.com)

FSx hardware, the good the bad and the expensive.

The hardware you need to run FSx in confort are now pretty clear as the beta testers are free to discuss it, let’s assume it won’t get slower in the release version, right? Now the good news is that the hardware you need can be bought (remember that was actually not the case for some previous version!) and that you you’ll be running a plenty fast FSx with just under 1000 Euro/Dollar. A lot of money, but you’ll get a system that is faster then anything you could have bought 12 months ago. Now any way you look at that, it’s a good deal (see our proposed setup). But why did simmers get so behind on hardware? We used to have the baddest kick-ass hardware to run our sim but these days it seems just about any kid who plays first person shooters has better hardware. Where did we loose it?

I believe we lost it soon after FS2004 was launched. We had to take a deep hit in framerate with FS2002, it was not a very good version of FS as you might remember. It was just slow and certainly to slow for the kind of scenery shown. FS2004 looked a lot better and was about as fast as FS2002. At the same time AMD became a serious competitor for Intel and we saw a large decrease in price. You might remember that time when MHz ruled and Intel and AMD released chip after chip. Memory also was cheap (unless you fell for the Rambus trap) and around 2004 there was not a single graphics card that really made sense for FS as it was truly CPU limited. So we all bought the largest monitor we could afford and spend a small fortune on yokes, rudders, throttle setups.

Now MS changed the game. Flight Simulator needed to get on the same level as other games. It needed to be visually more attractive, more game-like so it would attract a new group of customers. That also means the hardware specs could be lifted to what is mid to high end now and low to mid range 12 months from now. Microsoft made the right choice, do not blame them if you can’t run the game on full blast. FSx is NOT slow, it is just big and complex software.

Mathijs

Mathijs Kok is the manager of Aerosoft internal development department (and has just upgraded his hardware)