It’s difficult to review a product
Believe me – I am doing this for four years now and I still find it hard to write a review. There are so many fine products and by grading them I need to arrange them in 5 or 6 groups. (I use 1 to 5 star scale with 6 stars for exceptional products).
That is where the problem lies – how can I even compare Carenado Cessna 185 to Beech 1900 of the same developer? The first is pretty basic and the second is a complex aircraft and requires more work. So I wonder – should I give more stars to the 1900 simply because it’s complex? Piper Cub would not stand a chance – no matter how well it was done…
But I made my decision to write reviews and I decided on how I will rate products that I write about. This is my set of rules. There may be exceptions, but as I read my old reviews – I am pretty consistent.
I rate the result, not the effort
That is my rule number one. What it means – I review a small plane like Piper Cub and a large jest like Boeing 777 without any consideration to the amount of work that was necessary to develop it. A simple plane that flies realistically, has nice cockpit and a great model will get 5 or 6 stars. Boeing 777 that comes without FMS will get 3… maybe less. And it doesn’t matter that is a good looking add-on and flies like it should. It lacks something that is important – something that should be included. It’s the developer’s decision to take on such a challenge.
Bugs and errors influence my rating only if they influence my experience
I reviewed a plane that was missing a compass. A compass!!! It’s a stupid bug. And a serious one. But it was an airliner and frankly speaking – if not for a meticulous check of the cockpit I would not notice that it is missing. I almost never use compass in an airliner – with a nice HSI or compass rose displayed on Nav Display I don’t need it. This bug (and a few similar minor issues) did not prevent me from giving BAe 146 5 stars. On the other hand – faulty fuel system in Battlefield Airlifter was one of many reasons lowering it’s rating.
Promises vs features
I review how developer fulfilled his promises. I hate to read “FMC” in product description only to find that it was explained in fine print as a remodeled FS GPS. I hate fine print and I hate surprises so if someone sells quality I expect quality. On the other hand – if someone sells “lite scenery” or “fun line” – I will consider it in my review and give it considerably higher rating.
5 or 6 star scale
Think of my stars as the hotel stars. 3 is something nice, but not that fancy. 4 is something where you can stay if you are on a business trip. Still – nothing special but you can not complain. 5 is above that. Basically 5 star hotel provides everything that a 4 star would, but they go this extra mile if you need something. Also – you may have minor issues with a 4 star hotel but they are unacceptable in a 5 star one. 6 stars… well – it’s where people you know from TV or cinema will stay.
So these are my positive ratings. There are two negative (1-2). Why only two if I have four positive? I don’t think it’s worth to spend my and yours time explaining why a product that get’s 1,5* is better then something with 1 star rating. Both products are crappy.
Reviews related FAQ:
Why so many good reviews?
I love flying good planes. I love flying over well designed sceneries. I simply have no time to spend flying over something bad or even mediocre if I can spend this time better. And reviewing takes time. So that is not that I give only good ratings. It is just that I usually choose good add-ons to review. Most of products I review end with 4 or 5 stars.
Why do you criticize minor details?
In my reviews I write about everything that I noticed – so you will know what I am fond of and what makes me angry. I want you to be well informed after reading my review. That is why you will find criticism even in my 6 star reviews (at least in some of them). That is something that I miss in many reviews I read, but-hey – review is what it is called – a “review”. Not an epitaph.
Scale in detail
6****** – an extraordinary product. There are (almost) no mistakes here. Everyone who makes add-ons can be jealous of that it represents and the flight sim enthusiast can experience much more than average add-on would allow. If it’s a plane – it flies perfectly, it’s systems are simulated in depth – developers pobably went outside of FS confines to code it correctly. It also looks great. It needs something special to get the last star – usually it features something uncommon in FSX – damages and failures in planes, sloping runway in scenery, etc…
5***** – a very good product. Almost no bugs or mistakes. A perfect example how an add-on should be published, but lacks this “special something” that would get a sixth star.
4**** – a good product. I cut out the word “very” from the description. I could also write – “market standard” or “market average”. Nothing that would make you fall of your chair. On the other hand – it is good. May have a few flaws, but nothing that would make you angry.
3*** – for fun. For example a plane that looks nice but it is full of bugs. It is your decision to have fun and overlook that it consumes much more fuel that it should, or flies much faster, or that you can not change anything in fuel system because someone never coded it properly. Still – it has some indisputable advantages that may convince you to try it.
2** – if you need it… Go for it if you need it. You want to fly over Poland and need a good scenery – take VFR Poland. Not because it’s good (it is not) but since you need it… go for it.
1* – rubbish! Do not buy, do not download it for free, keep away. You have been warned.