One of the most challenging aspects of joining the shooting community is sorting out which sources of information are best applied to your particular pursuits. I remember how frustrating it was, when I became a serious student of defensive shooting and concealed carry in my early twenties, that experts in the field disagreed on so many fundamental points. Seemingly simply things such as suitable handgun calibers, gun types and holster positions drove endless debate.
So, why is it that experts don’t agree? When I started working on this piece I came up with a dozen reasons off the top of my head. But here are just three of them along with a bit of advice for dealing with these professional differences of opinion.
Variations in Test Parameters and Goals
Years ago, a friend of mine said that getting what you want out of life isn't always about finding answers—the world is filled with answers. The trick is asking the right questions so you get the answers you need. If a firearm is being tested as a hunting option, then the tester is going to ask questions and use test parameters that differ from those used to judge a competition or self-defense platform. Changing the parameters used for testing will yield different answers to the same questions. Here's an example of what I mean that got me into trouble with some readers.
Back in 2015 I wrung out a pair of Hi-Point semi-automatics, a pistol and a carbine chambered in .45 ACP that could share the same single-stack magazines. I already had several answers in hand before heading to the range. Among the least expensive new production firearms on the market, Hi-Point models are either beloved by their owners for their low price and reliability or jeered at by critics for their homely appearance and less-than-stellar accuracy. When fired at standard test distances (25 yards for pistols, 50 yards for carbines) the five-shot group sizes were nothing to write home about.
Hi-Point Firearms 4595TS PRO Carbine (top), JHP45 Pistol (bottom)
However, the unanswered question that lead to this review was inspired by a short clip of exterior security camera footage posted with a short news piece that I saw online. The mute, grainy black-and-white footage caught a small group of home invaders breaking into a trailer through the front door. Nothing happens for a moment or two, and then suddenly the would-be assailants come flying out of the door at medal-winning speeds. It turned out that the occupants of the trailer were children and a single mother who defended them with a Hi-Point carbine. Chilling stuff.
Recognizing that many home defenders just can't cough up the two grand or so required for a high-end combat carbine with all the tactical trimmings, I wanted to know just how much defensive potential the Hi-Points are really capable of. As a result, I shifted the metrics I would use at the range in order to get the answer I hadn't found anywhere else. In short, I found that if you use five-shot groups between 3" to 3.5" in size at a given distance as a starting point (that's my personal standard, and not everyone agrees with it), the Hi-Point pistol I tested was suitable for personal protection at distances of around 15 yards and the carbine out to around 25 yards. In other words, they were good enough for short-range, close-quarters combat distances.
At least one reader commented that I was skewing my testing process in order to make Hi-Points look good because I said they are suitable for self-defense at shorter then typical test distances. At the same time, I confirmed that those particular semi-automatics did not produce an acceptable degree of defensive accuracy at the commonly accepted 25-yard pistol and 50-yard carbine distances. I'm still not sure why someone would see this piece as somehow making the Hi-Points out to be better performers than they actually were. But I did get the answer I was looking for to a specific question. Whether or not the answer is useful is up to the reader to decide.
The moral of the story is that when you are evaluating a particular piece of shooting information, make sure that that the reviewer is asking the questions and using parameters that meet your requirements. Otherwise, their results may not be useful in your particular situation.
Standardized Ballistic Testing vs. Informal Test Materials
When striving to determine the performance of a particular bullet design, manufacturers and government agencies often rely on consistent, repeatable testing methods conducted in laboratory type settings using specified conditions, materials and methods. For the sake of simplicity, let’s call this Standardized Ballistic Testing. Like any other scientific procedure, standardized tests allow for different groups of researchers to produce measurable results that are considered valid and comparable to the data of those who ascribe to the same testing procedures.
Among the most well known and widely recognized bullet test standards is the FBI's handgun ammunition testing protocols, which include the use of 10-percent animal-protein gelatin. I've been fortunate enough to visit ammunition manufacturing facilities, where I observed these strict FBI protocols in action. They proved to be as exhausting as they are effective. While the data is quite useful, it's a difficult and expensive set of standards to meet, especially for those of us working outside of a dedicated multi-million dollar facility with a gel-preparation support staff.
Front and side views of a pottery clay block shot by a Winchester .410 Super X 3-pellet 000 Buckshot shell fired from a 2" barrel revolver at a distance of 10 feet.
For those who can't afford the FBI approach to bullet testing, commercially produced gel-type test materials are available as a substitute 10-percent gelatin. These include Clear Ballistics synthetic gel (click here to see some recent results) and Corbin SIM-TEST Ballistic Media (a favorite of Mythbusters and similar television shows). These gels do not spoil or stink like 10-percent gelatin, they have a much wider operational temperature range, and can be melted down and re-molded several times which greatly reduces costs. They are not cheap to start with, however; the casting process is time consuming and neither material behaves exactly like 10-percent gelatin, so results may differ a bit. They are accepted as useful for bullet testing by many, but they have yet to gain a 'standard' status.
Some writers, product testers and videographers choose to not work with gels of any type. For some it's the expense and inconvenience while others opine that gel testing is not as informative as some folks believe. Since the bodies of game animals we hunt and the assailants we defend against, are made of various layers of differing materials, consistent blocks of gel don't tell the whole story. What does that leave bullet testers to work with? Open up Youtube and you'll see an exceptionally broad range of informal test material in use under a variety of conditions. Water jugs, wet pulp paper (phone books, news print, magazines), tightly packed stacks of paper plates, ceramic plates, concrete blocks, dry wall, pottery clay blocks, caste iron skillets, mixed material targets and all kinds of groceries from pork chops to watermelons.
Informal test materials have the distinct advantages of being cheap, easy to find and just about anyone can use them (if the range master will let you). The down side to using hardware and grocery store supplies as bullet stops is material consistency. Your watermelon may be heavier than mine, or your water jugs made of a thicker or tougher plastic, or your paper plates were plastic-coated while mine are not—you get the drift. Differences in test materials will affect the test results.
Clear Ballistics synthetic gel with Mossberg .410 Bore 500 Persuader
But let's not throw out the baby with the bath water here. Although informal test material results may not meet certain standardized criteria, they can be useful within the parameters of a particular test if it is conducted properly. Here's what I mean:
I once watched a 10-minute video test online in which the host fired one single round each of two different hunting rifle calibers into a stack of paper plates (it was something like a .308 vs. a .45-70 Govt.). After each round was fired into virgin material, the plates were counted to see how many the bullet penetrated. Sure enough, one bullet penetrated more deeply than the other. So what did we learn? Not much that was useful as far as I could see. All the audience was told was the two cartridge calibers and the rifles they were fired from. No bullet weights, no velocity information or descriptions of the bullet types. The use of paper plates as the test material was not the issue in this case, it was the lack of comparative data.
Over at our sister site, American Hunter, Aaron Carter wrote up a top-notch bullet test of half a dozen varmint hunting .22 Mag loads using a malleable block of pottery clay. Within that one test, it was easy to compare each round's performance because Aaron clearly defined his test parameters. The clay provided tangible results that could be measured and compared, such as wound cavity size and penetration depths. Cartridge specifications were also provided, including bullet weights, velocity and bullet type. He included photos of the shot clay block so that we could see how it deformed, which is information that can't be as accurately represented with words. Does this bullet test meet certain scientific requirements? No. But I did I walk away from the review with a much better idea of what to expect from these loads when hunting varmints.
Personal Preferences
There's no way around it. Gun reviews and product tests are all conducted by people. People have opinions which in turn are driven by their personal preferences. Some shooting preferences are based in hands-on experience (gun X fell apart when I needed it most while gun Y sacked a trophy elk for me). Other preferences are more mercurial and difficult to measure (balance, feel and appearance).
For the moment, let's go with something that influences everyone's personal preferences: aesthetics. I remember when a particular handgun was released a few years ago. I was excited about its innovative design and mixed materials construction which included steel, aluminum and polymer components. However, in one of the first articles I read the reviewer used the word "ugly" to describe the firearm as many times as possible. This low level of enthusiasm for the gun’s appearance translated into a lackluster review overall. ‘Yeah, sure it goes bang like it ought to but, whatever, Americans like ugly for some reason… .’
Innovative designs or cosmetic eyesores? It's up to the buyer to decide (From left: Heizer PKO45, Chiappa Rhino 200DS, Glock Gen5 G26)
Does the fact that a given platform is seen as ugly by a reviewer mean it's a "bad" gun? Of course not! But things like aesthetics, length of pull, grip shape, felt recoil and hard-to-reach controls, all of which are subjective qualities, are likely to get just as much ink as the gun’s positively perceived features. One person’s I-don’t-like-it is another person’s perfect fit.
If you're new to the shooting sports, or you’re striving to learn more about a specific gun make or model, one of the best ways to sort out a reviewer’s personal preferences from the data that's relevant to your needs is to read (or watch) multiple reviews from a variety of trusted sources. Look for patterns in the commentary related to quality, reliability and handling characteristics. If four out of five reviews say that Model X has a smooth trigger, then it probably does.
