Among the 20-year old arguments we still haven't resolved in infosec is this one. Part of the inconclusiveness is that both sides of the argument are right, depending on unspoken premises. We all assume your premises are the same as mine.
Some organizations haven't yet gotten the basics down. They need pentests doing the basics. Other organizations have solved what hackers did yesterday and want pentesters doing things that hackers will do tomorrow.
The underlying thread brings up a slightly different topic. Defenders are treated worse than dogs. I'm referring to sniffer dogs in airports that search bags for drugs and bombs. The dogs become bored and disillusioned unless they are occasionally given successes.
You need to regularly give sniffer dogs a bag containing explosive materials or rugs. Likewise, you need to feed defensive teams random pentesters who they can catch.
But while organizations need practice at what they are designed to do, they also need pentesters thinking outside the box doing oddball things that challenge their preconceptions.
Too often pentester contracts are designed to limit the pentester to doing only the things the organization knows they can defend against. Orgs need unrestricted pentests where the pentesters are allowed to do anything.
I don't want to attack your main web server. I want to attack that server somebody in IT setup for retirees, which is not strictly part of the company, yet (once popped) has access to the internal corporate network.
Adversaries like Anonymous or the GRU are often found having used the simplest of techniques, like SQL injection or phishing. The lesson many take from this is that they only need to defend against such simple techniques.
But in truth, they could do more -- but the simplest techniques worked. In other words, this reflects the maturity of the defenders more than the maturity of the attackers. If suddenly every fixed phishing, such hackers wouldn't go away, they'd simply get more advanced.
We hear about Anonymous and GRU hacks that used simple techniques like SQL injection and phishing. We don't hear about NSA hacks. It doesn't mean the NSA isn't hacking, it means we don't hear about it. 
Anyway, the point of this thread is that we've been debating for 20 years what a "pentest" ought to do, and we still haven't come to consensus on this. I don't suppose we ever will.
