Here is how to review a pentest and figure out if it is a network security assessment or an AppSec security review.
When you look at a pen test, you can tell very quickly if it was done by somebody who understands AppSec (somebody who can code), or by somebody who is approaching the problem from an network security point of view (usually running lots of tools).
The first main points to notice are if they asked for the source code, and if they performed an threat model on the target application. If they didn't, then it is most likely going to be a network security assessment.
You can also learn a lot from the findings. If you see lots of network scans results (for example open ports, exposed services, out-of-date server), or heavy usage of tools like like ZAP/Burp/WebInspect/Nessus used a lot, or lots of blackbox/browser-based findings, these are very good clues that a network security test was actually carried out.
Clues from a stronger AppSec will be lots of root-cause analysis (with code references and code snippets), specific code patterns that were found to be dangerous, multiple scripts/mini-tools created during the assessment, and ideally, a number of Tests delivered.
These Tests should provide evidence of the findings reported (which will allow developers to replicate those finding) and the exploit vectors that were not successful (which shows the coverage of the assessment).
In the cases where they didn't have the code, but you see lots of customisations, custom scripts, and all sorts of tools that allowed them to really test the application, then that is also somebody who is doing AppSec.
As with most everything you get what you pay for, and if you don't invest in making sure your Security Reviews are performed to the highest possible level, then don't be surprised when the value you get is really low.
(from Software Quality book)
