Before I get into this subject I should point out that this article is really meant to highlight problems with the majority of companies out there. Some open source organisations and projects tend to do better - mainly because they cannot afford to pay for security and as a result tend to find interesting ways to solve this problem. No pain, no gain as they say and unfortunately this is the best way to learn and grow.
Typically security issues are stored in reports. Vulnerability details and examples are often encapsulated as appendices Once the report is obtained, all the information is processed and fed to the various people who need to do the fixes. Once implemented, these fixes are sent back for re-validation to either the internal security team or the penetration testing company, which performed the test.
From my personal experience I see several problems with this approach. First, all the data is encapsulated statically into a flat report structure. Second, some issues require extra steps in order to be re-tested, which may not be clearly explained in the report and therefore difficult to fix. Finally, there is a high possibility that some issues may not be properly fixed at the end and therefore sent back to the developer again with more comments and suggestions.
It may not be immediately obvious to you, unless you have been in the field for as long as I have, but this is a broken process. The sad reality of the situation is that this is how typically things are done. This game of ping-pong is the reason why security still feels difficult. Developers typically do not want to deal with security and security people get frustrated and stop giving a shit. No one offers any new solutions and the security process is generally stuck in a limbo.
Back To The Origins
I believe that one of the fundamental reason for this situation can be traced back to the way information security services has been for the last 12 years. Penetration testing is still an exclusive service and just like any other consultancy service at the end of the day the product comes in the form of reports. The report represents everything - from outline of the found issues to generic recommendations and examples of best practices. However, the report is simply designed for one purpose only and that is to highlight the problems. The provided recommendations are often as generic as they can get because at the end of the day penetration testers do not know the business nor the technology in order to suggest any sensible solutions. Pointing the issues is easy - fixing them takes a lot of logistics and effort especially in large organisations.
The problem is magnified by the fact that the very same people who used to work for security consultancies, writing reports all day long, now work for business in charge of the vulnerability remediation process. It is no surprise that the skills learned as penetration testers in the field are now used to solve the big security problems. However, it is all the same. The focus is still at pointing out the issues, and while the static report structure may now be replaced by spreadsheets or some kind of vulnerability management system, the situation remains.
Getting To Know Your Developers
It doesn't have to be this way. Let's take a step back and analyse why information security services exists on first place. If there are no developers, there will be no code and if there is no code there will be no vulnerabilities and no one to sell information security services. Information security, although very specialised, is in fact no different that quality assurance testing. QA works with developers. Information Security doesn't. There is this thing called SDLC (Secure Development Lifecycle) but I've worked for organisation with very SDLC centric process and still broken security. Security issues are typically piled on top of each other stored in a decentralised systems away from the eyes of the developers. And if no one can fix the issues, which is typical, there is a risk acceptance process, which is often abused beyond reason.
Why is that? Your developers are the people who are supposed to fix the problem and they use all kinds of tools to make their lives easier and less stressful. Why security is not part of this process? Where is the need for risk registers and secretive vulnerability management solutions or spreadsheets shared only among security personal? Why there is a need for risk acceptance? If you organisation is serious about security than your process needs to be oriented towards supporting your development structure in every possible way to make security as painless as possible.
Face to Face With Your Problem
Your developers make use of tons of tools to improve the quality of their code. Bug tracking systems are used to track defects and features. Unit testing is done for a reason and that is to find bugs early during the development process because bugs are inevitable. Continues Integration works because some products are way too complex to deal with individually and you need some kind of standardised workflow to get you going. All of these tools are designed to aid better code. Security is typically not part of it.
Change is behind the corner and it starts by not thinking in terms of reports. If you cannot action on the information that is encapsulated inside these reports than they are less useful than toilet paper.
So what do you do? My humble advice comes in 5 steps:
Step 1: Vulnerabilities should go directly into your bug tracking system and recorded as defects. The next time you hire external security consultancy to do a penetration test, save yourself some time and money and request all issues to be entered in a format suitable for importing into your bug tracking system used by your development team. If possible, ask the penetration testers to enter the issues as soon as they are identified. Make sure that all issues are bundled to the correct milestones. Some issues cannot be fixed immediately because they are not a priority and that is OK. However, specify the milestone by when they must be fixed.
Step 2: You need unit testing. Someone has to write unit tests for security defects because they tend reappear sometimes even days after they are fixed. I can guarantee that the more unit tests you have the better quality your code will be and less bugs will generally occur now and in the future. It is the most proactive thing to do.
Step 3: You need security tools in your Continues Integration/Delivery pipeline. If you don't have such a thing perhaps you should start thinking about setting one up since it is very useful unless you are running a very small development team. Static code analysers should be executed before every build. Blackbox assessment tools, such as scanners and fuzzers, should execute during the unit testing stage. Concentrate on the low handing fruit first. Don't try to find everything because this is very time consuming. It is better to be able to identify 5 types of vulnerabilities with no false positives instead of 50 types of vulnerabilities with 80% false positive rate. Don't listen to vendors who tell you that their products are false positive free. Their software is not perfect and their developers are just normal developers. Mistakes will happen.
Step 4: Equip your developers with basic security tools like scanners, bruteforcers, fuzzers or whatever else they need to do the bare minimum and capture the low hanging fruit as early as possible. If your developers cannot do it then QA should be able to do it. It is really not that difficult and even if your people may not have extensive experience in security they will get the grip of the basic concepts very quickly.
Step 5: You still need expertise from internal and external security consultants but make sure that their process is inline with your development process. At the end of the day the goal is to fix the bugs not to collect exotic and sexy security vulnerabilities. I know that this makes one feel powerful and important, especially if you are still young and unexperienced, but trust me when I say that you will get numb towards such things as you grow. The important thing is to keep security issues at bay and this directly translates into getting them fixed.
Here is a list of a two things to avoid doing:
- Do not invest into an expensive vulnerability management system. You still may need such a system for some things, which are not directly related to your business, but unless you have hundreds of issues, you don't really need it. Seriously, most of the time there is no need for such a thing.
- Do not underestimate security. Even when you think that you have figured all out you will find through painful experience that you had it all wrong. Some people find joy in taking apart applications and network infrastructures and they will eventually find a way in. It is inevitable. Fortunately, there are not that many people like that so your better security practices act as a filter. If somebody is going to break your stuff at least hope that he or she is the best one in field.
I Am Almost Done
I wouldn't even bother writing this article if I did not feel strong about this subject. I was debating if I should post it on GNUCITIZEN or here. I end up posting it here because the tools I am building for Websecurify are in fact designed to solve the problems I mentioned above. Our reasons are not charitable but the goal is to make a dent for a change and improvement and if these tools can help achieve this goal than perhaps some day we will move on and find other problems to solve.