Penetration tests need to accomplish business goals, not just check for random holes. Here's how to get the most value for your efforts.
Why are you performing
penetration tests? Whether you're using an internal team, outside experts or a combination of the two, are you simply satisfying regulatory or audit requirements, or do you actually expect to improve enterprise security?
We asked penetration testing experts for guidance on how to improve your program to get the most benefit for your time, money and effort. If you turn to outside expertise, their advice will show you what to expect and demand from consultants. The following 10 tips will show you understand the goal and focus of your testing; develop effective testing strategies; make effective use of your personnel; and make the most effective use of pen test results to remediate issues, improve processes and continuously improve enterprise security posture.
Penetration Test Tip 1: Define Your Goals
Penetration testing—really, all information security activity—is about protecting the business. You are taking on the role of attacker to find the vulnerabilities and exploiting them to determine the risks to the business and making recommendations to improve security based on your findings. Attackers are trying to steal your data—their techniques are a means to an end. So too, penetration testing: It's not about the cool technical things you can do to exploit a vulnerability; it's about discovering where the business risk is greatest. "If can't express things in terms of my business, you're not providing me value," said Ed Skoudis, founder and senior security consultant at InGuardians. "Don't tell me you've exploited a vulnerability and gotten shell on that box without telling me what that means for my business."
Also see Network stress test tools: dos and don'ts on CSOonline.com
With that understanding, from a more tactical perspective, penetration testing is a good way to determine how well your security policies, controls and technologies are actually working. Your company is investing a lot of money in products, patching systems, securing endpoints etc. As a pen tester, you are mimicking an attacker, trying to bypass or neutralize security controls.
"You're trying to give the company a good assessment if their money is being well spent," said Alberto Solino, founder and director of security consulting services of Core Security.
The goal should not be to simply get a check box for pen testing to meet compliance requirements, such as PCI DSS. Pen tests should be aimed at more than discovering vulnerabilities (vulnerability scanning should be part of a pen testing program but is not a substitute). Unless the testing is part of a sustained program for discovering, exploiting and correcting security weaknesses, your money and effort will have gained you at best that check mark, and at worst, a failed audit by a sharp assessor.
Penetration Test Tip 2: Follow the data
Organizations have limited budget and limited resources for pen testing, regardless of whether you are conducting internal tests, hiring outside consultants or using a combination of both. You can't conduct penetration tests across your entire IT infrastructure, spanning hundreds or thousands of devices, yet pen testers will often be told to try to compromise devices across an extensive range of IP addresses. The result is likely to be the most cursory of testing regimens, yielding little or no value. You can't even expect to conduct vulnerability scans and remediate flaws across a very large number of devices in a reasonable amount of time and at reasonable cost. "In many cases customers have thousand of IP addresses they want us to pen test," said Omar Khawaja, Global Products Manager, Verizon Security Solutions. "We could run vulnerability tests and see what's most vulnerable, but they may not be the most important to your organization."
Step back and ask, "What am I trying to protect?" What critical data is at risk: credit card data, patient information, personally identifiable customer information, business plans, intellectual property? Where does the information reside? Do you even know every database, every file repository and every log store that contains sensitive data? You may not know, but chances are an attacker will find it.
So, the first critical step is to narrow the scope of pen testing is data discovery: determining which sensitive data is at risk and where it is. Then the task is to play the role of attacker and figure out how to get at the prize. (Read
Red team versus blue team for more ideas on this approach.)
"The idea to mimic what a real attacker will do during time frame agreed to with the customer," said Core Security's Solino, "not to find all the possible problems."
Penetration Test Tip 2: Follow the data
Organizations have limited budget and limited resources for pen testing, regardless of whether you are conducting internal tests, hiring outside consultants or using a combination of both. You can't conduct penetration tests across your entire IT infrastructure, spanning hundreds or thousands of devices, yet pen testers will often be told to try to compromise devices across an extensive range of IP addresses. The result is likely to be the most cursory of testing regimens, yielding little or no value. You can't even expect to conduct vulnerability scans and remediate flaws across a very large number of devices in a reasonable amount of time and at reasonable cost. "In many cases customers have thousand of IP addresses they want us to pen test," said Omar Khawaja, Global Products Manager, Verizon Security Solutions. "We could run vulnerability tests and see what's most vulnerable, but they may not be the most important to your organization."
Step back and ask, "What am I trying to protect?" What critical data is at risk: credit card data, patient information, personally identifiable customer information, business plans, intellectual property? Where does the information reside? Do you even know every database, every file repository and every log store that contains sensitive data? You may not know, but chances are an attacker will find it.
So, the first critical step is to narrow the scope of pen testing is data discovery: determining which sensitive data is at risk and where it is. Then the task is to play the role of attacker and figure out how to get at the prize. (Read
Red team versus blue team for more ideas on this approach.)
"The idea to mimic what a real attacker will do during time frame agreed to with the customer," said Core Security's Solino, "not to find all the possible problems."
Penetration Test Tip 3: Talk to the Business Owners
Work with the business people. They know what is at risk—what data is critical, what applications create and interface with that data. They will know at least the more obvious places in which the data resides. They will tell you which applications must be kept up and running.
You'll learn much of what you need to know about the threat level associated with particular applications, the value of the data and the assets that are important in the risk equation.
An important part of this process is to work with people who understand the business logic of the application. Knowing what the application is supposed to do and how it's supposed to work will help you find its weaknesses and exploit them.
"Define the scope that includes critical information assets and business transaction processing," said InGuardians' Skoudis. "Brainstorm with the pen test team and management together."
Skoudis also suggests asking for management to give their worst case scenario, "what's the worst thing that could happen if someone hacks you?" The exercise helps scope the project by determining where "the real crown jewels" are.
Penetration Test Tip 4: Test Against the Risk
The value of the data/applications should determine the type of testing to be conducted. For low-risk assets, periodic vulnerability scanning is a cost-effective use of resources. Medium risk might call for a combination of vulnerability scans and manual vulnerability investigation. For high-risk assets, conduct exploitative penetration testing. For example, the security director for a large university said they started performing pen testing to meet PCI DSS requirements. Once that program was in place, it became the model for testing a potential attacker's ability to penetrate their systems. The university classifies data as public, internal, sensitive and highly sensitive.
For information that's highly sensitive, we perform pen testing under much the same guidelines as PCI," he said. "We back off from there, based on some specific criteria and some subjective judgment that goes into what level of pen testing, if any, will be done for system."
So, for example, on the lower end of the risk spectrum the university will test a random sample of systems and/or applications, depending on criteria for a particular category and time and budget constraints. With tens of thousands of devices on a campus network, even a low-level scan of all of them would be infeasible.
"You can test on a business system that has a clear owner and systems administrator," he said. "But when you have 3,000 Wiis attached to the network, you don't want to scan those and figure out who they belong to."
Penetration Test Tip 5: Develop attacker profiles
Your pen testers need to think like and act like real attackers. But attackers don't fit into one neat category. Build profiles of potential attackers. External attackers may have little or no knowledge of your company, perhaps just some IP addresses. They may be former employees or work for partners or service providers and have considerable knowledge of the inside of your network. An insider may be a systems administrator or DBA with privileged access and authorization and knows where critical data resides.
Motive is a factor in developing profiles. Is the attacker after credit card numbers and PII that can be turned into cash?
Intellectual property to sell to a competitor or gain a business advantage? The attacker may be politically/ideologically or competitively motivated to bring your Web application down. He may be an angry ex-employee who wants to "get back at the company."
Work with business owners to help fashion these profiles and learn what types of potential attackers they are most concerned about.
The profile narrows the focus of the pen testing, and tests will vary based on each of these multiple profiles.
"We get a snapshot of what a particular attacker can do against a target, and we don't mix results," said Core Security's Solino. "For every profile, we get the result of the pen test and do another profile."
Penetration Test Tip 6: The More Intelligence the Better
Information gathering is as much a part of the process as the actual exploit—identify devices, operating systems, applications, databases, etc. The more you know about a target and its connected systems, the better chance you have of breaking in.
Each step may yield valuable information that will allow you to attack another asset that will eventually get you into the target database, file share etc. The information will allow you to narrow the search for exploitable vulnerabilities. This reconnaissance is typically performed using automated scanning and mapping tools, but you can also
use social engineering methods, such as posing as a help desk person or a contractor on the phone, to gather valuable information.
"We're increasingly starting to do social engineering," said Verizon's Khawaja. "It's essentially reconnaissance—performed with the permission of the customer—to let us find everything in the environment that could assist us in breaking in."
Multi-stage penetration testing typically is a repeated cycle of reconnaissance, vulnerability assessment and exploitation, each step giving you the information to penetrate deeper into the network.
Penetration Test Tip 7: Consider All Attack Vectors
Attackers can and will exploit different aspects of your IT infrastructure, individually or, frequently, in combination to get the data they are seeking. Thorough pen tests will leverage any and all of these potential attack vectors, based on the attacker's end goal, rather than the vulnerability of each.
"A few years ago we would do network penetration testing, and application pen testing and wireless pen testing, and then we stepped back and said 'that makes absolutely no sense," said Solino. "The bad guy doesn't say, 'I can only break into a system using the network.'"
Successful pen tests, like real attacks, may leverage any number of paths that include a number of steps till you hit pay dirt. A print server may not seem particularly interesting, but it may use the same admin login credentials as a database containing credit card information.
"Pen testers find flaws and exploit them, then pivot from that machine to another machine, to yet another," said InGuardians' Skoudis.
An attack on a Web application might fail in terms of exploitation, but yield information that helps exploit other assets on the network. Or an attacker might get information about employees without high privileges, but with access to the internal network that act as a springboard.
Also see How to compare and use wireless intrusion detection systems
So, a critical resource may not be directly assailable, but can be compromised through other systems.
For example, said Khawaja, Verizon pen testers were unable to directly compromise a Web server that had access to a sensitive database. If the testers focused narrowly on testing the Web application on that server, the conclusion would be that the data was safe. But by taking a data-centric approach, they discovered that the Web server was connected to a second Web server, which had a critical vulnerability that an attacker could exploit to gain access to the first Web server and, hence, the database. (Read more about Web application attacks in
How to evaluate and use Web application security scanners.)
"We care about anything that isn't cordoned off from the network segment we are targeting," he said. "Are there any network controls to prevent an attacker from jumping from a vulnerable low-value system to a more critical system?"
That being said, there are valid cases for vector-specific testing. For example, a company may be particularly concerned about
wireless security, because it knows it has been somewhat lax in this area or may have recently installed or upgraded WLAN infrastructure. But even if you are confident that a particular vector is safe—for example , if the wireless network is isolated from the credit card database—don't be too sure. Attack paths can be complex and byzantine.
Penetration Test Tip 8: Define the Rules of Engagement
Pen testing simulates attack behavior, but it is not an attack. Whether you are conducting in-house testing or contracting with a consultant, you need to establish parameters that define what can and cannot be done, and when, and who needs to know.
The latter depends on whether you are conducting white box or black box testing. In the former case, there's probably an acknowledgement that the security program of the company (or a particular department or business unit) needs a lot of work, and the pen testing is open process known to all involved.
On the other hand, black box testing is more clandestine, conducted more like a real attack—strictly on a need to know basis. You are determining how good the company's people are at their jobs and the effectiveness of the processes and systems supporting them.
"Whether it's the operations center, or the investigative response team or physical security guards, everyone has to pretend it's just another day at the office," said Verizon's Khawaja.
Typically, companies will perform white box testing first to learn the security issues that have to be addressed. Subsequently, black box testing will help determine if the initial findings have been effectively remediated. Sometimes, for example, a CSO will want to know not only how vulnerable critical systems are, but how good their personnel are at detecting and responding to an attack.
In either case, certain key people need to be involved to avoid problems that might impact the business or undermine the testing. At least one person in the target environment who is involved in the change control process should be in the loop, said InGuardians' Skoudis. Under the rules of engagement, for example, the company may permit the pen testers to install software on the target devices to do more in-depth pivoting, but at least that one person has to be involved to make sure that the testers are not stopped by dropping their IP address from a router ACL or
invoking a firewall rule.
In both white box and black box scenarios, Skoudis recommends daily briefings with the test stakeholders to let them know what the testers are doing. For example, the rules of engagement may allow the pen testers to exploit vulnerabilities, but the briefing can be used to give folks a heads up that they are about to do it.
"It builds bridges," he said. "It shows the pen testers are not a distant, evil group that is out to 'catch me.' Rather, it's all about transparency and openness."
The rules of engagement also may set limits on what may and may not be exploited, such as client machines, or techniques that may or may not be used, such as social engineering.
Penetration Test Tip 9: Report Findings and Measure Progress
The goal of penetration testing is to improve your security posture, so if you are conducting internal tests, your report should provide useful, actionable and specific information.
"The goal is to help improve security, for management to make decisions to improve business and help the operations team improve security," said InGuardians' Skoudis.
You should provide an executive summary, but the heart of your reporting should include detailed descriptions of the vulnerabilities you found, how you exploited them and what assets would be at risk if a real attack took place. Detail every step used to penetrate, each vulnerability that had to be exploited, and, most important, perhaps, all the attack paths.
"The beauty of identifying the attack path is that it allows you to solve specific problems by breaking the path," said Core Security's Solino.
Be very specific about recommendations. If architectural changes are required, include diagrams. Explain how to verify that a fix is in place (use this command, or that tool to measure). In cases where multiple systems are involved, explain how to mass deploy a fix, using GPOs if possible.
Make sure that each recommended remediation includes a caveat that the solution is thoroughly tested before it is implemented in a production environment. Enterprise IT infrastructure may be very complex.
"This is a huge issue," said Skoudis. "You don't know all the subtleties. You don't want to break production."
Penetration testing should not be a one-time exercise, and successive results should be compared. If you are performing internal testing, put together deltas to measure how your people are addressing issues. If the problems from the last test—or the last two—remain unaddressed, you may have a problem. Perhaps the software patching program isn't working as it should, or developers are not being
properly trained to write secure code.
"What we're looking for are trends," said the university security director. "It's just like you would treat an audit report. If you have repeat findings, it indicates you might have a more serious problem."
Penetration Test Tip 10: Decide Who Your Pen Testers Are
The decision to use in-house staff for pen-testing depends on the size of your organization, the value of the information you are trying to protect and where you want to put your internal resources. A company may have a dedicated pen testing team or a group within the security team. An internal team is in a better position to conduct regular testing. If your organization is large and distributed, create mechanisms and promote an environment in which information can be shared.
"If have internal community that can share information, make sure they have a strong knowledge base backed up by mature knowledge management systems," said Verizon's Khawaja . "You want to make sure that what happened in your Beligian unit doesn't happen in Brazil."
Even if you do some in-house testing, there are good reasons for hiring consultants to perform at least some of the work. Some regulations require external companies to perform pen tests; consider that insiders may have too much information about the target systems, as well as a vested interest in the outcome. So, beyond compliance requirements, it's a good idea to bring a fresh view from the outside periodically.
For the same reasons, if you do hire outside testing consultants, rotate among vendors, just as would with auditors every few years.
"Bringing in outside people gives an added degree of confidence in the results," said the university security director. "There's no perception of conflict of interest."—
For your internal team, look for the right blend of knowledge and curiosity.
A good training candidate, said Core's Solino, has a strong knowledge of networking and application protocols as a foundation. Mostly, he looks for curiosity and a hacker mentality.
"It's IT knowledge and that attitude, a specific mindset that denies something is secure and says, 'Go for it!'"
"This is an art," said Skoudis. "Although there are tools and methodologies, you have to be creative in finding problems in target systems and applications."
