What CEOs Do, and How They Can Do it Better

Why did you come in late on Tuesday? Did you really need an hour and a half for lunch on Wednesday? Why wasn't that report done by Thursday? For most of us, justifying our schedules is an expected part of the job.
But what employee hasn't looked at the closed door of the corner office and wondered what the boss is doing all day. For all of the minute-to-minute monitoring of employee performance from the time of Henry Ford onward, it's amazing how little any of us really know about how CEOs of major companies spend their time.
"Fundamentally, it's because no one knows what a CEO should do," says Harvard Business School professor Raffaella Sadun. "Most of the time it's difficult to codify the qualities of a good manager."

"We went in with the curiosity of trying to understand the life of a CEO"

Despite that difficulty, however, it's self-evident that the way a CEO chooses to spend his or her time has much more of an effect on a company's success or failure than if a middle manager spends a half hour more at lunch. With that in mind, Sadun and three colleagues-Oriana Bandiera and Andrea Prat of the London School of Economics and Luigi Guiso of the European University Institute—set out to get to the bottom of CEO time management by following nearly 100 top managers in Italy, as reported in a recent paper with the deceptively simple title, What Do CEOs Do?
"We had no way of knowing what we were going to find," says Sadun. "We went in with the curiosity of trying to understand the life of a CEO."
But what they did discover should help CEOs learn to be more effective with their time, and provide boards with a new tool to help assess the effectiveness of their chief executives.

Under a microscope

Of course, it's not so easy to codify all of the many actions a CEO could take during the course of a day—attending meetings, reviewing a marketing campaign, schmoozing clients on the golf course. So Sadun and her colleagues instead divided up activities with a much simpler measure of looking at the people with whom a CEO spent time.
After all, the boss is in a unique position within a firm not only to spend time with employees, but also with the outside world, making connections and gathering information. However, not all of the time the boss spends with outsiders might help the firm, especially if a CEO's and a company's interests are not aligned.
"CEOs should be working with both constituencies, insiders and outsiders," says Sadun. "However, if there are governance issues, there might be the possibility that the CEO is in the outside world more for his or her personal benefit than for the benefit of the firm."
In order to test whether this was true, the researchers enlisted 94 CEOs of major Italian corporations who agreed to put their lives under the microscope for a period of a week at a time. The CEO's personal assistant was asked to record every activity the boss engaged in that lasted at least 15 minutes.
Tabulating the data, the researchers discovered that the vast majority of a CEO's time, some 85 percent, was spent working with other people through meetings, phone calls, and public appearances, while only 15 percent was spent working alone. Of the time spent with others, chief execs spent on average 42 percent with only "insiders" (employees or directors of the CEO's firm); 25 percent with insiders and outsiders together; and 16 percent with only outsiders. (Exact numbers varied dramatically among the sample, with some CEOs spending more than 20 hours a week outside the office, while others spent almost none.)
Next, the researchers crunched a number of factors measuring company performance—for example, profits per employee—in order to see which CEOs were more productively using their time.

Better on the inside

Their first finding, which might seem unsurprising, was that the top managers who spent more time at work were more productive than those who spent less time at work. In fact, Sadun and company found, for every 1 percent increase in hours worked, there was a 2.14 percent increase in productivity. "That's never been shown before, so that was reassuring," Sadun says.
Likewise, time spent with insiders was strongly correlated with productivity increases. For every 1 percent gain in time spent with at least one insider, productivity advanced 1.23 percent. Less reassuring, however, was that the time CEOs spent with outsiders had no measurable correlation with firm performance.

"It's a way to monitor where the efforts of the CEO are going"

In a final measure of CEO's performance, the researchers rated firms based on the quality of governance, measuring a variety of factors such as the size of the board, the presence of at least one woman on the board, ownership, whether the company was based in another country, and if so, the general level of governance in that country. Again they found a clear correlation: in companies with stronger governance, CEOs spent more time with insiders and less time with outsiders, and at the same time were more productive.
"There are some industries where a CEO really needs to be outside, so we don't need to be proscriptive, but if you were taking these results literally it would tell you that since a CEO's time is constrained, he should be mindful of the time spent with his own employees," says Sadun.
In extrapolating from the data, Sadun cautions the sample size used in the study was relatively small (though exponentially bigger than any past research on the topic), and that the results of the study (especially when it comes to the link between CEO time use and firm performance) should for the moment be interpreted as suggestive correlations rather than firm causality statements. Even so, encouraged by the results of the initial study, the group is planning to continue along this line of research by expanding the data collection in other countries (India, China, and the US) in order to increase the sample as well as to take cultural differences into account.
Sadun says that the group has received nothing but positive feedback from the anonymous CEOs who participated in the study. In keeping with the adage that "it's lonely at the top," many of the managers studied had little idea of how they could make their time more productive. Sadun hopes that the information will be equally helpful for boards in evaluating the performance of their CEOs.
"It's a way to monitor where the efforts of the CEO are going, and to get them understanding that perhaps spending too much time on the outside might not be as beneficial as they might think," she says.
If nothing else, next time employees ask the question "What is the boss doing with all of his time?" at least they'll have an answer.

About the author

Michael Blanding is a freelance writer who lives in Boston.

The Ten Worst Things to Put on Your Resume

According to a 2010 Accountemps survey, 28% of executives say the resume is where most job seekers make mistakes in the application process. But what exactly constitutes a mistake?
We talked with career coaches and resume writers to find ten gaffes that will guarantee that your resume never makes it past round one.
Related: Explaining Job Gaps in Your Resume | How to Gather References and Make Them Work For You


1. Unnecessary Details About Your Life
There are a few personal details you should include on a resume: full name and contact information, including email, phone number and address. But beyond that, personal details should be kept to a minimum. If the prospective employer wants to know more than the minimum, they will ask you or figure it out for themselves.
"Your age, race, political affiliation, anything about your family members, and home ownership status should all be left off your resume," says Ann Baehr, a certified professional resume writer and president of New York-based Best Resumes. "What's confusing is that [a lot of personal information is] included on international CVs. In the U.S., including [personal data] is a no-no because it leaves the job-seeker open to discrimination."
The exception to the rule: If you're looking to work for an organization closely tied to a cause, you may consider including your race, political party, or religious beliefs.
"Personal data may suggest a bias, unless what you want to do next is directly tied to one of those categories, because it shows aligned interest," says Roy Cohen, a New York City career coach and author of The Wall Street Professional's Survival Guide. So, unless you're looking to work for a religious, political, or social organization, you're better off keeping personal philosophies to yourself.


2. Your Work Responsibilities as a Lifeguard When You Were 16...
"Don't include information that will not advance you in your work goals," says Rena Nisonoff, president of The Last Word, a resume-writing and job-coaching company in Boston. "Anything extraneous should be left off your resume." That includes hobbies and irrelevant jobs you held many years ago.
Unless you're an undergraduate student or a freshly minted professional, limit your work history to professional experience you've had in the past 10 to 15 years (or greater, if it was a C-level position).


3. A Headshot
In some industries, being asked for and including a headshot is commonplace, but unless you're a model, actor, or Miss America, the general rule of thumb is that photos should be left out.
"To many [hiring managers], including a headshot feels hokey," says Cohen. It can give off the wrong impression, and isn't a job-seeking tactic that's customarily received well.
Furthermore, it's illegal for employers to discriminate against job candidates based on appearance, so attaching a headshot can put employers in an awkward position, says Nisonoff. Unless it's specifically requested, and it's relevant to the job at hand, keep your appearance out of it.


4. Salary Expectations
Most job candidates feel uneasy discussing salary requirements. For good reason: Giving a number that's too high or too low can cost you the job. You should keep it out of your application materials entirely, unless the hiring manager asks for it.
"If they specifically ask for it, you should give them a range," says Nisonoff, but even still, that information should be reserved for the cover letter and not put on the resume. If you have the option, save that discussion for a later stage of the interviewing process, ideally once the interviewer brings it up.


5. Lies
This should really go without saying, but career coaches and resume writers alike report that the line between embellishment and fabrication is often crossed by job applicants -- and that they've seen it cost their clients jobs.
One of the most common areas in which people fudge the facts is the timeline of their work history.
"A client of mine who worked for a Wall Street firm had moved around quite a bit," says Cohen. The client, who was a registered representative, intentionally excluded a former employer from his resume, and covered it up by altering the dates of employment at other firms. "Registered representatives leave a FINRA trail, and when his resume was checked against his FINRA trail, [the company] saw he had left off a firm and they pulled the offer," Cohen explains.
Whether it's using false information to cover a blemish or exaggerate success, there's no room to lie on your resume. No matter how miniscule the chance is that you'll be caught, you should always represent yourself as accurately as possible.


6. Things That Were Once Labeled "Confidential"
In many jobs, you will handle proprietary information. Having inside information from your positions at previous employers might make you feel important -- but if you use that information to pad your resume, chances are it will raise a red flag.
"Confidential information should never be shared, it shows poor judgment," says Cohen.
If you're sharing the names of your clients, in-house financial dealings, or anything else that might be for your eyes only, it can backfire in two ways. The prospective employer will know that you can't be trusted with sensitive information; and your current (or former) employer might find out what you have been sharing and it could be grounds for dismissal or even a lawsuit.


7. If You Were Fired From a Job -- and What You Were Fired For
Your resume should put you in a positive light. Including that you were let go for poor performance, stealing from the company, or any other fault of your own will have the exact opposite effect.
"Leave out information about a situation that positions you negatively, such as 'I got fired' or 'I mishandled funds,'" says Cohen. "Anything that suggests you used poor judgment in your current or former job."
Following this advice does not violate the rule about lying (No. 5). If you're asked to explain why you left a job, you need to bite the bullet and be straightforward, but until then, make sure you're putting your best foot forward.


8. Overly Verbose Statements
There is a pretty fine line between selling yourself and overselling yourself. Too many resumes overstate the importance of job responsibilities.
"Job seekers with limited experience [try] to put themselves in a 'management' light," says Baehr, using phrases like "'Spearheaded high-profile projects through supervision of others, leading by example.'" Keep your flair for the dramatic to a minimum, so resume readers can get a picture of what your real responsibilities were with your past or current company.


9. "References Available Upon Request" and Your Objective
The age-old "references available upon request" has become archaic. You should have solid references lined up from the get-go, so when the hiring manager asks for them, you're ready to share them.
"It's not really an option," says Baehr. "If they want your references, they're going to get them."
Also nix the objective statement. It's not really necessary to explain your career goals unless you are a recent graduate or are switching careers. If necessary, work your objective into a summary of your qualifications, says Cohen.
"It explains what you want, which may not be readily apparent from the resume," he says, "and it also tells a story to explain why you want to make the career change."


10. TMI
Too much information is almost never a good idea. It's particularly bad when it's put in front of hiring managers who are busy, tired, and quite frankly, probably not going to read your resume word-for-word. If you put too much information in your resume, recruiters will likely not read it at all or just scan it quickly.
"Far too much detail is damaging because it won't get read," says Cohen. "It suggests that you get lost in seeing the forest for the trees and also suggests an attachment to information. It's a burden to the reader, and these days, readers of resumes don't want to be burdened."

How to Select a Great Domain Name for Your Company

Here's what to look for in a good domain name.

1. A good domain name is relatively short. A short name -- if you can get it -- is important for several reasons. It is easy to fit into logos, makes a better brand, is more easily recognizable, and is harder to misspell. Some companies have 50-character domain names spelling out their whole company name. That's unwise. Long domain names don't fit in forms, on billboards, or in Google PPC ads. Keep them relatively short.
2. A good domain name is memorable. You remember generic names, such as Art.com and Garden.com. But you also remember more unique names such as Amazon.com, Google.com, and FogDog.com. Putting together strange combinations of words is fun and can be very productive. It helps if it rhymes like FogDog, or repeats sounds such as Google, or is sing-songy like WilsonWeb. Say your prospective domain name out loud to listen to its sounds. See if your tongue gets twisted around any syllables. Whatever your domain name, it should stick in the mind.
3. A good domain name isn't easily confused with others. In their desperation to find a domain name, some grasped at hyphenated names and put "the" in front of a word, as in TheStandard.com. The problem is confusion. Trademark laws are designed to prevent customer confusion. If the holder of a similar domain name is first to trademark his combination, it could threaten your domain name, or at least your ability to use it as a brand. Be sure to check with the US Patent and Trademark database (www.uspto.gov/main/trademarks.htm) or the trademark database for your country. Another consideration is how you'll need to say your domain name over the phone. If you always have to say "spelled ding-hyphen-doodle.com" you'll soon wish you'd left out the hyphens. Do your best to find a name that can't be confused.
4. A good domain name is hard to misspell. If people can misspell something, they will. The longer and more complex your domain name, the harder it is for your customers to type it in correctly. Many of them can't type well to start with, so to type in a long name may lose you lots of business. At the low price of domain names, it may pay you to purchase the misspellings of a domain name, too. This way you'll get the traffic intended for your site and discourage poachers from buying up the variants. Poachers can be driven off by lawsuits if you have trademark protection, but you don't want that hassle.
5. A good domain name relates to your business name or core business. It's best if your domain name can be guessed from your company name. But in your search for a domain name, don't give up if you can't find the domain for your exact business name. Find functional names, names that describe your uniqueness, names that express an emotion or attitude.
6. A good domain name sounds solid to your target audience. If possible, get a .com domain or the domain that has the most respect in your country. You can get a .biz or .info, or .cc, .ws, .tv, and .to. (The latter are the country top level domains of the small nations of Cocos (Keeling) Islands, (Western) Samoa, Tuvalu, and Tonga, respectively). The problem is that the general public, in the US anyway, is accustomed to .com, or maybe .net (though .net and .org aren't nearly as well regarded). Offbeat domain names sound ... offbeat and suspect. Your main domain should be the one that people expect it to be. In the US, that's probably .com. In France it would be .fr. If you want to appeal to an international audience, .com is probably best. Having said that, I think it's wise to buy up other common domain name endings. They're cheap. If you become successful you'll wish you had kept them away from poachers. This helps your main domain name stay unique.

Getting the Creative Process Going

Several tools can help you in the process of coming up with a company name or a domain name -- most of them owned by domain registrars to attract business. Some of the best are no longer available online, but here are a few:
DomainFellow.com (www.domainfellow.com) lets you start with a keyword and then adds up to 1,000 popular prefixes or suffixes to that word. For each combination you can see which domains are available and which have been taken.
MakeWords.com (www.makewords.com) lets you select the length and language of a possible domain name, plus a prefix and suffix. Then it generates possible matches and tells you if they are available.
NameBoy (www.nameboy.com) is one of the oldest and best. You enter a primary and a secondary word to work with, select the type of site you have, and whether or not you want to turn on the hyphenation and rhyming features, then let 'r rip! It works like one of those children's books that allows you to play with multiple head, body, and leg combinations.
DomainNameSoup.com (www.domainnamesoup.com) allows you to mix and match words with domain names. Tools allow you to experiment with various combinations of hyphens, prepending and appending words, changing letters of a domain anem, searching for typos, trying multiple choices, looking for synonyms, jumbling word combinations, etc. For the results they'll tell you what's available and what's not.
Be aware! The above services don't necessarily have the best or lowest cost domain registrar services. For that I recommend GoDaddy (www.wilsonweb.com/afd/godaddy.htm), the one I currently use.

Domain Names as Branding

Don't forget that your domain name automatically becomes your brand-name, whether you intend it or not. It will forever after affect how your company is perceived.
The time you take to select your name is time well spent. The money you spend for marketing or branding consultants to help you get the very best name is a good investment, too.

How to Use Network Behavior Analysis Tools

 Network behavior analysis tools can help tune operations as well as improve security. Here are five tips for getting the job done.
What's happening on the enterprise network—or more to the point, what's occurring on the network that should not be—is a major concern of security executives. If someone is trying to hack in, or a virus or worm is spreading, or a denial-of-service attack is underway, there might be evidence of these types of activities before they become a major problem.
Network behavior analysis (NBA) technology helps organizations detect and stop suspicious activity on corporate networks in a timely manner—possibly preventing, or at least limiting, serious damage from attacks. NBA is designed to give security managers a level of network visibility they need in order to make sure security threats are quickly identified and remedied.
The products analyze network traffic through data gathered from devices such as IP traffic flow systems, or via packet analysis. They use a combination of signature and anomaly detection to alert security and network managers of any activity that appears to be out of the norm, providing a view of the network that lets managers analyze activity and respond before there's damage to systems and data.
"A key benefit of NBA systems is the [network] visibility that they provide," says Lawrence Orans, research director at Gartner, who leads the firm's NBA coverage. Orans says this visibility helps in two areas: network operations (for example, troubleshooting and performance) and security (i.e. malware monitoring and detecting unwanted applications).
NBA can be used to detect behavior that might be missed by other security technologies such as intrusion prevention systems (IPS), firewalls and security information and event management (SIEM) systems, according to Gartner. Those technologies might not identify threats that they are not specifically configured to look for. Gartner says NBA is suitable as a complementary technology to intrusion detection and prevention software, which is effective for addressing network attacks that can be positively identified.
Vendors addressing the network behavior analysis market include many of the broader, established network and security companies as well as niche players that specialize in the technology. Those that focus specifically on NBA are Arbor Networks, Lancope, Mazu Networks and Q1 Labs. Companies including Cisco Systems, Internet Security Systems (part of IBM), NetFort Technologies, Sourcefire and Securify (to be acquired by Security Computing) also offer products with some type of NBA capabilities.
Among the common functionality and features of behavior analysis systems are the use of network flow data to identify suspicious behavior on the network and where it's coming from; mitigation to stop malicious activity and fix network problems; and reports on all network configurations and user behavior.
Orans says some NBA vendors are enhancing their products by adding identity capabilities. "Specifically, some vendors have added the ability to map a user [identification] to an IP address," he says. "This provides the benefit of quickly identifying a user who is responsible for anomalous or malicious traffic." So, instead of being notified that a particular IP address is exhibiting anomalous behavior, a manager can know exactly which user in the organization is conducting the anomalous behavior.
"This is especially valuable for forensic analysis," Orans says. "If you are using an NBA system to analyze a breach that occurred in the past—maybe three months ago—then it is often difficult to map the IP address, which is assigned dynamically, to a user. It's difficult unless your NBA system can do it for you.
Before deploying NBA, security managers need to figure out which system is a good fit for their network and how best to use the technology. Here are five tips on evaluating, purchasing and implementing NBA offerings.
1. Before putting in NBA, first deploy intrusion prevention technology.
"NBA systems are best for organizations that have already implemented IPS systems" and are looking for more visibility into their network and network traffic, Orans says. "NBA is not something that you do before IPS or instead of IPS. It is done afterward because it provides visibility."
After successfully deploying IPS and firewalls with appropriate processes for tuning, analysis and remediation, consider adding behavior analysis to identify network events and behavior that are undetectable using other techniques, Orans says. He notes that the size of an organization does matter when it comes to NBA.
"NBA is for large enterprises, it's not for SMBs," Orans says. "The expertise and experience level needed to tune an NBA solution and interpret its results is beyond most SMB network and security professionals."
2. Conduct a thorough analysis prior to selecting a vendor's offering.
It might sound obvious, but NBA systems can cause more harm than good if they're not carefully selected based on the needs of the organization, existing network components, level of in-house expertise, etc.
When evaluating NBA systems, make sure they meet the organization's requirements for analysis and reporting, and can be integrated with existing networks. Also, consider how easy or difficult the system is to calibrate and use.
"Think of all the devices you need to collect flows from," says John Kindervag, senior analyst, security and risk management, at Forrester Research in Cambridge, Mass. "Will they all support sending flows? Will enabling flows on the device negatively impact its performance?"
Depository Trust & Clearing Corporation (DTCC), a New York—based firm that provides clearing, settlement and information services for a variety of financial instruments including equities, corporate and municipal bonds, and government and mortgage-backed securities, evaluated several NBA vendors and reviewed market research on the technology within its security department, before selecting a product from Mazu Networks, says Neil Wasserman, vice president, Core and Smart Network Services at DTCC.
"We installed a Mazu demo and ran it through a rigorous evaluation," Wasserman says. "The product met our requirements—and the rest is history."
3. Test before broad rollout.
Experts say it's important to thoroughly test an NBA system before moving ahead with a full-scale implementation. That way, security managers can see what kind of actual reporting they will get on network activity.
"The only way to properly evaluate the tools [is] to install them in your live production network," Kindervag says. "Any other evaluation methodology, lab, etc., will not provide true results."
AirTran Airways, Orlando, Fla., a low-fare airline designed for business travelers, had vendor Lancope conduct an onsite proof-of-concept trial of its StealthWatch product before the system was rolled out broadly, says Michelle Stewart, manager of information security at AirTran. The proof-of-concept "had no impact [on] our production environment and demonstrated the effectiveness of the reporting."
During the implementation, AirTran worked closely with a Lancope engineer and deployed the system according to Lancope best practices, Stewart says.
AirTran's security team uses StealthWatch for network monitoring, reporting and forensics. The network team uses the system to troubleshoot behavior-related network issues, Stewart says. Managers can examine granular data about network behavior by zone, node and user, and collect historical data to view trends.
4. Tune NBA systems to cut down on false positives.
Experts says it's important to take the time to effectively tune NBA systems to gather relevant network data and help reduce false positives.
If an organization fails to fine-tune NBA systems adequately, it might have to contend with a lot of false-positive readings that overburden the network and security managers who need to examine all the alerts.
"We did this tuning exercise immediately upon implementation, and it proved extremely valuable," Stewart says. "After segregating our network geographically and logically into zones, we examined the behavior within our high-risk zones for volume and type of traffic. In several cases, the port/protocol information we were given from our application vendors was found to be incomplete, but by using StealthWatch we were able to properly fingerprint the application behavior."
After tuning the zone behavior policies appropriate to the high-risk zones, "our alarm count was much more manageable and useful," Stewart says. "This information also allows us to properly plan WAN bandwidth growth, as we can determine how much legitimate network traffic is required for business."
5. Use NBA data to help determine network usage patterns.
Stewart says it was important that AirTran managers spend as much time as necessary reviewing the behavior data gathered to appropriately classify zones, zone policies and services.
"We discovered a great deal about ports, protocols and chattiness of third-party applications during this exercise," Stewart says. "Our zones include geographic segregation, allowing both security and networking to quickly review and treat WAN location issues. We defined server zones by behavior, allowing more granular control over alerting."
Also, in using NBA systems, it's important to create focused views and logical groupings within the tool that make sense, says Wasserman. "Strive for ease of use and an easy understanding of common sense nomenclature and device or host groupings," he says. "Limit the number of flows that need to be queried or viewed" in order to get useful network information on a more timely basis.
That way, NBA can provide not only greater network visibility, but an effective way to deal with trouble when it arises.

Firewall audit tools: features and functions

Why would anyone need firewall audit software? If you're already jugging hundreds of rules on multiple firewalls, here's what these tools can do for you.

Firewall audit tools automate the otherwise all-but-impossible task of analyzing complex and bloated rule sets to verify and demonstrate enterprise access controls and configuration change-management processes.
Although the market has been driven by compliance—it was essentially created by PCI DSS—these tools can also allow organizations to improve network performance, reduce downtime, improve security and reassign staff from shooting down firewall issues and analyzing configurations to taking on tasks that help grow the business.
The problems are familiar to organizations of all sizes—from those with just one or two overtaxed and inefficient firewalls, to large, distributed enterprises with scores or hundreds of firewalls administered by many business units, often all following different policies that may have been written before the units' acquisitions.

Also see Firewall audit dos and don'ts for practical implementation advice

Not long ago, 200-300 rules was considered excessive. Now, it's not unusual for firewalls to have many hundreds or even thousands of rules, many of which were rendered obsolete when IT operations added new rules to meet business requests but neglected to remove any old ones. Analyzing configurations for a few firewalls, let alone hundreds, has grown beyond the capacity of human computation.

Firewall Audit Tools: Key Benefits and Use Cases

Business efficiency and security may be the goals, but regulatory requirements frequently open up the budget. The firewall audit market, pegged by Forrester Research at $25 million to $30 million in 2009, is fueled by PCI DSS requirements to review firewall and router configurations every six months. These controls also typically come under scrutiny during internal, partner and other regulatory audits.
Enterprises exhaust countless man-hours analyzing firewall and router configurations to produce audit reports, only to realize that they do not have a firm grasp on their network access controls and the change-management processes that enable them.
"How do you demonstrate that a 2,000-rule set is robust and secure?" says a security officer for a telecommunications company, which uses SkyBox Security's SkyBox Assure solution. "It's impossible to do manually."
These automated tools run complex algorithms that evaluate the actual rules against corporate policies and best practices to identify gaps, verify changes and produce audit reports. They enable organizations to verify and document the entire configuration-management lifecycle to demonstrate to auditors that practice follows policy, and that changes were completed as authorized and grant the intended access.
"There's nothing more embarrassing or devastating to an organization than when you tell an auditor, 'This is how we do it,' and when they look, there is no semblance of what you said," says Jeff Sherwood, principal security strategist for H&R Block, a Secure Passage customer. "Now we can come out of the gate and say, 'This is what we do and here is proof we do it.'"
While compliance automation may be sufficient justification for their implementation, firewall audit tools also offer tangible business benefits that go beyond surviving the audit ordeal.
Performance and Optimization: This is a prime function of all these tools. Firewall performance degrades because excessive rules eat up CPU cycles, and critical access rules are situated too far down in the hierarchy because when additions were made, the focus was on speed of implementation, rather than on optimizing the configuration. Firewall audit tools clean up redundant rules and requests for service that have already been enabled, and flag rules that apply to objects that are no longer in use or even in existence.

Also see SIEM Dos and Don'ts

Optimizing firewalls and network devices can improve performance problems that companies might otherwise have had to throw new hardware at. Benefits will be even more noticeable as traffic increases.
Business Continuity: Performance and optimization issues can seriously slow or even bring down critical business processes. This costs the business not only revenue, but also the man-hours it must spend to deal with the problems.
"Before, our team was heavily weighted—30 percent of their time—to firefighting, toward fault analysis and fault fixing," says Colin Miles, corporate network manager for U.K.-based Virgin Media, a Tufin Technologies user with a network infrastructure that includes more than 100 firewall pairs. "Since Tufin was implemented, that's turned to proactive capability, rule-based efficiency and optimization of the network, driving toward people savings."
Security: Complex configurations make security analysis very difficult. Obsolete or misconfigured rules can be exploited to give attackers access to sensitive data. Firewall administrators under pressure to fulfill business requests are likely to err on the side of granting too much access rather than too little. Firewall audit tools improve security by determining optimal rules and detecting unused and misconfigured rules.
Firewall Upgrade and Migration: Upgrading firewalls and consolidating onto fewer platforms create excellent opportunities for organizations to use an audit tool. It's a good time to cost-justify configuration cleanup and firewall optimization, rather than carrying over the old infrastructure's issues. Since these products support multiple firewall platforms, they are well-suited for consolidation, streamlining the configurations on each and translating them onto the new platform. Virgin Media, for example, consolidated from numerous legacy platforms brought over through corporate acquisition to Check Point firewalls for its dynamic environments and Cisco for more static conditions.
Change Management: Change-management policies and processes can fall short when requests are made out-of-band, which happens when either someone fails to follow procedure or there's an urgent need to enable or restore service for critical business processes. Several vendors have complementary workflow products that automatically document all configuration changes and reconcile them with ticketing systems.

Firewall audit dos and don'ts

Firewall audit products are maturing, but the product class is still a relatively young, small market, defined by compliance requirements. You have a fairly limited choice of vendors, including Tufin Software Technologies, AlgoSec, Secure Passage and Athena Security, which all come with firewall audit pedigrees, and RedSeal Systems and Skybox Security, which are primarily vendors of risk-mitigation tools, and so go beyond firewall audit to feature sophisticated risk-assessment and risk-management capabilities.
Take the time to define your requirements, narrow down your choices and put candidates to the test.

See the companion article Firewall audit tools: features and functions on CSOonline.com

DO look at platform and device coverage. These products generally support all the major firewall vendors and some others, as well as major network devices, so you should be covered. Take both present and future needs into account. For example, you may run a single platform across the organization now, but future acquisitions may run on other vendors' infrastructures. These tools should be able to help whether you plan to migrate onto a single platform or continue to manage several while still realizing the efficiencies they promise. See if the vendor has a software development kit that can allow it to integrate with unsupported platforms.
Check that coverage for network devices is included. There are a couple of considerations here. First, it may be important to you to clean up and optimize access control lists on your routers, and second, routers are increasingly featuring more built-in security capabilities.
DON'T overlook scalability. Those vendors that focus largely on enterprise deployments claim they can scale up to thousands of devices. Determine what that actually means in terms of management and the ability to perform under stress.
"In addition, the magnitude of environment brings huge demands on technology and methods that can be used," says the telecommunications company security officer. "What in a smaller company can be rock solid may not be applicable in a big environment. You need be cautious about the limitation of technology."
Choose with growth in mind. Even if a product scales to your current requirements, how well-suited is it to meet greater demands as the business grows, services are added, acquisitions are integrated and traffic increases?
DON'T buy more than you need. Some of these products are aimed at complex, heterogeneous environments with hundreds of firewalls and network devices. Measure the tool's capabilities and cost against your environment. If your firewall environment is relatively simple and static and your traffic is fairly predictable, choose a less-expensive product that you can apply initially for your optimization project and periodically to keep your firewalls under control.
DO put these products to the test once you narrow your choices to those that claim to meet most of your requirements.
"Pick two or three of your favorites and bake them off in real-world situations," says John Kindervag, senior analyst at Forrester Research. "The nice thing about firewall-auditing products is that you can test them on a live production environment because they are passive tools."
Kindervag recommends testing how well they do at finding unused rules, optimizing configurations and so on, then comparing reports.
"Run the results by your firewall guru or bring in one who can say, 'Yes, that's a good rule change,'" he says.
You can also determine whether they actually scale and deliver analysis at the speeds they claim and what kind of hardware they'd require.
DO determine your reporting requirements and evaluate the products' capabilities accordingly. Audit reports should come first and foremost for most organizations. Evaluate the quality of summary reports—are they sufficient to prove that your control policies are, in fact, carried out?
Also, make sure that you can produce satisfactory reports on demand in response to specific auditor queries. Some products offer regulation-specific reports, usually for PCI DSS, which may be useful.
Since these are management tools, you'll want to see useful operational reporting that quickly lets you see what has been done and what needs to be addressed. Make sure the reports deliver the information you want at the level of detail you need. For example, rule usage can change over time. A rule that was optimally placed at first may become a bottleneck as it's hit with more and more traffic, and may need to be moved up in the hierarchy.

Also see How to Use Network Behavior Analysis Tools

Finally, high-level reports can demonstrate overall improvements in efficiency and security, as well as highlight which business units may be lax in properly managing their networks.
DO consider workflow integration. Most vendors offer complimentary workflow products to integrate their core capabilities with change-management workflow tools, such as ticketing systems. This may not be important if your organization has a well-defined process and supporting tools, either homegrown or commercial. But some companies find this capability useful in automating their change-management programs.
DON'T give short shrift to hardware, especially if you are running one of these products in a virtual environment in which resource-sharing may be an issue.
Make sure you have enough CPU and memory muscle to support the product under live conditions, and make provisions for growth as traffic increases.
Alternatively, you could go with one of the three appliance-based solutions Tufin offers in addition to its software.
DO review and refine your policies and procedures before buying and deploying a firewall audit product.
Enterprise IT governance and information security is built on well-defined policies and processes. Technological tools reduce error, improve efficiency and automate analysis that frustrates manual efforts, but you won't get their full benefit if you are simply throwing technology at a problem. Every organization is different, but here are some basic guidelines:

• Examine corporate practices and procedures across business groups and departments. Make sure they can be applied across the organization while allowing for acceptable deviations to meet specialized needs.
• Create a process that is documented at each step and holds each stakeholder accountable.
• Where possible, express requests in terms of business need, rather than in narrow IT terms.
• Have a team that evaluates requests in terms of adherence to corporate policy.
• Conduct both business- and technology-based risk assessments. Implementation should be dependent on passing the risk assessment.
• Test implementation for final sign-off by both IT and the business owner.
• Document.
• Rinse and repeat.

Penetration tests: 10 tips for a successful program

Penetration tests need to accomplish business goals, not just check for random holes. Here's how to get the most value for your efforts.

Why are you performing penetration tests? Whether you're using an internal team, outside experts or a combination of the two, are you simply satisfying regulatory or audit requirements, or do you actually expect to improve enterprise security?
We asked penetration testing experts for guidance on how to improve your program to get the most benefit for your time, money and effort. If you turn to outside expertise, their advice will show you what to expect and demand from consultants. The following 10 tips will show you understand the goal and focus of your testing; develop effective testing strategies; make effective use of your personnel; and make the most effective use of pen test results to remediate issues, improve processes and continuously improve enterprise security posture.

Penetration Test Tip 1: Define Your Goals

Penetration testing—really, all information security activity—is about protecting the business. You are taking on the role of attacker to find the vulnerabilities and exploiting them to determine the risks to the business and making recommendations to improve security based on your findings. Attackers are trying to steal your data—their techniques are a means to an end. So too, penetration testing: It's not about the cool technical things you can do to exploit a vulnerability; it's about discovering where the business risk is greatest. "If can't express things in terms of my business, you're not providing me value," said Ed Skoudis, founder and senior security consultant at InGuardians. "Don't tell me you've exploited a vulnerability and gotten shell on that box without telling me what that means for my business."
Also see Network stress test tools: dos and don'ts on CSOonline.com
With that understanding, from a more tactical perspective, penetration testing is a good way to determine how well your security policies, controls and technologies are actually working. Your company is investing a lot of money in products, patching systems, securing endpoints etc. As a pen tester, you are mimicking an attacker, trying to bypass or neutralize security controls.
"You're trying to give the company a good assessment if their money is being well spent," said Alberto Solino, founder and director of security consulting services of Core Security.
The goal should not be to simply get a check box for pen testing to meet compliance requirements, such as PCI DSS. Pen tests should be aimed at more than discovering vulnerabilities (vulnerability scanning should be part of a pen testing program but is not a substitute). Unless the testing is part of a sustained program for discovering, exploiting and correcting security weaknesses, your money and effort will have gained you at best that check mark, and at worst, a failed audit by a sharp assessor.

Penetration Test Tip 2: Follow the data

Organizations have limited budget and limited resources for pen testing, regardless of whether you are conducting internal tests, hiring outside consultants or using a combination of both. You can't conduct penetration tests across your entire IT infrastructure, spanning hundreds or thousands of devices, yet pen testers will often be told to try to compromise devices across an extensive range of IP addresses. The result is likely to be the most cursory of testing regimens, yielding little or no value. You can't even expect to conduct vulnerability scans and remediate flaws across a very large number of devices in a reasonable amount of time and at reasonable cost. "In many cases customers have thousand of IP addresses they want us to pen test," said Omar Khawaja, Global Products Manager, Verizon Security Solutions. "We could run vulnerability tests and see what's most vulnerable, but they may not be the most important to your organization."
Step back and ask, "What am I trying to protect?" What critical data is at risk: credit card data, patient information, personally identifiable customer information, business plans, intellectual property? Where does the information reside? Do you even know every database, every file repository and every log store that contains sensitive data? You may not know, but chances are an attacker will find it.
So, the first critical step is to narrow the scope of pen testing is data discovery: determining which sensitive data is at risk and where it is. Then the task is to play the role of attacker and figure out how to get at the prize. (Read Red team versus blue team for more ideas on this approach.)
"The idea to mimic what a real attacker will do during time frame agreed to with the customer," said Core Security's Solino, "not to find all the possible problems."

Penetration Test Tip 2: Follow the data

Organizations have limited budget and limited resources for pen testing, regardless of whether you are conducting internal tests, hiring outside consultants or using a combination of both. You can't conduct penetration tests across your entire IT infrastructure, spanning hundreds or thousands of devices, yet pen testers will often be told to try to compromise devices across an extensive range of IP addresses. The result is likely to be the most cursory of testing regimens, yielding little or no value. You can't even expect to conduct vulnerability scans and remediate flaws across a very large number of devices in a reasonable amount of time and at reasonable cost. "In many cases customers have thousand of IP addresses they want us to pen test," said Omar Khawaja, Global Products Manager, Verizon Security Solutions. "We could run vulnerability tests and see what's most vulnerable, but they may not be the most important to your organization."
Step back and ask, "What am I trying to protect?" What critical data is at risk: credit card data, patient information, personally identifiable customer information, business plans, intellectual property? Where does the information reside? Do you even know every database, every file repository and every log store that contains sensitive data? You may not know, but chances are an attacker will find it.
So, the first critical step is to narrow the scope of pen testing is data discovery: determining which sensitive data is at risk and where it is. Then the task is to play the role of attacker and figure out how to get at the prize. (Read Red team versus blue team for more ideas on this approach.)
"The idea to mimic what a real attacker will do during time frame agreed to with the customer," said Core Security's Solino, "not to find all the possible problems."

Penetration Test Tip 3: Talk to the Business Owners

Work with the business people. They know what is at risk—what data is critical, what applications create and interface with that data. They will know at least the more obvious places in which the data resides. They will tell you which applications must be kept up and running.
You'll learn much of what you need to know about the threat level associated with particular applications, the value of the data and the assets that are important in the risk equation.
An important part of this process is to work with people who understand the business logic of the application. Knowing what the application is supposed to do and how it's supposed to work will help you find its weaknesses and exploit them.
"Define the scope that includes critical information assets and business transaction processing," said InGuardians' Skoudis. "Brainstorm with the pen test team and management together."
Skoudis also suggests asking for management to give their worst case scenario, "what's the worst thing that could happen if someone hacks you?" The exercise helps scope the project by determining where "the real crown jewels" are.

Penetration Test Tip 4: Test Against the Risk

The value of the data/applications should determine the type of testing to be conducted. For low-risk assets, periodic vulnerability scanning is a cost-effective use of resources. Medium risk might call for a combination of vulnerability scans and manual vulnerability investigation. For high-risk assets, conduct exploitative penetration testing. For example, the security director for a large university said they started performing pen testing to meet PCI DSS requirements. Once that program was in place, it became the model for testing a potential attacker's ability to penetrate their systems. The university classifies data as public, internal, sensitive and highly sensitive.
For information that's highly sensitive, we perform pen testing under much the same guidelines as PCI," he said. "We back off from there, based on some specific criteria and some subjective judgment that goes into what level of pen testing, if any, will be done for system."
So, for example, on the lower end of the risk spectrum the university will test a random sample of systems and/or applications, depending on criteria for a particular category and time and budget constraints. With tens of thousands of devices on a campus network, even a low-level scan of all of them would be infeasible.
"You can test on a business system that has a clear owner and systems administrator," he said. "But when you have 3,000 Wiis attached to the network, you don't want to scan those and figure out who they belong to."

Penetration Test Tip 5: Develop attacker profiles

Your pen testers need to think like and act like real attackers. But attackers don't fit into one neat category. Build profiles of potential attackers. External attackers may have little or no knowledge of your company, perhaps just some IP addresses. They may be former employees or work for partners or service providers and have considerable knowledge of the inside of your network. An insider may be a systems administrator or DBA with privileged access and authorization and knows where critical data resides.
Motive is a factor in developing profiles. Is the attacker after credit card numbers and PII that can be turned into cash? Intellectual property to sell to a competitor or gain a business advantage? The attacker may be politically/ideologically or competitively motivated to bring your Web application down. He may be an angry ex-employee who wants to "get back at the company."
Work with business owners to help fashion these profiles and learn what types of potential attackers they are most concerned about.
The profile narrows the focus of the pen testing, and tests will vary based on each of these multiple profiles.
"We get a snapshot of what a particular attacker can do against a target, and we don't mix results," said Core Security's Solino. "For every profile, we get the result of the pen test and do another profile."

Penetration Test Tip 6: The More Intelligence the Better

Information gathering is as much a part of the process as the actual exploit—identify devices, operating systems, applications, databases, etc. The more you know about a target and its connected systems, the better chance you have of breaking in.
Each step may yield valuable information that will allow you to attack another asset that will eventually get you into the target database, file share etc. The information will allow you to narrow the search for exploitable vulnerabilities. This reconnaissance is typically performed using automated scanning and mapping tools, but you can also use social engineering methods, such as posing as a help desk person or a contractor on the phone, to gather valuable information.
"We're increasingly starting to do social engineering," said Verizon's Khawaja. "It's essentially reconnaissance—performed with the permission of the customer—to let us find everything in the environment that could assist us in breaking in."
Multi-stage penetration testing typically is a repeated cycle of reconnaissance, vulnerability assessment and exploitation, each step giving you the information to penetrate deeper into the network.

Penetration Test Tip 7: Consider All Attack Vectors

Attackers can and will exploit different aspects of your IT infrastructure, individually or, frequently, in combination to get the data they are seeking. Thorough pen tests will leverage any and all of these potential attack vectors, based on the attacker's end goal, rather than the vulnerability of each.
"A few years ago we would do network penetration testing, and application pen testing and wireless pen testing, and then we stepped back and said 'that makes absolutely no sense," said Solino. "The bad guy doesn't say, 'I can only break into a system using the network.'"
Successful pen tests, like real attacks, may leverage any number of paths that include a number of steps till you hit pay dirt. A print server may not seem particularly interesting, but it may use the same admin login credentials as a database containing credit card information.
"Pen testers find flaws and exploit them, then pivot from that machine to another machine, to yet another," said InGuardians' Skoudis.
An attack on a Web application might fail in terms of exploitation, but yield information that helps exploit other assets on the network. Or an attacker might get information about employees without high privileges, but with access to the internal network that act as a springboard.
Also see How to compare and use wireless intrusion detection systems
So, a critical resource may not be directly assailable, but can be compromised through other systems.
For example, said Khawaja, Verizon pen testers were unable to directly compromise a Web server that had access to a sensitive database. If the testers focused narrowly on testing the Web application on that server, the conclusion would be that the data was safe. But by taking a data-centric approach, they discovered that the Web server was connected to a second Web server, which had a critical vulnerability that an attacker could exploit to gain access to the first Web server and, hence, the database. (Read more about Web application attacks in How to evaluate and use Web application security scanners.)
"We care about anything that isn't cordoned off from the network segment we are targeting," he said. "Are there any network controls to prevent an attacker from jumping from a vulnerable low-value system to a more critical system?"
That being said, there are valid cases for vector-specific testing. For example, a company may be particularly concerned about wireless security, because it knows it has been somewhat lax in this area or may have recently installed or upgraded WLAN infrastructure. But even if you are confident that a particular vector is safe—for example , if the wireless network is isolated from the credit card database—don't be too sure. Attack paths can be complex and byzantine.

Penetration Test Tip 8: Define the Rules of Engagement

Pen testing simulates attack behavior, but it is not an attack. Whether you are conducting in-house testing or contracting with a consultant, you need to establish parameters that define what can and cannot be done, and when, and who needs to know.
The latter depends on whether you are conducting white box or black box testing. In the former case, there's probably an acknowledgement that the security program of the company (or a particular department or business unit) needs a lot of work, and the pen testing is open process known to all involved.
On the other hand, black box testing is more clandestine, conducted more like a real attack—strictly on a need to know basis. You are determining how good the company's people are at their jobs and the effectiveness of the processes and systems supporting them.
"Whether it's the operations center, or the investigative response team or physical security guards, everyone has to pretend it's just another day at the office," said Verizon's Khawaja.
Typically, companies will perform white box testing first to learn the security issues that have to be addressed. Subsequently, black box testing will help determine if the initial findings have been effectively remediated. Sometimes, for example, a CSO will want to know not only how vulnerable critical systems are, but how good their personnel are at detecting and responding to an attack.
In either case, certain key people need to be involved to avoid problems that might impact the business or undermine the testing. At least one person in the target environment who is involved in the change control process should be in the loop, said InGuardians' Skoudis. Under the rules of engagement, for example, the company may permit the pen testers to install software on the target devices to do more in-depth pivoting, but at least that one person has to be involved to make sure that the testers are not stopped by dropping their IP address from a router ACL or invoking a firewall rule.
In both white box and black box scenarios, Skoudis recommends daily briefings with the test stakeholders to let them know what the testers are doing. For example, the rules of engagement may allow the pen testers to exploit vulnerabilities, but the briefing can be used to give folks a heads up that they are about to do it.
"It builds bridges," he said. "It shows the pen testers are not a distant, evil group that is out to 'catch me.' Rather, it's all about transparency and openness."
The rules of engagement also may set limits on what may and may not be exploited, such as client machines, or techniques that may or may not be used, such as social engineering.

Penetration Test Tip 9: Report Findings and Measure Progress

The goal of penetration testing is to improve your security posture, so if you are conducting internal tests, your report should provide useful, actionable and specific information.
"The goal is to help improve security, for management to make decisions to improve business and help the operations team improve security," said InGuardians' Skoudis.
You should provide an executive summary, but the heart of your reporting should include detailed descriptions of the vulnerabilities you found, how you exploited them and what assets would be at risk if a real attack took place. Detail every step used to penetrate, each vulnerability that had to be exploited, and, most important, perhaps, all the attack paths.
"The beauty of identifying the attack path is that it allows you to solve specific problems by breaking the path," said Core Security's Solino.
Be very specific about recommendations. If architectural changes are required, include diagrams. Explain how to verify that a fix is in place (use this command, or that tool to measure). In cases where multiple systems are involved, explain how to mass deploy a fix, using GPOs if possible.
Make sure that each recommended remediation includes a caveat that the solution is thoroughly tested before it is implemented in a production environment. Enterprise IT infrastructure may be very complex.
"This is a huge issue," said Skoudis. "You don't know all the subtleties. You don't want to break production."
Penetration testing should not be a one-time exercise, and successive results should be compared. If you are performing internal testing, put together deltas to measure how your people are addressing issues. If the problems from the last test—or the last two—remain unaddressed, you may have a problem. Perhaps the software patching program isn't working as it should, or developers are not being properly trained to write secure code.
"What we're looking for are trends," said the university security director. "It's just like you would treat an audit report. If you have repeat findings, it indicates you might have a more serious problem."

Penetration Test Tip 10: Decide Who Your Pen Testers Are

The decision to use in-house staff for pen-testing depends on the size of your organization, the value of the information you are trying to protect and where you want to put your internal resources. A company may have a dedicated pen testing team or a group within the security team. An internal team is in a better position to conduct regular testing. If your organization is large and distributed, create mechanisms and promote an environment in which information can be shared.
"If have internal community that can share information, make sure they have a strong knowledge base backed up by mature knowledge management systems," said Verizon's Khawaja . "You want to make sure that what happened in your Beligian unit doesn't happen in Brazil."
Even if you do some in-house testing, there are good reasons for hiring consultants to perform at least some of the work. Some regulations require external companies to perform pen tests; consider that insiders may have too much information about the target systems, as well as a vested interest in the outcome. So, beyond compliance requirements, it's a good idea to bring a fresh view from the outside periodically.
For the same reasons, if you do hire outside testing consultants, rotate among vendors, just as would with auditors every few years.
"Bringing in outside people gives an added degree of confidence in the results," said the university security director. "There's no perception of conflict of interest."—
For your internal team, look for the right blend of knowledge and curiosity.
A good training candidate, said Core's Solino, has a strong knowledge of networking and application protocols as a foundation. Mostly, he looks for curiosity and a hacker mentality.
"It's IT knowledge and that attitude, a specific mindset that denies something is secure and says, 'Go for it!'"
"This is an art," said Skoudis. "Although there are tools and methodologies, you have to be creative in finding problems in target systems and applications."