Along with triage and a strong talent pool of security researchers to draw on, customer success management (CSM) is an indispensable pillar underpinning successful Bug Bounty Programs (BBPs).
We quizzed the head of YesWeHack’s CSM team, Selim Jaafar, about the CSM role, the keys to a flourishing BBP, and how his team ensures programs continually evolve to meet their security goals.
Selim joined YesWeHack in 2019 as our first customer success manager after roles in security consultancy, project management, communications and customer support.
How does the CSM team typically ensure a successful BBP launch?
Selim: It’s about finding an optimal balance in terms of scopes, rewards and rules. You want consistent results that build a use case for Bug Bounty as well as giving customers practical knowledge about running a program effectively, while being conservative enough to avoid a ‘big bang’ effect that overloads the customer with vulnerability reports or rapidly exhausts the budget.
It’s also important to help the customer adopt some basic good practices, notably with regards to collaboration with hunters. Pre-launch, we familiarise customers with:
- How Bug Bounty works and the pitfalls to avoid
- Program types and their requirements and pros/cons – such as public or private; grey-box or black-box; focused or wildcard (all assets in scope)
- Processing vulnerability reports and communicating with bug hunters to avoid miscommunications, mishandled reports or disputes
- Using the platform to build strong processes and reduce time-to-fix and time-to-payout
We also help the customer identify potential scopes; calibrate rules and rewards; finetune the program description for maximum clarity and fidelity to the customer’s goals; set up roles and workflows; shortlist prospective hunters; and brief the triage team.
To summarise: we ensure that the program reflects customer objectives, and is sufficiently detailed and attractive to hunters so as to promote their engagement and adherence to the rules.
How does the CSM support the customer post-launch?
Selim: The CSM guides the security team in handling the first tranche of reports, and setting high standards in terms of communication, qualifying reports, managing delays and so on.
These early reports often generate many practical and theoretical questions about how best to handle communications with both triage and hunters, as well as considerations on platform functionality. So the training process, which identifies and tackles real-world problems and builds internal knowhow, continues beyond launch.
Initial reports can vary a lot in terms of quantity, quality and diversity. As CSMs, it’s important to evaluate our first impressions. Are the results consistent with expectations? And how were the results processed behind the scenes? This sets the tone for the security team’s capacities and constraints.
We can then give better directions to the customer, whether it’s to refine, slow down, extend or boost the program. We leverage our expertise to help the customer optimise their metrics in tune with their ambitions.
We often propose tailored program development plans that enable the organisation to harness the community’s talents in an achievable and productive way. We are then proactive in regularly reviewing and optimising the plan in collaboration with the customer to ensure it continues to align with their goals.
How can friction in customer-hunter relationships arise and how can you or the customer avoid or mitigate these problems?
Selim: Disputes typically arise due to miscommunications, response times, and divergent CVSS assessments or interpretations of scopes and program rules. There are also challenging edge cases that are invariably addressed by frank and friendly communication, and an empathic and constructive mindset (on both sides).
We intervene at different junctures to varying degrees, depending on the situation, either to pre-empt a problem, resolve a dispute or draw lessons to prevent recurrences of the problem.
We continuously strive to prevent problems with effective training, by instilling best practices and by encouraging clear program specifications. Nevertheless, there are always unanticipated challenges – and it’s our duty to help both parties find common ground and sensible solutions.
The success of Bug Bounty Programs ultimately hinges on mutual trust. As a man-in-the-middle, it’s up to us to set standards for improving the framework, rules and processes that can be leveraged to prevent and settle disputes.
What are the most common mistakes made by customers?
Selim: Most misconceptions or mistakes come from transplanting traditional offensive security approaches to the Bug Bounty framework. Yes, Bug Bounty has similarities to pentests, but Bug Bounty success particularly hinges on engendering mutual trust and carefully incentivising and encouraging the engagement of hunters.
While we strive to acculturate the customer to the crowdsourced testing model during the launch phase, we sometimes have to help them remedy issues such as:
- Scopes being unaligned with testing goals. For example, testing a mobile app without considering webservices/APIs; a complex scope fragmented into small pieces so the hunter cannot develop a consistent attack scenario; or insufficient detail about the scopes that allows for misinterpretation – all of which can lead to inconsistent results and problems with out-of-scope interpretations.
- Inappropriate reward grid. For instance, applying low to average rewards to mature, complex and hardened scopes, or vice versa. Offering low or zero rewards to medium severity issues is also detrimental, especially when such vulnerabilities can be chained for higher impact and can help hunters familiarise themselves with the program and its scopes.
- Unsuitable testing conditions. These typically arise from a mismatch between expectations and means. If your objective is to find complex vulnerabilities, for instance, then black-box testing a web application is usually unproductive.
- Misconceiving hunters as adversaries. It’s legitimate to worry that the goals of hunters and customers might conflict. However, hunters are in reality incentivised to pursue the same primary objective as customers: unearthing vulnerabilities with real business impact. By understanding hunters’ motivations and pain points, customers will see clearly how an open, honest and collaborative relationship – within, of course, the framework of clearly defined rules of conduct – is necessary to produce the best results and avoid problems.
To prevent such issues surfacing, the CSM team must be detail-oriented in managing customer expectations and helping the customer continuously optimise rules, scopes and testing conditions in line with their goals.
How has the CSM operation improved during your more than six years at the helm?
Selim: We’ve come a long way not just as a company but as a team too. We have gained a lot of useful experience, as have many of our customers, some of which we’ve been working with for several years.
We’ve developed all kinds of programs, with various setups, strategies and learning curves. Over time our team has grown, we’ve refined our processes, and we’ve built a set of best practices. However complex the customer’s testing needs, and whatever their sector, development model or security maturity, we feel we can handle it.
Finally, our job is fundamentally about providing human, expert support. However, we continually seek ways to automate and streamline processes – which frees us up to focus on delivering best-in-class support.