How data science drives value for private equity from deal sourcing to post-investment data assets

Jaclyn Rice Nelson

Data science can bring tremendous value to the world of private equity (PE). From investment sourcing to due diligence and analyzing post-investment data assets, the range of challenges is matched by the rich data and potential for enormous impact.

This explains why such projects are a favorite among Tribe AI’s community of data scientists and ML engineers, and I am not immune. I was particularly excited to sit down with legendary data scientist and Tribe AI advisor Drew Conway, head of data science at Two Sigma Investments, to talk all things data and investing, including:

  • How PE investors and PE-backed companies can use data to build a competitive advantage
  • Why PE is one of the most exciting places to apply data science

Here are some highlights from my recent fireside chat with Drew Conway.

In private equity, there’s a huge gap between leveraging data science and being data-driven

Often we think that traditional, well-established PE firms aren’t using data, but that’s just not true. They’re all leveraging data. Real estate and PE in general are data-rich. Firms know that and use it—but for most, their use comes post hoc. They’ve used their traditional approach—relying on intuition, their networks, and boots on the ground—to identify an opportunity and to diligence that opportunity. It’s only then that they begin to use data.

Not only is this approach ripe for confirmation bias, but it’s simply not a data-driven approach. Data isn’t helping them identify new opportunities or evaluate whether one opportunity is better than another. This is where most of the opportunity lies in private investing and where firms like Two Sigma have found tremendous value.

Data science can broaden the investing funnel and surface the most attractive opportunities

When it comes to data-driven investing, the value is about widening the funnel. The worst thing that can happen to an investor is to miss an opportunity. But when you widen the funnel, then you also need to be able to assess how good that opportunity is relative to your strategy. That’s where data science can be incredibly powerful.

There are broad top-down analyses and data science tools that you can build to inform investors’ decisions at the strategic level. And there are much more bottom-up analyses that you can do to help inform their decisions at a tactical level.

The thing they all share is that there’s always a human in the loop—it’s really about augmenting investor decision making with data. Market selection is an example; it’s really about helping investment teams focus their time on the areas that will have the highest impact.

Public data holds a huge opportunity for forecasting

Investment firms heavily leverage public data. For example, labor market data can help them understand why certain labor markets are growing. Consumer behavior and demographic data like U.S. Census data can help them understand how consumers interact with products. Combining all these data sources together provides a unique view of the market.

Forecasting is a great example. We use data to give investors a long-term view of where the market is going. When you invest in a building, you have to hold it for several years. That means our forecasting model has to account for that range of time, which is incredibly challenging.

The most exciting part is that it’s been proven that you can use data science to scale an investment strategy once you’ve interrogated its bona fides with data. For example, maybe that’s investing in a certain kind of property asset. Data science allows you to rapidly and at scale identify where similar opportunities are in the market. So it’s not just on-off deals in the pipeline anymore. This scaling is what gives newer, data-driven players an advantage over established players.

Data science can impact all aspects of private equity

Data science isn’t just influential in investment decisions; it can have a massive impact on PE-backed companies and how they are run. Like private equity firms, but unlike startups, PE-backed companies are often more mature and therefore have quite a bit of data. This data can prove extremely valuable when thinking about where the best markets are for them to open locations, how to think about marketing and ways to reach their customers, dynamically price their products based on user behavior, or even better understand who their customers are. We’ve seen companies generate huge returns and drive real impact by turning their existing data into insights and optimizing operations.

If data science doesn’t drive business value, it doesn’t matter

There’s so much that data can tell an investor, but ultimately if it doesn’t move the needle, then it’s a slide in a pitch deck and nothing more. The most important thing you can do to ensure a return on your investment in data science is to apply data science to the right problems and understand how insights would actually influence the investing process. Simply put, designing data science solutions is like designing products: It starts with understanding the user.

In private investing, decisions are made in a discretionary way—humans are sitting down and deciding whether to invest in a company. So all the work data scientists do has to have a front-end interface that can be understood by investors, rather than just technologists. This is especially different from the public markets, where models are interacting with the market directly. Data-enabled private equity investing remains human-in-the-loop, so the key to success is ensuring that your data scientists are deeply integrated with investors and marrying their data expertise with the deep domain expertise of the investors themselves.

Private equity investing is one of the most exciting places to be a data scientist

What I love to think about is how data can be used to understand human decision making at scale. Why do groups of people make choices? If that’s something that you have even an inkling of interest in, there’s truly no better industry to be a data scientist. There are so many different types of questions both internally—Why would we value a business one way versus the market?—and externally to understand why certain assets behave in the way they do. All of it is driven by human behavior.

Watch the full fireside chat with Drew Conway, “Using Data to Drive Private Equity: Lessons, Trends, and Opportunities for Data Scientists.”

Want more insights? Join Tribe AI and Scale AI for Applied AI: a series of conversations around how ML is accelerating change across industries. You’ll hear from technical experts at top companies on how they’re using data to drive impact, operationalizing ML solutions, and accelerating adoption across fintech, healthcare, investing, media, and more.

Related Stories

Applied AI

An Actionable Guide to Conversational AI for Customer Service

Applied AI

A Deep Dive Into Machine Learning Consulting: Case Studies and FAQs

Applied AI

Everything you need to know about generative AI

Applied AI

Key Generative AI Use Cases From 10 Industries

Applied AI

What the OpenAI Drama Taught us About Enterprise AI

Applied AI

Why do businesses fail at machine learning?

Applied AI

How to Evaluate Generative AI Opportunities – A Framework for VCs

Applied AI

A primer on generative machine learning models for music production

Applied AI

No labels are all you need – how to build NLP models using little to no annotated data

Applied AI

Understanding MLOps: Key Components, Benefits, and Risks

Applied AI

Key Takeaways from Tribe AI’s LLM Hackathon

Applied AI

How to build a highly effective data science program

Applied AI

10 ways to succeed at ML according to the data superstars

Applied AI

AI in Private Equity: A Guide to Smarter Investing

Applied AI

Machine Learning in Healthcare: 7 real-world use cases

Applied AI

AI in Banking and Finance: Is It Worth The Risk? (TL;DR: Yes.)

Applied AI

How 3 Companies Automated Manual Processes Using NLP

Applied AI

8 Ways AI for Healthcare Is Revolutionizing the Industry

Applied AI

A Guide to AI in Insurance: Use Cases, Examples, and Statistics

Applied AI

AI in Construction in 2023: Use Cases and Benefits

Applied AI

How the U.S. can accelerate AI adoption: Tribe AI + U.S. Department of State

Get started with Tribe


Find the right AI experts for you


Join the top AI talent network

Jaclyn Rice Nelson
Jackie spent the majority of her career at Google partnering with enterprise companies and incubating new products. She was an early employee at CapitalG, Alphabet’s growth equity firm, where she built a fifty-thousand-person expert network and advised growth-stage tech companies like Airbnb on scaling their technical infrastructure, data security, and leveraging machine learning for growth.