Data Demystified, Part 4: Building an efficient data team

The Data Stack has changed substantially in the last few years. So have the data teams that build, deploy, and manage them.

The old strategy of hiring a data engineer first isn’t necessarily a good call for many startups and early-stage companies, especially since modern cloud-based data stack technologies simplify some of the more technical processes, like data transformation and data modeling.

That doesn’t mean there’s no room for data engineers on your data team. What it does mean is that you should rethink past norms if you want to build an effective, efficient data team that can deliver insights, drive action, and deliver value without breaking your budget. 

If you’re a data analytics and business intelligence (BI) leader who’s ready to start building a team, this is the Team AirOps recommended approach.

When is it time for a startup to build a data team?

Like so many questions in the data analytics world, the answer to this one is “it depends,” but here’s a rule of thumb you can use:

If your startup or early-stage company is in the market for (or already has) a data analytics and BI program powered by a modern data stack, there are certain data team roles that can add value to the business. 

Who to hire for your data team

Titles and roles in the data and business intelligence domain can be very vague and there’s lots of disagreement about roles and responsibilities. As an example, “Data Scientist” is often used inconsistently to describe a multitude of roles within a data organization, sometimes even within the same company.

Since data scientist is a vague title, we suggest that startups and early-stage companies consider these three main roles*:

• Data analysts
• Data engineers
• Analytics engineers

* Just kidding, there’s actually one role that we recommend 99% of the time. Still, it’s worth learning about each of these roles so that you can be prepared as your data team grows.

Data analysts synthesize and analyze data to solve problems and identify key insights into the business. They also communicate this information to company leadership and other internal stakeholders (this is often the most difficult part of their job).

Data engineers are responsible for your data infrastructure. Their job is to build the systems that turn raw data into consumable analytics, which includes responsibilities like building data ingestion pipelines and managing data warehouse permissions.

Analytics engineers operate on a continuum – they have technical skills and business acumen. They have a unique ability to move up or down as needed to solve problems. They possess deep knowledge of data modeling and data engineering concepts and can build and maintain a modern data stack. They also understand how the business works and are comfortable collaborating with stakeholders. They’re always wondering, “What is the highest business priority and how am I serving that?”

Here’s how dbt breaks down some of the differences between each role:

The ideal first hire for (almost) any data team

In general, it’s best to hire data generalists, not data specialists. That’s why the ideal first hire for virtually every data team is a skilled analytics engineer.

Many roles and responsibilities within the data team can be replaced or augmented with tooling… but not this one. For example, the conventional work of a data engineer includes building and maintaining ETL pipelines. Now, there are powerful ETL tools that have out-of-the-box connectors for the vast majority of SaaS applications and production databases that businesses use. In our preferred modern data stack, the data transformation piece is handled by a tool like dbt. 

You may eventually decide to hire for additional data team roles, but you can get really far with an analytics engineer who knows their way around the cloud-based technologies in a modern data stack.

This is a big change – only a few years ago it was common practice for a data engineer to be the first hire on a new data team. 

How an analytics engineer sets a data team up for success

‍

We like to use a crawl → walk → run analogy to illustrate how an analytics engineer can facilitate the success of your organization’s data function:

🍼 Crawling is having your analytics engineer set up foundational data models

🚶‍♀️Walking is creating mechanisms that enable business users to work with data.

⚡️Running is doing the “cool stuff,” like building machine learning (ML) models and integrating artificial intelligence (AI) into your product.

The right analytics engineer will help you move through the crawling phase and into the walking phase, where the real magic happens: When business users can work with data, it allows operations teams, bizops teams, and other functions to self-serve many of their own data needs. 

Eventually, these business end-users fill the data analyst role because they have everything they need to be more effective at using data in their jobs. This is how the “great analytics engineer + right tooling” combo can also help you stretch your budget by delaying the hiring of additional data analysts and data engineers. \

Note that ML and AI are super cool technologies that have generated a lot of buzz. However, their impact isn’t significant for most companies, unless they’re a component of the product itself. Don’t worry about running or sprinting to an AI-powered finish line, because that’s not going to unlock wins and efficiencies for your data team. 

The most common mistakes startups make with their data teams

By now you know that you should consider hiring an analytics engineer, but what are some of the major shouldn’ts to keep in mind when it comes to building a data team?

In our collective Team AirOps experience, these are the four most common mistakes that we see: 

1. Setting out to build a complicated ML model or the most comprehensive dashboarding tool/spreadsheet ever seen.
2. Being too business-focused in the beginning and hiring a BI developer when it’s not necessary.
3. Trying to solve every problem in the business at once (much like in data modeling, a modular approach that focuses on core business needs works best)
4. Overhiring.

Each of these mistakes has a common theme: They begin at the end and require you to work backward towards a huge goal, which is problematic for a few reasons:

• It leads to shortcuts, like using a production database as a data warehouse or not building a flexible, reusable data model
• It’s a lot of ad hoc work that’s not reproducible
• It’s also just a lot of work in general
• Sometimes, there just isn't enough data to science

It’s tempting to think about all of the amazing things that your fledgling data team can accomplish, but as one of our favorite analytics engineers once said: “Don’t blow it and try to boil the ocean.” 

This applies to both the engineering and business sides of the fence. For example, trying to solve every business-related problem from the get-go can lead to bad practices like decentralizing data and business logic into BI platforms. Visualization tools are a great place for business users to go for a look at trends in data, but relying too heavily on BI limits the business's ability to interact with and understand the data.

At the risk of sounding like lobbyists for Big Analytics Engineering, just hire an analytics engineer and stick to the crawl → walk approach that we described in the previous section. It’s the easiest way to carve out a path of efficiency and avoid biting off more than you can chew.

Hiring best practices for analytics engineers (and other data team roles)

Hiring is rarely easy, but there are several things that you can do to increase the odds of attracting the right people to your job openings. 

First, start small. Don’t try to fill a bunch of data roles at once. Slow and steady really does win the race here, and it’s generally much more effective to start with one person (like a well-rounded analytics engineer) versus hiring a team right off the bat.

You still need to find the perfect analytics engineer, though. To do that, dbt recommends including these five things in your job descriptions:

• Background on your company, the role, and your data stack
• Eligibility requirements, including technologies and frameworks the candidate will need to know
• Responsibilities of the role with as much specificity as possible 
• The hiring process candidates will go through, including the number of interviews and whether there will be a technical assessment
• A 30/60/90 day plan that explains how you intend to ramp up this new role

Very few job listings include all five of these things. It’s especially uncommon to see those last two suggestions in a job post. And while it may sound like a lot, hitting each point won’t require a dissertation-length description: For examples of solid (and succinct) data analytics job descriptions, check out dbt’s guide to writing data, analytics engineer, and data analyst job descriptions.

For the 30/60/90 day plan and onboarding in general, be sure to be realistic. Most startups, early-stage companies, and other organizations have many analytics-related items on their wish lists. We find that a reasonable, modular approach makes the first 30 days go a lot easier. 

Here are some examples of responsibilities you might include in those early months + detailed guides that outline the steps involved in each one:

• Manage and maintain the organization’s data stack: How to Build a Modern Data Stack 
• Deploy an effective warehousing strategy: Best Cloud Data Warehouses: Top Provider Comparison
• Build foundational data models with the help of dbt: How to Build Your Foundational Data Models 
• Collaborate with stakeholders to build a metrics framework to guide the data analytics function: Building Your Metrics Framework: A 7-Step Guide 

Where to find candidates for your data team

If you want to proactively recruit potential talent, consider getting involved with some of these communities:

• The Locally Optimistic community (they have a Slack channel and a great blog – bookmark this article on how to Run Your Data Team Like A Product Team; it has useful advice on how to frame your data team’s purpose.)
• Dbt’s Slack community
• In-person dbt meetups and other events held around the world

Data analytics and BI are rapidly evolving spaces. If you’re a leader who wants to recruit, hire, and manage an effective data team, you’ll learn a ton from these resources.

Once you have a data team, here’s how to build trust

You’ll most likely be asked to prove the ROI of your data team at some point. Since the ROI of a data team is often measured indirectly and can’t always be tied to dollars and cents, trust is an essential part of the equation. 

You want to build trust as quickly as possible and quick wins are the best way to do it. Quick wins turn your analytics engineer (and the data function as a whole) into a force multiplier that can help people use data to do their jobs more effectively. When your data team can show that it has empowered business users and/or business units to effectively use data to drive positive outcomes, trust is soon to follow. 

A quick win could be driving an outcome that is not game-changing in scope, but that still brings significant value to a part of the business. For example, optimizing sales execution by giving sellers full visibility into their pipeline and allowing them to more prioritize opportunities more effectively. While this is specific to the sales team, it will turn those users into data promoters within your organization. Word-of-mouth is an important tool when building a data-driven culture and quick wins help the message spread organically.

Before you can start racking up the quick wins that will generate trust and lead to a positive ROI, you need to create good communications mechanisms between the data team and business teams. And who’s the perfect person to take on the job of developing those communications mechanisms? An analytics engineer, of course!

Thanks to their ability to oscillate between technical savviness and business acuity, an analytics engineer is the perfect person to lead the charge and develop these important relationships between the business and the data team. 

There are tons of different ways to facilitate communication, so tailor your approach to the company’s internal culture. No matter how you approach it, though, we recommend keeping things simple. 

We aren’t the only ones with this advice, either. The quote below is from an interview for an upcoming AirOps blog on measuring the ROI of your data function. It’s perfect advice for anyone trying to build an efficient data analytics team that collaborates closely with business stakeholders:

“I know this sounds really simple, but having the analyst engineer talk with the business on a regular cadence is the way to get that done. The only people that are going to tell you how valuable your work is, are the people that actually are using it.” - Chris Meier, Senior Analytics Engineering Manager, Bambee

‍

‍