How to build data products for the next decade

To show an example of what’s possible, we spoke with Malcolm Angus, Head of Analytics at OGM. As an SEO agency working with high-growth B2B companies, OGM provides internal and external users with dashboards and self-service tools. Here, we’ll walk through the key takeaways and explore how a small team at OGM builds data products that serve all their varied users.

The data you provide through your product can help your users make more-informed business decisions. However, it’s not enough to just have that data available somewhere in the product—users have to be able to access and use it. With traditional BI, dashboard discovery—that is, finding the dashboard that actually contains the data a user wants—can be a big problem. Analysts might create a bunch of dashboards for the team to view, but as more dashboards get built, knowing where to look for what data can become confusing for the team. As Malcolm put it, “How do you take users who are not trained in a dashboard, let alone 20 different dashboards, and route them to the correct one?” Guiding the user to the insights they’re looking for makes the whole product more useful.

OGM solved their discovery problem by building a search bar powered by GPT-4 that allows users to ask for the data they want. The search passes the query to OpenAI’s API, which then identifies the right dashboard for the requested data and provides a deep link to it. It even applies relevant filters and brings the user directly to the corresponding tab. For example, searching “May vs June 2023 performance considering seasonality” will return their Performance over Time dashboard with the proper date ranges set. Now, their users don’t have to be trained on all their dashboards, or even know if the dashboard they want exists. This tool helps users find the data they need more quickly and lets them see the full value of their data.

Once a user finds the right dashboard, the UI needs to support users in getting what they need from the data in that dashboard. For instance, you might consider customizing your chart designs to make insights clearer. While a basic line chart will display your data, being thoughtful about how the information is displayed can highlight the insights users are looking for. To improve the UI, remember, too, to think about organization. Malcolm joked, “Users would always ask us to add just one more visualization, and next thing you know the dashboard starts looking like a newspaper.” He recommended creating tabbed pages inside your dashboards to help you include many visualizations while avoiding clutter.

In OGM’s case, they built a custom visualization for their campaign performance dashboard to make it easier to spot anomalies. It shows traffic forecasts and the bounds for the expected range, so it’s clear when the observed outcome didn’t match the prediction.

A great product should provide a tailored experience to everyone who uses it, whether they’re the CEO of your company or an external vendor. For data products specifically, you may find yourself with a wide range of users, and those users might have different levels of access to data and varied needs. This means you’ll need to control what dashboards and tools you allow each user to see, and make sure they can access the most relevant things.

In addition to mitigating critical privacy and safety concerns, permission groups are a useful tool for this targeting. OGM uses them to create bespoke experiences for both internal and external users. Their internal employees need access to a holistic view of all their clients’ data in order to understand how OGM is performing as an SEO agency. Their external clients, on the other hand, should only see performance data for their own company. By using permission groups, OGM can ensure users only access the data relevant to their role.

Going one further, the permission groups can support a convenient, efficient experience for users. In OGM’s case, they set up the UX to adapt to the groups the user is in. When Malcolm logs in, the app shows him admin tools and key internal dashboards. When an external client logs in, they might be shown an overview of their metrics.

Modern data stacks can include a variety of sources, and next-generation data products need to be able to connect to all of them, not just traditional databases. For instance, you might want to incorporate information from a third-party source, such as a vendor providing demographic data for a city, without having to first scrape the source to store the data in a database. One way to make it possible to pull in data from all these connections is by allowing your data product to call out to external APIs. OGM shared a particularly clever approach to forecasting based on this ability to call an API.

To create a dashboard of forecasts with a product that can only read from a database, you would have to pre-generate the forecasts in your data pipeline and then pull them into the dashboard. Malcolm warned, “Think about the number of possible forecasts you’d have to produce–hundreds or thousands or more. It’s a waste of resources.” OGM instead built a service on top of Meta Prophet that they could call via API. This allowed them to create forecasts on the fly and then display them in custom visualizations.

In addition to reading, allowing users to also write data opens up new use cases for your products. This lets users analyze their data and act on it from the same place, giving them a complete data story. End users could also enrich their own data if they need to, adding additional information like context or annotations.

For OGM, reading data for visualizations is a core part of their product, but they also use Retool’s ability to write to their databases. They wanted to allow clients to configure the data pipeline, like defining key metrics to track, but quickly hit issues managing configurations. “Our original idea was giving internal employees a spreadsheet where they could, say, configure the goals for a client. That wasn’t the best process, though, and we couldn’t give clients direct access due to privacy concerns.” Now, they’ve created configuration forms inside their app. They’ve also set up input validation and customized the forms to include tutorials directly on the page, making it easier to use and ensuring configurations are correct.

Remember that when you’re building out your data product, it won’t be fully formed right out of the gate. Building quickly and iterating can help your product get more useful with less effort. To start, Malcolm recommended prototyping with your users to help you ensure you’re serving their needs. As he explained, “we just start with a query and a visualization, then go back to the stakeholders and ask how it fits their expectations.” This quick iteration loop moves the product in the right direction without investing effort into larger features that may not be needed.

Then, once you have the core needs addressed, you can start to build out the product around it. Malcolm continued that once they’re ready to think about the product, “that’s when we’ll think about tabs, about filtering, about the URL parameters we want to expose.” For these features, iterating on them and getting feedback from users can guide development toward what they find most useful.

While there’s no telling exactly where the future will lead, you can be ready for it by building data products that are useful to users in a world of abundant data and that can adapt to the changing landscape. If you want to see Malcolm’s full explanation about how OGM thinks about this challenge, check out the webinar.

If OGM’s solutions here inspire you, book a Retool demo and feel free to reference this post.