Established in 2008 in Sydney, Deputy is an Australian technology company that provides cloud-based workforce management and scheduling platforms. Its 150 employees serve customers in more than 70 countries around the world. It needed to replace its data warehouse and after an eight-week proof of concept, it chose Snowflake. Thomas Ward, Director, Data, Insights and Analytics at Deputy tells SME.tech about the benefits Deputy has seen since it chose Snowflake.
Deputy traditionally relied on product usage data to provide insights and guide future development. Its legacy data warehouse could not scale to accommodate increasing volumes of data streamed from several sources daily. Analysts were hindered by performance degradation and maintenance downtime. They were frustrated because they could not access historical data.
Managing the data warehouse was becoming an increasingly time-consuming task. The organisation was spending more time doing that than undertaking analytics. Upgrading to the next tier of its existing data warehouse would have doubled its costs. The Deputy analytics team looked at several options to replace its data warehouse and chose Snowflake.
As well as the growing quantity of data, Deputy knew it had issues with its design which negatively impacted performance. Rather than change this immediately, it opted to run the existing design on Snowflake to see how differently it would perform. Snowflake was able to run the existing design with improved performance and no downtime. This convinced Deputy senior management that Snowflake’s cloud data platform was the best option.
The team then migrated the entire eight terabyte data warehouse to Snowflake in just one day. Overall, its monthly costs have been reduced by 40% but, just as importantly, it has allowed Deputy to regain control over its data. The fact that its storage and compute resources have now been separated means it can tailor its resources to match demands. Snowflake also enables the analytics team to perform data transformation tasks within the warehouse, reducing complexity and processing time.
Resource monitoring ensures the company doesn’t exceed its allocated resources during busy periods. Managers can readily view and reallocate resources if required. Logging features allow change tracking while query profiling ensures optimal query design and shows users the best way to extract value from the data warehouse.
A significant and unexpected benefit of migrating to Snowflake has been a culture shift where everyone, not just the analytics team, is analysing and using data.
Work is now underway to deliver a self-service capability. This means everyone can access data rather than having to submit a request and wait.
Thomas Ward, Director, Data, Insights and Analytics at Deputy, told SME.tech more about why the company chose Snowflake.
What were the main benefits which attracted the company to Snowflake?
Cost was one, and predictability of costs sitting alongside that, driven by Snowflake’s separation of storage and compute. We can pay for only the performance we need, independent of the storage that we require. I think confidence in the support model played a role, as well as confidence in the technology itself and the product roadmap. The direction that it is going with the product, in general, aligns with our interests.
How easy was the implementation?
The implementation of Snowflake itself is a separate question from how easy the migration was. I think doing a migration of a data warehouse is never really going to be easy. That’s unavoidable regardless of what platform you’re moving to. The part of it that was Snowflake-specific was pretty straightforward. Again, the support was very good – it was in-house support, rather than being outsourced to a third party.
Why have the monthly costs been reduced?
We achieved a cost reduction on the order of about 60% below our previous solution, almost immediately upon completing the migration to Snowflake. And we’ve achieved further reduction since then through introducing various efficiency improvements to the data warehouse and our pipelines. Initially, the cost reductions came from two sources, which were the separate storage and compute. In our previous data warehouse, we were paying significantly more than we really needed, just because there was no flexibility in terms of the billing there.
And the second major thing was the ability within Snowflake to assign different compute resources to specific user roles and tasks. This also was a factor in why our previous solution was relatively inefficient from a cost standpoint. We had to size the warehouse based on the requirements of the most demanding users, and therefore over-resource for low priority use cases, because we couldn’t provision them differently.
Why do you think migrating to Snowflake has been a culture shift for everyone, not just the analytics team?
Probably the biggest thing that has caused a culture shift based on Snowflake is the improved performance that’s enabled things like being able to separately provision resources for different user groups’ general performance improvements. And then the redesign of the warehouse itself, the structure of the warehouse and the data that sits on it is not necessarily a Snowflake thing but having a platform that is as performant and cost-effective as Snowflake has allowed us to do things that were probably cost-prohibitive previously. And that generally enables data sharing across the board. It can therefore be said to contribute to a culture shift in terms of just data-driven decision making more broadly.
That being said, the data warehouse sits behind the scenes for a lot of people in there. And you do get users who don’t really know the difference between the data warehouse and Tableau, for example, and who are not necessarily going to understand what the cause of the improved performance is. They’ll just see that data is suddenly more accessible and more available for them.
How will the migration help Deputy grow?
Migrating to Snowflake has resulted in two big things that support Deputy’s growth. Scalability is one. We’ve talked about the improved cost-effectiveness and improved ability to allocate resources for specific purposes. Those two things are pretty critical in enabling scalability and the use of data.
And the second one is interoperability. And that refers to the ease of integrating different data sources. Snowflake Data Sharing in particular, I would call out here as a service that we can use to make data more available to us internally, as well as if we want to provide a service even for our own customers, to see the data that they’re generating through use of our product.
What are the organisation’s plans for using data to derive business insights and contribute to overall goals and investment plans?
The progression of data maturity inside of Deputy is, on the lowest level, just making data available for reporting. So having access to a set of numbers in a dashboard, or some other way, that tells you the state of the business as they exist today – that’s the minimum, so you just have a window into what’s going on. The next step is identifying what are the points of leverage, where applying that data can contribute to better decision making in a high-impact way and having dedicated teams of analysts, or other people who are knowledgeable about interpreting data contribute to making those decisions.
The third stage is making data itself a source of value for the business. So, can we directly monetise data? Can we incorporate it as part of the product? That’s probably more difficult and has more requirements in terms of enabling real-time and complexity of the analysis and models that go into generating that value for customers. A bit more complexity in terms of security considerations, but it’s probably the final stage that we would look to.