Storyblocks Partners with Confluent to Redesign Its Data Pipeline and Slash Technical Debt

Ranked the fourth-fastest growing media company in the U.S. by Inc. Magazine, Storyblocks was born out of a desire to solve the problem of a lack of accessible, affordable stock for creatives. It’s the first unlimited download subscription-based provider of stock video and audio, claiming over 100,000 customers in the television and video production industry, including NBC and MTV, plus tens of thousands of hobbyists looking to enhance their video projects and productions. Storyblocks’ subscribers can download an unlimited number of clips from a vast and rapidly growing library of stock video, production music, motion backgrounds, sound effects, special effects, and more.

As the company transitioned from scrappy disruptor to major industry player, it began to experience issues with a monolithic application they had built when the company was started. They also experienced other issues later with synchronous REST API calls between services, after they had split the monolith into microservices. Namely, developers and data engineers couldn’t resolve issues fast enough or iterate on their search functionality with sufficient agility, and were taking on a significant amount of technical debt as an increasing amount of data threatened to slow down the productivity and time to market for new features.