Don’t miss out on Current in New Orleans, October 29-30th — save 30% with code PRM-WEB | Register today

Demo

Why Confluent over MSK for Streaming Data Pipelines at Security Scorecard

Watch Now

When a business wants to understand their credit risk, they consult one of three major credit scoring companies. When companies want to see the security risks of their 1st party and 3rd party systems in their data infrastructure, they turn to SecurityScorecard. Accurate, up-to-date data is the lifeblood of any modern business, and it needs to come from myriad sources. SecurityScorecard continuously monitors and rates the cyber posture of more than 12 million companies worldwide.

Up until a few years back, the teams at SecurityScorecard relied on batch pipelines to push data to and from Amazon S3 as well as expensive REST API based communications to carry data between systems. They also spent significant time and resources on Kafka upgrades and cluster management initially using Amazon MSK before turning to Confluent’s fully managed solution.

SecurityScorecard’s Brandon Brown, lead of the company’s Pipelines team, pivoted away from antiquated batch processing and toward Confluent’s streaming approach for the development of a new Automatic Vendor Detection (AVD) product. His team continues to leverage the full breadth of Confluent Cloud’s connectors to deliver several new use cases, including:

  1. AVD to onboard new sources of data quickly via topics and refresh that data every hour
  2. A source of truth using PostgreSQL and pushing data to downstream systems using CDC connectivity
  3. A future use case using ksqlDB to have consolidated observability, and save on infrastructure costs

Join us and hear the reasons why SecurityScorecard switched to Confluent.

Brandon Brown is a Senior Engineering Manager at SecurityScorecard where he leads the team who provide the scores at the core of our ratings platform. These scores guide cybersecurity leaders worldwide in reducing their risk and addressing risk in their vendors. His team also drives secure data sharing and advances company-wide adoption of streaming technologies. With 10+ years of experience in software development, Brandon has experience across the full SDLC specializing in Data Pipelines over the last 8+ years. His language of choice is Scala but he has a soft spot for SQL. He’s contributed to top open source projects such as Debezium and was an early contributor to the ZIO ecosystem. He’s passionate about Kafka and Hybrid Data Pipeline strategies. Outside of work he’s helping raise his young son and catching live music or talking about movies