Live demo: Kafka streaming in 10 minutes on Confluent | Watch now

Online Talk

Streaming Data Pipelines for Automatic Vendor Detection in 5 Steps

Available On-demand

When a business wants to understand their credit risk, they consult one of three major credit scoring companies. When companies want to see the security risks of their 1st party and 3rd party systems in their data infrastructure, they turn to SecurityScorecard. Accurate, up-to-date data is the lifeblood of any modern business, and it needs to come from myriad sources. SecurityScorecard continuously monitors and rates the cyber posture of more than 12 million companies worldwide. Up until a few years back, the teams at SecurityScorecard relied on batch pipelines to push data to and from Amazon S3 as well as expensive REST API based communications to carry data between systems. They also spent significant time and resources on Kafka upgrades when using Amazon MSK.

SecurityScorecard’s Brandon Brown, lead of the company’s Pipelines team, pivoted away from antiquated batch processing and toward Confluent’s streaming approach for the development of a new Automatic Vendor Detection (AVD) product. His team continues to leverage the full breadth of Confluent Cloud’s connectors to deliver several new use cases, including:

  1. AVD to onboard new sources of data quickly via topics and refresh that data every hour
  2. A source of truth using PostgreSQL and pushing data to downstream systems using CDC connectivity
  3. A future use case using ksqlDB to have consolidated observability, and save on infrastructure costs

Watch now to hear the reasons why SecurityScorecard switched to Confluent.

Helpful Resources:

Presenters

Brandon Brown

Senior Staff Software Engineer, SecurityScorecard

Bharath Chari

Team Lead, Solutions Marketing, Confluent

Watch Now

Additional Resources

cc demo
kafka microservices
Image-Event-Driven Microservices-01

Additional Resources

cc demo
kafka microservices
microservices-and-apache-kafka