LIC Transforms Dairy With Real-Time Data

Livestock Improvement Corporation (LIC), a farmer-owned New Zealand co-op, realized it needed to reimagine its data strategy to better serve dairy farmers nationwide.

With 10,500 farmer shareholders, LIC generates vast volumes of data—from milk analysis to animal genetics—and its legacy pipelines couldn’t keep up. Complex, multi-hop architecture, delayed insights, fragmented views, limited support, and disconnected systems strained the ability of farmers to get the relevant information they needed to run their operations.

“We wanted to evolve from traditional ETL and batch processes to a unified streaming platform that empowers real-time insights and faster outcomes,” explained Vik Mohan, LIC Principal Technologist.

The Challenge: Herds of Data, Siloed and Stalled

As LIC grew across R&D labs, farms, and international operations, so did its data sprawl. Insights critical to herd health, breeding decisions, and productivity were trapped across disconnected systems. This meant:

• Data silos slowed insight delivery

• Batch processing delayed decisions

• Fragmented systems hindered proactive farmer support

The Vision: A Unified Streaming Platform

To eliminate delays and build a single source of truth, LIC transitioned to an event-driven architecture powered by Confluent. The goal? Deliver transformed, trusted, and real-time data to every corner of the organization—securely and at scale.

“We’re shifting from multi-hop batch flows to real-time stream processing,” Mohan noted. “This helps us respond faster and serve our farmers better.”

From Multi-Hop to Real-Time Flow

LIC replaced sequential batch layers with a unified streaming pipeline, using Confluent Cloud and Apache Kafka®. Now, events flow continuously—no more intermediate landing zones, no more lag. Business wins include:

• Real-time herd health alerts

• Timely breeding insights

• Scalable integration with R&D and partner systems

With Apache Flink®, raw data is transformed on the fly—cleaned, enriched, and published for immediate use across systems.

“We built traceable, reusable business events and a common data model that any team or system can access,” Mohan added.

The Impact: Stronger Farms, Smarter Decisions

The transformation is already driving measurable outcomes, including faster, data-driven recommendations to farmers, reduced redundancy through centralized pipelines, and greater agility across services and operations. And with a shared streaming backbone, LIC can now federate partner data, accelerate innovation, and scale support without scaling complexity.