The Data Glossary

Application security refers to the different sets of processes, practices, and tools maintaining the security of the software application against any external threat or vulnerability.

ASPICE is a framework designed to assess and enhance the software development processes within the automotive industry.

In today’s fast-paced environment, success in software development depends significantly on development speed, reliability, and security.

CQRS is an architectural design pattern that helps handle commands to read and write data in a scalable way. Learn how it works, its benefits, use cases, and how to get started.

Data fabric architectures enable consistent data access and capabilities across distributed systems. Learn how it’s used, examples, benefits, and common solutions.

Also known as dataflow or data movement, data flow refers to how information moves through a system. Learn how it works, its benefits, and modern dataflow solutions.

Good data flow enhances the efficiency, scalability, responsiveness, and reliability of systems.

This guide explores essential data integration best practices that will help you streamline your processes and maximize the value of your data assets.

Data integration security protects sensitive information during the integration process.

If computer networks were cities, routing would be the interstates and freeways connecting them all, and vehicles would be the data packets traveling along those routes.

Data serialization can be defined as the process of converting data objects to a sequence of bytes or characters to preserve their structure in an easily storable and transmittable format.

A database is a collection of structured data (or information) stored electronically, which allows for easier access, data management, and retrieval. Learn the different types of databases, how they're used, and how to use a database management system to simplify data management.

A Distributed Control System (DCS) is a control system used in industrial processes to manage and automate complex operations.

Event sourcing tracks the current state of the system, and how it evolves over time. Learn how event sourcing works, its benefits and use cases, and how to get started.

ELT (Extract, Load, Transform) is a data integration process where raw data is loaded first and transformation happens after.

Fast-paced businesses rely on smooth data exchange and analysis between systems.

FIPS or Federal Information Processing Standards is a set of publicly announced standards developed by the National Institute of Standards and Technology (NIST).

Apache Flume is an open-source distributed system designed for efficient data extraction, aggregation, and movement from various sources to a centralized storage or processing system.

IaC is a transformative approach to managing IT infrastructure by allowing organizations to define and provision their resources through code rather than manual processes.

Legacy systems can often be dusty relics of a bygone era, leaving you with outdated technology.

Apache Kafka's backup mechanisms are essential components of a robust data infrastructure strategy.

Apache Kafka is the most commonly used stream processing / data streaming system. Learn how Kafka benefits companies big and small, why it's so popular, and common use cases.

A Kafka Dead Letter Queue (DLQ) is a special type of Kafka topic where messages that fail to be processed by downstream consumers are routed.

A Kafka message key is an attribute that you can assign to a message in a Kafka topic. Each Kafka message consists of two primary components: a key and a value.

The Kafka message size limit is the maximum size a message can be to be successfully produced to a Kafka topic.

A partition key in Apache Kafka is a fundamental concept that plays a critical role in Kafka's partitioning mechanism.

Apache Kafka partition strategy revolves around how Kafka divides data across multiple partitions within a topic to optimize throughput, reliability, and scalability.

A Kafka Sink Connector is part of Kafka Connect, the robust API for data integration. It makes data integration easy by providing automation of data movement both in and out of Kafka.

Kafka Source Connectors are integral components of Kafka Connect, designed to import data into Kafka topics from various data sources.

Apache Kafka Streams is a client library for building applications and microservices that process and analyze data stored in Kafka.

Kafka Topic Naming convention keeps your data organized and makes it easier to understand, scale, and maintain.

Message brokers facilitate communication between applications, systems, and services. Learn how they work, their use cases, and how to get started.

While message brokers and message queues are often used interchangeably, understanding their distinct characteristics and use cases can help you make better architectural decisions.

Microservices refers to an architectural approach where software applications are composed of small, independently deployable services that communicate with each other over a network.

Apache NiFi is an integrated data logistics platform for automating the movement of data between disparate systems. It can handle various data types and support various protocols. Learn how it's used, and how it works.

The National Institute of Standards and Technology's Secure Software Development Framework NIST SSDF is a set of guidelines that are intended to assist organizations in developing their software securely.

Observability is the ability to measure the current state or condition of your system based on the data it generates. With the adoption of distributed systems, cloud computing, and microservices, observability has become more critical, yet complex.

OWASP is a non-profit organization dedicated to improving the security of software.

Pub/sub is a messaging framework commonly used for inter-service communication and data integration pipelines. Learn how it works, with examples, benefits, and use cases.

RabbitMQ is a message broker that routes messages between two applications. Learn how RabbitMQ works, common use cases, pros, cons, and best alternatives.

RabbitMQ and Apache Kafka are both open-source distributed messaging systems, but they have different strengths and weaknesses.

RAG leverages real-time, domain-specific data to improve the accuracy of LLM-generated responses and prevent hallucinations. Learn how RAG works with use case examples.

Real-time data (RTD) refers to data that is processed, consumed, and/or acted upon immediately after it's generated. While data processing is not new, real-time data streaming is a newer paradigm that changes how businesses run.

A complete comparison of Kafka vs Redpanda and two cloud Kafka services - Confluent vs Redpanda. Learn how each works, the pros and cons, and how their features stack up.

Refactoring is an important part of software development that optimizes the code's internal structure without changing how the application works on the outside.

REST API stands for Representational State Transfer. Learn more about REST API, how it simplifies server communication, and how it leverages large-scale data.

Shift Left in data integration is derived from the software engineering principles of Shift Left Testing where tests are performed earlier in the software development lifecycle to improve quality of software, accelerate time to market, and identify issues earlier.

SAST is a method that checks for security flaws in code before it reaches production.

Stream processing allows for data to be ingested, processed, and managed in real-time, as it's generated. Learn how stream processing works, its benefits, common use cases, and the best technologies to get started.

Streaming analytics is an approach to business analytics and business intelligence where data is analyzed in real-time. Learn how streaming analytics works, common use cases, and technologies.

Streaming data pipelines move data from multiple sources to multiple target destinations in real time. Learn how they work, with examples and demos.

What is ETL vs ELT streaming, and how are they different from streaming ETL pipelines? Learn the differences between data pipeline and integration tools, their processes, and which to choose.

Technical debt is a concept that originated in software engineering and refers to the future costs due to shortcuts taken in system development.