Terminology

It's often useful to create terms to help describe abstract ideas within software. While Hydra strives to follow industry standard terminology as closely as possible, there are some terms we use that help describe certain components of our system. Think of this page like a glossary, you'll find many helpful terms and definitions here.

DSL/Job Config

Currently referred to as DSL, or Domain Specific Language, this is the JSON structure that you'll use to submit jobs with Hydra-Streams. The DSL is actually more of a specific configuration rather than a fully featured DSL, so sometimes you'll hear the terms used interchangeably. We'll favor Job Config in the future.

Replication

This term refers to the practice of taking data from a data store and "replicating" that data to potentially many systems. You can think of it like a lightweight ETL (Extract-Transform-Load) process.

Compaction

This refers to the idea of compacting data, or aggregating it by some key. This is useful in the context of Kafka, for example, where compaction allows a topic to issue updates to a piece of data when a primary key is specified, conceptually deleting old entries for that data. You can think of compaction like an upsert (insert/update) in relational database terms.

Entity

In the Hydra ecosystem, we treat an entity as an object (or just a thing, really) that:

• has some state
• can evolve or change its state over time (mutable)
• doesn't contain state "history", in other words, any give entity contains only a snapshot of the current state

Event

Conceptually, you can think of an event like "a stream of what happened over time". An event:

• is immutable
• tracks changes over time, a change is persisted as a new event