Case Study - Redesign of IoT Data Ingestion System

The challenge

Taqt’s IoT data ingestion system had evolved rapidly alongside product needs.
Over a few years, the growing number of device types, firmware versions and network operators introduced many specific rules, making the system increasingly complex to maintain and risky to evolve.

Incoming data flows came in heterogeneous formats, with parts of the business logic embedded directly in the processing layer. This approach reduced code readability, made testing more difficult and limited the platform’s ability to scale and support new use cases.

Our approach

We redesigned the data ingestion process as a modular, extensible and decoupled processing pipeline, relying on principles from Domain-Driven Design (DDD) and a clear separation of responsibilities.

Each incoming payload is first normalised and validated using typed DataFrames that explicitly represent domain structures. The data is then processed through a chain of responsibility composed of specialised handlers, dynamically selected based on the device type, firmware version and network context.

The entire processing flow is built on an asynchronous architecture using messages and queues, ensuring both system resilience and scalability as data volumes grow.

The implemented solution

• A unified ingestion architecture for all IoT devices
• A modular chain-of-responsibility for data processing
• Clear separation between validation, parsing and business logic
• Native support for multiple firmware versions, without breaking changes
• Full traceability of incoming data and generated events

Results

This redesign strengthened the platform’s technical foundation while preparing it for new use cases and a steady increase in IoT data volumes.

Teams now work with a system that is readable, testable and easy to extend, capable of integrating new devices and new business rules without requiring a global redesign of the architecture.