We are always listening to our customers. One of the most common challenges we hear is the cost, complexity, and potential incompleteness of transferring data to the cloud. Consequently, we developed an innovative addition to our Industrial Data Fabric solution, Element Unify – Time Series Data Mover (TSDM).
Taking a Quick Step Back
The “Industrial Data Fabric” or IDF is fast emerging as a phrase of choice for many industrial CIO/CDO/Data architects. It is a data architecture pattern that addresses the issues with Data Lakes quickly turning into Data Swamps. Turns out moving siloed data to the cloud moves the silo as well with the data. IDF is an approach that emphasizes data operations over data integration. It leverages not just metadata within a silo but also the relationship between these data models across the silos. Often referred to as Contextualization; this approach to data management, delivers greater value from the combined IT/OT/ET data via modern analytical tools. Modern IDF stacks maintain this unified model in a knowledge graph. This enables data consumers to use the same source data in any use case, enabling faster scale up at lower cost. Unify is an industry leading engine for creating and governing these knowledge graphs within an IDF stack.
Moving Data is Expensive
Major vendor migration solutions are costly and confusing and require significant labor to implement. Another common approach, writing custom Python code, is time consuming – and could end up being even more costly since the code is “one-and-done” and can not be reused for other data projects. Additionally, working bottom up with siloed data also results in moving all the data all the time since you don’t know what you need or don’t need until you get to the last mile/application configuration.
Data is Incomplete
Merely relocating data to the cloud does not render it usable. Data silos are also transferred to the cloud alongside the data which may lack necessary context and require processing via mash-ups before it can be utilized. However, the mash-up code is often not standardized or reusable, resulting in additional time and expenses. To derive value from the data, it must undergo contextualization, or mashing, in the cloud.
Reuse and Recycle
Element Unify TSDM is an easy-to-use single tool for managing your model and your time series data. With TSDM, the data can be mixed, matched, or mashed prior to being stored in your preferred data lake or data warehouse. This eliminates the need for the application team to spend time wrangling the data, allowing them to deliver intelligence and value to users.
As your data migrates to the cloud, you can add supplementary metadata to facilitate seamless integration with the rest of your data ecosystem. This minimizes the significant effort typically needed to verify data during runtime.
The power of this capability expands when viewed within the context of the Industrial Data Fabric. Unify TSDM allows for a flexible, powerful and scalable management of knowledge graphs. This mix-n-match of knowledge from the graph with data from the data streams results in ready to use data products. Additionally, since we are approaching this as a data operations problem instead of a data integration problem, we gain reusability, efficiency and cost savings.
The initial release of TSDM will include:
- Connector for PI Data Archive to import time series events with associated PI tags
- Updates to Unify to store and process streaming time series data
- Ability to use Unify transform blocks to help with no-code mashup of time series data with model metadata
- Unify Connector for AWS RedShift to export time series data streams
- Unify Connector API for exporting time series data streams to other systems
The general availability of Element Unify 5.0 with TSDM is March 15, 2023.
Want to Learn More?
Read the press release: New Capabilities in Element Unify 5.0 Streamline the Process of Moving Industrial Data to the Cloud
Register to view a live demo of Unify or sign up for a free trial to use the software for 30 days. You can also sign up for a free trial or purchase a Personal License from the AWS Marketplace.
Questions? Please contact us.
