Poradmin
The term “data pipe” refers a series processes that collect raw data and transform it into an app-friendly format. Pipelines can be batch or real-time. They can be deployed in the cloud or on-premises and their software is commercial or open source.
Similar to how a pipeline transports water from a river to your house, data pipelines bring data from one layer (transactional or event sources) to another (data lakes and warehouses). This allows analytics and insights to be derived from the data. In the past, data transfers required manual processes such as daily uploads of files or lengthy waiting times to get insights. Data pipelines are a replacement for manual processes and allow companies to transfer data more efficiently and with less risk.
A virtual data pipeline provides large infrastructure savings in terms of storage costs in the datacenter and remote offices. It also reduces hardware, network and management costs associated with the deployment of non-production environments, such as test environments. Automation of data refresh, masking, and access control based on roles as well as customizing and integrating databases can help to reduce time.
IBM InfoSphere Virtual Data Pipeline (VDP) is a multi-cloud copy-management solution that decouples test and development environments from production infrastructures. It uses patented snapshot and changed-block tracking technology to capture application-consistent copies of databases and other files. Users can instantly create masked, near-instant virtual copies of databases from VDP to VMs and mount them in non-production environments to start testing within minutes. This is especially useful for accelerating DevOps and agile methodologies as in accelerating time to market.
Sobre o autor