Data Integration
Big data, the Internet of Things (IoT), software as a service (SaaS), cloud activity, and more created an explosion in the number of data sources and the sheer volume of data existing in the world. Historically most of this data has been collected and stored in stand-alone silos or separate data stores. Data integration is the process of discovering, moving, and combining data from multiple sources to drive insights and power machine learning and advanced analytics.
Data integration is especially important as your business pursues digital transformation strategies, since your ability to improve operations, boost customer satisfaction, and compete in an increasingly digital world requires insight from all your data.
Google Cloud's data integration solution is a suite of loosely coupled but tightly integrated services that include:
- Cloud Data Fusion: a fully managed, cloud-native data integration service that helps users efficiently build and manage ETL/ELT data pipelines
- Cloud Composer: a fully managed workflow orchestration service built on Apache Airflow to manage and orchestrate the end-to-end data and process life cycle
- DataStream: a serverless and easy-to-use change data capture and replication service
- Dataplex: an intelligent data fabric to discover, manage, monitor and govern distributed data at scale
- Dataflow: a fully managed streaming analytics service that minimizes latency, processing time, and cost
- Pub/Sub: an asynchronous and scalable messaging service used for streaming analytics and data integration pipelines
- Dataproc: a fully managed Spark and Hadoop service for batch processing, querying, streaming, and machine learning
