google-cloud-dataproc

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

[DEPRECATED] Kubernetes operator for managing the lifecycle of Apache Flink and Beam applications.

Run in all nodes of your cluster before the cluster starts - lets you customize your cluster

BigQuery data source for Apache Spark: Read data from BigQuery into DataFrames, write DataFrames into BigQuery tables.

Libraries and tools for interoperability between Hadoop-related open-source software and Google Cloud Platform.

Cloud Dataproc: Samples and Utils

Tools for creating Dataproc custom images

A sample demo to check latest spark, big query connector and scala 2.12

Sua missão será criar um ecossistema de Big Data usando o Google Cloud Platform (GCP). Para isso, o expert te ensinará a configurar o Google Cloud Dataproc, um Hadoop totalmente gerenciado, usando seus créditos gratuitos da GCP.

Streaming JSON data to Spark or Google Cloud Dataproc.

Projeto do Curso "Criando um Ecossistema Hadoop Totalmente Gerenciado com Google Cloud Dataproc" do Bootcamp Data Engineer da Digital Innovation One

This project explores the core concepts of distributed data processing using the MapReduce programming model , implemented with Python via Hadoop Streaming , and deployed on a multi-node Google Cloud Dataproc cluster.