MapReduce, Spark, Java, and Scala for Data Algorithms Book

Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University

hadoop-cos（CosN文件系统）为Apache Hadoop、Spark以及Tez等大数据计算框架集成提供支持，可以像访问HDFS一样读写存储在腾讯云COS上的数据。同时也支持作为Druid等查询与分析引擎的Deep Storage

HADOOP 3.1.0 winutils

Export Hadoop YARN (resource-manager) metrics in prometheus format

This is projects of Cloud Computing Course

Containerized Apache Hive Metastore for horizontally scalable Hive Metastore deployments

A Spark application to merge small files on Hadoop

Some simple, kinda introductory projects based on Apache Hadoop to be used as guides in order to make the MapReduce model look less weird or boring.

This repository showcases a Medallion Architecture Data Lakehouse designed for both batch and real-time processing of e-commerce and marketing data. It supports comprehensive data analysis, reporting, and monitoring, providing a scalable solution for deriving insights from integrated datasets.

This repository provides a guide to preprocess and analyze the network intrusion data set using NumPy, Pandas, and matplotlib, and implement a random forest classifier machine learning model using Scikit-learn.

An python implementation of Minimal Mapreduce Algorithms for Apache Spark

A fast, scalable and distributed community detection algorithm based on CEIL scoring function.

The implementation of Apache Spark (combine with PySpark, Jupyter Notebook) on top of Hadoop cluster using Docker

Simplified Hadoop Setup and Configuration Automation

Set of Input Formats for Hadoop Streaming

Kubernetes operator for managing the lifecycle of Apache Hadoop Yarn Tasks on Kubernetes.

Apache Hadoop Cluster configuration with original apache/hadoop:3.4.1 docker image (with YARN)

This project simulates a real-world enterprise data migration and modernization strategy. It extracts transactional data from a simulated "On-Premise" environment (hosted on AWS EC2), performs heavy distributed processing using a Hadoop/Spark cluster, and ultimately serves the data via a Cloud-Native, serverless architecture to optimize costs .