GitHub Repo
https://github.com/slatawa/Forex-Currency-Processing-Airflow-Hdfs-Hive-Spark
We build a Forex-currency rates pipeline to get currency rates from an external API and load the data into HDFS from where we use pyspark job to massage the data and insert it into a Hive table. The objective of this pipeline is to get the data ready for any downstream machine learning pipeline.
