Data Pipeline

#aws #data

Data Pipeline VS. Glue:

  • Data pipeline has more control over the orchestration and code (not just Spark)
  • Glue automatically scales so does its underlying Spark cluster