Senior Software Engineer (Big Data)

What You’ll Achieve: • Design, develop, and maintain scalable data processing solutions across on-premises and cloud environments using Python and Apache Spark. • Optimize and fine-tune Spark jobs for performance, including resource utilization, shuffling, partitioning, and caching, to ensure maximum efficiency in large-scale big data environments. • Design and implement scalable, fault-tolerant data pipelines with end-to-end monitoring, alerting, and logging. • Leverage AWS cloud services (2 y…