Spark & Scala Developer

Job Title : Scala and Spark Developer

Job Description :

We are seeking a talented and experienced Scala and Spark Developer to join our dynamic team. As a key member of our development team, you will play a crucial role in designing, implementing, and maintaining scalable and high-performance data processing applications using Scala and Apache Spark.

Responsibilities :

Design, develop, and implement distributed data processing applications using Scala and Apache Spark.

Collaborate with data engineers, data scientists, and other stakeholders to understand and define requirements for data processing tasks.

Optimize and tune Spark applications for performance and efficiency.

Develop and maintain ETL processes for large-scale data pipelines.

Work with large datasets and implement efficient data processing algorithms.

Troubleshoot and resolve issues related to data processing and performance.

Stay abreast of industry trends and advancements in Spark and Scala technologies.

Collaborate with cross-functional teams to integrate Spark applications into existing systems.

Requirements :

Proven experience as a Scala Developer with a focus on Spark development.

Strong programming skills in Scala and a solid understanding of functional programming concepts.

Experience with Apache Spark, including Spark SQL and Spark Streaming.

Proficiency in building and optimizing Spark jobs for large-scale data processing.

Knowledge of data modeling, ETL processes, and data warehousing concepts.

Familiarity with distributed computing and cluster management.

Experience with big data technologies such as Hadoop and Hive is a plus.

Excellent problem-solving and analytical skills.

Strong communication and collaboration skills.