Lead Data Engineer

About the job Lead Data Engineer

Role: Lead Data Engineer

Location: Fremont, CA(3 days onsite, 2 days work from home)

Duration: Full Time

Job Description:

• Candidate should Provide technical expertise in needs identification, data modelling, data movement, translating business needs into technical solutions with adherence to established data guidelines and approaches from a business unit or project perspective. Oversee and govern the expansion of existing data architecture and the optimization of data query performance via best practices. The candidate must be able to work independently and collaboratively
• 10+ Years of experience as a Data Engineer
• Strong technical expertise in Spark and Scala is must
• Strong experience with Python
• Strong expertise in ETL process and with various data model concepts
• Knowledge of star schema and a snowflake schema
• Good to have knowledge on AWS services such as S3, Athena, Glue, EMR/Spark with major emphasis on S3 and Glue
• Experience with Big Data Tools and technologies

Key Skills:

• Strong technical expertise in Spark and Scala is must
• Good Understanding of data structures and data analysis using Python

Responsibilities:

• Understand and translate business needs into data models supporting longterm solutions.
• Analyze data-related system integration challenges and propose appropriate solutions.
• Assist with and support setting the data architecture direction (including data movement approach, architecture / technology strategy, and any other data-related considerations to ensure business value).