Data Engineer

Section

Details

Job Title

Senior Data Engineer

Job Type

Full Time

Client Location

Sunnyvale, CA

Work Arrangement

Remote

Duration

Full Time

Pay Rate / Salary

$120-$150k/yr.

Job Summary

• The Data Engineer will be responsible for designing, building, governing, and optimizing data platforms, pipelines, and integrations to ensure secure, reliable, and scalable data availability for analytics, reporting, AI/ML, and business operations.

Key Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines.
• Automate data ingestion from multiple data sources.
• Ensure reliable, high-performance, and scalable data movement.
• Develop and optimize PySpark-based data processing frameworks.
• Build and manage Airflow workflows and orchestration pipelines.
• Work with Hive and Kafka-based batch and streaming data solutions.
• Implement data quality, monitoring, governance, and operational controls.
• Support production incidents, performance tuning, and continuous improvement initiatives

Required Qualifications

• Strong Functional Knowledge in Retail (Search & Personalization) Added advantage.
• Strong experience with large-scale GCP Data Engineering solutions.
• Hands-on experience with Airflow DAG development and support
• Experience building and optimizing Spark/PySpark data pipelines.
• Exposure to Hive-based data processing and Kafka-driven ingestion frameworks.
• Experience supporting analytics, personalization, recommendation, or search platforms is highly desirable
• Google Cloud Platform (GCP)
• PySpark
• Python
• Airflow
• Hive
• Kafka