Software Data Engineer

Job Description

Job Description

Software Data Engineer

Expected start date : Dec 1st

Duration of engagement : 3 months, with potential extension

Work location : Alpharetta, GA or Plano TX.

Work location model : Hybrid, 3-days in office

About the Role :

The ideal candidate will be responsible for designing and maintaining modern, scalable data solutions on Azure using Databricks. This includes building data pipelines, ETL / ELT workflows, and architectures such as Data Lakes, Warehouses, and Lakehouses for both real-time and batch processing. The role involves integrating large datasets from diverse sources, implementing Delta Lake, and preparing data for machine learning through feature stores.

Key Responsibilities :

• Design, develop, and optimize scalable data pipelines and ETL / ELT workflows using Databricks on Azure
• Build and maintain modern data architectures (Data Lake, Data Warehouse, Lakehouse) for real-time streaming and batch processing on Azure
• Implement data integration solutions for large-scale datasets across diverse data sources using Delta Lake and other data formats
• Create feature stores and data preparation workflows for machine learning applications on Azure
• Develop and maintain data quality frameworks and implement data validation checks
• Collaborate with data scientists, ML engineers, analysts, and business stakeholders to deliver high-quality, production-ready data solutions
• Monitor, troubleshoot, and optimize data workflows for performance, costefficiency, and reliability
• Implement data governance, security, and compliance standards across all data processes
• Create and maintain comprehensive technical documentation for data pipelines and architectures

Required Qualifications :

Preferred Qualifications :

Educational Background :