Job Title: Lead Data Engineer
Location: Chicago, IL (Onsite for the first month, then mostly remote) | Local or nearby
Duration: Contract-to-Hire (3 6 months)
Local Candidates Only: Must be local to the Chicagoland area and available for onsite interviews.
Must-Have Requirements
10+ years of overall IT experience, with strong Data Engineering and ETL expertise.
4+ years of hands-on Databricks experience, specifically on AWS. Azure or other-platform experience alone will not qualify.
Strong expertise in AWS services, including S3, IAM, and networking.
Hands-on experience with Apache Spark, Delta Lake, Unity Catalog, and Databricks Lakehouse architecture.
Strong SQL, Python, data modeling, and distributed data processing skills.
Experience with ETL/ELT pipelines, batch and streaming data processing, CI/CD, and DevOps.
Knowledge of Delta Live Tables (DLT), Databricks SQL, workflow orchestration, and medallion architecture.
Experience leading technical design, establishing engineering standards, mentoring team members, and collaborating with business stakeholders.
Role Overview
Lead the design and implementation of client's modern AWS and Databricks Lakehouse platform. Own the end-to-end ETL/ELT lifecycle, build scalable and reliable data pipelines, optimize performance and costs, and support enterprise analytics, reporting, and AI initiatives.
Interview Process
1. 60 90-minute technical interview with the Director, preferably onsite.
2. 30 45-minute behavioral and team-fit interview with the Manager.
Important Notes
Candidates must be local to the Chicagoland area.
The first month is onsite in downtown Chicago; the role will then be mostly remote, with onsite visits as needed.
This is a contract-to-hire opportunity, with conversion expected within 3 6 months.
Candidates must have recent, hands-on Databricks-on-AWS experience. Databricks on Azure or another platform is not a substitute.