Data Engineer

Job Description

This is a remote position.

Data Engineer

Summary

This pivotal role is dedicated to architecting, constructing, and maintaining the robust data infrastructure necessary to drive organizational insights. The successful candidate will serve as a cornerstone for data accessibility and integrity, ensuring that complex datasets are efficiently processed and transformed into actionable intelligence. By bridging the gap between raw data sources and analytical applications, this position plays a critical part in supporting strategic decision-making and optimizing operational workflows across the enterprise.

We are seeking a Data Engineer to join our Data & AI practice, building modern data platforms for our clients on Microsoft Azure and Azure Databricks.

This role is well suited to an ambitious, client-focused technologist who wants hands-on experience with the latest Microsoft data platform technologies - Microsoft Fabric, and Azure Databricks - to deliver scalable pipelines that power analytics and AI.

You will work alongside architects, data scientists, and client stakeholders to ingest, clean, prepare, and deliver trusted data, and to build the APIs that connect that data to downstream applications and reporting.

Responsibilities

• Design, develop, and deploy scalable data pipelines to facilitate the seamless movement of information from diverse sources to centralized repositories.
  
• Collaborate closely with data scientists and analysts to understand specific data needs and deliver high-quality, reliable data solutions.
  
• Monitor and optimize the performance of existing data systems to ensure maximum efficiency, reliability, and security.
  
• Implement data governance policies and standards to maintain data accuracy, consistency, and compliance with regulatory requirements.
  
• Troubleshoot and resolve complex data issues, including data quality problems, pipeline failures, and integration challenges.
  
• Document technical processes, data architectures, and system specifications to ensure knowledge transfer and operational continuity.
  

Requirements

Data Pipeline Delivery

• Design, build, and maintain batch and streaming data pipelines using Microsoft Fabric, Azure Data Factory, and

Azure Databricks, with Python, PySpark, and SQL.

• Ingest data from structured, semi-structured, and unstructured sources - databases, APIs, files, and event streams -

into governed lakehouse environments.

• Support the full data journey on client engagements: ingestion, cleaning, preparation, transformation, and delivery of

analytics-ready datasets.

• Implement transformation logic aligned to Medallion (Bronze, Silver, Gold) architecture patterns in Azure Data Lake

Storage (ADLS Gen2), OneLake, and/or Delta Lake.

• Build and consume APIs and reusable data services that expose data to reporting, applications, and downstream

analytics.

• Write clean, well-tested code and participate in peer code reviews.

Data Modeling and Quality

• Apply data modeling fundamentals (dimensional, normalized, star-schema) to structure data for analytics, Power BI,

and reporting use cases.

• Implement data quality checks, validation rules, and monitoring to ensure pipelines are accurate, complete, and

reliable.

• Troubleshoot data pipeline failures, performance bottlenecks, and data discrepancies, escalating and resolving

issues promptly.

• Document data flows, data models, and transformation logic for technical and client audiences.

Collaboration and Client Delivery

• Work directly with client stakeholders, analysts, and data scientists to understand data requirements and translate

them into technical solutions.

• Collaborate with senior engineers and architects to ensure your work aligns with broader platform and governance

standards.

• Support data security, access control, and governance practices (e.g., Microsoft Purview, role-based access).

• Get exposure to how data is activated for front-line systems and early Data Agent/Copilot-style use cases.

• Participate in agile ceremonies - sprint planning, stand-ups, and estimation - as part of a delivery team.

Qualifications

• 1-3 years of professional experience in data engineering or a related hands-on technical role (relevant internships

and academic projects considered for entry-level candidates).

• Proficiency in Python and SQL, with hands-on experience writing production-quality code.

• Experience with PySpark and an understanding of distributed data processing.

• Exposure to the Microsoft Azure data ecosystem (Azure Data Factory, Microsoft Fabric, ADLS Gen2) and/or Azure

Databricks; Delta Lake and Unity Catalog a plus.

• Working knowledge of data modeling concepts and data warehousing fundamentals.

• Understanding of ETL/ELT concepts and the end-to-end data lifecycle from ingestion through activation.

• Familiarity with building or consuming APIs for data integration.

• Familiarity with version control (Git) and basic CI/CD concepts (Azure DevOps a plus).

• Strong analytical and problem-solving skills with close attention to data accuracy.

• Comfortable communicating with both technical peers and, over time, client stakeholders.

• Relevant certifications (e.g., Microsoft Azure Data Engineer Associate) are a plus but not required.