Data engineer [h1.location_city]
[job_alerts.create_a_job]
Data engineer • philadelphia pa
[last_updated.last_updated_variable_hours]
Location : Philadelphia, PA (Hybrid).The ideal candidate is a strong technical problem-solver, highly collaborative, and proactive in automation and process improvements.
Data Pipeline Management & S...[show_more][last_updated.last_updated_variable_days]
Please Contact : To discuss this amazing opportunity, reach out to our Talent Acquisition Specialist.Contract for Data Analytics Engineer.
Please let me know if you or any of your friends would be i...[show_more][last_updated.last_updated_variable_days]
About Us : BW Design Group is a fully integrated architecture, engineering, construction, system integration, and consulting firm committed to helping our clients realize their most critical goals ...[show_more][last_updated.last_updated_30]
This position is on-site 2-3 days in Philadelphia.Pathology is at the center of medicine and is undergoing a profound transformation as the final frontier of digitization in healthcare.We started P...[show_more][last_updated.last_updated_30]
Our client, an organization located in Center City, Philadelphia has retained Top Stack to assist them in hiring a Data Engineer to join ...[show_more][last_updated.last_updated_30]
Known for being a great place to work and build a career, KPMG provides audit, tax and advisory services for organizations in today's most important industries.
Our growth is driven by delivering re...[show_more][last_updated.last_updated_variable_days]
This role is responsible for developing data structures and pipelines aligned to established standards and guidelines to organize, collect, standardize, and transform data that helps generate insig...[show_more][last_updated.last_updated_variable_days]
Design, build, test, and maintain scalable data management systems and ETL / ELT pipelines using PySpark and Databricks.Utilize AWS cloud services for data storage, computation, and orchestration.Col...[show_more][last_updated.last_updated_30]
Employment Type : Full-Time, Mid-level.Department : Business Intelligence.CGS is seeking a passionate and driven Data Engineer to support a rapidly growing Data Analytics and Business Intelligence pl...[show_more][last_updated.last_updated_30]
Location : Camden, NJ [Hybrid] Employment Type : Contract Job ID : 7912 Date Added : 02 / 05 / 2024 Thank you for applying.
Someone will be contacting you shortly.A new application was submitted.Overview PS...[show_more][last_updated.last_updated_30]
MKS2 Technologies, LLC, an award-winning high growth small business, creates innovative and customer-centric technology solutions in the areas of Cyber Security, Instructional Design and Training, ...[show_more][last_updated.last_updated_variable_days]
We are seeking a Data Pipeline / ETL Engineer to design build and maintain the data ingestion and transformation processes that power IntegriChains analytics and SaaS platform.Youll work closely wi...[show_more][last_updated.last_updated_variable_days]
Data Engineer – Job Opportunity.Please note this role is open for US citizens or Green Card Holders only.We’re looking for a Data Engineer to help build and enhance scalable data systems that power...[show_more][last_updated.last_updated_variable_days]
We are seeking a passionate and skilled Senior Data Engineer to join our dynamic team in Philadelphia, PA.In this role, you will lead the design and implementation of advanced data pipelines for Bu...[show_more][last_updated.last_updated_variable_days]
Junior Data Scientist with Data Engineering Skills.Junior Data Scientist with Data Engineering skills.This is an exciting opportunity for recent graduates or early-career professionals who are pass...[show_more][last_updated.last_updated_30]
We are looking for a passionate certified Data Engineer.The successful candidate will turn data into information, information into insight and insight into business decisions.Data analyst responsib...[show_more][last_updated.last_updated_30]
Join a global community of talented professionals to shape the future of AI.Earn up to $15 USD / hr and additional rewards based on quality of submission.
Outlier is committed to improving the intelli...[show_more][last_updated.last_updated_variable_hours]
Employment Type : Full-Time, Mid-level.Department : Business Intelligence.CGS is seeking a passionate and driven Data Engineer to support a rapidly growing Data Analytics and Business Intelligence pl...[show_more][last_updated.last_updated_30]
The Office of Innovation & Technology (OIT) is the central IT agency for the City of Philadelphia headed by the Chief Information Officer (CIO).
OIT oversees all major information and communicat...[show_more][last_updated.last_updated_variable_days]
- [promoted]
Data Engineer
EXLPhiladelphia, PA, United States[job_card.full_time]
Data Analytics engineer
Generis Tek IncPhiladelphia, PA, United States[job_card.permanent]
Controls Engineer-Data Center
Barry-WehmillerPhiladelphia, PA, United States[job_card.full_time]
Senior Data Engineer, Real World Data
ProsciaPhiladelphia, PA, US[job_card.full_time]
Data Engineer
Top StackArdmore, PA, US[job_card.full_time] +1
[filters_job_card.quick_apply]
Manager, Data Operations, Data Engineer
KPMGPhiladelphia, PA[job_card.full_time]
- [promoted]
Data Engineer
ComcastPhiladelphia, Pennsylvania, US[job_card.full_time]
Data Engineer
Cynet SystemsPhiladelphia, PA[job_card.full_time]
Data Engineer
CGSPhiladelphia, Pennsylvania, United States, 19102[job_card.full_time]
Data Engineer
PSCICamden, NJ[job_card.temporary]
Application Data Platform Engineer
MKS2 TechnologiesPhiladelphia, PA, US[job_card.full_time] +1
- [promoted]
Data Pipeline, ETL Engineer
IntegriChainPhiladelphia, Pennsylvania, USA[job_card.full_time]
- [promoted]
Data Engineer
Realtime RecruitmentPhiladelphia, PA, United States[job_card.full_time]
- [promoted]
Senior Data Engineer
Eigen XPhiladelphia, PA, United States[job_card.full_time]
Data Engineer
SynergisticITPhiladelphia, PA, US[job_card.full_time]
Data Engineer
ForhyrePhiladelphia, PA, US[job_card.full_time]
- [promoted]
- [new]
Junior Data Support Engineer
Scale AIPhiladelphia, Pennsylvania, United States[job_card.full_time]
Data Engineer
Contact Government Services, LLCPhiladelphia, PA, US[job_card.full_time]
Data Engineer / Integration Specialist
City of PhiladelphiaPhiladelphia, PA, US[job_card.full_time]
Data Engineer
EXLPhiladelphia, PA, United States[job_card.variable_days_ago]
[job_preview.job_type]
- [job_card.full_time]
[job_card.job_description]Design, develop, and optimize PySpark jobs for efficient large-scale data processing. Diagnose and resolve complex Spark performance issues such as data skew, shuffle spill, executor OOM errors, slow-running stages, and partition imbalance. Use Databricks for Spark job orchestration, workflow automation, and cluster configuration. Debug and manage Spark on Kubernetes , addressing pod crashes, OOM kills, resource tuning, and scheduling problems. Work with MinIO / S3 storage for bucket management, permissions, and large-volume file ingestion and retrieval. Partner with onshore business stakeholders to clarify requirements and convert them into well-defined technical tasks. Provide daily coordination and technical oversight to offshore engineering teams. Participate actively in design discussions and technical reviews. Maintain accurate and detailed documentation, runbooks, and troubleshooting guides. Contribute to process improvements that enhance operational stability and engineering efficiency. PySpark : Advanced proficiency in transformations, performance tuning, and Spark internals. SQL : Strong analytical query design, performance tuning, and foundational data modeling (relational & dimensional). Python : Ability to write maintainable, production-grade code with a focus on modularity, automation, and reusability. Kubernetes : Experience with Spark-on-K8s, including pod diagnostics, resource configuration, and log / monitoring tools. Databricks : Hands-on experience with cluster management, workflow creation, Delta Lake optimization, and job monitoring. MinIO / S3 : Familiarity with bucket configuration, policies, and efficient ingestion patterns. DevOps : Experience with Git, CI / CD, and cloud environments (Azure preferred).
Job Title : Data Engineer
Location : Philadelphia, PA (Hybrid)
Experience : 5+ years
Role Summary :
We are seeking an experienced Data Engineer with strong expertise in PySpark and data pipeline operations . This role focuses heavily on performance tuning Spark applications , managing large-scale data pipelines, and ensuring high operational stability. The ideal candidate is a strong technical problem-solver, highly collaborative, and proactive in automation and process improvements.
Key Responsibilities :
Data Pipeline Management & Support
- Operate and support Business-as-Usual (BAU) data pipelines, ensuring stability, SLA adherence, and timely incident resolution.
- Identify and implement opportunities for optimization and automation across pipelines and operational workflows.
Spark Development & Performance Tuning
Platform & Tool Management
Collaboration & Communication
Documentation & Operational Excellence
Required Skills & Qualifications :
Primary Skills (Must-Have)
Secondary Skills (Highly Desirable)