DATA ENGINEER -
Job Description
ROLE: DATA ENGINEER - Sparx, Hadoop, OzoneCH, Flink
Job Title: Data Engineer (Spark, Hadoop, OzoneCH)
Location: Berkley Heights, NJ
Role Overview:
We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark, Hadoop ecosystem, and Apache Ozone. The ideal candidate will design, develop, and optimize large-scale data processing systems, ensuring high performance, scalability, and reliability for enterprise-level applications.
Key Responsibilities:
Design and implement distributed data processing solutions using Apache Spark, Hadoop, Flink
Develop and maintain Spark applications for data transformation, aggregation, and ETL processes using Scala, Java, or Python
Utilize Apache Ozone for storing large-scale datasets, ensuring efficient data access and management in a distributed environment
Manage and optimize HDFS and Apache Ozone, Kafka for scalable and fault-tolerant storage.
Develop ETL pipelines for batch and real-time data ingestion and transformation.
Implement and ensure data validation, data security, integrity, and compliance across big data platforms.
Monitor and troubleshoot performance issues in large-scale clusters.
Collaborate with data scientists, analysts, and application teams to deliver high-quality data solutions.
Automate workflows and improve operational efficiency using scripting and orchestration tools.
Required Skills & Qualifications:
Strong expertise in Apache Spark (Core, SQL, Streaming).
Hands-on experience with Hadoop ecosystem (HDFS, YARN, MapReduce).
Proficiency in Apache Ozone for object storage and integration with Hadoop.
Solid programming skills in Java , Scala , or Python.
Experience with Hive, HBase , and Kafka is a plus.
Knowledge of cluster management and resource optimization.
Familiarity with Linux/Unix environments and shell scripting.
Understanding of data security, governance, and compliance standards.
Experience with cloud-based big data platforms
Exposure to containerization (Docker, Kubernetes) for big data workloads.
Knowledge of CI/CD pipelines for data engineering projects.
Behavioral Skills:
Good Communication skills
5 days Work from Office at Berkley Heights, NJ
Team Player
Ability to work in a changing environment
Strong problem solving and analytical skills
Ability to work independently or within a team
Manage day-to-day challenges and communicate developmental risks with the technical team
Qualifications:
Bachelor's degree in computer science, Software Engineering, or a related field.
Proficiency in business process modeling and documentation tools.
Product implementation experience is preferred
Data Engineer with Hadoop - • Berkeley Heights, NJ, United States
Job Title AWS Data Integration & ETL Analyst Overview / Summary Seeking a professional with strong SQL and ETL architecture expertise to partner with source system owners, business stakeholders, AI... Show more
Director Of Enterprise Data & Analytics Strategy.At EisnerAmper, we look for individuals who welcome new ideas, encourage innovation, and are eager to make an impact.Whether you're starting out in ... Show more
Review and label digital content including text, images, and documents.Every task you complete helps improve how technology interprets information and performs in practical settings.Detail-oriented... Show more
At Goldbelly, we believe food brings people together.We connect people with their greatest culinary desires within and beyond local communities.We empower food makers of all sizes and deliver their... Show more
Senior Software Engineer – Parsippany, NJ Dvelop softwre slutions; dsign/dvelop web applcations; intgrate systems using Microsft tchnologies & public cloud srvices & prform anlysis, dsign, cding, t... Show more
Who We Are :Managed Health Care Associates, Inc.MHA) provides care communities access, solutions, and insights to help them run their businesses more effectively.Our members include post-acute prov... Show more
PTC) is a leading E-Verified IT staffing and Solutions Company located in New Jersey, USA.Entry Level Full Time Business Analyst / BA Positions.Precision Technologies (PTC) is a leading E-verified ... Show more
Director For Data Architecture And Engineering.At GAF, we cover more than buildings.No matter what role, tenure, or track, under this roof you are empowered to be there for your teammates, your cus... Show more
Irys, originally launched as Bundlr, is an L2 protocol that enhanced the scalability of permanent data storage, handling over 96% of transactions on a major storage provider.Recognizing limitations... Show more
We are looking for a Mobile Applications Engineer to support our mobile app suite of products and develop automation tools to ensure quality control on high end medical mobile applications.The idea... Show more
Since its launch 6 years ago, over 60MN users have earned and withdrawn over $250MN! The platform is rated 4.TrustPilot with over 230k+ reviews, establishing Freecash as one of the highest-rated op... Show more
Director Of Commercial Data, Systems, And Business Intelligence.The Director of Commercial Data, Systems, and Business Intelligence is accountable for developing and overseeing the approach, delive... Show more
Hybrid minimum 4 days per week at various sites in the NY/NJ area.The Data Center Project Manager (M&E) is responsible for the end-to-end delivery of mechanical, electrical, and technology-infrast... Show more
We have an urgent position listed below.Please send your most recent resume along with the expected rate if you are interested.Job Role: Snowflake Solutions Architect Location: IFF office, Union Be... Show more
Technical Business Analyst Investments Data Platform.SGA is seeking a Technical Business Analyst Investments Data Platform for a long-term consulting assignment with one of our premier insurance ... Show more
Description:Job DescriptionData and Analytics Consulting Partner who will play a key role in driving solution architecture design, evaluation, and selection, buy vs.Partner with Key Roles (Business... Show more
Competencies: Digital : SaaS HCM-Oracle Talent Management Cloud.Develop and modify TCC integration scripts, understanding complex requirements and defining technical solutions.Take ownership of ass... Show more
Skill PM: Project Delivery Technology Databricks, Cloud Infrastructure, Agile Required:.Must have technical expertise in Databricks.Possesses in-depth knowledge of data processing techniques and ha... Show more
IT services company that specializes in cybersecurity, Big Data/databases, software development, systems testing, STIG Compliance training, closed-circuit television/security cameras and access con... Show more
Lead Data Engineer - Remote (Raleigh or Jersey City)Join Arch Insurance Group Inc.Lead Data Engineer to oversee the technical design and implementation of data solutions, champion best practices in... Show more
