Position: Senior Data Engineer – (Hadoop/Big Data, AWS, Python, Kinesis)
Location: Irvine, CA - Onsite role
Position Type: Contract to Hire
Responsibilities
You will be responsible for all aspects of data acquisition, data transformation, analytics scheduling and operationalization to drive high-visibility, cross-division outcomes. Investigate, evaluate, test and recommend technical solutions for future systems.
They will support software developers, database architects, data scientists on data initiatives and will contribute optimal data delivery architecture.
What you will be doing:
Data Operations
Lead the creation of data environments and/or data sets to serve a wide range of data users, including but not limited to Data Scientists, Data Analysts, Business Analysts etc.
Perform offline analysis of large data sets using components of a big data software ecosystem.
Validate the solution of root cause analysis escalated by various technical staff in multiple organizations and with differing levels of expertise.
Investigate, evaluate, test and recommend technical solutions for future systems.
Data Management
Own product data sets from the definition phase through to production deployment (end-to-end).
Provide solutions for the design and implementation of Hadoop EMR Cluster/ Big Data Infrastructure.
Deploy Hadoop/Big Data/Spark and database storage Infrastructures in AWS cloud.
Monitor HDFS/Hadoop/Spark and related software releases, third-party utilities with emphasis on overall system performance.
Lead and develop tools and procedures to monitor and automate system tasks on servers and clusters
Lead and collaborate with other teams to design, develop, and deploy data tools that support both operations and product use cases.
Data Design
Lead and design distributed, scalable, and reliable data pipelines that ingest and process data at scale and in real-time.
Lead and design big data technologies and prototype solutions to improve data processing architecture.
Qualifications
What we are looking for:
Bachelor’s degree in computer science, computer engineering, or a related technical field.
12+ years of professional experience as a data software engineer; or 16+ years of related experience as a data software engineer in lieu of 4-year degree.
2+ years of experience with AWS cloud or other cloud Big Data computing design, provisioning, and tuning.
Related AWS certification, preferred.
Previous experience as a Data Engineer / Database Administrator and/or Business Intelligence Analyst.
Expertise in database concepts, object and data modeling techniques and design principles.
Expertise in database architectures, software, and facilities
Expertise with programming languages - Python (required), Scala, Ruby, R Database technologies - SQL, performance tuning concepts, AWS RDS, RedShift, MySQL
Expertise with big data batch processing tools: Hadoop MapReduce, ElasticSearch, PIG, Hive, Cascading/Scalding, Apache Spark, AWS EMR
Expertise with stream-processing systems: Kinesis, Kafka, MQTT
Expertise with relational NoSQL databases including DyanamoDB
Expertise in writing JSON, XML, YAML and other data definition schemas
Excellent verbal and written communication skills necessary to effectively collaborate in a team environment and present and explain technical information and provide advice to management.
Ability to work on advanced complex technical projects or business issues requiring state of the art technical knowledge or industry.
Ability to work on significant and unique issues where analysis of situations or data requires an evaluation of intangibles.
Exercises independent judgment in methods, techniques and evaluation criteria for obtaining results.
Ability to lead and mentor junior engineers and colleagues.
Principal Data Engineer – (Hadoop/Big Data, AWS, Python, Kinesis) - Irvine, CA - Onsite - Contract to Hire - AS • Irvine, CA, United States
We are seeking a highly skilled and experienced Data Engineering Lead / Architect to join our dynamic team.The ideal candidate will have a proven track record of designing, building, and maintainin... Show more
JOB DESCRIPTION Req ID:354633 We are currently seeking a Healthcare Data Engineer - REMOTE to join our team in Irvine, California (US-CA), United States (US).Day to Day Responsibilities:Uses a work... Show more
Senior Project Manager R&D | Medical Device New Product Development.Now Hiring: Senior Project Manager R&D | Medical Device New Product Development San Clemente, CA (Hybrid) Competitive Salary | ... Show more
Principal Data Scientist, Clinical (U.Remote & Temp to Hire)Position OverviewAct as a strategic collaborator across Medical Affairs and cross-functional teams, helping to translate complex clin... Show more
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
We are urgently seeking people interested in taking market research studies for well known brands.If you are a self-starter, looking for flexible hours throughout the week, this may be for you! Ear... Show more
Data EngineerThe Data Engineer will support the team in measuring and managing material readiness using custom tools, databases, data lakes, data warehouses, and business intelligence interfaces in... Show more
Must reside in PST Time Zone and be a US Citizen Job Overview We are seeking a skilled Data Engineer with 3-5 years of experience specializing in data analysis, writing queries, query optimization,... Show more
Product Testers are wanted to work from home nationwide in the US to fulfill upcoming contracts with national and international companies.We guarantee 15-25 hours per week with an hourly pay of bet... Show more
The Trade Compliance Analyst is responsible for ensuring ICU Medicals exports and dangerous goods movements are in compliance with local government laws and regulations governed by U.Customs and Bo... Show more
As an Order Picker at Walmart, you will be responsible for accurately and efficiently selecting merchandise from storage locations to fulfill customer orders or replenish inventory for Walmart stor... Show more
Amazon warehouse associates hiring in your area.If you are looking for outstanding pay, flexible part-time, full-time and seasonal schedules, plus benefits for you and your family, the Amazon Logis... Show more
The R&D Project Coordinator will support the Business Development R&D department.The Project Coordinator is responsible for supporting the development and implementation of products through adheren... Show more
Location: Irvine, California Salary: ***K/Y.Hands-on Engineering Manager to lead AI Platform and GenAI product engineering initiatives.Lead delivery execution for AI Platform initiatives.Contribute... Show more
Manager, Data ScienceFind joy in serving others with IEHP! We welcome you to join us in healing and inspiring the human spirit and to pivot from a job opportunity to an authentic experience!The Man... Show more
Location: Clemente, CA (Hybrid).Duration: 11 Months Pay Rate: $55-$57 on W2 per hour.Position Summary: The Senior Project Manager oversees enterprise type projects.Projects are often complex in nat... Show more
Cloud & API Platforms Practice Lead (AVP).The Cloud & API Platforms Practice Lead (AVP), in Perficient's Services/Delivery Organization, blends broad solution and practice leadership with technolog... Show more
Global Oracle Integration Program Manager.Location: San Clemente, CA - Hybrid.The Senior Program Manager Lasso will report to the ICU Global IT PMO Director and will be responsible for the plannin... Show more
Senior Program Manager - Lasso.The Senior Program Manager Lasso will report to the ICU Global IT PMO Director and will be responsible for the planning, implementation, and tracking of a large glob... Show more
About the job Cloud Engineer (Remote)WE ARE HIRING !My recruiter Inc caters to grow the business to next level by providing professional services to empower the organizational values with right sui... Show more
