Talent.com
Data Engineer
Data EngineerForhyre • San Francisco, CA, US
Data Engineer

Data Engineer

Forhyre • San Francisco, CA, US
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Job Description

Job Description

We are looking for a passionate certified Data Engineer. The successful candidate will turn data into information, information into insight and insight into business decisions.

Data analyst responsibilities include conducting full lifecycle analysis to include requirements, activities and design. Data analysts will develop analysis and reporting capabilities. They will also monitor performance and quality control plans to identify improvements.

Primary skillset: Python/Pyspark & Azure, ADF, SQL, SQL Server, Data Warehousing, ETL
Secondary: Databricks
Nice to have: Informatica/ETL.

Responsibilities

  • Design and develop ETL processes based on functional and non-functional requirements in python / pyspark within Azure platform
  • Understand the full end to end development activities from design to go live for ETL development and Azure platform
    Recommend and execute improvements
  • Document component design for developers and for broader communication.
  • Understand and adopt an Agile (SCRUM like) software development mindset
  • Follow established processes/standards, business technology architecture for development, release management and deployment process
  • Execute and provide support during testing cycles and post-production deployment, engage in peer code reviews.
  • Elicit, analyze, interpret business and data requirements to develop complete business solutions, includes data models (entity relationship diagrams, dimensional data models), ETL and business rules, data life cycle management, governance, lineage, metadata and reporting elements.
  • Apply automation and innovation on new and on-going data platforms for those development projects aligned to business or organizational strategies.
  • Design, develop and implement reporting platforms (e.g. modeling, ETL, BI framework) and complex ETL frameworks that meet business requirements.
  • Deliver business or enterprise data deliverables (that adhere to enterprise frameworks) for various platforms/servers/applications/systems.

Requirements

  • Proven working experience as a data engineer
  • Bachelor degree or equivalent in Computer Science
  • Skilled in Python object-oriented programming
  • Skilled in AWS Compute such as EC2, Lambda, Beanstalk, or ECS
  • Skilled in AWS Database products such as Neptune, RDS, Redshift, or Aurora
  • Skilled in AWS Management and Governance suite of products such as CloudTrail, CloudWatch, or Systems Manager
  • Skilled in Amazon Web Services (AWS) offerings, development, and networking platforms
  • Skilled in SQL
  • Skilled in Jenkins
  • Skilled in JSON
  • Skilled in discovering patterns in large data sets with the use of relevant software such as Oracle Data Mining or Informatica
  • Skilled in cloud technologies and cloud computing
  • Experience using software and computer systems' architectural principles to integrate enterprise computer applications such as xMatters, AWS Application Integration, or WebSphere
  • Determining causes of operating errors and taking corrective action
  • Experience in the process of analyzing data to identify trends or relationships to inform conclusions about the data
  • Skilled in creating and managing databases with the use of relevant software such as MySQL, Hadoop, or MongoDB
  • Programming including coding, debugging, and using relevant programming languages
  • Communication including communicating in writing or verbally, copywriting, planning and distributing communication, etc.
  • Ability to frame ideas as systems and analyzing the inputs, outputs, and process
  • Experience helping an organization to plan and manage change in effort to meet strategic objectives
  • Adept at managing project plans, resources, and people to ensure successful project completion
  • Working with people with different functional expertise respectfully and cooperatively to work toward a common goal

[job_alerts.create_a_job]

Data Engineer • San Francisco, CA, US

[internal_linking.similar_jobs]

Data Engineer

Perfict Global, Inc.San Francisco, CA, United States
[job_card.full_time]

Hybrid - Monday/Friday Remote + Onsite.San Francisco, CA on Tuesday/Wed/Thursday.You will build systems, core libraries and frameworks that power our batch and streaming Data and ML applications.Th...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer

ROBOTSan Francisco, CA, United States
[job_card.full_time]

Data Engineer Opportunity At Kiwibot.Kiwibot is at the forefront of robotic delivery solutions, transforming urban logistics with autonomous technology.Our mission is to make last-mile delivery eff...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer

AdobeSan Francisco, CA, United States
[job_card.full_time]

Adobe Customer Solutions is looking for a full time Data Engineer with experience in building data integrations using AWS technology stack as part of the team's Data as a Service portfolio for Adob...[internal_linking.show_more]

 • [job_card.promoted]

Lead Data Engineer

Nuna IncSan Francisco, CA, United States
[job_card.full_time]

At Nuna, our mission is to make high-quality healthcare affordable for everyone.We are dedicated to tackling one of our nation's biggest problems with ingenuity, creativity, and a keen moral compas...[internal_linking.show_more]

 • [job_card.promoted]

Senior Staff Data Engineer

SoFiSan Francisco, CA, United States
[job_card.full_time]

Employee Applicant Privacy Notice.Shape a brighter financial future with us.Together with our members, we're changing the way people think about and interact with personal finance.We're a next-gene...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer-III

Hire TalentSan Francisco, CA, United States
[job_card.full_time]

Job Title: Data Engineer - III.We are looking to hire a contractor to support data and analytics work for a start-up team at *** that works on building a point of sale system that we sell to restau...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer

Stepping Up SolutionsSan Francisco, CA, United States
[job_card.full_time]

RadiumOne is a global programmatic ad buying platform.RadiumOne is the 6th largest web property in the U.We build intelligent software that automates media buying, making big data actionable for ma...[internal_linking.show_more]

 • [job_card.promoted]

Lead Data Engineer

Q-CellsSan Francisco, CA, United States
[job_card.full_time]

This role provides technical leadership in designing and optimizing ETL/ELT frameworks using Azure data services (Fabric, Data Lake, Data Factory), integrating data from ERP, CRM, and operational s...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer

RedolentSan Bruno, CA, United States
[job_card.full_time]

Designs, develops, and implements Hadoop eco-system based applications to support business requirements.Follows approved life cycle methodologies, creates design documents, and performs program cod...[internal_linking.show_more]

 • [job_card.promoted]

AWS Data Engineer

Diverse LynxSan Francisco, CA, United States
[job_card.full_time]

Design and manage AWS-based data infrastructure using services such as S3, Glue, EMR, Redshift, Athena, Lambda, Kinesis, and RDS.Build and maintain scalable, fault-tolerant data pipelines (batch an...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer

AEG PresentsSan Francisco, CA, United States
[job_card.full_time]

In order to be considered for this role, after clicking "Apply Now" above and being redirected, you must fully complete the application process on the follow-up screen.Swish Analytics is a sports a...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer (Core Platform & Optimization)

Hilbert's AISan Francisco, CA, United States
[job_card.full_time]

The Mission: Build the High-Performance Refinery.At Hilbert's AI, we shorten "months" to "minutes.To achieve this, our internal data systems must be more than just pipelines-they must be a high-vel...[internal_linking.show_more]

 • [job_card.promoted]

Senior Data Engineer

Triumph Arcade, IncSan Francisco, CA, United States
[job_card.full_time]

Triumph makes mobile gaming more thrilling by letting players wager -- and win -- real money, play in mass multiplayer games, and compete in social tournaments.We've built the top app in our App St...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer

GridCARERedwood City, CA, United States
[job_card.full_time]

GridCARE solves data center developers' most urgent bottleneck immediate access to power through a pioneering physics-based generative AI platform that unlocks gigawatts of near-term capacity from ...[internal_linking.show_more]

 • [job_card.promoted]

Founding Data Engineer

Assort HealthSan Francisco, CA, United States
[job_card.full_time]

Our mission is to make exceptional healthcare accessible anytime, anywhere, for anyone.At Assort Health, we believe healthcare should feel effortless and connected quick answers, clear communicatio...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer, Analytics

OpenAISan Francisco, CA, United States
[job_card.full_time]

The Applied team works across research, engineering, product, and design to bring OpenAI's technology to consumers and businesses.We seek to learn from deployment and distribute the benefits of AI,...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer

Mercor AlabasterSan Francisco, CA, United States
[job_card.full_time]

Mercor is defining the future of work.We partner with leading AI labs and enterprises to provide the human intelligence essential to AI development.Our vast talent network trains frontier AI models...[internal_linking.show_more]

 • [job_card.promoted]

Data Engineer

Purple DriveSan Francisco, CA, United States
[job_card.full_time]

The ideal candidate would be responsible for designing, developing, testing, deploying, and maintaining enterprise-level applications using the Snowflake platform.Knowledge of SQL language and clou...[internal_linking.show_more]