Talent.com
Opplane Portugal Unipessoal Lda
Senior Data EngineerOpplane Portugal Unipessoal Lda • Santa Clara, California, United States
No longer accepting applications
Senior Data Engineer

Senior Data Engineer

Opplane Portugal Unipessoal Lda • Santa Clara, California, United States
30+ days ago
Job type
  • Full-time
  • Remote
  • Quick Apply
Job description

We are looking for a Senior Data Engineer with minimum 5+ years of professional experience to design, build, and maintain scalable, high-performance data platforms that power our analytics and machine learning initiatives. You will work in a distributed, data-intensive environment and collaborate with cross-functional teams to ensure data accuracy, reliability, and security.

Roles & Responsibilities:

  • Design, develop, and maintain scalable data processing pipelines and workflows using frameworks such as Apache Spark, PySpark, and Apache Beam.
  • Build and maintain microservices in Python that serve data-driven features in production.
  • Develop internal tools to support CI/CD pipelines, experiment tracking, and data versioning.
  • Collect, process, and integrate large datasets from multiple sources, including databases, file systems, and APIs.
  • Ensure data integrity, consistency, and quality through robust validation and monitoring processes.
  • Optimize data systems for performance, scalability, and high availability.
  • Implement best practices for data security, access control, and privacy.
  • Collaborate with data scientists, analysts, and engineers to support analytics and ML workflows.
  • Lead complex migration initiatives involving the transition from on-premise Cloudera (HDFS, Hive, Impala, HBase) environments to cloud-native platforms (Snowflake or Databricks), ensuring zero data loss and minimal downtime.
  • Architect and build a large-scale, greenfield data platform from the ground up, including the setup of end-to-end infrastructure, data ingestion layers, transformation frameworks, and high-throughput integrations with Snowflake or Databricks.

Must have:

  • Strong understanding of distributed systems and modern data architectures.
  • Proven, hands-on experience with large-scale data platform migrations, specifically transitioning from Cloudera (CDH/HDP) to either Snowflake or Databricks. This includes re-architecting Hive/Impala workloads, migrating HDFS data, and refactoring legacy ETL processes to leverage cloud-native features (e.g., Snowpipe, Auto-ingestion, or Delta Live Tables).
  • Deep technical expertise in building and orchestrating a high-performance data platform from scratch, including the implementation of complex integrations (Reverse ETL, CDC, API ingestions) specifically targeting Snowflake (using Snowpipe Streaming, Tasks, Streams) or Databricks (using Auto Loader, Delta Sharing, Unity Catalog).
  • 5+ years of professional experience in software engineering or data engineering.
  • Strong software engineering skills with Python in large-scale, high-performance production environments.
  • Hands-on experience with Spark/PySpark and other big data frameworks.

About Opplane

Opplane specializes in providing advanced data-focused solutions for financial services, telecommunication, and reg-tech to accelerate their digital transformation journey. Opplane leadership team is comprised of Silicon Valley serial entrepreneurs and experienced executives. Its expertise comes from years of specific industry experience at some of the world’s top companies, such as PayPal, Xerox Parc, Amazon, Wells Fargo, SoFi in the areas of product management, data technology, data governance, data privacy, security, machine learning, and risk management.

Global & Multicultural – Diverse perspectives, global collaboration (US, Portugal, India and Singapore offices)
Startup Energy – Fast-moving, impact-driven environment
Ownership Mindset – Engineers own what they build
Collaborative & Friendly – Open, curious, and supportive culture



Create a job alert for this search

Senior Data Engineer • Santa Clara, California, United States

Similar jobs

Senior Staff Engineer, Agentic AI Engineering

WaymoMountain View, CA, United States
Full-time

Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver.Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on buildin... Show more

 • Promoted

Data Engineer

SynergisticITFremont, CA, United States
Full-time

Launch Your Data Career with Proof, Not PromisesAt SynergisticitIf you're serious about starting a high-impact, high-paying career in Data Science, Analytics, or Engineering, it's time to stop gues... Show more

 • Promoted

Data Engineer - Remote

EXPERISSunnyvale, CA, United States
Remote
Full-time

Data EngineerWe're hiring a Data Engineer to design, build, and operate the data pipelines and platform that power analytics and AI / ML use cases.You'll own batch and streaming ingestion, transfor... Show more

 • Promoted

Data Engineer- Mountain View, CA

Staffing the UniverseMountain View, CA, United States
Full-time

Data EngineerLocation:Mountain View, CADuration:ContractRate:DOERequired Skills:10years of overall experience in data management space and at least 5 years of working in large data sets in a data l... Show more

 • Promoted

Data Engineer

InterSourcesFremont, CA, United States
Full-time

Certified Databricks ConsultantLocation:TX/GA/CADuration:Long-termClient:IBM/AT&TSkills:Need certified Databricks consultant with good experience in Spark and PySpark.Experience with Azure, Pyt... Show more

 • Promoted

Senior Engineer: AI Applications SF, NYC, Remote

SwayablePalo Alto, CA, United States
Remote
Full-time

About SwayableSwayable is a fast-growing cloud data product that doubles the impact of advertising by empirically measuring how it changes people's opinions.Our partners include Amazon, Meta, IBM, ... Show more

 • Promoted

Senior Staff Engineer, AI Core

NubankPalo Alto, CA, United States
Full-time

Nu is the leading digital bank in Latin America, serving 135 million customers across Brazil, Mexico, and Colombia.The company has been leading an industry transformation by leveraging data and pro... Show more

 • Promoted

Senior Staff Engineer, Wafer-Scale Bring-Up

Cerebras Systems, Inc.Sunnyvale, CA, United States
Full-time

This role involves close collaboration across chip design, software development, and productization to deliver end-to-end solutions.The ideal candidate will have a strong educational background in ... Show more

 • Promoted

Lead Data Engineer

QcellsSanta Clara, CA, US
Full-time

This role provides technical leadership in designing and optimizing ETL/ELT frameworks using Azure data services (Fabric, Data Lake, Data Factory), integrating data from ERP, CRM, and operational s... Show more

Lead AI Data Engineer (Remote)

Allocate IncPalo Alto, CA, United States
Remote
Full-time

SummaryAllocate is looking for a Lead AI Data Engineer to design and implement the foundational data infrastructure that will power our AI-driven portfolio analysis and investment recommendation ag... Show more

 • Promoted

Remote Senior Data Engineer - AI-Powered Pipelines

Altimate.aiSunnyvale, CA, United States
Remote
Full-time

A leading AI-driven data solutions company in California is seeking a Senior Software Engineer to enhance AI capabilities and build scalable data infrastructures.You will design robust data systems... Show more

 • Promoted

Data Engineer

ShiftCode AnalyticsSan Ramon, CA, United States
Full-time

Data EngineerLocation:San Ramon, CA (100% Onsite)Duration:6-12 MonthsPositions:2Start Date:1/9/23Job Description:Integrate, extract the data using ETL and inject using AWS, build pipeline from AWS ... Show more

 • Promoted

Software Principal Engineer- Data Storage

DellCupertino, CA, United States
Full-time

Software Principal EngineerThe Software Engineering team delivers next-generation application enhancements and new products for a changing world.Working at the cutting edge, we design and develop s... Show more

 • Promoted

Senior Hadoop Architect -- Remote Data Platform Lead

Tredence Inc.San Jose, CA, United States
Remote
Full-time

A consulting firm is seeking a Principal Hadoop Architect to lead the optimization of a large-scale Big Data ecosystem.This role involves defining design standards and ensuring a highly optimized o... Show more

 • Promoted

Data Engineer

Staffing the UniverseSanta Clara, CA, United States
Full-time

Data EngineerLocation:RemoteExperience:4YearsAbout the Role:A Fintech is seeking a strong and passionate data engineer with experience in large scale system implementation, with a focus on complex ... Show more

 • Promoted

Senior Software Engineer - Data Infrastructure

Applied IntuitionSunnyvale, CA, United States
Full-time

Senior Software Engineer - Data InfrastructureApplied Intuition, Inc.Founded in 2017 and now valued at $15 billion, the Silicon Valley company is creating the digital infrastructure needed to bring... Show more

 • Promoted

Remote Data Engineer

Global Channel ManagementSanta Clara, CA, United States
Remote
Full-time

About the job Remote Data EngineerRemote Data Engineer needs 10 years experienceRemote Data Engineer requires :Must be local top Santa ClaraCloud Platform Expertise (GCP Focus) :Extensive hands-on ... Show more

 • Promoted

Data Engineer (26144-1) Santa Clara, CA

ESR HealthcareSanta Clara, CA, United States
Full-time

Lead governance, planning, and cross-functional coordination for the two company IT integration, ensuring strategic alignment, stakeholder engagement, and Day 1/Day 2 readiness across all technical... Show more

 • Promoted

Data Engineer (Python, Data Engg, Kafka) | Hybrid

Kaav Inc.Sunnyvale, CA, United States
Full-time

Marketing Technology Team PositionThe Marketing Technology team has the mandate of increasing the rate of customer acquisition, retention, and revenue by optimizing products and inventing new marke... Show more

 • Promoted

Staff Software Engineer, Data (Bay Area / Paris / Remote)

GenesisAISunnyvale, CA, United States
Remote
Full-time

What You'll DoDesign, build, and maintain large-scale data pipelines (batch and streaming) for robotics foundation model training and evaluation at petabyte scaleOwn core data infrastructure :data ... Show more