Talent.com
Lead Data Platform Engineer
Lead Data Platform EngineerAlpaca • New York, NY, US
Lead Data Platform Engineer

Lead Data Platform Engineer

Alpaca • New York, NY, US
[job_card.30_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
[job_card.job_description]

Job Description

Job Description

Who We Are :

Alpaca is a US-headquartered self-clearing broker-dealer and brokerage infrastructure for stocks, ETFs, options, crypto, fixed income, 24 / 5 trading, and more. Our recent Series C funding round brought our total investment to over $170 million, fueling our ambitious vision.

Amongst our subsidiaries, Alpaca is a licensed financial services company, serving hundreds of financial institutions across 40 countries with our institutional-grade APIs. This includes broker-dealers, investment advisors, wealth managers, hedge funds, and crypto exchanges, totalling over 6 million brokerage accounts.

Our global team is a diverse group of experienced engineers, traders, and brokerage professionals who are working to achieve our mission of opening financial services to everyone on the planet . We're deeply committed to open-source contributions and fostering a vibrant community, continuously enhancing our award-winning, developer-friendly API and the robust infrastructure behind it.

Alpaca is proudly backed by top-tier global investors, including Portage Ventures, Spark Capital, Tribe Capital, Social Leverage, Horizons Ventures, Unbound, SBI Group, Derayah Financial, Elefund, and Y Combinator.

Our Team Members :

We're a dynamic team of 230+ globally distributed members who thrive working from our favorite places around the world, with teammates spanning the USA, Canada, Japan, Hungary, Nigeria, Brazil, the UK, and beyond!

We're searching for passionate individuals eager to contribute to Alpaca's rapid growth. If you align with our core values—Stay Curious, Have Empathy, and Be Accountable—and are ready to make a significant impact, we encourage you to apply.

Your Role :

We are seeking a Lead Data Engineer to design and develop the data management layer for our platform to ensure its scalability as we expand to larger customers and new jurisdictions. At Alpaca, data engineering encompasses financial transactions, customer data, API logs, system metrics, augmented data, and third-party systems that impact decision-making for both internal and external users. We process hundreds of millions of events daily, with this number growing as we onboard new customers.

We prioritize open-source solutions in our data management approach, leveraging a Google Cloud Platform (GCP) foundation for our data infrastructure. This includes batch / stream ingestion, transformation, and consumption layers for BI, internal use, and external third-party sinks. Additionally, we oversee data experimentation, cataloging, and monitoring and alerting systems.

Our team is 100% distributed and remote.

Things You Get To Do :

  • Design and oversee key forward and reverse ETL patterns to deliver data to relevant stakeholders.
  • Develop scalable patterns in the transformation layer to ensure repeatable integrations with BI tools across various business verticals.
  • Expand and maintain the constantly evolving elements of the Alpaca Data Lakehouse architecture.
  • Collaborate closely with sales, marketing, product, and operations teams to address key data flow needs.
  • Operate the system and manage production issues in a timely manner.

Who You Are (Must-Haves) :

  • 7+ years of experience in data engineering, including 2+ years of building scalable, low-latency data platforms capable of handling >
  • 100M events / day.

  • Proficiency in at least one programming language, with strong working knowledge of Python and SQL.
  • Experience with cloud-native technologies like Docker, Kubernetes, and Helm.
  • Strong hands-on experience with relational database systems.
  • Experience in building scalable transformation layers, preferably through formalized SQL models (e.g., dbt).
  • Ability to work in a fast-paced environment and adapt solutions to changing business needs.
  • Experience with ETL technologies like Airflow and Airbyte.
  • Production experience with streaming systems like Kafka.
  • Exposure to infrastructure, DevOps, and Infrastructure as Code (IaaC).
  • Deep knowledge of distributed systems, storage, transactions, and query processing.
  • How We Take Care of You :

  • Competitive Salary & Stock Options
  • Health Benefits
  • New Hire Home-Office Setup : One-time USD $500
  • Monthly Stipend : USD $150 per month via a Brex Card
  • Alpaca is proud to be an equal opportunity workplace dedicated to pursuing and hiring a diverse workforce.

    Recruitment Privacy Policy

    [job_alerts.create_a_job]

    Data Platform Engineer • New York, NY, US

    [internal_linking.similar_jobs]
    Senior Software Engineer (Data Platform)

    Senior Software Engineer (Data Platform)

    Rokt • New York, NY, US
    [job_card.full_time]
    We are Rokt, a hyper-growth ecommerce leader.Rokt is the global leader in ecommerce, unlocking real-time relevance in the moment that matters most. Rokt’s AI Brain and ecommerce Network powers...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Data Engineer

    Senior Data Engineer

    January • New York, New York, United States
    [job_card.full_time] +1
    At January, we're fixing what's broken in credit.Our data-driven platform rebuilds trust, delivers results, and helps millions move toward brighter financial futures while bringing humanity to cons...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Data Engineer

    Data Engineer

    MetroPlus Health Plan • New York, NY, United States
    [job_card.full_time] +1
    Water Street, 7th Floor, New York, NY 10004 .The position of Data Engineer in the Analytics and Reporting Department is responsible in managing encounter data submission to NYSDOH, CMS and other...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Data Engineer

    Data Engineer

    Soho Dragon • New York, New York, United States
    [job_card.full_time]
    SoHo Dragon represents a tech-driven, AI-native pharmaceutical company that’s shaping the future of healthcare.We’re looking for a mid-to-senior-level Data Engineer to play a key role in building a...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Data Engineer, Platform

    Data Engineer, Platform

    Basis Research Institute • New York, NY, United States
    [job_card.full_time]
    AI research organization with two mutually reinforcing goals.This means to establish the mathematical principles of what it means to reason, to learn, to make decisions, to understand, and to expla...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Data Engineer

    Data Engineer

    Tabs • New York, New York, United States
    [job_card.full_time]
    AI-native revenue platform for modern finance and accounting teams.Tabs agents automates the entire contract-to-cash lifecycle, including billing, collections, revenue recognition, and reporting, t...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Platform Engineer

    Platform Engineer

    Lovefreedom Solution • New York, NY, US
    [job_card.full_time]
    As a Platform Engineer, you'll play a crucial role in building and maintaining the foundation, our core AI technologies that power our suite of AI-native applications.This position offers an ex...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    AI Engineer - Data Platform

    AI Engineer - Data Platform

    Traversal • New York, NY, United States
    [job_card.full_time]
    AI Site Reliability Engineer (SRE) for the enterprise—already trusted by some of the largest companies in the world to troubleshoot, remediate, and even prevent the most complex production incident...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Data Engineer

    Data Engineer

    10a Labs • New York, New York, United States
    [job_card.full_time]
    Labs is an applied research and AI security company trusted by AI unicorns, Fortune 10 companies, and U.We combine proprietary technology, deep expertise, and multilingual threat intelligence to de...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Cloud Data Engineer

    Cloud Data Engineer

    Govx • New York, New York, United States
    [filters.remote]
    [job_card.full_time]
    The Cloud Data Engineer is part of a Data team that is responsible for supporting, modernizing, and transforming our data and reporting capabilities across our products by implementing a new modern...[show_more]
    [last_updated.last_updated_variable_days] • [promoted]
    Lead Data Analytics

    Lead Data Analytics

    Altice USA • Long Island City, NY, United States
    [job_card.full_time]
    Driven by the power of Optimum, Optimum Media is our innovative multiscreen advertising sales and media consultancy business servicing small to medium business as well as national, political, media...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Data Engineer, Product Platform

    Senior Data Engineer, Product Platform

    Match Group • New York, New York, United States
    [job_card.full_time]
    Hinge is the dating app designed to be deleted.In today's digital world, finding genuine relationships is tougher than ever. At Hinge, we’re on a mission to inspire intimate connection to create a l...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Data Engineer

    Data Engineer

    Coast • New York, New York, United States
    [job_card.full_time]
    Coast is re-imagining the trillion-dollar U.B2B card payments infrastructure, with a focus on the country’s 500,000 commercial fleets, 40 million commercial vehicles, and many million commercial dr...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Data Engineer, Data Infrastructure

    Senior Data Engineer, Data Infrastructure

    Zocdoc • New York, New York, United States
    [job_card.full_time]
    Healthcare should work for patients, but it doesn’t.In their time of need, they call down outdated insurance directories. Then wait weeks for the privilege of a visit.Then wait in a room solely desi...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Data Engineer

    Senior Data Engineer

    Plum Inc • New York, New York, United States
    [filters.remote]
    [job_card.full_time]
    PLUM is a fintech company empowering financial institutions to grow their business through a cutting-edge suite of AI-driven software, purpose-built for lenders and their partners across the financ...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Sr Google Cloud Platform Data Engineers

    Sr Google Cloud Platform Data Engineers

    Epathusa • New York, New York, United States
    [job_card.full_time]
    Seeking Senior Google Cloud Platform Data Engineers to design, develop, and maintain robust data pipelines that transform raw data into valuable insights, as well as : . Design, develop, and maintain...[show_more]
    [last_updated.last_updated_variable_hours] • [promoted] • [new]
    Data Engineer

    Data Engineer

    Fanduel • New York, New York, United States
    [job_card.full_time]
    Our roster has an opening with your name on it.At FanDuel, our data platform underpins critical business operations that need to run with maximum efficiency and scalability.As our business continue...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Data Engineer II

    Data Engineer II

    Capital Rx • New York, NY, United States
    [job_card.full_time]
    Judi Health is an enterprise health technology company providing a comprehensive suite of solutions for employers and health plans, including : . PBM) solutions to self-insured employers,.Enterprise H...[show_more]
    [last_updated.last_updated_1_day] • [promoted]