Talent.com
Data Engineer (Middle) ID43228
Data Engineer (Middle) ID43228AgileEngine • West Palm Beach, FL, us
[error_messages.no_longer_accepting]
Data Engineer (Middle) ID43228

Data Engineer (Middle) ID43228

AgileEngine • West Palm Beach, FL, us
[job_card.variable_days_ago]
[job_preview.job_type]
  • [job_card.full_time]
  • [filters_job_card.quick_apply]
[job_card.job_description]

Job Description

AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI / ML, and our people-first culture has earned us multiple Best Place to Work awards.

WHY JOIN US

If you're looking for a place to grow, make an impact, and work with people who care, we'd love to meet you!

ABOUT THE ROLE

As a Middle / Senior Data Engineer , you’ll help build a high-impact Thematic Research Platform that processes large-scale financial data and powers daily insights for researchers and data scientists. You’ll design and optimize ETL pipelines, develop cloud data warehousing solutions, and shape AWS-based infrastructure using tools like Airflow, Spark, and Terraform. This role offers the chance to work on complex data ecosystems, drive performance improvements, and collaborate closely with cross-functional teams in a technically rich, mission-driven environment.

WHAT YOU WILL DO

  • Build and support ETL pipelines;
  • Monitor data pipelines, identify bottlenecks, optimize data processing and storage for performance and cost-effectiveness;
  • Collaborate effectively with cross-functional teams including data scientists, analysts, software engineers, and business stakeholders;
  • Work with Terraform to build AWS infrastructure;
  • Analyze sources and build Cloud Data Warehouse and Data Lake solutions.

MUST HAVES

  • 3+ years of experience with Python ;
  • 3+ years of experience in a Data Engineering role;
  • Proficiency in programming languages commonly used in data engineering such as Python, SQL, and optionally Scala for working with data processing frameworks like Spark and libraries like Pandas;
  • Proficiency in designing, deploying, and managing data pipelines using Apache Airflow for workflow orchestration and scheduling;
  • Ability to design, develop, and optimize ETL processes to move and transform data from various sources into the data warehouse, ensuring data quality, reliability, and efficiency;
  • Knowledge of big data technologies and frameworks such as Apache Spark for processing large volumes of data efficiently;
  • Extensive hands-on experience with various AWS services relevant to data engineering, including Amazon MWAA, Amazon S3, Amazon RDS, Amazon EMR, AWS Lambda, AWS Glue, Amazon Redshift, AWS Data Pipeline, Amazon DynamoDB;
  • Deep understanding and practical experience in building and optimizing cloud data warehousing solutions;
  • Ability to monitor data pipelines, identify bottlenecks, and optimize data processing and storage for performance and cost-effectiveness;
  • Excellent communication skills to collaborate effectively with cross-functional teams;
  • Bachelor’s degree in computer science / engineering or other technical field, or equivalent experience;
  • Upper-intermediate English level.

    • NICE TO HAVES

  • Familiarity with the fintech industry, understanding of financial data, regulatory requirements, and business processes specific to the domain;
  • Documentation skills to document data pipelines, architecture designs, and best practices for knowledge sharing and future reference;
  • Experience with GCP services relevant to data engineering;
  • Experience with Snowflake;
  • Experience with OpenSearch or Elasticsearch;
  • Experience using Jupyter for data analysis;
  • Experience with Bitbucket and Bamboo;
  • Experience with Terraform.

    • PERKS AND BENEFITS

  • Professional growth : Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps.
  • Competitive compensation : We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities.
  • A selection of exciting projects : Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands.
  • Flextime : Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office – whatever makes you the happiest and most productive.

    • Requirements

    3+ years of experience with Python; 3+ years of experience in a Data Engineering role; Proficiency in programming languages commonly used in data engineering such as Python, SQL, and optionally Scala for working with data processing frameworks like Spark and libraries like Pandas; Proficiency in designing, deploying, and managing data pipelines using Apache Airflow for workflow orchestration and scheduling; Ability to design, develop, and optimize ETL processes to move and transform data from various sources into the data warehouse, ensuring data quality, reliability, and efficiency; Knowledge of big data technologies and frameworks such as Apache Spark for processing large volumes of data efficiently; Extensive hands-on experience with various AWS services relevant to data engineering, including Amazon MWAA, Amazon S3, Amazon RDS, Amazon EMR, AWS Lambda, AWS Glue, Amazon Redshift, AWS Data Pipeline, Amazon DynamoDB; Deep understanding and practical experience in building and optimizing cloud data warehousing solutions; Ability to monitor data pipelines, identify bottlenecks, and optimize data processing and storage for performance and cost-effectiveness; Excellent communication skills to collaborate effectively with cross-functional teams; Bachelor’s degree in computer science / engineering or other technical field, or equivalent experience; Upper-intermediate English level.

    [job_alerts.create_a_job]

    Data Engineer • West Palm Beach, FL, us

    [internal_linking.related_jobs]
    Lead Machine Learning Engineer

    Lead Machine Learning Engineer

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    Launch Potato is a digital media company.As The Discovery and Conversion Company, Launch Potato is a leading connector of advertisers to customers at all parts of the consumer journey, from awarene...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Machine Learning Engineer

    Principal Machine Learning Engineer

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Lead Data Analyst, Marketing

    Lead Data Analyst, Marketing

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    Launch Potato is a digital media company.As The Discovery and Conversion Company, Launch Potato is a leading connector of advertisers to customers at all parts of the consumer journey, from awarene...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Machine Learning Engineer, Ad Performance

    Principal Machine Learning Engineer, Ad Performance

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Lead Machine Learning Engineer, Recommendation Systems

    Lead Machine Learning Engineer, Recommendation Systems

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Engineer

    Senior ML Engineer

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    Launch Potato is a digital media company.As The Discovery and Conversion Company, Launch Potato is a leading connector of advertisers to customers at all parts of the consumer journey, from awarene...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Machine Learning Engineer, Ad Performance

    Senior Machine Learning Engineer, Ad Performance

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Lead ML Engineer, Ad Performance

    Lead ML Engineer, Ad Performance

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Staff DevOps Engineer, Data

    Staff DevOps Engineer, Data

    GoodLeap • West Palm Beach, Florida, United States, 33401
    [job_card.full_time]
    About GoodLeap : GoodLeap is a technology company delivering best-in-class financing and software products for sustainable solutions, from solar panels and batteries to energy-efficient HVAC, heat pu...[show_more]
    [last_updated.last_updated_30]
    Lead ML Engineer

    Lead ML Engineer

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    Launch Potato is a digital media company.As The Discovery and Conversion Company, Launch Potato is a leading connector of advertisers to customers at all parts of the consumer journey, from awarene...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Lead ML Engineer, Recommendation Systems

    Lead ML Engineer, Recommendation Systems

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal Machine Learning Engineer, Recommendation Systems

    Principal Machine Learning Engineer, Recommendation Systems

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior Machine Learning Engineer

    Senior Machine Learning Engineer

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Lead Machine Learning Engineer, Ad Performance

    Lead Machine Learning Engineer, Ad Performance

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal ML Engineer, Recommendation Systems

    Principal ML Engineer, Recommendation Systems

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Lead ML Engineer

    Lead ML Engineer

    Vaga Para Lead Ml Engineer Na Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Senior ML Engineer, Recommendation Systems

    Senior ML Engineer, Recommendation Systems

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]
    Principal ML Engineer

    Principal ML Engineer

    Launch Potato • West Palm Beach, Florida, United States
    [job_card.full_time]
    M+ monthly visitors through brands such as.As The Discovery and Conversion Company, our mission is to connect consumers with the world’s leading brands through data-driven content and technology.He...[show_more]
    [last_updated.last_updated_30] • [promoted]