Talent.com

Big data engineer Jobs in USA

Create a job alert for this search

Big data engineer • usa

Last updated: 6 hours ago

Big Data Engineer

SGARockville, MD
Full-time

Software Guidance & Assistance, Inc.This role is hybrid ( days/week Onsite in either Rockville or Tysons office).Design, develop, and maintain large-scale data processing pipelines using Big Data t... Show more

Data Engineer Senior (Hadoop/Big Data)

PNC Bank NATwo PNC Plaza
$110,500.00 yearly
Full-time +1

Leads in developing, supporting and implementing data solutions for multiple applications in order to meet business objectives and user requirements.Leverages technical knowledge and industry exper... Show more

Software Engineer (Big Data)

PlanIT GroupFort Worth, TX
Full-time

We are seeking a motivated and experienced Software engineer to design, develop, and deliver automated system capabilities and big-data processing architectures.The selected candidate will be respo... Show more

Senior Software Engineer (Big Data)

KDA Consulting IncChantilly, VA, US
Full-time
Quick Apply

Full Stack Developer with Big Data expertise.This role will focus on designing, developing, and maintaining scalable applications that process and analyze large volumes of data in support of nation... Show more

Senior Software Engineer (Big Data)

EpsilonWestminster, Colorado, United States
$88,900.00 yearly
Full-time

Design, develop, and maintain scalable data processing solutions across on-premises and cloud environments using Python and Apache Spark.Optimize and fine-tune Spark jobs for performance, including... Show more

AWS Big Data Lead Software Engineer

J.P. MorganColumbus, Ohio, US
Full-time

Design & build new applications utilizing leading edge technologies and modernize existing applications.Implement batch & real-time software components consistent with architectural best-practices ... Show more

Big data Engineer

Akaasa TechnologiesWilmington, DE, United States
Full-time
Quick Apply

Req-104768-1</p> <p class="x">Big data Engineer </p> <p><b>Job location -Wilmington DE or will also consider candidate from NJ who can relocate and nearby to... Show more

Lead Big Data Engineer

S&P GlobalNew York, NY, US
Full-time

Grade Level (for internal use):.Our enterprise data management team is at the forefront of designing and developing cutting-edge data solutions that drive strategic business outcomes across the org... Show more

Big Data Engineer II

Amazon Web Services, Inc.Dallas, Texas, USA
Full-time

The AWS Sales Marketing Global Services Ops (SMGS Ops) Revenue team publishes mission-critical Sales Revenue data product including, Unified Estimated Revenue, GAAP Sales Revenue, Quota Sales Retir... Show more

Associate Principal Engineer -- BIG-DATA-ENGINEER

NagarroAtlanta, Georgia, US
Full-time

Oversee the delivery of data projects, including data pipelines, warehouses, lakes, and analytics platforms.Collaborate with architects, data engineers, and data scientists to deliver robust and sc... Show more

Big Data PM

ClifyXTorrance, CA, United States
Full-time

Description: Skills: 1) Project manager with PMP or equivalent certification preferred.Understanding Big data technologies.Must have delivered big data projects in last 2 years.Desired to have expo... Show more

Big Data Engineer

LiniumITChicago, Illinois, United States
Full-time

Our client, a premier investment bank, is looking for a Big Data Engineer to join their growing team in Chicago, IL.Currently our client spends a lot of time trying to answer client's questions by ... Show more

Big Data Engineer

Bright Vision TechnologiesSan Jose, Evergreen, CA, US
$100,000.00 yearly
Full-time
Quick Apply

Big Data Engineer – Remote Bright Vision Technologies is a technology consulting and software development company delivering cloud, AI, data, and enterprise solutions across the United States... Show more

 • New!

Sr. Big Data Engineer

Conch Technologies IncRockville, MD, United States
Full-time
Quick Apply

Hi</div> </div> <div class="gmail_quote" style="text-align:start; text-indent:0px; -webkit-text-stroke-width:0px"> <div dir="ltr"> <div clas... Show more

Manager, Big Data

Beyond SOFWashington, DC, United States
Full-time

The Big Data Manager is responsible for overseeing the development and implementation of big data solutions to support business operations.Main responsibility and duties include overseeing the deve... Show more

Data Engineer-GCP, Big Query - 69309

PRIMUS Global ServicesUnited States
Full-time
Quick Apply

Data Engineer-GCP, Big Query (Remote)</div> <div> </div> <div>We have an immediate need for an experienced <b>GCP BigQuery Data Engineer</b> with strong experti... Show more

Big Data Architect

ITL USATexas, US
Full-time

Overview The Strategic Technology Group (STG) unit at Infosys is designed for power programmers and senior technologists who want to go beyond traditional development roles and take on complex, lar... Show more

Big Data Architect

GIGATEC EngineeringAnnapolis Junction, MD, US
Full-time
Quick Apply

Perks and Benefits 100% Paid Healthcare 10% 401k in every paycheck 100% Fully Vested!.NOTE  – Our positions require a  Top Secret clearance , as well as the favorable  completi... Show more

Big Data Engineer, AVP

BarclaysHanover Township, New Jersey, US
Full-time

Big Data Engineer, AVP in Whippany, NJ (multiple positions available):.Analyze system requirements to automate and improve functionality and test, maintain, monitor, and enhance radial and BDH envi... Show more

People also ask
The top 10 cities are:
Big Data Engineer

Big Data Engineer

SGARockville, MD
30+ days ago
Job type
  • Full-time
Job description

Software Guidance & Assistance, Inc., (SGA), is searching for a Big Data Engineer for a Contract assignment with one of our premier Regulatory clients in Rockville, MD or Tysons, VA.

This role is hybrid ( days/week Onsite in either Rockville or Tysons office)

Responsibilities :
  • Design, develop, and maintain large-scale data processing pipelines using Big Data technologies (, Hadoop, Spark, Python, Scala).
  • Implement data ingestion, storage, transformation, and analysis of solutions that are scalable, efficient, and reliable.
  • Stay current with industry trends and emerging Big Data technologies to continuously improve the data architecture
  • Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
  • Optimize and enhance existing data pipelines for performance, scalability, and reliability.
  • Develop automated testing frameworks and implement continuous testing for data quality assurance.
  • Conduct unit, integration, and system testing to ensure the robustness and accuracy of data pipelines.
  • Work with data scientists and analysts to support data-driven decision-making across the organization.
  • Ability to write and maintain automated unit, integration, and end-to-end tests
  • Monitor and troubleshoot data pipelines in production environments to identify and resolve issues.
Requirements :
  • Bachelor's degree in Computer Science, Information Systems or related discipline with at least five () years of related experience, or equivalent training and/or work experience; Master's degree and past Financial Services industry experience preferred.
  • Demonstrated technical expertise in Object Oriented and database technologies/concepts which resulted in deployment of enterprise quality solutions.
  • Past experience with developing enterprise quality solutions in an iterative or Agile environment.
  • Extensive knowledge of industry leading software engineering approaches including Test Automation, Build Automation and Configuration Management frameworks.
  • Strong written and verbal technical communication skills.
  • Demonstrated ability to develop effective working relationships that improved the quality of work products.
  • Should be well organized, thorough, and able to handle competing priorities.
  • Ability to maintain focus and develop proficiency in new skills rapidly.
  • Ability to work in a fast paced environment.
  • Experience with object oriented programming languages such as Java, Scala or Python.
Essential Technical Skills:

Big Data technologies
• Experience with Big data technologies such as Hadoop, Spark, Hive & Trino
• Evaluate understanding of common issues like:
◦ Data skew and strategies to mitigate it.
◦ Working with massive data volumes in PetaBytes.
◦ Troublehsooting job failures due to resource limitations, bad data, scalability challenged.
• Look for real-world debugging and mitigation stories.

AI Skills
• Prompt Engineering: Proficiency in crafting effective prompts for AI coding assistants and analysis tools
• AI Workflow Design: Experience redesigning development processes to leverage AI capabilities
• Data Analysis: Ability to interpret AI-generated insights and translate them into actionable team improvements
• Change Management: Experience leading teams through AI adoption and workflow transformation

SQL Skills (Window Functions, Joins, Complex Queries)
• Assess comfort with SQL window functions, multi-table joins, aggregations.
• Provide examples or ask them to write/optimize SQL queries on the spot.
• Probe how they handle edge cases like NULLs, duplicates, ordering, etc.

Apache Spark (Development, Internals & Tuning)
• Test their understanding of Spark's core architecture — executors, tasks, stages, DAG.
• Focus on Spark performance tuning techniques: partitioning, caching, broadcast joins, etc.
• Ask scenario-based questions on troubleshooting slow running/stuck jobs or resource issues in Spark.
• Explore their experience optimizing Spark jobs for large-scale datasets.

Cloud Technologies
• Check exposure to AWS services like S, EMR, Glue, Lambda, Athena, etc.
• Ask how they've used S with Spark (, dealing with file formats, consistency issues).
• EKS, Serverless knowledge, etc.

Programming - Python or Scala
• Assess ability to write clean, modular, and performant code.
• Look for experience in functional programming concepts (, immutability, higher-order functions).
• Ask about real-world use cases where they wrote scalable data processing code.
• Evaluate understanding of collections, concurrency, and memory management.

Preferred Skills:
• Experience with managing production data pipelines/ETL systems
• Experience with CI/CD
• Experience writing test cases
• AWS certifications