Senior Software Engineer - Data Lakehouse InfrastructureTRM Labs • Worcester, MA, United States
Senior Software Engineer - Data Lakehouse Infrastructure
TRM Labs • Worcester, MA, United States
[job_card.variable_days_ago]
[job_preview.job_type]
- [job_card.full_time]
[job_card.job_description]Bachelor's degree (or equivalent) in Computer Science or a related field. A proven track record, with 5+ years of hands-on experience in architecting distributed system architecture, guiding projects from initial ideation through to successful production deployment. Exceptional programming skills in Python, as well as adeptness in SQL or SparkSQL. Versatility that spans the entire spectrum of data engineering in one or more of the following areas : In-depth experience with data stores such as ClickHouse, ElasticSearch, Postgres, Redis, and Neo4j. Proficiency in data pipeline and workflow orchestration tools like Airflow, DBT, Luigi, Azkaban, and Storm. Expertise in data processing technologies and streaming workflows including Spark, Kafka, and Flink. Competence in deploying and monitoring infrastructure within public cloud platforms, utilizing tools such as Docker, Terraform, Kubernetes, and Datadog. Proven ability in loading, querying, and transforming extensive datasets The Data Platform team is the funnel between all of TRM's data world and product world. We care about all layers of stack including petabyte of data stores, pipelines, and processes. We have quite a big scope as a the team with new and exciting projects every quarter. As a result, we collaborate across the board with most teams at TRM. We believe in async communication and are also not afraid to jump on a quick huddle if that helps to move things faster. We are both scrappy when the situation demands and also process-oriented when we need to achieve our OKRs. We are always looking for people who can elevate the quality our tech and our execution. If you enjoy a remote-first and async friendly environment to achieve efficacy and efficiency at petabyte scale, our team could be a great pick for you! Team members are based in the US across almost all timezones! Our on-call tends to be in EST / PST shift, whatever suits you best. We do try to reserve some overlap in the day for meetings. Our north star - no IC spends more than 3-4 hours / week in meetings. Build scalable engines to optimize routine scaling and maintenance tasks like create self-serve automation for creating new pgbouncer, scaling disks, scaling / updating of clusters, etc. Enable tasks to be faster next time and reducing dependency on a single person. Identify ways to compress timelines using 80 / 20 principle. For instance, what does it take to be operational in a new environment? Identify the must have and nice to haves that are need to deploy our stack to be fully operation. Focus on must haves first to get us operational and then use future milestones to harden for customer readiness. We think in terms of weeks and not months. Identify first version, a.k.a., skateboards for projects. For instance, build an observability dashboard within a week. Gather feedback from stakeholders after to identify more needs or bells and whistles to add to the dashboard. Impact-Oriented Trailblazer : We put customers first, driving for speed, focus, and adaptability. Master Craftsperson : We prioritize speed, high standards, and distributed ownership. Inspiring Colleague : We value humility, candor, and a one-team mindset. Purpose and Mission : Have a real-world impact, from disrupting terrorist networks to returning stolen funds. Inspiring Colleagues : Collaborate with industry leaders and learn something new daily. Personal Growth : We're not just using technology; we're inventing it and taking category-defining products to market. Remote First : Our HQ is online. Tools like Zoom and Slack ensure seamless collaboration, but we also value in-person interactions, organizing regular meetups and offsites for team bonding. Clear communication is key in our decentralized setup. With tools like Slack, Loom, and voice notes, we document meetings and decisions, promoting transparency and efficiency. Ownership & TRM Speed : Small teams drive big goals at TRM. Every team member has significant ownership and responsibility, fostering an environment of initiative and direct impact. While there's no strict clocking in or out, we expect team members to balance personal schedules and team needs. We move "surprisingly fast" while maintaining a high bar in service of our customers and mission. This can feel both intense and rewarding. Our unique approach to projects emphasizes quick wins, rapid iterations, and constant feedback. Diversity and Inclusion : Diversity at TRM encompasses backgrounds, experiences, and perspectives. Every day is an opportunity to learn from a colleague, whether they're a law enforcement expert or a tech pioneer. Annual Company Offsite : Once a year, TRMers come together from around the world to attend a 1-week mandatory company offsite. This is designed to foster in-person relationships, creativity, and strategic alignment. Employee attendance is required of all TRMers.
Software Engineer
TRM is on a mission to build a safer financial system for billions of people. We deliver a blockchain intelligence data platform to financial institutions, crypto companies, and governments to fight cryptocurrency fraud and financial crime. We consider our business and our profit as a way to move towards our mission sustainably and at scale.
The Data Platform team collaborates with an experienced group of data scientists, engineers, and product managers to build highly available and scalable data infrastructure for TRM's products and services. As a Software Engineer on the Data Platform team, you will be responsible for executing mission-critical systems and data services that analyze blockchain transaction activity at petabyte scale, and ultimately work to build a safer financial system for billions of people.
The impact you'll have here :
- Build highly reliable data services to integrate with dozens of blockchains.
- Develop complex ETL pipelines that transform and process petabytes of structured and unstructured data in real-time.
- Design and architect intricate data models for optimal storage and retrieval to support sub-second latency for querying blockchain data.
- Oversee the deployment and monitoring of large database clusters with an unwavering focus on performance and high availability.
- Collaborate across departments, partnering with data scientists, backend engineers, and product managers to design and implement novel data models that enhance TRM's products.
What we're looking for :
About the Team :
Learn about TRM Speed in this position :
About TRM's Engineering Levels :
Engineer : Responsible for helping to define project milestones and executing small decision decisions independently with the appropriate tradeoffs between simplicity, readability, and performance. Provides mentorship to junior engineers, and enhances operational excellence through tech debt reduction and knowledge sharing.
Senior Engineer : Successfully designs and documents system improvements and features for an OKR / project from the ground up. Consistently delivers efficient and reusable systems, optimizes team throughput with appropriate tradeoffs, mentors team members, and enhances cross-team collaboration through documentation and knowledge sharing.
Staff Engineer : Drives scoping and execution of one or more OKRs / projects that impact multiple teams. Partners with stakeholders to set the team vision and technical roadmaps for one or more products. Is a role model and mentor to the entire engineering organization. Ensures system health and quality with operational reviews, testing strategies, and monitoring rigor.
Life at TRM Labs
Leadership Principles :
Our Leadership Principles are foundational element of our strategy, guiding how we make decisions, how we treat each other, and how we behave day-to-day.
Build a Career :
Joining TRM means being part of a mission-driven team comprised of industry leaders. At TRM, you'll experience :
Work Environment :
Join Us :
We're looking for team members who love building from the ground up and want to work hard in a fast-paced and ambitious environment. We are remote-first, with exceptionally talented team members located around the world. If you like solving tough
[job_alerts.create_a_job]
Senior Software Engineer Infrastructure • Worcester, MA, United States
[internal_linking.similar_jobs]
Software Principal Engineer / Software Senior Engineer - Data Path.Data Protection Software Engineering.Working at the cutting edge, we design and develop software to protect data hosted across On-...[show_more]
Diversity - Innovation - Caring - Global Collaboration - Winning Spirit- High Performance.At Boston Scientific, we'll give you the opportunity to harness all that's within you by working in teams o...[show_more]
Dephy is looking for a candidate who was born to be an engineer : someone that has an urge to understand how things work, and who gets excited when there is no known solution to the problem at hand....[show_more]
About Commonwealth Fusion Systems : .Commonwealth Fusion Systems is on a mission to deliver the urgent transition to fusion energy.
Combining decades of research, top talent, and new technologie...[show_more]
Earn at least $1740 driving with Uber when you complete your first 114 trips in 30 days.Driving is an easy way to boost your income while maintaining the flexibility your schedule requires (gig, pa...[show_more]
Hutchinson Aerospace and Industry,.GREAT PEOPLE , INSPIRED to INNOVATE, that are ENGAGED, ENABLED, & ENERGIZED to make an IMPACT.
As a growing engineering driven manufacturing company, we serve the ...[show_more]
FishEye Software is seeking Software / Hardware Integration Engineers to join the Radar Infrastructure Development and Integration Team.
Our team is responsible for providing the operating system (OS)...[show_more]
Location- On site (Smithfield, RI).The Data Governance Engineer plays a pivotal role in ensuring the integrity, quality, and security of organizational data assets.
This position will collaborate wi...[show_more]
The Senior Quality Engineer II will assist the Quality Manager relating to activities such as returns, source inspection, other support as required, and maintain Quality Management System objective...[show_more]
As a recognized leader in engineering, planning, and design services, BETA Group, Inc delivers transformative transportation solutions across the Northeast.
With a legacy of technical excellence, in...[show_more]
Implements Microservice principles architecture using Spring WS, Rest patterns, OAuth, Swagger, and Apache Camel.Crafts and develops Java, Spring, and RESTful APIs.
Programs in Java and JavaScript b...[show_more]
We are seeking a Cloud Engineer to join American Tower's Information Technology ("IT") organization's Data Center and Cloud Platform Services Corporate IT team.
The team focuses on implementing and ...[show_more]
Minimum 10 years’ experience within End User Technologies and delivering capabilities such as Active Directory, Azure AD, Group Policy, SCCM, Application packaging, Microsoft O365, Standard Operati...[show_more]
Are you a passionate Civil Engineer looking for the next step in your career? At WDA Design Group, we don’t just design projects—we shape communities.
As a well-established, multi-discip...[show_more]
The Leonardo DRS Naval Power Systems business is a trusted provider of innovative naval gas and hybrid power systems, control technology and support solutions for the U.
Navy and navies around the w...[show_more]
We are looking for a motivated and skilled Technical Lead to join a client in the environmental space.In this role, you will combine hands-on development expertise with leadership responsibilities,...[show_more]
Join our team of innovators and get a chance to work alongside a team of talented, curious people that are passionate about designing and deploying solutions that are accelerating humanity's expans...[show_more]
At Viasat, we're on a mission to deliver connections with the capacity to change the world.For more than 35 years, Viasat has helped shape how consumers, businesses, governments and militaries arou...[show_more]
Software Principal Engineer - Data Path
Dell • Hopkinton, MA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Senior Systems Engineer, R&D
Boston Scientific • Marlborough, MA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Senior Controls Engineer
Dephy, Inc. • Town of Boxborough, MA, US
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Enterprise Cloud Data Engineer
Commonwealth Fusion Systems • Devens, MA, US
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Earn At Least $1740 For Your First 114 Trips, Guaranteed.
Uber • Greenville, NH, US
[job_card.full_time] +1
[last_updated.last_updated_variable_days] • [promoted]
Senior Project Engineer
Hutchinson Aerospace & Industry • Hopkinton, MA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Infrastructure Engineer
FishEye Software • Marlborough, MA, USA
[job_card.full_time]
[filters_job_card.quick_apply]
[last_updated.last_updated_30]
Data Governance Engineer (39035)
Neighborhood Health Plan of Rhode Island • Smithfield, RI, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Senior Quality Engineer
Diamond Antenna and Microwave • Littleton, MA, US
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Traffic Project Engineer
BETA GROUP INC • Worcester, MA, US
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Senior Software Engineer / Developer
Fidelity Investments • Smithfield, RI, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Cloud Engineer
American Tower Corporation • Westborough, MA, United States
[job_card.full_time]
[last_updated.last_updated_variable_hours] • [promoted] • [new]
Sr. Infrastructure Engineer
Ovation WPS • Northborough, Massachusetts, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Civil Engineer
WDA Design Group • Westborough, MA, US
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Senior Systems Engineer
Leonardo DRS • Fitchburg, MA, United States
[job_card.full_time]
[last_updated.last_updated_30] • [promoted]
Lead Software Engineer
Robert Half • Hudson, MA, US
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]
Sr. Embedded Software Engineer (Embedded Software Engineer, Sr)
Redwire Space • Marlborough, MA, United States
[job_card.permanent]
[last_updated.last_updated_30] • [promoted]
Senior Cloud Engineer
Viasat • Marlborough, MA, United States
[job_card.full_time]
[last_updated.last_updated_variable_days] • [promoted]