Job Description: The role
Senior hands-on engineer to own a data and event backbone - how events flow, and how live and historical data are separated, stored, secured, and served. Accountable for delivering their scope, built correctly, securely, and on time. A hands-on expert engagement: building, not advising, requiring real streaming and data-modeling depth.
This is core software development and transaction-processing work - building the operational data backbone of a live system. It is not an analytics, data-warehousing, or business-intelligence role; the data modeling here is transactional (OLTP-style), not dimensional / reporting modeling.
Must-have summary - the hard bar
A candidate must clear all of these to be a fit:
- 6 8 years hands-on, currently building - recent, personally-built and delivered work they can speak to in depth.
- Streaming / event-processing depth - partitioning, ordering, consumer semantics, replay (not batch-ETL-only).
- Strong SQL and relational data modeling / projection design - the read layer is relational; SQL-light is not a fit.
- Object storage and lakehouse depth, including its security posture - not just basic file-store usage.
- Streaming-to-object-store integration - moving the event log to archival storage with idempotent / exactly-once delivery.
- Data-plane security - tenant / row-level isolation, object-storage security (encryption, immutability, public-access blocking), and encryption in transit.
- Test judgment for data properties - event replay, projection-versus-log correctness, idempotency, cross-tenant isolation.
- AI-assisted-development fluency and verification - directs and verifies AI-generated code and tests, catching plausible-but-wrong output.
- Strong Python; serverless-first, container fluency, local cloud emulation.
- Ownership and delivery discipline; clear communication; can hold core consistency under direction when the lead is unavailable.
Core skills - full detail
- 6 8 years hands-on, with recent work you personally built and delivered.
- Depth in streaming / event-processing platforms - partitioning, ordering, consumer semantics, replay.
- Streaming-to-object-store integration - moving the event log into archival object / lakehouse storage reliably, with idempotent / exactly-once delivery and schema handling.
- Strong relational database proficiency - the read / projection layer is relational and central to the role.
- Strong read-model / projection design and data modeling.
- Object storage and lakehouse depth - bucket / prefix design, partitioning, schema evolution, snapshot / lifecycle management, and columnar query over it.
- Data-plane security across all three stores:
- Tenant / party data isolation - row / tenant-level isolation, enforced server-side and per request.
- Object-storage security - public-access blocking, bucket / prefix isolation, encryption with managed keys, immutability where retention requires it, and lifecycle / tiering.
- Encryption at rest and in transit across the relational store, the event log, and object / lakehouse storage.
- Sound data-lifecycle judgment - what to cache, project, or archive, and why; hot vs. cold separation.
- Security verification of AI-generated code - catches data-exposure and access-control flaws in generated output before they land.
- Testing the hard data-plane properties - event ordering / replay, projection-versus-log correctness, idempotency / exactly-once, and cross-tenant isolation (proving data can't leak).
- Test strategy and verification judgment - reviews generated tests for genuine coverage rather than green-but-hollow passing.
- Hands-on with automated testing frameworks - unit / integration testing, service mocking / stubbing, and test-data generation, alongside local cloud emulation.
- Ownership and delivery discipline - accountable for getting work to done under time pressure.
- Clear communicator - surfaces risk and status clearly.
- Strong Python proficiency.
- Serverless-first cloud-native build - object storage and serverless compute as primary building blocks.
- Container fluency - containerized local development and container-image packaging of compute.
- Local cloud emulation for development and testing.
- Fluent with modern AI-assisted development tooling - directs and verifies AI-generated code with rigor.
- Comfortable applying an established architectural decision framework under direction - able to hold core consistency when the lead is unavailable.
Advantageous
- Cloud data services.
- Key management / secrets handling for data stores.
- Data retention / records-lifecycle and immutability experience.
- Container orchestration - good to know, not required.
- High-volume IoT / telemetry data.
- Observability / distributed-tracing tooling.
- Domain exposure in a data-intensive, operationally complex industry.
Assessment
A deep-dive on a data / event system you personally built - including how you moved the event stream into archival storage, isolated tenants, secured object storage, and protected data at rest and in transit - plus how you'd verify a data-layer implementation is correct and judge whether its test suite proves the hard properties (replay, projection correctness, isolation).
