Senior Data Engineer

Accountabilities: You will own the design, development, and operation of production-grade data infrastructure, ensuring that data systems remain scalable, reliable, efficient, and performant. Your responsibilities will include: Design, develop, and maintain scalable, high-throughput data pipelines and platform systems supporting blockchain data across more than 30 networks. Build backend data services and data loaders using Go and Python. Design and evolve analytical data models and workloads in ClickHouse, optimizing schemas and queries to achieve sub-second performance. Develop incremental, idempotent data pipelines using Databricks to process billions of records and tens of terabytes of information. Continuously improve platform scalability, reliability, data freshness, processing performance, and infrastructure cost efficiency. Investigate complex performance bottlenecks and reliability issues, implementing effective solutions and following them through to resolution. Collaborate with product, analytics, and investigations teams to understand data requirements and translate them into scalable platform architectures. Contribute to technical direction through architecture discussions, design documentation, technical reviews, and code reviews. Mentor fellow engineers and promote strong development, operational, and engineering practices. Implement monitoring and instrumentation for production systems, maintain operational runbooks, and conduct blameless post-incident reviews. Manage data backfills, pipeline failures, and processing consistency across distributed environments. Apply production-scale testing, performance profiling, and continuous optimization to maintain high engineering standards. Leverage AI-assisted development tools while applying sound technical judgment when reviewing and integrating generated code. Requirements The ideal candidate is an experienced data engineer with strong backend development capabilities, deep expertise in large-scale data processing, and a hands-on approach to production operations. You should bring: At least 5 years of experience building and operating production data systems, including ETL and ELT pipelines. At least 2 years of experience owning systems that other engineers or teams depend on. Expert-level Python skills and strong SQL proficiency, including experience writing typed, tested code and working with asyncio and concurrency. Demonstrated experience building incremental data models and processing datasets containing billions of records. Production experience with Go, particularly developing and operating high-throughput backend services and data loaders. Strong hands-on experience with ClickHouse, including MergeTree schema design, sort-key optimization, materialized views, bulk loading, and query performance tuning. Production experience with Databricks, including PySpark, Delta Lake, and Unity Catalog, as well as designing incremental and idempotent tables and optimizing compute costs. Practical experience managing large-scale data pipelines involving billions of records and tens of terabytes. Strong understanding of partitioning, clustering, file sizing, and their impact on infrastructure costs and query latency. Production experience with Kafka, Airflow, and Kubernetes, including delivery semantics, idempotent writes, backfills, and failure recovery. Experience running containerized data workloads using Docker and Helm. Familiarity with Google Cloud Platform, which is preferred, or AWS. Strong operational and communication skills, including system instrumentation, technical documentation, runbook creation, and post-incident reviews. Ability to produce clear architecture and design documents that support effective collaboration across distributed teams and time zones. Additional experience that would be advantageous includes: Familiarity with blockchain data structures and models, including EVM, UTXO, and non-EVM networks. Experience retrieving blockchain data from nodes through JSON-RPC or gRPC. Knowledge of graph databases, distributed key-value stores, entity resolution, or large-scale clustering. Experience with infrastructure-as-code, GitOps, and observability tools such as Terraform, Helm, ArgoCD, OpenTelemetry, Prometheus, and Grafana. Experience designing load tests and building platforms supporting multiple data sources, datasets, or product teams. Familiarity with AI-assisted software development. Previous experience in fintech, regulatory compliance, or Anti-Money Laundering (AML). Benefits Industry-leading compensation packages. Generous equity participation, offering an opportunity to share in the company's growth. Comprehensive health insurance coverage. Flexible paid time off. Learning and professional development initiatives. Flexible working hours designed to support work-life balance. Regular team-building activities and opportunities for collaboration. A supportive environment that encourages open conversations about mental health and employee well-being. Opportunities for professional growth and increased technical ownership within a rapidly growing organization. A collaborative, senior-level engineering environment where technical contributions directly influence product and platform development. How Jobgether works: We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team. We appreciate your interest and wish you the best! Why Apply Through Jobgether? Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time. #LI-CL1