About the Opportunity

We are seeking a Senior Data Platform Engineer to build and improve the infrastructure that enables data teams to process, manage, and consume large volumes of information reliably. This is a hands-on engineering position focused on data platform architecture, distributed processing, pipeline orchestration, performance optimisation, and operational reliability.

You will work with data engineers, analytics professionals, machine learning specialists, and software developers to create reusable platform capabilities that reduce operational complexity and improve the efficiency of data-driven products.

The successful candidate will bring strong engineering fundamentals, practical experience with modern data infrastructure, and the ability to make sound technical decisions across scalability, reliability, maintainability, and cost.

Key Responsibilities

Data platform architecture

• Design and improve scalable data platforms supporting batch processing, streaming workloads, analytical storage, and downstream data services.
• Develop reusable components and shared frameworks that simplify pipeline implementation across teams.
• Evaluate architectural options for data ingestion, processing, storage, orchestration, and access.
• Establish appropriate patterns for schema management, data contracts, partitioning, retention, and data lifecycle management.
• Contribute to technical design reviews and document architectural decisions and their trade-offs.

Distributed processing and pipeline engineering

• Build and optimise data pipelines using Python, SQL, and suitable distributed processing technologies.
• Develop batch and streaming workflows based on platform requirements and business latency needs.
• Work with technologies such as Apache Spark, Kafka, Flink, or equivalent tools where appropriate.
• Improve pipeline throughput, processing efficiency, fault tolerance, and recovery behaviour.
• Investigate data loss, processing delays, duplicate events, schema incompatibilities, and distributed system failures.
• Support efficient ingestion from application databases, APIs, event streams, and cloud storage.

Reliability, observability, and automation

• Implement monitoring, alerting, structured logging, and operational dashboards for data infrastructure.
• Establish data quality controls and service-level indicators for critical pipelines.
• Improve deployment processes through CI/CD, automated testing, infrastructure as code, and version-controlled configuration.
• Investigate production incidents, identify root causes, and implement corrective measures.
• Improve resource utilisation and manage processing costs without compromising data quality or reliability.
• Strengthen access controls, secrets management, encryption practices, and auditability.

Engineering collaboration

• Work with analytics and machine learning teams to provide dependable, well-documented datasets and platform services.
• Review code and technical designs, share engineering practices, and support the development of colleagues through technical collaboration.
• Identify opportunities to reduce duplicated work and improve the consistency of data engineering practices.
• Evaluate new tools and architectural patterns based on measurable benefits, operational complexity, and long-term maintainability.

Technical Qualifications

• Significant professional experience in data engineering, platform engineering, distributed systems, or a related field.
• Strong proficiency in Python and SQL, with experience writing maintainable production code.
• Practical experience designing and operating data pipelines in production environments.
• Strong understanding of data modelling, analytical storage, distributed processing, and workflow orchestration.
• Experience with cloud platforms such as AWS, Microsoft Azure, or Google Cloud.
• Familiarity with technologies such as Apache Spark, Kafka, Airflow, dbt, Snowflake, Databricks, or equivalent tools.
• Experience with Docker, CI/CD pipelines, automated testing, and infrastructure automation.
• Understanding of monitoring, incident investigation, system resilience, and performance optimisation.
• Knowledge of data governance, access management, privacy, and secure platform design.
• Ability to communicate complex engineering decisions clearly and work across multidisciplinary teams.

Desirable Experience

Experience with lakehouse architecture, real-time data processing, event-driven systems, data mesh principles, Kubernetes, Terraform, or cloud-native data services is advantageous. Experience supporting machine learning pipelines or high-volume financial and transactional datasets will also be beneficial.

Measures of Success

Success in this role will be demonstrated through improved data platform reliability, reduced pipeline failure rates, efficient processing performance, reusable engineering capabilities, and better observability across production workloads. You will also contribute to sound architectural decisions that support future growth without creating unnecessary operational complexity.

What We Offer

• A technically challenging individual-contributor role with substantial engineering ownership.
• Opportunities to shape data platform design and reliability practices.
• Exposure to distributed systems, cloud infrastructure, and modern data architecture.
• Collaboration with experienced professionals across engineering, analytics, and machine learning.
• Competitive remuneration reflecting senior technical expertise and responsibilities.

Ideal Candidate

The ideal candidate is a technically mature engineer who can balance architectural thinking with hands-on implementation. You should be comfortable diagnosing complex production issues, evaluating trade-offs, and building platform capabilities that are reliable, secure, and maintainable. You value clear documentation, measurable engineering improvements, and practical solutions to complex data infrastructure problems.

Application

Submit a current CV highlighting production data platform experience, relevant technologies, architectural contributions, and examples of reliability or performance improvements. Applicants may include links to appropriate technical work or professional projects where confidentiality permits.

Are you interested in this position?

Apply by clicking on the “Apply Now” Button below!

#JobsHubEstonia #GlobalRecrument
#CareerOpportunities #HiringNow
#JobSeekersNetwork #EstoniaJobs
#RecruitmentServices #EmploymentPortal.