Senior Data Pipelines Architect
For our customer, a large consultancy company, we are recruiting for multiple roles. They will be replacing existing consultancy agencies for one large new force. The project will be taking 5+ years and the main goal is to update and deploy all new software on set top boxes for cable
companies around Europe. Any video streaming through any portal (Set top box, Web, App) will be processed through this project.
Team size of teams are on average 6 people including PO. The teams are distributed over Europe and Asia, and there is a core team working from the HQ in Amsterdam.
In this case, we are looking for a Senior data pipelines Architect.
· Senior Data Architect who will be responsible for leading Big Data Platform Development Programs from Client location working closely with Business and existing vendor teams.
· In collaboration with Product Engineering, and colleagues in Service Management and Operations, work to reduce or eliminate manual or repetitive tasks, improving the end-to-end health and performance of the underlying platforms.
· Aiding in identification of solution requirements and key case-studies/scenarios to architect data solution for business transformation.
· To ensure that the infrastructure is developed in a manner that supports delivery of services to multiple platforms with appropriate levels of resilience and performance.
· Conduct solution architecture review/audit calculate and present ROI.
· Development of proposals for implementation and design of scalable big data architecture, along with Product Owners.
· Review and audit of existing solution, design, and system architecture.
· To ensure that the business has a common, consistent, and flexible platform that continuously delivers the tools, interfaces, and components that drive our products and services.
· Design and develop in-life excellence solution like Fault identification, tolerance, and recovery automations.
· Strong ‘hands-on’ experience as a Big Data Architect with a solid design/development background with Scala & Python with Spark.
· Broad experience working with Design & Architecture of open source Big Data Platform.
· Wide experience in analysis, design, implementation, deployment as well as troubleshooting and rebuilding distributed Linux based Big Data solutions on premises.
· Experience in continuous delivery tools and technologies (Ansible, Jenkins – CICD).
· Experience working with Containers and Resource Management systems: Docker, Mesos, Kubernetes
· Queues and Stream processing: Kafka Streams; Spark Streaming, Akka & Akka
· Working knowledge with one of the Cloud Services – GCP/AWS.
· Deep understanding of ElasticSearch stack.
· Working knowledge with HDFS & Clickhouse.
· Good in Agile development methodology.
· Experienced in Telecom domain & Big Data Architecture design experience on: Lambda, Kappa, Data Lake.
· Visas are paid in full, plus 30% ruling request
· 20 vacation days per plus national holidays + expat desk is arranged
· €1000,- starting bonus
· Health care insurance -Benefit in Kind- €1500 gross
· Private School allowance 2 kids maximum 5K per child per year
· We upfront rent plus deposit, payback in 5-6 months at no extra costs