Systems Integration Advisor

Job Description: Working closely with the Financial Crime team, orchestrating the cleansing and enriching data within our AWS environment. The focus is on a smooth transition, ingesting, processing, enriching, and validating data, and building an AWS data lake infrastructure : Linked to DAS programme.
Tasks:
:
Contributing to the technical design and development of ELT pipelines using AWS services, understanding the end:to:end architecture and process flow.
:
Ensuring that the technical deliverables are consistent with the original solution specified by the stakeholders.
:
Creating functional and technical documentation supporting best practices.
:
Resolving complex data:related issues, collaborating cross:functionally for effective solutions.
:
Gathering business requirements, performing detailed analysis, and producing associated functional specifications.
:
Configuring Control:M jobs with parameters and dependencies, defining predecessors and successors, as necessary.
:
Working effectively with a cross:functional team.
Skills Looking For:
:
AWS (Glue, EMR, Athena, Lake Formation, Step Functions, Lambda, ...etc.)
:
Spark (Scala and PySpark)
:
Scripting language: Python
:
SQL
:
Terraform
:
Git/GitHub
:
Big Data Expertise (HDFS, Hive, Sqoop, Impala, Hue)
Skills Nice to Have:
:
Batch script
:
CI/CD (GitHub workflows)
:
Control:M
Job Type: Permanent