Senior Data Engineer / Data Engineer (Department of Data Science, SGH)

Job Description
You will support the design, implementation and maintenance of data flow channels and data processing systems that support the collection, storage, batch and real-time processing, and analysis of information in a scalable, repeatable and secure manner. You will focus on defining optimal solutions to data collection, processing and warehousing. You will also focus on collecting, parsing, managing, analysing and visualising large sets of data to turn information into insights accessible through multiple platforms. You will design, code and test data systems and work on implementing them into the internal infrastructure, improving data reliability and quality. Most of all, you enjoy reducing complexity and building automated systems.

Job Requirements

• Bachelor’s Degree in Computer Science, Electronics or Electrical Engineering, Software Engineering, Information Technology or other related technical disciplines.
• Preferably 1-3 (Analyst) or 3-5 (Senior Analyst) years of relevant experience.
• Prior experience in a healthcare setting would be an advantage but is not required.
• Fluency in command-line shell scripting, Python and/or R.
• Experience in several of the following skill clusters:
  • Relational databases, data marts and data warehouses – experience with Oracle and/or PostgreSQL platforms; crafting secure and effective SQL queries
  • NoSQL databases – key-value, document, graph, column family
  • Reproducible computing – version control, software packaging, continuous integration/continuous delivery, process monitoring and containerization
  • Automated workflows - programmatically author, schedule and monitor workflows using Apache Airflow, Azure Logic Apps or similar
  • Distributed computing – distribute jobs for concurrent processing using Apache Spark, Databricks or similar
  • Streaming data – process continuous, serial port or micro-batch online data
• Ability to work independently as well as in a team.
• Knowledge in cloud technologies is preferable.
• Strong analytical and problem-solving skills.
• Excellent written and verbal communication skills.