Data Engineer

SP Staffing Services (P) Ltd
  • Chennai
  • 15-20 lakh
  • 3-6 years
  • Views
  • 17 Jan 2017

  • Software Design & Development

  • IT/ Technology - Software/ Services
Job Description

Responsibilities :

* Architect, deploy and maintain high-throughput data pipelines and ETL workloads that will streamline different data sources .
* Working alongside Data Scientists to build production-ready Machine Learning solutions.
* Convert Machine Learning algorithms, proof-of-concepts and AI products into highly scalable and secure customer-facing APIs with strict SLAs.
* Versioning and continuous integration of data science deliverables.
* Management of compute cluster and data warehousing tools.

Qualifications :

* 3 - 6 years of work experience.
* Educational background (or relevant programming experience) in computer science/engineering.
* Strong programming expertise (preferably in Python) and SQL knowledge.
* Proficiency in distributed computing, big data query engines such as Hive and cluster management and AWS cost optimization.
* Experience building RESTful APIs, architecting complex systems with batch and real-time workloads, monitoring and instrumenting production systems.
* Experience in MapReduce (preferably Spark/PySpark), SQL, relational database design, messaging brokers (preferably RabbitMQ) and data warehousing tools (preferably Redshift).
* Experience in shell scripting and version control with git .
* Ability to thrive in a dynamic startup environment with cross-functional teams.
* Strong communication and interpersonal skills and the ability to iterate with feedback from various stakeholders.

About Organisation

SP Staffing Services (P) Ltd