Data Engineer

SP Staffing Services (P) Ltd
  • Chennai
  • 10-15 lakh
  • 3-6 years
  • Views
  • 17 Jan 2017

  • Software Design & Development

  • IT/ Technology - Software/ Services
Job Description

Responsibilities
Architect, deploy and maintain high-throughput data pipelines and ETL workloads that will streamline different data sources
Working alongside Data Scientists to build production-ready ML solutions
Convert ML algorithms, proof-of-concepts and AI products into highly scalable and secure customer-facing APIs with strict SLAs
Versioning and continuous integration of data science deliverables
Management of compute cluster and data warehousing tools

Qualifications
3+ years of work experience
Educational background (or relevant programming experience) in computer science/engineering
Strong programming expertise (preferably in Python) and SQL knowledge
Proficiency in distributed computing, big data query engines such as Hive and cluster management and AWS cost optimization
Experience building RESTful APIs, architecting complex systems with batch and real-time workloads, monitoring and instrumenting production systems
Experience in MapReduce (preferably Spark/PySpark), SQL, relational database design, messaging brokers (preferably RabbitMQ) and data warehousing tools (preferably Redshift)
Experience in shell scripting and version control with git
Ability to thrive in a dynamic startup environment with cross-functional teams
Strong communication and interpersonal skills and the ability to iterate with feedback from various stakeholders


About Organisation

SP Staffing Services (P) Ltd