Building, deploying and managing large scale Hadoop based data infrastructure and data pipelines and products.
Administration and support of Hadoop and associated infrastructure.
Maintaining, optimizing and monitoring large scale distributed applications and services. Troubleshooting OS, services, system bottlenecks and application integration.
Building automation configuration management for daily tasks.
Designing, integrating and documenting technical components, dependencies of big data platform.
Ensuring best practices that can be adopted in BigData stack and share across teams. Helping our engineers and data scientists build software that scales in terms of
performance and stability
Establishing best practices inside the organization, proving that they work and then bringing them to other teams at Lithium
Big data and related technologies including: Hadoop, Hive, HBase, Kafka, Elastic Search, Spark, NoSQL (Solid understanding of the internals, usage and operational requirements)
System administration of Linux/Unix and networking strongly preferred.
Experience with automation configuration management tools such as Chef, Puppet or Salt
Experience in Cloud and Datacenter routing, VLANs, VPN.
Excellent knowledge of Hadoop and RDBMS integration points with enterprise BI and EDW tools.
Familiarity with OS internals, concepts of distributed data management, and design/implementation tradeoffs in building clustered, high- performance, fault-tolerant distributed systems software.