Role : Lead Data Engineer – Consultant
Location : Sheffield, UK
Is it Permanent/ Contract: Contract
Is it Onsite/Remote/Hybrid: Hybrid (3 days weekly from office)
We are seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas:
- Extensive enterprise experience with Hadoop, Spark, and Splunk.
- Proficiency in object-oriented and functional scripting, particularly in Python.
- Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL).
- Experience integrating large, disparate datasets using modern tools and frameworks.
- Strong background in building and optimizing ETL/ELT data pipelines.
- Familiarity with source control and implementing Continuous Integration, Delivery, and Deployment via CI/CD pipelines.
- Experience supporting and collaborating with BI and Analytics teams in fast-paced environments.
- Ability to pair program and work effectively with other engineers.
- Excellent analytical and problem-solving abilities.
- Knowledge of agile methodologies such as Scrum or Kanban is a plus.
- Comfortable representing the team in standups and problem-solving sessions.
- Capable of driving the creation of technical test plans and maintaining records, including unit and integration tests, within automated test environments to ensure high code quality.
- Promote SRE (Site Reliability Engineering) culture by addressing challenges through data engineering.
- Ensure service resilience, sustainability, and adherence to recovery time objectives for all delivered software solutions.