Recruitment news
Senior Data Engineer
Remuneration: | cost-to-company |
Location: | Cape Town |
Job level: | Senior |
Type: | Permanent |
Company: | THE SKILLS MINE (PTY) LTD |
Requirements
- Five-plus years of experience as a data engineer or relevant experience
- Proven experience in designing and implementing data pipelines
- Experience with end-to-end testing of analytics pipelines
- Expertise in managing and optimising Elasticsearch clusters, including performance tuning and scalability
- Strong proficiency with data extraction, transformation, and loading (ETL) processes
- Familiarity with data modelling and schema design for efficient data storage and retrieval
- Good programming and scripting skills using languages like Python, Scala, or Java
- Knowledge of DevOps and automation practices related to data engineering
- Experience with data engineering in an agile / scrum environment
- Familiarity with ksqlDB / Kafka or other stream processing frameworks
- Familiarity with data lakes and querying them
- Experience with integrating machine learning models into data pipelines
- Kafka / ksqlDB
- Python
- Redis
- Elasticsearch, cluster management and optimisation
- AWS S3
- PostgreSQL
- AWS
- Designing, implementing, and maintaining robust data pipelines, ensuring the efficient and reliable flow of data across systems
- Developing and maintaining Elasticsearch clusters, fine-tuning them for high performance and scalability
- Collaborating with cross-functional teams to extract, transform, and load (ETL) data into Elasticsearch for advanced analytics and search capabilities
- Troubleshooting data pipeline and Elasticsearch issues, ensuring the integrity and availability of data for analytics and reporting
- Participating in the design and development of data models and schemas to support business requirements
- Continuously monitoring and optimising data pipeline and Elasticsearch performance to meet growing data demands
- Collaborating with data scientists and analysts to enable efficient data access and query performance
- Contributing to the evaluation and implementation of new technologies and tools that enhance data engineering capabilities
Posted on 18 Sep 12:42, Closing date 18 Oct
Recent jobs by The Skills Mine | City | Date posted | |
Senior Data Engineer | Cape Town | 18 Sep 12:42 | |
Coordinator - Marketing and Events | Johannesburg | 16 Sep 09:25 | |
Senior Back-end Engineer | Cape Town | 13 Sep 15:50 | |
Regional Sales Manager | Durban | 10 Sep 16:21 | |
Customer Success Manager | Durban | 10 Sep 13:22 | |
Web Developer and SQL Admin | Johannesburg | 10 Sep 13:20 | |
Full Stack Developer - Ruby on Rails | Johannesburg | 06 Sep 15:29 | |
Regional Product Manager - (6 Months FTC) | Johannesburg | 04 Sep 13:56 | |
HR and Talent Acquisition Business Partner | Johannesburg | 04 Sep 09:22 | |
Portfolio Manager - Credit Risk | Johannesburg | 03 Sep 16:28 | |
More jobs... |
Senior Data Engineer
Requirements Five-plus years of experience as a data engineer or relevant experienceProven experience in designing and implementing data pipelinesExperience with...
Senior Python JavaScript AWS Software Engineer
Are you a Senior Python JavaScript Software Engineer looking for a new opportunity in an environment that empowers salesforce to reach every customer, through an...
Senior Back-end Engineer
Requirements Over five years of experience in back-end web development.In-depth expertise in PostgreSQL and RDBMS in general.Experience working with message brokers and...
Senior DevOps Engineer
New work in at an on-demand delivery and digital tech shop in SA, owned by one of our major retail groups. The current need is for a DevOps engineer with key...
Java System Integration Engineer
Become a member of a team enthusiastic about the challenges posed by integrating complex systems comprising diverse components and subsystems currently in search for...
Senior DevOps Engineer
New work in at an on-demand delivery and Digi tech shop in SA, owed by one of our major retail groups.
Data Engineer - SQL DBA
A Fintech department, delivering financial information to the company that is timely, accurate, reliable, and relevant data originating from a multitude of external...
Software Developer /Data Engineer
A dynamic client who leverages the power of AI to analyse and interpret data with unprecedented accuracy, speed and gives actionable improvement suggestions are looking...
Mechanical Engineer
Our seeking a highly skilled and experienced Mechanical Engineer to join our team on a world-changing mission.
Data Engineer
A fintech department, delivering financial information to the company that is timely, accurate, reliable, and relevant data originating from a multitude of external...
Senior Data Engineer
A fintech department delivering timely, accurate, reliable, and relevant financial information from various external sources is seeking a senior data engineer.
Office Administrator & Co-Ordinator
Office Administrator amp Co-Ordinator - George Western Cape nbsp
Point of Sales Cashier & Administrator
Point of Sales Cashier amp Administrator - George Western Cape nbsp
Office Administrator & Co-Ordinator
Office Administrator amp Co-Ordinator - George Western Cape nbsp
Point of Sales Cashier & Administrator
Point of Sales Cashier amp Administrator - George Western Cape nbsp