The Data Loading Service, loads in tax return data of any format for managing manipulating, storage and analytics. This solution was designed to be metadata driven, to be able to load any tax return dataset into a MYSQL database to them be passed downstream to other systems.

This project was delivered using the Scrum Agile Methodology using Jira as the main requirement monitoring software and documentation was delivered on Confluence (Atlassian).

The team that I worked within also managed multiple other downstream systems and therefore had changes in the product backlog relating to these.

My role as the senior data engineer was to apply enhancements that were listed as a priority by the product owner. This included but not limited to: front-end dashboarding changes to which the users process the tax returns through; optimisation of the Pentaho jobs and transformations; ingestion of new types of tax return; fixes to the current solution; filtering sole traders and encryption of data based around individual records, Auditing reports for data flowing through all of these systems and the capturing of this data.  

It consisted of a team of 5-10 data team developers, project delivery manager, QA and business analyst.