Data wrangling at scale: The experience of EW-Shopp

This paper presents a subsystem of a comprehensive platform dedicated to data transformation, linking and extension of large data sets. Furthermore, we detail and discuss both the main requirements that have led to the design and development of the platform, and the devised approach, which is a direct outcome of the requirement elicitation and discussion phase. In particular, the platform supports both design and run time aspects of the data transformation process, which is reflected in the architecture. Some initial tests have been carried out on a prototype implementation of our architecture on data sets of ~1TB featuring promising performance.