From Ad-Hoc Data Analytics to DataOps
The collection of high-quality data provides a key competitive advantage to companies in their decision-making process. It helps to understand customer behavior and enables the usage and deployment of new technologies based on machine learning. However, the process from collecting the data, to clean and process it to be used by data scientists and applications is often manual, non-optimized and error-prone. This increases the time that the data takes to deliver value for the business. To reduce this time companies are looking into automation and validation of the data processes. Data processes are the operational side of data analytic workflow. DataOps, a recently coined term by data scientists, data analysts and data engineers refer to a general process aimed to shorten the end-to-end data analytic life-cycle time by introducing automation in the data collection, validation, and verification process. Despite its increasing popularity among practitioners, research on this topic has been limited and does not provide a clear definition for the term or how a data analytic process evolves from ad-hoc data collection to fully automated data analytics as envisioned by DataOps. This research provides three main contributions. First, utilizing multi-vocal literature we provide a definition and a scope for the general process referred to as DataOps. Second, based on a case study with a large mobile telecommunication organization, we analyze how multiple data analytic teams evolve their infrastructure and processes towards DataOps. Also, we provide a stairway showing the different stages of the evolution process. With this evolution model, companies can identify the stage which they belong to and also, can try to move to the next stage by overcoming the challenges they encounter in the current stage
Sat 27 JunDisplayed time zone: (UTC) Coordinated Universal Time change
15:00 - 18:00 | Session 3ICGSE Research Papers / ICGSE Experience Reports / ICSSP / ICGSE [Joint Event] - ICSSP/ICGSE / ICGSE Journal First at ICSSP-ICGSE Chair(s): Fabio Calefato University of Bari, Paul Clarke , Marco Gerosa Northern Arizona University _ ICGSE Theme: Onboarding and community evolution ICSSP Theme: Machine Learning, AI and Microservices Architectures | ||
15:00 5mDay opening | OpeningRecorded ICGSE [Joint Event] - ICSSP/ICGSE | ||
15:05 15mResearch paper | TasRec: A Framework for Task Recommendation in Crowdsourcing ICGSE Research Papers Kumar Abhinav Accenture Labs, Gurpriya Kaur Bhatia Indraprastha Institute of Information Technology, Delhi, Alpana Dubey Accenture Labs, India, Sakshi Jain Accenture, Nitish Bhardwaj Accenture Technology Labs | ||
15:20 15mTalk | From Art to Science: Evolution of Community Development ICGSE Journal First | ||
15:35 15mFull-paper | Process Implications of Executable Domain Models for Microservices Development ICSSP | ||
15:50 15mFull-paper | Do Instance-level Review Diagrams Support Validation Processes of Cyber-Physical System Specifications ICSSP Marian Daun University of Duisburg-Essen, Jennifer Brings University of Duisburg-Essen, Thorsten Weyer University Koblenz-Landau | ||
16:05 15mFull-paper | Onboarding Bot for Newcomers to Software Engineering ICSSP James Dominic Clemson University, Charles Ritter Clemson University, Paige Rodeghero Clemson University | ||
16:20 15mExperience report | How do newcomers learn work process in Global Software Development (GSD)? A survey study from the perspective of newly project leaders ICGSE Experience Reports Raquel Cunha SIDIA Institute of Science and Technology, Fernanda Souza SIDIA Institute of Science and Technology, Franciney Lima SIDIA Institute of Science and Technology, Bruno Bonifácio Universidade Federal do Amazonas - UFAM | ||
16:35 15mExperience report | Designing Engineering Onboarding for 60+ Nationalities ICGSE Experience Reports Julian Harty Commercetest Limited | ||
16:50 15mFull-paper | From Ad-Hoc Data Analytics to DataOps ICSSP Aiswarya Munappy Chalmers University of Technology, David Issa Mattos Chalmers University of Technology, Jan Bosch , Helena Holmström Olsson Malmö University, Anas Dakkak Ericsson | ||
17:05 15mFull-paper | Emerging and Changing Tasks in the Development Process for Machine Learning Systems ICSSP Hanyan Liu Chalmers | University of Gothenburg, Samuel Eksmo Chalmers | University of Gothenburg, Johan Risberg IBM Sweden, Regina Hebig University of Gothenburg | ||
17:20 15mFull-paper | Developing ML/DL Models: A Design Framework ICSSP |