Statistical Scalability for Streaming Data

The profusion of sensor-based technologies in research and industrial systems is having a major influence on our ability to derive real-time insight and understanding about the world around us. However, this deluge of data streams also brings with it significant statistical challenge.

In the past, there has been a tendency to focus purely on algorithmic scalability. However, increasingly we find new statistical challenges arising, requiring novel solutions. For example, the statistical analysis of data streams can involve computational and memory constraints due to a need to process data at source on limited hardware. An alternative challenge can arise with the problem of synthesizing information across multiple related streams, such as those observed in digital networks. Research is therefore needed to find more robust and effective methods for scaling data and, in particular, streaming data.

This workshop is part of the six month programme at the Isaac Newton Institute (INI) on Statistical Scalability. It follows on from a highly successful knowledge exchange workshop in February which focused on Big Data and the role of statistical scalability..

Aims and Objectives Challenges in streaming data arise in numerous fields – consumer products, financial transactions, computer network traffic, transport and communications networks and energy systems are just some of them. As with statistical scaling generally, this requires an integrated approach.

This knowledge exchange event by the Turing Gateway to Mathematics will harness expertise from the research being undertaken as part of the INI Statistical Scalability Research Programme. It will also highlight experience, expertise and challenges from a number of key stakeholders from the following areas:

Exploration and Geology Energy and Environment Communications The problems faced can often be generic and so have relevance to numerous other sectors and end user applications. Additionally, real applications of streaming data involve specialist streaming infrastructure, hardware and software. It is therefore envisaged that this event will be of interest to a wide audience including those working in multiple business and industrial sectors, Government and the public sector.