Portal de Periódicos da CAPES

OnlineStats.jl: A Julia package for statistics on data streams

2020; Open Journals; Volume: 5; Issue: 46 Linguagem: Inglês

10.21105/joss.01816

2475-9066

Josh Day, Hua Zhou,

Advanced Database Systems and Queries

The growing prevalence of big and streaming data requires a new generation of tools.Data often has infinite size in the sense that new observations are continually arriving daily, hourly, etc.In recent years, several new technologies such as Kafka (Apache Software Foundation, n.d.-a) and Spark Streaming (Apache Software Foundation, n.d.-b) have been introduced for processing streaming data.Statistical tools for data streams, however, are under-developed and offer only basic functionality.The majority of statistical software can only operate on finite batches and require re-loading possibly large datasets for seemingly simple tasks such as incorporating a few more observations into an analysis.OnlineStats is a Julia (Bezanson, Edelman, Karpinski, & Shah, 2017) package for highperformance online algorithms.The OnlineStats framework is easily extensible, includes a large catalog of algorithms, provides primitives for parallel computing, and offers a weighting mechanism that allows new observations have a higher relative influence over the value of the statistic/model/visualization.