System that replaces human intuition with algorithms outperforms human teams

[Jim__]

From phys.org:

[center][/center]

...

MIT researchers aim to take the human element out of big-data analysis, with a new system that not only searches for patterns but designs the feature set, too. To test the first prototype of their system, they enrolled it in three data science competitions, in which it competed against human teams to find predictive patterns in unfamiliar data sets. Of the 906 teams participating in the three competitions, the researchers' "Data Science Machine" finished ahead of 615.

In two of the three competitions, the predictions made by the Data Science Machine were 94 percent and 96 percent as accurate as the winning submissions. In the third, the figure was a more modest 87 percent. But where the teams of humans typically labored over their prediction algorithms for months, the Data Science Machine took somewhere between two and 12 hours to produce each of its entries.

...

For instance, one table might list retail items and their costs; another might list items included in individual customers' purchases. The Data Science Machine would begin by importing costs from the first table into the second. Then, taking its cue from the association of several different items in the second table with the same purchase number, it would execute a suite of operations to generate candidate features: total cost per order, average cost per order, minimum cost per order, and so on. As numerical identifiers proliferate across tables, the Data Science Machine layers operations on top of each other, finding minima of averages, averages of sums, and so on.

It also looks for so-called categorical data, which appear to be restricted to a limited range of values, such as days of the week or brand names. It then generates further feature candidates by dividing up existing features across categories.

more ...

[DetlefK]

1. A data-mining algorithm once came in second in a fantasy-football league:

I once had a statistics-professor whose research-group worked on data-mining algorithms, especially neural networks.

He once told the story how they had tested the NN:
They made two instances of the NN. One was fed with historical data about all matches of all teams of the Bundesliga (german national league of soccer). The other one was fed with historical data and predictions by sports-"experts".
Then they launched a fantasy-soccer league in their group and started making predictions for each game.
The NN that operated on historical data only came in second. It performed better than the NN that had historical data and "experts"-input.

What was really interesting was, how the NN was capable of predicting the outcome of a game between two teams based on decades-old data, even though the rosters had changed throughout the whole time.

[eridani]

2. MIT's Science Machine

http://www.manufacturing.net/news/2015/10/mit-system-successfully-processes-big-data-without-human-involvement

A system designed by Massachusetts Institute of Technology researchers could produce far more efficient data analysis while removing most of the need for human involvement in the process.

Although computers and sensors can collect unfathomable amounts of data, human eyes are generally needed to find patterns within that information.

MIT’s Data Science Machine, however, utilized algorithms to successfully — and quickly — analyze Big Data without human intervention. The system successfully outperformed hundreds of human teams in recent data science competitions.

"We view the Data Science Machine as a natural complement to human intelligence," said Max Kanter, whose master's thesis provided the basis for the project.

The Data Science Machine can track correlations between entries throughout a database, as well as limiting categories such as times or labels.

The system subsequently produces a series of patterns to feature, then narrows that list by further identifying correlations and implementing sample data.

MIT researchers entered the system in three different data science competitions featuring a total of 906 human teams. The Data Science Machine finished ahead of 615 teams, and its predictions were 87 percent, 94 percent and 96 percent as accurate as the winning entries.