When human judgment and big data intersect there are some funny things that happen.
There are a number of fascinating stories included in 'The Human Face of Big Data' that represent some of the most innovative applications of data that are shaping our future.
Films are big hits when they touch a lot of people. Things are not funny in a vacuum, they're funny because we respond to some personal dislocation, some embarrassment, some humiliation, some pain we've suffered, or some desire we have.
Big data has been used by human beings for a long time - just in bricks-and-mortar applications. Insurance and standardized tests are both examples of big data from before the Internet.
People think 'big data' avoids the problem of discrimination because you are dealing with big data sets, but, in fact, big data is being used for more and more precise forms of discrimination - a form of data redlining.
Fashion is just a name for one of the things that happen where mind and body intersect.
While many big-data providers do their best to de-identify individuals from human-subject data sets, the risk of re-identification is very real.
One of the myths about the Internet of Things is that companies have all the data they need, but their real challenge is making sense of it. In reality, the cost of collecting some kinds of data remains too high, the quality of the data isn't always good enough, and it remains difficult to integrate multiple data sources.
The ability to collect, analyze, triangulate and visualize vast amounts of data in real time is something the human race has never had before. This new set of tools, often referred by the lofty term 'Big Data,' has begun to emerge as a new approach to addressing some of the biggest challenges facing our planet.
The promoters of big data would like us to believe that behind the lines of code and vast databases lie objective and universal insights into patterns of human behavior, be it consumer spending, criminal or terrorist acts, healthy habits, or employee productivity. But many big-data evangelists avoid taking a hard look at the weaknesses.
Some things happen without you putting an effort, while some others things do not happen despite we wanting it to happen.
Big data is great when you want to verify and quantify small data - as big data is all about seeking a correlation - small data about seeking the causation.
MapReduce has become the assembly language for big data processing, and SnapReduce employs sophisticated techniques to compile SnapLogic data integration pipelines into this new big data target language. Applying everything we know about the two worlds of integration and Hadoop, we built our technology to directly fit MapReduce, making the process of connectivity and large scale data integration seamless and simple.
If we study learning as a data science, we can reverse engineer the human brain and tailor learning techniques to maximize the chances of student success. This is the biggest revolution that could happen in education, turning it into a data-driven science, and not such a medieval set of rumors professors tend to carry on.
We get more data about people than any other data company gets about people, about anything - and it's not even close. We're looking at what you know, what you don't know, how you learn best. The big difference between us and other big data companies is that we're not ever marketing your data to a third party for any reason.
One [Big Data] challenge is how we can understand and use big data when it comes in an unstructured format.