I am currently “reading” (listening to the audiobook actually) David Brooks’ Social Animal. First of, if you care about understanding human behavior in any realm, it is a fantastic read. I have always enjoyed Brooks’ columns at the NYT. His writing in this book is just as engaging and thought-provoking.
I listened to a segment today about epistemological modesty. In addition to introducing epistemological modesty, it included the idea that what we do know can be known in different ways. The example Brooks gives:
For example, if you were asked what day in the spring you should plant corn, you could consult a scientist. You could calculate the weather patterns, consult the historical record, and find the optimal temperature range and date at each latitude and altitude. On the other hand, you could ask a farmer. Folk wisdom in North America decrees that corn should be planted when oak leaves are the size of a squirrel’s ear. Whatever the weather in any particular year, this rule will guide the farmer to the right date.
For the big data practitioner, one of the keys to setting your work apart is finding out how you learn about those additional sources of patterns or wisdom (and thus data) so that you can go and acquire it and use it in your analyses.
And by the way, big data practitioners should add epistemological modesty to their personal toolkit. An over reliance on data without an appreciation of human behavior and context can lead to incorrect conclusions that are “supported by the data.”