Thought this might be of some interest to the forum (Commoncog is cited in the Director’s Note):
AI has elevated the distinction between what is tacit and what is not. Language models can summarize and automate, but when they attempt to create something that carries the signature of human craft, the result is often flat. Cedric Chin, a Singaporean analyst and one of the inspirations for this series, describes the process of acquiring tacit knowledge as the process of moving from conscious incompetence, to conscious competence, to, finally, unconscious competence.
The ceiling for LLMs, for now, seems to be conscious competence. They can capture and articulate procedure (or at least the median advice on any particular topic), but not judgment.
I can’t wait to watch it — though I have to figure out how to get away from my daughter long enough to watch it on a bigger screen!
This is really nice! I think there’s a great opportunity right now for someone to build a more general version of the Anthropic interviewer: https://www.anthropic.com/news/anthropic-interviewer to help capture tacit knowledge better in different domains.
Aye, I think there’s some work by researchers in the Naturalistic Decision Making community on building an AI-interviewer to run Cognitive Task Analysis interviews. I don’t know how that’s going though (and missed the chance to attend the NDM conference last year to ask!)
I’m also going to miss the next one, because fatherhood:
But I’m hopeful either a) someone here will attend and report back! And, b) that we’ll get more publicly available news soon.
