joelchan's working notes

Powered by 🌱Roam Garden

QUE - How might domain distance modulate the effects of analogies on creative output?

Synthesis

I think the first subclaim is the most obvious one: [[CLM - Analogical distance of inspirations for an idea are positively related to the idea's creativity]]. This is kind of a weak sauce claim, to be honest. A bit too general for my taste.

Case for the claim

[[EVD - highly novel papers had higher variance in their citation outcomes over a 15-year window, biased towards the higher impact tail of the distribution - @wangBiasNoveltyScience2017]]

[[EVD - In general, MechE students' ideas when given near analogies were at least as novel, and sometimes moreso, then ideas generated with far analogies, or seeing nothing - @tsengRoleTimingAnalogical2008]]

There is some weak support from an early experiment from @dahlInfluenceValueAnalogical2002 that everyone cites in the creativity/engineering design literature. Two core results relating to analogical distance come from the third experiment.

In the one case, analogical distance was measured post-hoc from the analogies that the sample of engineering students turned out to use (think of; not given). They found a positive correlation between % far analogy use and originality/value: [[EVD - designs that were measured during an ideation session to have come from a higher percentage of far analogies were perceived as more original and valuable by potential customer judges - @dahlInfluenceValueAnalogical2002]]

In the other case, analogical distance was not varied directly in terms of the stimuli, but was manipulated in a "one or more stimuli vs. no stimuli" way. All stimuli were near analogies. They found a negative effect of the near stimuli compared to seeing nothing: [[EVD - Engineering students who were given one or more near analogies as stimuli generated design ideas that were perceived as less original and valuable by potential customer judges, compared to being given no stimuli - @dahlInfluenceValueAnalogical2002]]

There's also some evidence from citation level data that looks at combinations of "knowledge domains" (with journals as a proxy) and finds correlations between having atypical or highly novel combinations of knowledge domains being related to being outliers in terms of citation impact

[[EVD - papers with high median conventionality and high tail atypical combinations of journals they cited were 2x more likely than average to be in top 5 percent of citation distribution - @uzziAtypicalCombinationsScientific2013]]

[[EVD - highly novel papers were more likely to be in the top 1% of citations in the long run, but not in the short run, and particularly in other fields - @wangBiasNoveltyScience2017]]

Some complicated, qualified support also comes from studies that find contingent benefits of far analogies

[[EVD - Far analogies' effects on MechE students' ideation fluency and diversity were different depending on timing: more ideas (but with more functional repeats) before ideation began, vs. more functionally distinct designs after problem solving began (during a break), both compared to seeing nothing - @tsengRoleTimingAnalogical2008]]

[[EVD - Far analogies' effects on MechE students' novelty of ideas were different depending on timing: more novel ideas when seeing them after problem solving began vs. before - @tsengRoleTimingAnalogical2008]]

Also some support from outsider innovation effects

[[EVD - Self-rated technical distance from open innovation contest problems was slightly positively correlated with submitting winning solutions - @jeppesenMarginalityProblemsolvingEffectiveness2010]]

And some support in the form of negative effects of near analogies

Near associated with fixation: [[EVD - Undergrad MechE students who saw near analogies (clocks) for a time-keeping problem preferred discrete (drip-based) solutions to continuous (e.g., flow/fill) solutions - @tsengRoleTimingAnalogical2008]]

Counterarguments

There is some counter-evidence from m/s/In vivo studies of science labs and design teams

For example, Dunbar traced the discovery process of four top molecular biology labs, and found that they rarely used far analogies to generate novel scientific concepts: [[EVD - Molecular biologists with a reputation for innovation rarely used very far analogies in their lab meetings while generating novel scientific concepts; instead, they relied mainly on analogies to the same or other biological organisms - @dunbarHowScientistsThink1997]]

This is a strong result, even though it isn't experimental, due to its high external validity. Despite lack of experimental controls, it at least puts a cap on the claim about far analogies being important/necessary for creative breakthroughs, at least in molecular biology.

A very similar result was found by Saner et al (1999), who found that far analogies were basically never used in reasoning in psychology lab meetings, but were used in colloquia while ideas were being communicated with new audiences: [[EVD - Far analogies were rare and never used in psychology lab group meetings for reasoning (vs. mere mentions); far analogies were rare in colloquia as well, but were frequently used for reasoning - @sanerAnalogiesOutBlue1999]]

Related to this is [[EVD - In a design team, concepts tended to be more similar to their immediately preceding concepts after far analogy use compared to using near or no analogies - @chanImpactAnalogiesCreative2015]]. This isn't *quite* a measure of novelty as we normally think of it (the reference for similarity is concepts generated in the last 10 or 5 lines). But it at least opposes the idea that you might be primarily using far analogies to get some distance from where you currently are.

Dunbar also claims that [[CLM - far analogies are systematically overrated in their importance for creative breakthroughs due to memory bias - @dunbarHowScientistsThink1997]]

One support for this is a specific anecdote of tracing a significant discovery through a series of (near) analogies, and then asking the scientist to recall/recount the origins of that discovery: Dunbar found that [[EVD - A molecular biologist who had made a major scientific conceptual change did not recall any of the spontaneous analogies used to enact that change - @dunbarHowScientistsThink1997]]

And some experimental counter-evidence showing equal effects of near analogies on novelty

[[EVD - In general, MechE students' ideas when given near analogies were at least as novel, and sometimes moreso, then ideas generated with far analogies, or seeing nothing - @tsengRoleTimingAnalogical2008]]

Synthesis queries

{{synthesis}}

match

Evidence

Conditions

0

That

true

Relation

Informs

Predicate

QUE - How might domain distance modulate the effects of analogies on creative output?

Predicate

analogical distanc

More precise claim! [[CLM - There is a "Sweet Spot" in Analogical Distance. Somewhat far analogies lead to better creative outcomes than very near or very far analogies]]

this is Supported By

[[EVD - Molecular biologists with a reputation for innovation rarely used very far analogies in their lab meetings while generating novel scientific concepts; instead, they relied mainly on analogies to the same or other biological organisms - @dunbarHowScientistsThink1997]]

[[EVD - papers with high median conventionality and high tail atypical combinations of journals they cited were 2x more likely than average to be in top 5 percent of citation distribution - @uzziAtypicalCombinationsScientific2013]]


@artsParadiseNoveltyLoss2018a for [[QUE - How might domain distance modulate the effects of analogies on creative output?]] | if inventor files a patent in a new field they haven't explored before, their patent tends to be more novel; there's also a citation penalty, but this gets canceled out if they collaborate with an expert in the new field

[[EVD - patents filed by inventors who was new to the patent's field tended to be more novel; this was less true if they collaborated with an expert in the new field - @artsParadiseNoveltyLoss2018a]]

[[EVD - patents filed by inventors who was new to the patent's field tended to receive slightly fewer citations, except when they collaborated with an expert in the new field - @artsParadiseNoveltyLoss2018a]]

@uzziAtypicalCombinationsScientific2013 for [[QUE - How might domain distance modulate the effects of analogies on creative output?]] | both combining highly atypical and conventional combinations of journals (as a proxy for domain knowledge) led to 2x more likelihood of being a "hit" compared to average; teams more likely to enter this state too

[[EVD - papers with high median conventionality and high tail atypical combinations of journals they cited were 2x more likely than average to be in top 5 percent of citation distribution - @uzziAtypicalCombinationsScientific2013]]

@kneelandExploringUnchartedTerritory2020 for [[QUE - How might domain distance modulate the effects of analogies on creative output?]] | teams with broader prior patent experience more likely to generate outlier patents; examples of long search paths and distant recombination leading to outlier patents

[[EVD - patenting teams with a greater number of prior patenting fields were more likely to generate patents that were technological outliers - @kneelandExploringUnchartedTerritory2020]]

@wangBiasNoveltyScience2017 for [[QUE - How might domain distance modulate the effects of analogies on creative output?]] | papers that make first-time difficulty-adjusted combinations of journals have higher (variance in) impact, mostly over long-run over short-run, frequently in lower-impact venues, and mostly in other fields

[[EVD - highly novel papers had higher variance in their citation outcomes over a 15-year window, biased towards the higher impact tail of the distribution - @wangBiasNoveltyScience2017]]

[[EVD - highly novel combinations of cited journals in a paper were almost always cross-disciplinary, but cross-disciplinary combinations were infrequently novel - @wangBiasNoveltyScience2017]]

[[EVD - highly novel papers were more likely to be in the top 1% of citations in the long run, but not in the short run, and particularly in other fields - @wangBiasNoveltyScience2017]]

[[EVD - highly novel papers were more likely to be published in lower impact journals - @wangBiasNoveltyScience2017]]

@tsengRoleTimingAnalogical2008 for [[QUE - How might domain distance modulate the effects of analogies on creative output?]] | near analogies suprisingly novel; quant/diversity benefit for far analogies (curated) only when encountered with open goal (not before)

@sanerAnalogiesOutBlue1999 for [[QUE - How might domain distance modulate the effects of analogies on creative output?]] | TLDR: infrequent mentions of far analogies amongst psychology labs, basically no actual use for reasoning in lab group meetings during actual working out of discovery, but did use in colloquia when presenting more finished work

George Mestral's invention of velcro by analogy to a burdock root seed stuck to his dog's fur @swearingenIdeaThatStuck2016

observation-notes Crowd workers generated the most novel and practical ideas when given analogical inspirations found from domain-independent, concrete constraint problem descriptions, compared to inspirations from domain-independent abstract constraints, domain-dependent concrete constraints, or domain-dependent abstract constraints (which were about equal)
@yuDistributedAnalogicalIdea2016a

These are technically not analogies, but are somewhat relatinoal in that the stimulus words are verbs

Concepts generated by engineering graduate students who received stimulus verbs that were antonyms of the core functions of a problem generated were judged to be more novel than students who receive stimulus verbs that were synonyms of the core functions of the problem. This effect was robust for two out of four problems studied.

Concepts generated by engineering graduate students who received stimulus verbs that were antonyms of the core functions of a problem generated were judged to be more useful than students who receive stimulus verbs that were synonyms of the core functions of the problem. This effect was robust for two out of four problems studied.

Concepts generated by engineering graduate students who received stimulus verbs that were antonyms of the core functions of a problem generated were judged to be more cohesive than students who receive stimulus verbs that were synonyms of the core functions of the problem. This effect was robust for only 1 out of 4 problems studied.

25 cases is also really really small for a correlation analysis.

I think the most we can say from this is an existence proof that relatively near analogies can frequently lead to quite innovative outcomes.

25 cases is also really really small for a correlation analysis.

I think the most we can say from this is an existence proof that relatively near analogies can frequently lead to quite innovative outcomes.

Telling that I previously classified this study as "anecdotal"! Yikes... This was a rigorously done multiple-case-study analysis.

@jeppesenMarginalityProblemsolvingEffectiveness2010 | more novel ideas / perspectives

Solvers who chose to submit their solutions to the 2nd stage of an innovation contest and self-rated themselves as more technically distant from the problem domain were slightly more likely to submit winning solutions for Innocentive science problem solving contests