Where do ideas come from?

All new projects – whether in the arts, the sciences, business, domestic or personal life – originate from ideas. Where do these come from?
Some just seem to arise out of the blue – transmitted, it is widely believed, through vibrations of energy from ‘the Universe’ or the collective unconscious. The observation that several people who are not in contact with one other can get the same idea at around the same time would be in keeping with this. This can also happen with animals, as with the ‘Hundredth Monkey’ effect in which several groups of monkeys living on different islands learned how to to wash potatoes. The biologist Rupert Sheldrake made extensive studies of such phenomena while researching the concept of morphic fields.
Fully-fledged ideas sometimes present themselves through dreams. Well-known people said to have found creative inspiration through this channel include Frederich Kekule (chemical structure of the carbon ring), Elias Howe (invention of the sewing machine), Robert Louis Stevenson (plot of Dr Jekyll and Mr Hyde) and Paul McCartney (composition of the song Yesterday).
Experiences during waking life – not just the major events, but everyday incidents such as a chance conversation with a stranger, a visit to a new place, seeing an unusual car numberplate – are a frequent source of ideas. I must have encountered many potential instances during my medical career, and though I never took most of them any further, it was the story of one particular patient which started me off on the research project about ‘Life events and breast cancer prognosis’ which was to occupy me for several years.
As another example, I once read a case report in the British Medical Journal which for some inexplicable reason stuck in my mind, providing the inspiration for the short novel Carmen’s Roses which I have finally published ten years later (I can’t say what the case report was about without giving away the plot). Many other writers of fiction have also found medical case histories to be a valuable source of material. The best-selling novel Everlasting Love was apparently based on a report in a psychiatric journal – though its author, Ian McEwen, later admitted that both the report and the journal were fictional too, which is perhaps just as well given the importance of respecting confidentiality where real patients are concerned.