Splendours and Miseries of The Brain

Posted on Jul 6, 2014

A few years ago I read ‘Splendours and Miseries of The Brain’ by Semir Zeki. Zeki explores what our appreciation of art and music might tell us about the brain. For example I recall a section where he writes that we enjoy works that are somewhat unfinished because it leaves room for our imagination to fill in the gaps. As far as I recall the writing is backed up by studies using brain scans to monitor people’s reception of artworks.

I haven’t looked back at the book since reading it, but I remember that although I found parts of it interesting, I ended up thinking “so what”? If we explain our experience of looking at art in terms of what is happening in the brain, it does not seem sufficient to encompass the subjective experience of art. I am not suggesting that there is something mystical about art (or love, which is the other topic of the book) but I am mindful that art is a satisfying, multimodal experience, whilst an explanation of what art does to the brain is not (well not in the same way at least). I write this as a reminder to myself, not to get bogged down in theories of language and perception. I write it as a reminder to play.

In the article by Robert Finlay that I referred to in my last post he quotes a poem from Keates. I also thought it worth repeating here.

Do not all charms fly
At the mere touch of cold philosophy?
There was an awful rainbow once in heaven:
We know her woof, her texture; she is given
In the dull catalogue of common things.
Philosophy will clip an Angel’s wings,
Conquer all mysteries by rule and line,
Empty the haunted air, and gnomed mine—
Unweave a rainbow. . . .

(John Keats, “Lamia,” in The Complete Poems of John Keats (New York: Modern Library, 1994), p. 155.

Basic Colour Terms

Posted on Jul 5, 2014

A second video I am doing research for at the moment concerns colour; specifically colour perception and colour terminology. I am interested in several related aspects, but haven’t quite worked out how (if!) they will come together into something coherent. This blog post is an attempt to summarise and work out where I’m up to with this research.

The development of colour terminology:

In ‘Through The Language Glass’ Guy Deutscher writes about colour terminology. He tells the story of how William Gladstone noticed a paucity of colour terms and their apparently erratic usage in the writing of Homer. Gladstone’s conclusion was that Homer had an underdeveloped ability to perceive colour, compared to present day colour perception. His idea was that colour perception had evolved over time. Deutscher goes through various twists and turns from this story but the upshot is that Galdstone was wrong to suggest a deficit of colour perception, but he was onto something when he noticed an underdeveloped language for colour.

Colour perception and terms for colour description do not develop together, people may be able to see colour as well as we do, but not have the need for the number of colour terms that we use. Deutscher makes the point that we live in a world saturated by synthetic, single colour items whereas the natural world is filled with things that are more variegated.

In the 60s Berlin and Kay studied speakers from 20 different languages directly and collated evidence from 98 languages over all. Their research suggested that colour terms are adopted in a consistent order. First black and white, then red, next yellow then green or green then yellow. Then blue and then brown. They suggested an order for up to 11 terms (next including purple, grey, orange and pink). Later experiments have not supported these 11 basic colours, but some evidence seems to support the order of black & white, red, yellow, green and blue acquisition. (For example, this colour order is used as the structure for this course about colour). The logic for this order of acquisition is to do with things in our natural environment – day/night, light/dark, blood, plants etc.

Deutscher is more measured in his remarks as to whether such an order can be considered universal. He notes that there is not necessarily equivalence between colour terms in different languages. Colour terms split up a continuum and they may vary in terms of what their focus is (what is taken to be the prototypical example of a colour) and in terms of where they set the boundary between one colour and another. He raises a potentially more interesting point about whether colour terminology can actually influence what we perceive.

The influence of colour terminology on perception:

Deutscher describes several experiments that investigate whether colour words actually influence colour perception. For example Russian has two distinct colour terms for colours for ‘blue’:  ‘siniy’ (dark blue) and ‘goluboy’ (light blue). In one experiment Russian and English speakers were shown 3 blue squares; 1 alone on a top row and a row of 2 below. One of the squares in the bottom row was identical to the upper square and participants had to indicate which one by pushing a button as quickly as they could. The reaction time of Russain speakers turned out to depend not just on the objective distance between shades but also on the borderline between siniy and goluboy.

If the mis-matching square was 2 shades along in the light direction (thus across the border into goluboy from the 2 identical squares) the average time it took Russians to press the button was shorter than if it was 2 shades along in the dark direction (thus all squares would be described as ‘siniy’). When English speakers were tested with the same set up there was no skewing effect. Here’s a better explanation from New Scientist.

Colour perception, light, the eye and the brain.

LIGHT: Isaac Newton was the first person to demonstrate colour as ‘original and connate properties’ of light. Previously it was thought that light was illuminent (white) and that it could take on other properties through its interaction with matter. From his work we know that colour is not inherent in objects, but seeing colour is to do with the quantity and kind of rays that objects reflect. Colour perception occurs with the interaction of light, our eyes and our brains.

THE EYE: Thomas Young (1773–1829) proposed that all color sensations could be derived from three processes in the retina that are sensitive, respectively, to blue, green, and red and James Clerk Maxwell who showed mathematically that light interacts with structures in the retina to give rise to the sensation of sight.

THE BRAIN: We perceive colour when photons received on the retina send nerve signals to the brain. I haven’t done much research around this process although have come across the suggestion that colour is interpreted differently by different parts of the brain (e.g. system 1 thinking – largely unconscious and automatic, and system 2 thinking – conscious and requiring more effort). The brain is where colour associations are elicited, where we draw upon prior knowledge and experience.

Newton was criticised by Goethe in his ‘Theory of Colours’. I have yet to read Goethe’s book but in an article “Weaving The Rainbow, Visions of Colour in World History” by Robert Finlay writes that: “For Netwton colours are in are heads and for Goethe colours are in the world.” Finlay traces both the physiological development of colour perception and the development of its cultural significance, exploring an interrelationship between what is in the world, what happens in our eyes and their interface with the brain. He suggests that the two can never be completely separated. (As an aside Finlay mentions how Newton identified seven primary colours in the spectrum to make it match with the seven notes in the musical scale at that time. Newton saw this correspondence as confirmation of the divine harmony of nature, thus cultural assumptions influenced his work too). As well as writing about Newton and Goethe, he cites Wittgenstein’s later book ‘Remarks on Colour’ (1950) which he says “took place upon a fault line dividing the followers of Isaac Newton (1642-1727) from those of Johann Wolfgang von Goethe (1749-1727).” Finlay’s article is in some sense an attempt to answer Wittgenstein’s question about whether it is possible to establish a natural history of colour.

Deutscher adds another dimension – language – to the study of colour and perception. It is in our minds that language and vision interact and this, I think, is the crux of what I am interested in. So my next step, perhaps, is to read Goethe and Wittgenstein’s ‘On Colour.’ I know the latter is meant to be notoriously difficult to read (I have seen some of the video where Gary Hill films his daughter reading it out loud!) But perhaps with the help of Deutscher and Finlay it will enable me to think through the relationship between language and vision a little bit more. We shall see!

Prepositions

Posted on Jun 25, 2014

Untitled 2I have been reading about prepositions, as research for a short video that I am going to make. In part this is a development from my recent video work Turning Tables, which was about the influence of language on egocentric or geocentric spatial thinking.

Whilst working on this video I came across articles describing experiments in which linguists experimentally test use of prepositions (in, on, over etc) in relation to different visual scenes. (For example see “The Seamntics of English Prepositions” by Andrea Tyler and Vyvyan Evans). It seems that until quite recently it was commonly held that prepositions mapped spatial relationships between objects, in a fairly straightforward way, as if prepositions would map easily onto Euclidean geometry. More recent experiments have shown that things are not quite so simple, our use of a particular term is also influenced by 1) dynamic-kinematic aspects of the scene. 2) knowledge of object function.

For example, in relation to the first point and looking at the image, top left, where the horizontal arrows represent movement of the bowl from side to side, people are more likely to identify the top ball (identified by the diagonal arrow) as being in the bowl when it moves with the bowl, rather than when the scene is static. In relation to the second point, whether you describe an object as a ‘plate’ or a ‘dish’ apparently influences whether someone will describe a second object as being ‘on’ or ‘in’ the plate/dish. People also respond differently to items stacked in a jug vs. in a bowl and the difference is more marked when the containers are filled with liquid (when the jug seems to be fulfilling its salient function).

But why am I interested in this? Well, it’s part of a broader and ongoing exploration of the relationship between language, perception and the world. Tyler and Evans write: “If one assumes that language directly reflects the real world, then one assumes that the objectively metric properties and principles of Euclidean geometry…will form the basis of linguistic descriptions of spatial scenes” (p.21)

However, as explained, experiments show that we do not base word choice only on geometry. So they posit the idea of ‘mental models’ which act as a kind of interim stage between perception and description of a scene. When we look at an arrangement of objects we compare it to a storehouse of mental models that we have built up through experience. The model for the word ‘in’ is based on containment and locational control. The model for the word ‘on’ is to do with support and locational control.

The crux of this is that there is a bodily and experiential basis for meaning. Grammar is not just a set of abstract rules, but is based on our interaction with the physical world. This is a central notion in a book that captured me a few years’ back, Stephen Pinker’s “The Stuff of Thought” (which incidentally also contained reference to some of the experiments outlined above, although I’d forgotten that). On the one hand, the idea that grammar derives from our interaction with the world and yet is insufficient to specify it completely seems very simple and intuitive, Tyler and Evans go on to say:

“Things meaningful to us precisely when have important consequences. e.g. if we let go of a cup without placing it on a larger object, it will fall to the floor.” (p.24). and quoting another linguist, Langacker: “Linguistic expressions are not meaningful in and of themselves, but only through the access they afford to different stores of knowledge that allow us to make sense of them.” (p.17).

On the other hand, exploration of the relationship between language, meaning and the world tap into larger questions about how the mind works and about the nature of being human. What is intrinsic, what is universal and what is learned?

Tyler and Evans talk about non-spatial uses of terms ‘in’ and ‘on,’ such as when we say were are ‘in trouble’ or ‘on the dole.’ Where some linguists have suggested these are distinct, unrelated uses, Tyler and Evans posit that these are metaphorical extensions of a primary, spatial meaning. For example, we use ‘in’ when we are contained by a particularly strong emotional state. This is an idea that I grasp in theory, but haven’t really got my head around how all the meanings of the presuppositions can be reached by this sort of metaphorical extension (take a look in a dictionary and each word has 10-15 meanings laid out). Perhaps some of them are easy to trace and some of them are more like dead metaphors, retained in language but we’re not sure quite where from.

The question remains of how I am going to turn some of these ideas into a video without it seeming really boring or didactic? I must remember that I am not doing linguistics, but using some ideas taken from the field to make an artwork. My current thinking is that I will play around with mocking up some of the in/on experiments and combine this with a soundtrack of a voice intoning some different uses of the words ‘in’ and ‘on,’ but I need to test this to see if it works.

Interstellar matter

Posted on May 31, 2014

This week I found myself in the pub with a group of astrophysicists – from undergraduate to head of department level. It’s funny to hear people talking casually about interstellar matter almost as an aside, as something they just know about. It left me wishing that I knew more about what happens beyond the earth, and about the building blocks of the universe.

I was asking the physicists about their methods; how do they observe things? How are new things discovered? They told me about some of their methods: spectroscopy involves measuring the emission of electromagnetic radiation from matter, morphology involves looking at the shape of bodies or collections of matter. You can test the chemistry of molecules in a lab by staging reactions on a small scale and measuring emissions, or you can observe stars and galaxies at a distance using telescopes to collect data on electromagnetic emissions and morphology. Observation is combined with mathematical modelling from the (sometimes sparse) data that has been collected.

Two things in particular struck me from peoples responses.

1) There seems to be a distinct split in physics between being an observer and being a theorist. Apparently very few people are both – you either work on the mathematics and the modelling, or you work on observation and data collection. Recently I listened to an old episode of ‘In Our Time’ on optics and in this episode they guests were talking about the separate paths of practice and theory in the development of lenses. Lenses were being made (e.g. for glasses) by artisans, whilst separately philosophers and mathematicians were writing treatises on optics without actually practically testing any of their theories. The programme suggested that it was really only with the enlightenment and the work of Galileo and Kepler that these two separate paths came together. Of course now theorists and observers work closely in the same university department, but I was surprised that this split endures.

2) Interpreting data beyond our visual capabilities can become second nature, like speaking a foreign language. Talking a bit about the large telescopes used to collect data, I asked whether they actually go and look through the telescopes (they do sometimes) or if data was collected and disseminated to different researchers (this is also often true). As a layperson I tend to think of telescopes in terms of magnification – visually observing distant planets. But although some observation is visual – looking at shape and colour – when considering the whole electromagnetic spectrum, most data is outside our visual range, so researchers receive it in numerical form. I asked whether, over time, interpreting this data becomes as straightforward as the visual interpretation that we all do habitually every day. One of the group suggested it’s like learning a foreign language – eventually you can think in this language.

I also had a discussion about how ideas progress and, from this meeting at least, it seemed that a theory often comes first (although in truth the process must be recursive). Then it’s a case of testing whether observed data supports that theory. Some of the Phd students present explained that when they want ‘telescope time’ they have to make a case to book this with a coherent proposal about what they are looking for. I also talked to one person a bit about the meeting point of process, of day to day work and its minutiae with the ambition of astrophysics to explain the universe – the bigger picture of discovery. Interestingly he said that, if you think too much about the big picture you will miss a lot of the detail. I can imagine trying to have both in mind at once, but one at the forefront of your mind and the other hovering in the background.