22 May 2011

Seeing Is...

Individually as a person, consciousness is the closest to our experience, but least known to our knowledge. Often people wrongly take this close subjective experience as objective knowledge of mind, hence believe in many crazy ideas for thousands of years. Now, research and evidences from neuroscience, cognitive science and AI show a different picture. This is looking at some simple details/facts about our vision as an example.

How do we see the world? Our eyes convert light waves into electrical signal and send it to the brain. From this signal, different patterns are recognized such as lines, edges, borders, shades, colors, objects, things, faces, persons, motion, etc. How are these patterns recognized by the brain? As we grow up, these patterns are slowly learned step by step and stored as NC-s (Neuronal Circuitry). These NC patterns are models of the world. Ultimately, we perceive these models. The following simple illusions vividly show the difference between the reality and our brain constructed models.

(Click on the pictures to maximize it)

(Both images are exactly same)

(Color of A and B are exactly same. Take a printout of it, then cut the two areas and place them side by side)

(This is just a static image, no motion)

(Hollow Face. This is not a trick. As face is usually projected outwards, our brain sees even the hollow side as the projected outward face )

If seeing is recognizing different patterns from the light signal, then how do we see immediately? We do not see immediately. It takes approximately 50 milliseconds to start recognize something. Different elements of visual recognition such as color, motion, etc. are recognized at different locations in the brain and it takes different time duration to recognize them (includes both serial and parallel computations). So, we actually never see live (always delayed) and never see continuously. A snapshot of visual image is taken approximately every 50 milliseconds and processed. Our brain creates an illusion of continuity. That is why we can enjoy movies – continuous snapshots of picture frames as motion picture!

If recognizing different things takes time, how do we recognize so many things so fast, for example when we enter a room? This is also an illusion. We do not recognize all things immediately. Based on the context and history, our brain uses already stored templates and fills different things one by one depending on the needs and importance. That is why it takes a while to find the differences between two similar images.

(Find the Face in the Beans)

(If you still can't find the face, a hint: at the bottom. Please try the following test sincerely)

(Major percentage of people miss it. Older people miss it easily. Though we seem to see the entire picture, our brain picks certain things and misses others depending on the situations)

Brain neuronal pulse cycle speed is in the order of tens to few hundreds of hertz (compare to Giga-hertz of current computer speed). Then, how does the brain recognize things so fast? As mentioned earlier, the brain uses many techniques such as parallel computation, template filling, etc. Why is neuronal pulse cycle speed limited? Neurons use chemical ions to charge and discharge to create electric pulses. Building up this charge potential (action potential) takes certain time. Moreover, the brain consumes huge amount of energy to function (as high as our legs). So, the brain normally works at certain minimum required speed that is needed for our normal day to day operations. But during certain high-alert critical situations (such as escaping from a danger or an accident), the brain works faster than normal speed, and hence it can process/recognize lot more information. During this period, things seem to go in slow motion. Action in the external world goes at the same speed. Since the brain processes it faster than usual speed, things seem to go in slow motion. Perhaps, most of the good sports players able to do this during simulated situations such as sports games. As we get older, this brain speed slowly goes down and hence older people perceive outside world as going faster.

For more information:

10 May 2011

Design of Intelligence

The simplest process to find solutions or design anything is a random process – design by a random chance! This is like winning a lottery ticket. This is like building a cubes-tower by throwing few cubes on a floor. If one repeats it many, many times, it may happen at one point. This is like how different sort of figures appear on clouds by random chances. Depending on many factors and conditions, the probability may vary from very low to very high – that is, from highly random to highly determined. (There is randomness in our universe at different levels. In any case, if anything is influenced by vast amount of factors, that would also produce characteristics of randomness)

It requires extremely suitable conditions including correct temperature and pressure for carbon atoms to combine together to form a diamond. However rare this might be, these conditions may happen somewhere, sometime in this vast universe, in this vast cosmological time, and hence we see diamonds in this universe. That is how everything in the universe might have formed including first DNA like molecule. But all the designs in this universe are only a very tiny fraction of countless other possibilities.

The specialty about DNA like structure is that it keeps the blueprint (or model or code) of the design – the design of the life form. As it multiplies (reproduction), the original design information mostly preserved except few random mutations (mistakes). Though most of the mutations are not suitable, some rare mutations may happen to create a design that is suitable to the environment. Hence those designs survive and move on to next generations; and others die out. Over the course of time, these suitable designs are accumulated and created life form from bacteria to fishes and plants to animals including humans. This design process is known as evolution.

Random chance is still the fundamental design discovery process. But evolution process has two important additional features:
  • It stores/accumulates the previous design information (in the DNA)
  • It shares the design information (Sharing happens only in sexual reproduction. Without this, good genes cannot spread across the population and bad genes cannot be eliminated. So, without sexual reproduction, i.e. without sharing design information, it is almost impossible for any complex organisms such as animals to evolve)
As evolution process stores the previous designs, it is essentially a learning process. The basic steps of any learning process:
  • Random Design: Randomly produce many, many designs (This occurs because of random mutations)
  • Selection: Select only the suitable designs (Only suitable to the environment survives)
  • Accumulation: Store and accumulate the suitable designs (in the DNA)
Because of these reasons, the evolution process produces from simple designs to more and more complex designs - as complex as human brain! The brain fundamentally simulates a similar learning process in its simulated virtual world. But the brain has two major advantages:
  • It can create random designs many, many times faster compare to reproduction and its mutations
  • It has way sophisticated storage than DNA.
The brain functions as per its neuronal circuitry (NC) or neural network. It comes with some inborn circuitry designed through life evolution process and others are created from life experience/learning. The brain learning process can be visually understood from watching how children learn to stand and walk.
  • Random Design: Trial and error method. Continuous training/practice (Trying different ways to stand and walk)
  • Selection: Select the suitable NC design that maximizes the correct outcome (Based on sensory input data especially by comparing the balancing signals – from the liquid levels of both ears; vision data might also be used. But what sensory data is relevant for a purpose itself is part of the learning process)
  • Accumulation: Store and accumulate the suitable NC designs (fine-tune and extend the NC)
The brain comes with some rudimentary inborn NC for stand and walk. Let us refer this NC1. NC1 is used when children try to stand and walk. When it happens to find a better balancing position, it may solidify those neuronal connections in the NC1. This would lead to a modified NC, NC2. This is known as online learning [NC1 --> NC2]. Apart from this, the brain may store the information about each body positions and corresponding sensory data in a temporary memory. Let us refer this as test data. When the body is in idle condition, as in sleep, the brain can take this test data and try it through NC2 many, many times. In each trial, the brain can randomly modify NC2 and test it with the test data and select the suitable NC designs and fine-tune and extend NC2 step by step. This is known as offline learning [NC2 --> NC3]. Offline learning should be one of the important functions of sleep (Likely happens during REM sleep). This process is repeated every day over and over for years for children to stand and walk normally. There is another important part to this process: adult/parent encouragement, supervision and help – without this, a child may be forever crippled.

Apart from genetic and other environmental differences that may lead to different brain capability and development, each child learns and modifies the NC differently as random design discovery will be different. At the end, though there are similarities across everybody's NC for stand and walk, there will be unique differences among them, which we call it individuality!

Learned NC is referred as memory. The brain memory is divided into two broad categories: Procedural memory (or implicit memory or muscle memory) and Declarative memory (or explicit memory). Learning to stand and walk is one example for procedural memory. This also includes running, dancing, swimming, bicycling, driving, etc. Procedural memory is created when we learn to do something. Declarative memory is created when we learn to recognize (or sense) sensory input data. This includes vision, hearing, sensing smell, sensing taste, etc. Learn to recognize fundamentally deals with pattern recognition.
  • Random Design: Continuous trial and error method (Different ways to recognize patterns)
  • Selection: Select the suitable NC designs that recognize different patterns (Based on sensory input data)
  • Accumulation: Store and accumulate the suitable NC designs (fine-tune and extend)

(*** Many children under certain age draw human picture without body. It likely that children may first learn to recognize important patterns such as face and limbs. And then slowly further learn to recognize other parts)

Step by step, this learning process slowly builds a model of the world as NC representations (Like how an audio sound is stored or represented in a magnetic disk or computer disk) with all its hierarchies, relationships, associations, abstractions (generalization), cause and affects, etc. This allows us to make sense of the world. This leads to reasoning and understanding. This allows us to learn to learn better (learning new learning processes and techniques). This allows us to plan and solve known, similar and related problems.

How can the brain solve a totally new problem? The brain has the world model. Now it can simulate the random (and evolution) process. It can throw all random crazy ideas and test it with the model continuously (as a background process). When an idea or solution happens to match and solve the problem, it is a eureka moment! The secret of a genius lies in how effectively someone's brain builds a better world model, which leads to high probability to solve a totally new problem and gain a new insight.

These abilities of the brain such as learning, abstract thinking, understanding, reasoning, planning, problem solving, etc. are collectively referred as intelligence. With this intelligence, humans create many designs with Planning and Control. And that is intelligent design!

May 15, 2011

Two major parts of computation:
  • Data (Declarative memory)
  • Process (Procedural memory)
Different types of learning:
  • Online learning
  • Offline learning (like REM sleep learning)
  • Supervised learning (like guidance from Parent, adult, teacher, etc.)
  • Unsupervised learning
Input --> Learning --> Data (changing to data and its hierarchies, relationships and associations)
Input --> Learning --> Process (changing processes including that are part of learning. That is changing processes recursively)

Some components of Learning:
  • Working memory (WM) is a kind of short term memory (STM) that is currently relevant and active
  • Context is some sort of background or domain information
  • Attention is a kind of filter that focuses the relevant information
Once certain thing is learned, it becomes a automatic process. Many such automatic processes can run in parallel without attention/consciousness awareness. Consciousness is a serial process that is associated with handling novel input and learning.

*** I think, this is Marvin Minsky observation.