Finished Mass Effect 3. If you're an active resident of the internets, you've probably heard that a whole lot of people aren't just displeased with the ending, they actively hate it -- to the point that Bioware is caving and preparing an alternate ending (try searching around if you want to know more). I'll admit that the ending feels a little contrived at points (and the post credits scene is laughably bad), but it could be waaaay worse.
Here's the thing to know: in the minds of the game's creators, the message of this series was never simply about survival. For them, the whole of the series was about organic vs synthetic life (and the presumed deficiencies of organics). In that light, the ending makes perfect sense.
Unfortunately, this overarching "organic vs synthetic" plot line was never made nearly as explicit as the "fight for survival" that dominates the rhetoric of the characters. Yes, the synthetic nature of the Reapers makes them a unique sort of enemy, but it's their unknowable-ness (a combination of their origin, their age, and their limited appearance), not just their unnaturalness that makes them daunting.
You could just as easily have had an advanced organic race that shows up every several thousand years to check in on the modern races and assimilate or destroy them as they see fit. The Reapers didn't need to be synthetic to be effective, they just needed to be insurmountably powerful. Being synthetic is one path to that. Simple sci-fi technobabble is another.
It would be hard to do more with just the Reapers themselves to enhance the clarity of the message that the writers were trying to deliver, though. They need to be somewhat remote so they can remain effective boogeymen. Besides, Sovereign and Harbinger both bring out the whole "gaaah, you are puny organics, we are your future, cower! COWER, I SAY!" rhetoric on those rare occasions where you do get to communicate.
In order to make this story work, you need to have a world that's built around paranoia about the limitations of organics from the start. You need to have the advanced races terrified about the technological and cultural plateaus that they've been sitting at, sometimes for millennia. Humanity has only been part of the interstellar scene for about 30 years by the end of Mass Effect 3, and yet we're as advanced as any other species in the galaxy.
You need to have more incidents of failures to contain or control artificial intelligence, possibly all having gone and merged with one another to form some heterogeneous variation on the Geth. You need to have these scarier Geth floating around the edges of "civilized" space before things ever start, using their unique intelligence to devise technologies and solutions and threats that we've never conceived of. The Reapers need to not be the only threat, but the biggest and most sudden. The Reapers need to be a realization for the existing fears that are already plaguing the population.
Instead, we have a peacetime galactic government that is focused on politics as usual and just the one incidence of failed A.I.. Worse, the Geth don't seem to be improving much upon the same technologies everybody else in the galaxy is using, and once Legion shows up to provides a mouthpiece for them, they suddenly become more accessible -- making it seem like collaboration could be a legitimate option if organics are willing to set aside their (limited) fears.
Fixing these problems of setting would be an important step, but the rest of the problem is harder to address. Part of the reason the basic survival narrative seems so dominant is because it's next to impossible to provide a gameplay goal built around anything else. This stripped down story was always going to engage the player's attention because surviving is what they spend roughly half of the game doing. The only solution I can see is to make fights with synthetics lifeforms more imposing. Make sure that they are so much harder to kill than simple security robots. Make these fights feel distinct and more tense. Don't just send Shepard and friends off to kill them in droves, but let a handful of Geth be a significant challenge.
With these (not insignificant) changes, I'm sure people would have still complained about the ending because it's an ending and it's never going to satisfy everybody (and there are some contrived elements to it). But the complaints would have been for different reasons and could at least have been in line with what the writers were envisioning. As it stands, I'm glad to know what I know and I'll be glad to let that knowledge shape how I play through the game in the future. It was a fun trip, even if I'm not certain it was as strong as either of the first two games.
Saturday, March 24, 2012
Thursday, March 22, 2012
Games, Education, and Exploration
Generally speaking, I don't believe in educational games. That's not to say that I didn't learn a lot of facts from games like Number Muncher or Mario's Time Machine, but games like that are glorified flash cards. I think that we can add an engagement factor and arbitrary context for learning strictly declarative knowledge just fine. What I don't believe in is games that help, by their very nature, in learning procedural knowledge or higher level relationships between concepts.
This wouldn't be an issue if I wasn't currently taking a course that is, theoretically, about making games to serve just this purpose. Twice a week, I get to sit and listen to stories about all of the funding that my professor has earned from various institutions trying to make these sorts of games.
There are two simple reasons that I don't believe in educational games. One is transference. Virtually all games, especially games exploring abstract concepts, employ some sort of metaphor. This means that whatever skills or concepts are being taught, ultimately, are being learned with regards to the metaphor. Knowing what you're supposed to actually learn, then, requires additional interaction with the domain that the lessons are supposed to be applied to. Moreover, it requires an active effort to identify the pieces of the analogy in place. All of this means that it's going to be very hard to make the game itself educational unless it's included as a piece of a larger, directed education plan. Either that, or the game needs to actively mix in these elements, which risks breaking engagement.
Second is that, my professor, at least, seems to lack any sense of distinction between "play" and "playing a game". Play is self-directed and exploratory. It's a matter of expressing and recognizing and responding to patterns that we've observed in an environment where we have complete control. This lets us experiment with ideas and see where the rabbit hole leads us (even if it leads us astray). This is incredibly useful in learning because learning is an exploratory activity. On the other hand, games, by their very nature, are goal based.
The purpose of a game, in the mind of the player, is to complete the game. This can mean gaining mastery of the systems in play, or it can mean butting your head against it until something works and then moving on. In cases where a game requires real world knowledge, people are willing to play lip service to that knowledge without taking it to heart (just like students are willing to put the answer the professor wants on a test or a paper without believing a word of it). In short, all we want to learn from a game is how to beat the game.
Play is educational. Playing a game is not. What we need aren't educational games, but educational toys. Toys can be used in the construction of games to push people towards a specific goal, and games can have their purpose subverted and turned into toys to promote experimentation, but they are very different things.
This wouldn't be an issue if I wasn't currently taking a course that is, theoretically, about making games to serve just this purpose. Twice a week, I get to sit and listen to stories about all of the funding that my professor has earned from various institutions trying to make these sorts of games.
There are two simple reasons that I don't believe in educational games. One is transference. Virtually all games, especially games exploring abstract concepts, employ some sort of metaphor. This means that whatever skills or concepts are being taught, ultimately, are being learned with regards to the metaphor. Knowing what you're supposed to actually learn, then, requires additional interaction with the domain that the lessons are supposed to be applied to. Moreover, it requires an active effort to identify the pieces of the analogy in place. All of this means that it's going to be very hard to make the game itself educational unless it's included as a piece of a larger, directed education plan. Either that, or the game needs to actively mix in these elements, which risks breaking engagement.
Second is that, my professor, at least, seems to lack any sense of distinction between "play" and "playing a game". Play is self-directed and exploratory. It's a matter of expressing and recognizing and responding to patterns that we've observed in an environment where we have complete control. This lets us experiment with ideas and see where the rabbit hole leads us (even if it leads us astray). This is incredibly useful in learning because learning is an exploratory activity. On the other hand, games, by their very nature, are goal based.
The purpose of a game, in the mind of the player, is to complete the game. This can mean gaining mastery of the systems in play, or it can mean butting your head against it until something works and then moving on. In cases where a game requires real world knowledge, people are willing to play lip service to that knowledge without taking it to heart (just like students are willing to put the answer the professor wants on a test or a paper without believing a word of it). In short, all we want to learn from a game is how to beat the game.
Play is educational. Playing a game is not. What we need aren't educational games, but educational toys. Toys can be used in the construction of games to push people towards a specific goal, and games can have their purpose subverted and turned into toys to promote experimentation, but they are very different things.
Sunday, March 18, 2012
Inventing Recipes: Huevos Rancheros Burger
If you've ever had a burger topped with a fried egg, then this variation should come as a no-brainer: top a black bean burger with a fried egg, some avocado, and salsa. This may be one of the most delicious things I've ever had. It's been knocking around in my head for ages, but my last couple of efforts to make black-bean burgers hadn't fared too well. A recent bout of falafel making gave me some ideas to fix that, though. This may be the best thing that has come out of me going vegetarian for lent.
Huevos Rancheros Burgers (feeds four):
Patties
2 15 oz cans black beans
1/2 large white onion
3 large cloves garlic
1.5 tsp salt
2 tsp cumin
1 cup frozen corn
3 Tbsp flour
Mince the garlic and onion; drain and rinse the beans. Mash the beans thoroughly and then fold everything together. Form into 4 large patties and grill, 5 minutes on a side, or cook in a pan with a bit of oil somewhat longer.
Salsa
1 15 oz can "fire roasted" tomatoes
2 large cloves garlic
1 large jalapeno pepper
2 Tbsp Cider Vinegar
Salt, Cumin, Coriander to taste (be somewhat generous with coriander and more restrained with cumin)
In a small, non-stick fry pan, roast the garlic (unpeeled) and the jalapeno. This should take 10-15 minutes over medium high heat. Turn everything on occasion. Once the garlic is a pleasant brown and the pepper is lightly charred on all sides, peel the garlic and remove the stem from the pepper. Add everything to a blender and liquify, seasoning as desired.
Others
1 avocado
4 buns
4 large eggs
Shredded cheese
Halve the avocado and remove the pit. Then, with the rind still on, slice perpendicular to the rind into strips without cutting through the avocado. If the avocado is sufficiently ripe, you should be able to remove slices with a spoon.
Fry the eggs to your desired done-ness (something slightly better than over medium works well), ideally keeping them individual. When you flip the eggs, top them with cheese so it melts.
Toast the buns. Then stack them with a black bean patty, an egg, slices of avocado, and salsa. Consume with great vigor.
Huevos Rancheros Burgers (feeds four):
Patties
2 15 oz cans black beans
1/2 large white onion
3 large cloves garlic
1.5 tsp salt
2 tsp cumin
1 cup frozen corn
3 Tbsp flour
Mince the garlic and onion; drain and rinse the beans. Mash the beans thoroughly and then fold everything together. Form into 4 large patties and grill, 5 minutes on a side, or cook in a pan with a bit of oil somewhat longer.
Salsa
1 15 oz can "fire roasted" tomatoes
2 large cloves garlic
1 large jalapeno pepper
2 Tbsp Cider Vinegar
Salt, Cumin, Coriander to taste (be somewhat generous with coriander and more restrained with cumin)
In a small, non-stick fry pan, roast the garlic (unpeeled) and the jalapeno. This should take 10-15 minutes over medium high heat. Turn everything on occasion. Once the garlic is a pleasant brown and the pepper is lightly charred on all sides, peel the garlic and remove the stem from the pepper. Add everything to a blender and liquify, seasoning as desired.
Others
1 avocado
4 buns
4 large eggs
Shredded cheese
Halve the avocado and remove the pit. Then, with the rind still on, slice perpendicular to the rind into strips without cutting through the avocado. If the avocado is sufficiently ripe, you should be able to remove slices with a spoon.
Fry the eggs to your desired done-ness (something slightly better than over medium works well), ideally keeping them individual. When you flip the eggs, top them with cheese so it melts.
Toast the buns. Then stack them with a black bean patty, an egg, slices of avocado, and salsa. Consume with great vigor.
Thursday, March 15, 2012
Thoughts from the Half-Way Point: Mass Effect 3
Unless I opt to move my 360 into my room, the NCAA tournament is going to put a massive damper on my ability to play Mass Effect 3 over the next several days. As such, I figure this is as good a time as any to sit back and reflect some on the experience so far. I'll try to avoid spoilers, but at this point, so much of the game depends on what you've done before, that there are only a few plot points that would obviously follow no matter what your Shepard has done.
That's really the first mark in the game's favor. After so many decisions in ME1 seemed to be largely without consequence beyond the cosmetic in ME2, everything you've done before suddenly takes on added significance. More important, the blatant good/bad dichotomy has lessened remarkably, as the bulk of decisions seem to be much more grey. To this end, even the Paragon/Renegade divide has been diminished, with many decisions simply serving to enhance your "reputation" -- a catchall that contributes to both scores. I finally have a compelling reason to revisit Mass Effect 1 and alter my decision making at certain junctures just to see how certain quests change and how that will affect my ability to prepare the galaxy at large.
The increased value of your decisions wouldn't matter if they didn't continue to contribute to strong storytelling, though, and I'm happy to say that they do. The overarching plot is, of course, a bit generic (help prepare an army to challenge a seemingly unstoppable enemy), but the power is in the setting and the bond that each player gets to establish with it. There are so many characters at this point and so much history that we've been exposed to that it's hard not to feel an attachment when a familiar face pops up or when you get closure from a character or event that you expected to be left by the wayside. Of course, not every character can be included in a satisfying way. Some additions are brief and seem forced in as nothing more than an excuse to let some secondary or tertiary character saunter by as a reminder of their existence and then contribute unseen to galactic readiness (this holds especially true of any character that showed up exclusively in DLC). But, we can't expect miracles from the writers, and satisfying everybody was never an option.
Honestly, I only have two real complaints about the story so far. First of all, too many characters have changed or accomplished too much since the end in Mass Effect 2. The game tells us that it's been six months since Shepard turned him/herself in for trial on Earth. How in the hell have some of these people picked up the positions they have since then? Grunt, Jack, even Garrus: I'm looking at you. Second, the game really needs to give a better glimpse into Cerberus' motivations. It sounds like that's going to play a much more serious role going forward. For those of you farther into the game than I, a certain major plot event just took place on the Citadel. I get that it's not meant to make immediate sense from the perspective of the player/our characters, but there's no coherent attitude/belief system of which the player is aware that would allow the event to make sense. People make choices for reasons other than "wouldn't it be dramatic if...", and that's the only attitude that could possibly seem to be driving the Illusive Man at this point.
With the important part out of the way, it's time to talk briefly about gameplay.
In terms of controls, we're mostly looking at a fine tuning of the systems in place in Mass Effect 2. Gunplay sometimes feels a bit loose, but that happens with most 3rd person shooters. The improved mobility is nice, and I find my infiltrator using his cloak to sprint up and beat down fools as often as to pick out a careful headshot. The changes made to armor sorts of abilities -- each carrying the option to purge the added defense in order to gain some other temporary advantage -- are fun and make them seem more active. I also like that these armor abilities (along with the infiltrator's cloak) don't incur their cooldown until they are purged or their period ends. It makes it a lot easier to use traditional powers in conjunction with these prolonged effect abilities. Combat, in general, is as fun as it's been, and that makes multiplayer a much better option than anybody would have expected.
First, though, I have a couple of complaints about things at the design level. The least important but most glaring of these is the new resource collecting system. You run around star systems using your sensors in an attempt to ping for things that could be useful in the preparations you're supporting. This isn't so bad by itself, but each time you run a scan, you make Reapers more aware of your activity in the region. Do it too often (basically, 3 times), and they're off and chasing you and you need to get your ass out of the system! Except...you don't. The game autosaves each time you enter a system. There's no good reason not to just go around firing off your scanners like crazy, determine where everything is, Reapers be damned, let them catch you, and then have that autosave reload so you can fire off your scanners in the two or three places you need to to collect everything from a system. It's a silly waste of time that tries to add a sense of high stakes and consequence where there are none.
Slightly more annoying is a shift in combat similar to some things I complained about in Dragon Age 2: namely, inexplicable waves of enemies. Reinforcements will jet pack in or jump down from balconies or come through otherwise sealed doors at every opportunity, just to inflate challenge. It makes it nearly impossible to develop any high level strategy the way that you sometimes needed to at the upper difficulties of Mass Effect 1 and that were occasionally beneficial in Mass Effect 2. It keeps the action going, and things are fun enough that I don't mind too much at the default difficulty, but I'm going to be cursing up a storm during my inevitable Insanity attempt. In the prior two games harder difficulties just started to feel natural once you adapted to what they expected of you. I don't see that happening here because the source of difficulty is different.
Finally, the designers seem to have responded in the worst possible fashion to the complaints that Mass Effect 2 lacked side quests. There are now dozens and dozens of meaningless fetch quests you can go on (largely connected to the new resource system) and lots of time consuming, fluffy interactions available at the Citadel (and possibly other hubs if there are any). The fetch quests remind me a lot of Dragon Age 2 and the various weird things you would find while you were going about your adventuring only to suddenly know that somebody would give you money/experience for them. In this case, at least you overhear conversations about the things to know that you're looking for them, but it's similarly arbitrary. The more traditional quests, while at least providing some sort of context, but they mostly involve talking to a sequence of people/terminals and then getting some sort of payoff when a tertiary character from some previous sidequest in a previous game shows up at the end.
The good news is that, other than establishing mood, all of these serve simply to increase the resources you can bring to the fight with the Reapers, meaning they can be avoided without consequence if you just sink some time into the multiplayer!
I haven't done much with the multiplayer so far. Spent an hour or so going through a few rounds with Andy before I'd really gotten into the single player. I'm looking forward to going back. For the most part, it's class-based horde mode with unlockable persistent upgrades in the form of both level gaining and new equipment. Each player gets to control a stripped down version of one of the base classes, each with three hotkeyed abilities and two weapons (or at least the Sentinel only gets two weapons, maybe the Soldier gets more?). You're dropped in hostile territory and confronted with a series of randomized enemy waves and objectives and asked to survive as many as you can, up to 10. The longer you last, the more experience you get and the more you improve the ability of the galaxy to fight off the Reapers. Good stuff, and I'm eager to do more once I buy a Gold subscription again (note to MS an EA: the 2-day pass you shipped with the game succeeded in ensuring that I'll reup for a full year in short order after letting my account lapse).
All said, I'm roughly 20 hours in, and I'm thoroughly enjoying Mass Effect 3. There are decisions that confuse me and a lot of additions that seem to exist purely to occupy your time, which is frustrating after 2 was so tight and focused. Still, I love this setting and these characters. I'm eager to see how the story of Commander Shepard wraps up in each of the four iterations I've created for it. I'll be sad to see it come to an end.
That's really the first mark in the game's favor. After so many decisions in ME1 seemed to be largely without consequence beyond the cosmetic in ME2, everything you've done before suddenly takes on added significance. More important, the blatant good/bad dichotomy has lessened remarkably, as the bulk of decisions seem to be much more grey. To this end, even the Paragon/Renegade divide has been diminished, with many decisions simply serving to enhance your "reputation" -- a catchall that contributes to both scores. I finally have a compelling reason to revisit Mass Effect 1 and alter my decision making at certain junctures just to see how certain quests change and how that will affect my ability to prepare the galaxy at large.
The increased value of your decisions wouldn't matter if they didn't continue to contribute to strong storytelling, though, and I'm happy to say that they do. The overarching plot is, of course, a bit generic (help prepare an army to challenge a seemingly unstoppable enemy), but the power is in the setting and the bond that each player gets to establish with it. There are so many characters at this point and so much history that we've been exposed to that it's hard not to feel an attachment when a familiar face pops up or when you get closure from a character or event that you expected to be left by the wayside. Of course, not every character can be included in a satisfying way. Some additions are brief and seem forced in as nothing more than an excuse to let some secondary or tertiary character saunter by as a reminder of their existence and then contribute unseen to galactic readiness (this holds especially true of any character that showed up exclusively in DLC). But, we can't expect miracles from the writers, and satisfying everybody was never an option.
Honestly, I only have two real complaints about the story so far. First of all, too many characters have changed or accomplished too much since the end in Mass Effect 2. The game tells us that it's been six months since Shepard turned him/herself in for trial on Earth. How in the hell have some of these people picked up the positions they have since then? Grunt, Jack, even Garrus: I'm looking at you. Second, the game really needs to give a better glimpse into Cerberus' motivations. It sounds like that's going to play a much more serious role going forward. For those of you farther into the game than I, a certain major plot event just took place on the Citadel. I get that it's not meant to make immediate sense from the perspective of the player/our characters, but there's no coherent attitude/belief system of which the player is aware that would allow the event to make sense. People make choices for reasons other than "wouldn't it be dramatic if...", and that's the only attitude that could possibly seem to be driving the Illusive Man at this point.
With the important part out of the way, it's time to talk briefly about gameplay.
In terms of controls, we're mostly looking at a fine tuning of the systems in place in Mass Effect 2. Gunplay sometimes feels a bit loose, but that happens with most 3rd person shooters. The improved mobility is nice, and I find my infiltrator using his cloak to sprint up and beat down fools as often as to pick out a careful headshot. The changes made to armor sorts of abilities -- each carrying the option to purge the added defense in order to gain some other temporary advantage -- are fun and make them seem more active. I also like that these armor abilities (along with the infiltrator's cloak) don't incur their cooldown until they are purged or their period ends. It makes it a lot easier to use traditional powers in conjunction with these prolonged effect abilities. Combat, in general, is as fun as it's been, and that makes multiplayer a much better option than anybody would have expected.
First, though, I have a couple of complaints about things at the design level. The least important but most glaring of these is the new resource collecting system. You run around star systems using your sensors in an attempt to ping for things that could be useful in the preparations you're supporting. This isn't so bad by itself, but each time you run a scan, you make Reapers more aware of your activity in the region. Do it too often (basically, 3 times), and they're off and chasing you and you need to get your ass out of the system! Except...you don't. The game autosaves each time you enter a system. There's no good reason not to just go around firing off your scanners like crazy, determine where everything is, Reapers be damned, let them catch you, and then have that autosave reload so you can fire off your scanners in the two or three places you need to to collect everything from a system. It's a silly waste of time that tries to add a sense of high stakes and consequence where there are none.
Slightly more annoying is a shift in combat similar to some things I complained about in Dragon Age 2: namely, inexplicable waves of enemies. Reinforcements will jet pack in or jump down from balconies or come through otherwise sealed doors at every opportunity, just to inflate challenge. It makes it nearly impossible to develop any high level strategy the way that you sometimes needed to at the upper difficulties of Mass Effect 1 and that were occasionally beneficial in Mass Effect 2. It keeps the action going, and things are fun enough that I don't mind too much at the default difficulty, but I'm going to be cursing up a storm during my inevitable Insanity attempt. In the prior two games harder difficulties just started to feel natural once you adapted to what they expected of you. I don't see that happening here because the source of difficulty is different.
Finally, the designers seem to have responded in the worst possible fashion to the complaints that Mass Effect 2 lacked side quests. There are now dozens and dozens of meaningless fetch quests you can go on (largely connected to the new resource system) and lots of time consuming, fluffy interactions available at the Citadel (and possibly other hubs if there are any). The fetch quests remind me a lot of Dragon Age 2 and the various weird things you would find while you were going about your adventuring only to suddenly know that somebody would give you money/experience for them. In this case, at least you overhear conversations about the things to know that you're looking for them, but it's similarly arbitrary. The more traditional quests, while at least providing some sort of context, but they mostly involve talking to a sequence of people/terminals and then getting some sort of payoff when a tertiary character from some previous sidequest in a previous game shows up at the end.
The good news is that, other than establishing mood, all of these serve simply to increase the resources you can bring to the fight with the Reapers, meaning they can be avoided without consequence if you just sink some time into the multiplayer!
I haven't done much with the multiplayer so far. Spent an hour or so going through a few rounds with Andy before I'd really gotten into the single player. I'm looking forward to going back. For the most part, it's class-based horde mode with unlockable persistent upgrades in the form of both level gaining and new equipment. Each player gets to control a stripped down version of one of the base classes, each with three hotkeyed abilities and two weapons (or at least the Sentinel only gets two weapons, maybe the Soldier gets more?). You're dropped in hostile territory and confronted with a series of randomized enemy waves and objectives and asked to survive as many as you can, up to 10. The longer you last, the more experience you get and the more you improve the ability of the galaxy to fight off the Reapers. Good stuff, and I'm eager to do more once I buy a Gold subscription again (note to MS an EA: the 2-day pass you shipped with the game succeeded in ensuring that I'll reup for a full year in short order after letting my account lapse).
All said, I'm roughly 20 hours in, and I'm thoroughly enjoying Mass Effect 3. There are decisions that confuse me and a lot of additions that seem to exist purely to occupy your time, which is frustrating after 2 was so tight and focused. Still, I love this setting and these characters. I'm eager to see how the story of Commander Shepard wraps up in each of the four iterations I've created for it. I'll be sad to see it come to an end.
Thursday, March 8, 2012
Computers, the Cortex, Prediction, and Intelligence
**Warning: This is a response to the book On Intelligence by Jeff Hawkins. It was originally written for a Cognitive Science course. Ahead lies 4000 words worth of historical computer science, bumbling neurobiology, and a bit of armchair philosophy. Read at your own peril**
Jeff Hawkins, author of On Intelligence has a bone to pick with both Artificial Intelligence researchers and neuroscientists: neither group, he claims, seems to be concerned with determining the nature of intelligence. For their part, computer scientists have long been resistant to the notion that the structure of the brain is important in promoting intelligent behavior. Neuroscientists, meanwhile, have not put forth as much energy as Hawkins would like into putting forth any sort of unified theory or framework of cognitive function.
The sins Hawkins accuses computer scientists of committing can seemingly be attributed to an over-reverence for Alan Turing. If it is true that a Turing Machine can compute anything that can be computed, and if any we believe that one of the primary functions of the brain is to compute, then it seems perfectly plausible to suggest that the brain is nothing more than a biological implementation of a Turing Machine. Since all Universal Turing Machines are effectively equivalent, then it further seems reasonable to insist that digital computers ought to be a perfectly suited to playing host to intelligence.
Hawkins also argues that Artificial Intelligence researchers were led astray from the very beginning by the Turing Test. Turing lived at a time when psychology was still dominated by behaviorism: the notion that intelligence could only be determined by action. The Turing Test endorses this thinking, and while it is not a benchmark that researchers seriously pursue, virtually all testing of machine intelligence that has followed in its wake is also focused on this input-output driven benchmarking.
The largely fruitless results of historical AI research suggest that maybe this is not the right way to do things. We have made programs and algorithms that can do all manner of seemingly complicated activities from playing chess to approximating optimized configurations for complex systems. The “simple” things that we would really like computers to do, though (vision, language acquisition, motor control) have made only minimal progress. If the brain is following formal, procedural algorithms in the fashion of a turing machine, we obviously have not found them. More likely, though, is that the brain does something different.
Hawkins suggests that the brain, while obviously capable of traditional computation, must also do something more elaborate, and we ought to try replicating that process if we want to create computers and/or programs that exhibit intelligence. Computer Scientists came up with this same idea decades ago and it led to the idea of Neural Networks. This was a good start, but it was not taken far enough. As soon as toy three-layer networks produced some interesting results, researchers returned to navel-gazing instead of taking further steps to mimic brain behavior. But who can blame them when we still understand so little about the brain?
This leads us to Hawkins’ frustrations with the neuroscience community. Chiefly, he thinks that we simply have not put in enough effort to determine how the brain, and specifically the cortex, works. We have gathered reams of experimental data about brain activity. We have rough maps of where all sorts of phenomena are processed in the brain -- from linguistic syntax to motor control to immediate optical stimulus. What we do not know is really anything concrete about how these signals interact to produce what we think of as intelligence.
This is not really surprising in itself. The brain is an incredibly complex and sensitive organ, and it is very hard to perform any sort of accurate experimentation on it. All of our current methodologies necessarily sacrifice either spatial or temporal resolution, and we really need both in order to say anything meaningful. Still, Hawkins would encourage boldness. We cannot spend all of our time simply collecting data without anything to use that data for, and our current theories are too low-level and low-risk to be of interest. We need a theory that can explain the phenomenon of intelligence as a whole, both in order to understand ourselves and in order to imbue this property on future generations of machines.
To this end, Hawkins suggests an overarching theory to describe the nature and purpose of brain function as it relates to intelligence. Such a theory, while likely flawed, would give direction to our research and change the way we think about our work, both in neuroscience and A.I.. Ultimately, On Intelligence is Hawkins’ attempt to lay out such a theory: the Memory Prediction Framework.
In short, the Memory Prediction Framework suggests this: The function of the brain, as leads to intelligence, is not simple computation. Computation would suggest nothing more than stimulus-response pairings. Instead, Hawkins claims that the human brain strives to use previously identified patterns in order to predict and change the future.
For Hawkins, the seat of what we think of as intelligence -- agency, intentionality, adaptability, even creativity and consciousness -- is the cortex. The cortex is not exclusive to humans, but the most notable difference in brain structure between us and other mammals is the sheer size of ours, thanks to the evolutionarily recent expansion known as the neocortex. Since humans are so far beyond other animals in our ability to understand and control our environments and pass on our knowledge to subsequent generations, this enlarged cortex ought to play the primary role.
Hawkins proposes that the chief function of the cortex is to contain a model of the world as we have experienced. At its most abstract, ignoring all of the biology involved, this is accomplished by forming what Hawkins calls invariant memories. We recognize patterns that occur together -- the shape of a hand, the sensation of heat on our skin, the sound of a musical interval -- and group the sensations together into a single mental concept, recognizable even when specifics -- the starting pitch, the orientation of the hand, the location of the burn -- change.
From these patterns, we construct composites of patterns. From letters we pick up words and phrases and stories. Eyes and nose and lips become a face. Intervals expand to phrases combine to make songs. Simple patterns, universally, become building blocks for more complicated concepts, both in the abstract or the specific. As we experience a pattern more, it becomes more accessible and more foundational to creating new patterns. In a world where we often encounter combinations of stimuli that seem completely unrelated, these models help us come to reasonable conclusions about our surroundings. If you hear an animal roar but look around and see that you are still in your kitchen, you don’t assume that a bear got into the house, but that somebody has the television on too loud.
The power of these models is apparent: if we know what follows from what we are experiencing right now, we know how to respond. We see a bottle teetering on a table and we can steady it before it falls. We see an old wounds threatening to reopen during an argument and we preemptively make peace. We build tools to initiate a chain of events that lead to a desired goal, whether that is an improved harvest or a man setting foot on the moon. By predicting how the environment will respond to our actions or what state naturally follows from the current one, we exhibit control over reality in a way that simple organisms simply cannot. We nudge the trajectory of the world to be more favorable for us.
What is even more remarkable is that we can create invariant “memories” about things we have never witnessed. We have developed tools, notably language, that allow us to pass on our experiences to others, providing them the ability to respond appropriately to situations before they have ever encountered them. Further, for lack of a better term, we have the ability to imagine. We can create mental worlds rooted in the model of reality that we have built but distinctly different. We can tweak parameters, ask ourselves how things would change if certain patterns coincided in a new way. We can test the outcome of a course of action without taking on the risk ourselves. This gives us our unique capacity for invention in both the practical and artistic senses.
All of this is well and good, and we can provide ample anecdotal evidence to convince ourselves of it based purely on reason. The question is, can it be supported by biology? Unfortunately, there is still so much about the brain, and especially the cortex, that is a mystery to us that little can be said conclusively. Hawkins does, however, offer up a description of what is known about the cortex and how this could endorse the Memory Prediction Framework. This is where my expertise wanes, but I will relay Hawkins’ lesson to the best of my ability:
Physically, the cortex is a thin coating of brain matter, consisting of “grey matter” (neurons) and “white matter” (axons connecting the neurons), surrounding the evolutionarily old brain. The cortex is divided physically into six layers of neurons. Each neuron is connected at many points to the neighbors in its own vertical column as well as to neurons in its neighboring columns and a plethora of other neurons distributed throughout the whole of the rest of the cortex. It is well known that, in the visual regions of the cortex, these connections form further topological hierarchies, and Hawkins thinks it natural that this should also be the case for other regions, as we will see. Signals begin in the lower levels of these hierarchies and then collect and travel upwards as more elaborate relationships between signals are processed.
These upwards connections have been studied widely, but the first important thing that Hawkins focuses on is the fact that there are actually more feedback connections traveling down the hierarchies than there are transmitting data forward. Most theories of the brain seem to discount the importance of these connections, but we will see shortly that they take on prominence in the Memory Prediction Framework.
There is one other important feature of cortical design that we have to discuss first. I already mentioned that the cortex is consistent in its physical makeup across its entirety. In the late 1970’s, this led researcher Vernon Mountcastle to put forth a theory that has largely been dismissed ever since: that there is no significant functional differentiation in the cortex. That is, all regions of the cortex, whether they process sight or language or movement follow some universal algorithm. This, in part, is why Hawkins assumes logical hierarchies within all regions of the cortex and not just the regions that process vision.
As evidence for this claim, Hawkins offers up two arguments. First, the extreme plasticity of the cortex. We know, for example, that violinists have larger areas of their cortex dedicated to controlling the movement of the fingers on their left hand. Individuals born deficient in one sense seem to have larger regions devoted to their others. In one particularly shocking experiment, a man turned blind was able to regain “sight” be having a camera send electrical impulses to his tongue. This somatosensory information was processed in the visual regions of the cortex.
Second, there is no reason to believe that the brain has any reason to process different senses differently. The brain itself is without any sense, after all, and all senses can ultimately be described in the same way. Whether it is a collection of light rays collecting on the retina or longitudinal wave colliding with the cochlea, all of our perception boils down to spatio-temporal patterns. If the brain is just processing spatio-temporal patterns at every turn (as the Memory Prediction Framework already suggests it does), then it should not matter what the input mechanism for these patterns is, just that the pattern is delivered for interpretation.
From here out, I will adopt this theory that all inputs to the cortex are equivalent. I may use language that seems pertinent to the way we perceive a particular sense, notably sight, but please understand that I am referring to any generic sense.
Most of what researchers have observed in the brain has been how these signals are propagated upward through the (logical) hierarchy. A particular image or impression is received from the sensory organs and is transmitted to the cortex. A subset of neurons immediately connected to that sense fires in accordance with the received image and, in firing, pass a signal upwards. The next layer receives this input pattern exactly as the lower layer did and again fires off a subset of its member neurons. This continues until the signal reaches layers high enough for us to make conscious sense of what is being perceived. These groupings of firing neurons, notably the particular groupings that result in recognition, embody the patterns that constitute our invariant memories.
What is interesting is that, while the lowest regions are rapidly modulating due to constantly changing input (because of the saccadic motion of the eye or the continuous flow of sound through the air or whatever else), higher regions -- regions where things in the world are recognized -- stay active for much longer. Obviously, then, these higher regions respond to increasingly general patterns. Hawkins also suggests that there are specific temporal patterns, that is, sequences of spatial patterns, to which these regions respond, and that they will remain active for as long as they receive the same expected repetition of spatial patterns.
Tying into the importance of these temporal patterns, Hawkins now comes back to the significance of the feedback connections, the ones connecting logically higher layers in the hierarchy to those closer to the actual perceived input. In short, Hawkins claim is that, once a pattern has been recognized at a higher level, it tells the levels below which input it is expecting to receive next. In effect, we prime ourselves to see what follows from what we are currently seeing, and we begin responding to it before it can even occur. This is the “prediction” element of the Memory Prediction Framework.
When the actual input defies the expected input, we are jarred out of the current pattern and the image once again propagates normally until we can replace our previous pattern one with one that more accurately represents what we are truly seeing and resume along a more appropriate course of action.
All of this feedback and neural priming predisposes us to see patterns with which we are already familiar. This can help us resolve ambiguities in the environment and perform on-the-fly error correction, but it can also cause us to gloss over seemingly unimportant distinctions. We see this any time we automatically correct a minor spelling or grammatical mistake or even when we try our hands at a “spot the differences” exercise. We are very good at seeing what we want to see because that makes our lives easier and our processing swifter.
So, we now understand how memories are activated and how they are used to predict the immediate future. But how are they formed? Hawkins endorses a simple mechanism known as Hebbian Learning. Simply put, when neurons fire at the same time, the synapses between them are strengthened. This means that each member of a neuron pattern firing increases the odds of the other neurons in the pattern firing as well. These patterns identify an element of a “memory”, and the patterns themselves are stored in the synapses. The more we see a pattern, the more likely we are to see it in the future, even if not all of the elements of that pattern are available in the immediately presented image.
Hawkins has a lot more to say about how the brain operates: the significance of the columnar alignment of neurons as a processing unit; the nature and method of inhibition between neurons and columns; role of the hippocampus as the topmost hierarchical element of the cortex; the importance of the thalamus as a gateway between cortical regions. This is where nuance that is beyond me comes into play, though, and I cannot hope to do all of it justice. I take Hawkins at his word in these arguments, in part because, he offers a number of testable hypotheses that need to be confirmed in order for the Memory Prediction Framework to stand up.
For instance, Hawkins suggests that we will eventually be able to identify the downwards cascades of “predictive” activity through neural hierarchies that should coincide with sudden understanding. Novel events, on the other hand, should be seen propagating upward toward the hippocampus. He identifies specific sorts of cells that should exist in particular cortical layers and which should show excitement in anticipation of an input or respond differentially depending on whether its input is expected or unexpected.
Either because of a lack of interest or continuing limitations in our monitoring technology, it would seem that none of Hawkins’ hypotheses have been either confirmed or refuted in the years since he wrote On Intelligence. If Hawkins really wants to promote his theories or provide some evidence of their correctness without waiting for technology to catch up, the best route might be through implementing them successfully in technology. To this end, Hawkins has already started a company called Numenta, with the goal of producing machine learning packages that model a variation on what he calls Hierarchical Temporal Memory -- a learning architecture that mimics his theories about the design and behavior of the cortex.
Hawkins talks about future generations of machines that utilize Hierarchical Temporal Memory to process patterns and make predictions about any phenomena imaginable. Computers are already better than we are at processing large amounts of data. The electrical signals that travel through a microprocessor are orders of magnitude faster than the electrochemical impulses that drive neural activity. There is no telling what we could discover with intelligent machines churning away and making “informed” predictions based on all of the information we could feed into them.
This is especially apparent when we consider the sorts of input such machines could process. We have already committed to the notion that senses are arbitrary and interchangeable, so why should machines be limited to making decisions based on our senses? Imagine computers that operate and make extrapolations based on input from novel senses, like sonar or barometric pressure, as easily as we do sight. Such computers would drastically increase our ability to, say, predict weather patterns or plan unmanned spaceflight. And that is just the beginning. Imagine how much more effective the approach could be if we actually had computer architectures that integrate memory and processing the way the brain seems to.
But one important question remains: would such a machine be intelligent? This is where we need to start holding Hawkins accountable for some of his early rhetoric. For all of his grand talk early going about the superiority of the brain, Hawkins still appears to believe that intelligence can be replicated by digital machines. This would not necessarily be human intelligence, complete with all of the intangible qualities that make life fascinating, but at least the pattern-driven, future predicting intelligence that he believes is the key to our success as a species. So, whether he commits to it directly or not, Hawkins ultimately believes after all that the brain is nothing more than an augmented Turing Machine; a Turing Machine optimized for the sorts of feedback driven, memory intensive algorithms he has described, but Turing Machine nonetheless.
Moreover, for all of his complaints about behaviorism, Hawkins is ultimately insisting that behavior is the ultimate benchmark of intelligence. He simply shifts the scale from the macro (measurable action) to the micro (method of signal processing). If prediction is the defining measure of intelligence, does the implementation matter? It seems unreasonable to suggest that it is, in which case Hawkins is not really offering us anything new. Prediction, of a sort, has been the goal of A.I. all along. Big Blue was able to “predict” the correct move at each juncture to beat Kasparov. In the abstract realm, the Chinese Room is able to “predict” an appropriate response to a written inquiry in an unknown language. We already have all kinds of learning systems, ranging from traditional Neural Networks to Bayesian modeling, that seek to make predictions based on previous observations without making any effort at faithfully modeling the cortex. Why are these implementations less capable of intelligence than Hawkins’?
Truly, as regards prediction, Hawkins’ solution is different only in approach. This approach may have advantages, it does not give us intelligence by itself. Since prediction alone would appear to be insufficient (unless we suddenly want to change our minds and ascribe intelligence to existing implementations (actual or theoretical) of A.I.), we need to look for another primary feature of intelligence and ask whether that is fundamental to the Memory Prediction Framework and Hierarchical Temporal Memory.
Off the cuff, I would argue this defining feature of intelligence is willfulness -- the ability to direct (if not select) our thoughts, to forge mental connections of our own volition rather than programmatically. Truth be told, I am not certain that the Hierarchical Temporal Memory can provide that or that the Memory Prediction Framework can explain it. Certainly, Hawkins does. He suggests that consciousness is simply “the feeling of having a [sufficient] cortex”. But, while his description of the brain and the neural connections within the cortex certainly provides mechanisms and conduits through which thought could be consciously directed, the driving force is nowhere to be found.
So, if Hawkins’ framework cannot successfully explain intelligence in individuals, we cannot expect it to imbue intelligence on machines. Hierarchical Temporal Memory does not create any stronger argument for understanding than passing the Turing Test. That does not mean his model is useless. The feedback and priming systems that Hierarchical Temporal Memory contains seem well suited to monitoring and making predictions about real time systems, whether it be vision or other crucial signal monitoring. Hawkins’ work may not be any less artificial than other routes that we have taken, but it is still a potentially beneficial supplement.
For the neuroscientist, Hawkins ideas would seem to posses more significant value. The importance of feedback, pattern recognition, and pre-conscious prediction in cognition make a world of sense at a cursory glance, even if the mechanisms that guide it may not uniform throughout the cortex. Even if we limit the Memory Prediction Framework to being a description of cortical function and not intelligence as a whole, though, Hawkins still runs into hot water. For one thing, he discounts the importance of the old brain in intelligence far more than can be acceptable. More than that, though, there has to be a reason that Mountcastle’s theories of universal cortical function have not gained more traction over the past 30-some years.
I may be willing to accept that uniformity could well be the norm for processing sensory input. Patterns in where we process the senses are easily explained by nerve connections from those senses, and all of the signals ultimately are translated into the same sorts neural firings. How does this explain issues like Brocha’s area, though? Why would virtually all humans process linguistic syntax, something with no direct connection to a single sense or the outside world at all, in the same region of the cortex? There does not seem to be an easy “path of least resistance” explination available here.
Since this, ultimately, is Hawkin’s goal, I have to end by giving him credit. His goal was never to provide a definitive answer to these problems, but a starting point. On Intelligence does provide and intriguing, if likely flawed, account of what intelligence could be. I suspect it will not stand the test of time, but it may well provide a useful stepping stone. By taking the risk of proposing not only a theory but standards by which it can be tested, Hawkins has left future researchers with the ammunition to tear apart his framework and iteratively replace it with one that lies closer to the truth. That risk can only lead us closer to the truth.
Bibliography
Hawkins, Jeff. On Intelligence (New York: St. Martin’s Press, 2004).
Hebb, D.O. The Organization of Behavior (New York: Wiley and Sons, 1949).
Mountcastle, Vernon B. “An Organizing Principle for Cerebral Function: The Unit Model and the Distributed System” in The Mindful Brain (Cambridge, Mass: MIT Press, 1978).
Legacy Content. Numenta. http://www.numenta.com/legacy.php. Accessed March 6, 2012.
Jeff Hawkins, author of On Intelligence has a bone to pick with both Artificial Intelligence researchers and neuroscientists: neither group, he claims, seems to be concerned with determining the nature of intelligence. For their part, computer scientists have long been resistant to the notion that the structure of the brain is important in promoting intelligent behavior. Neuroscientists, meanwhile, have not put forth as much energy as Hawkins would like into putting forth any sort of unified theory or framework of cognitive function.
The sins Hawkins accuses computer scientists of committing can seemingly be attributed to an over-reverence for Alan Turing. If it is true that a Turing Machine can compute anything that can be computed, and if any we believe that one of the primary functions of the brain is to compute, then it seems perfectly plausible to suggest that the brain is nothing more than a biological implementation of a Turing Machine. Since all Universal Turing Machines are effectively equivalent, then it further seems reasonable to insist that digital computers ought to be a perfectly suited to playing host to intelligence.
Hawkins also argues that Artificial Intelligence researchers were led astray from the very beginning by the Turing Test. Turing lived at a time when psychology was still dominated by behaviorism: the notion that intelligence could only be determined by action. The Turing Test endorses this thinking, and while it is not a benchmark that researchers seriously pursue, virtually all testing of machine intelligence that has followed in its wake is also focused on this input-output driven benchmarking.
The largely fruitless results of historical AI research suggest that maybe this is not the right way to do things. We have made programs and algorithms that can do all manner of seemingly complicated activities from playing chess to approximating optimized configurations for complex systems. The “simple” things that we would really like computers to do, though (vision, language acquisition, motor control) have made only minimal progress. If the brain is following formal, procedural algorithms in the fashion of a turing machine, we obviously have not found them. More likely, though, is that the brain does something different.
Hawkins suggests that the brain, while obviously capable of traditional computation, must also do something more elaborate, and we ought to try replicating that process if we want to create computers and/or programs that exhibit intelligence. Computer Scientists came up with this same idea decades ago and it led to the idea of Neural Networks. This was a good start, but it was not taken far enough. As soon as toy three-layer networks produced some interesting results, researchers returned to navel-gazing instead of taking further steps to mimic brain behavior. But who can blame them when we still understand so little about the brain?
This leads us to Hawkins’ frustrations with the neuroscience community. Chiefly, he thinks that we simply have not put in enough effort to determine how the brain, and specifically the cortex, works. We have gathered reams of experimental data about brain activity. We have rough maps of where all sorts of phenomena are processed in the brain -- from linguistic syntax to motor control to immediate optical stimulus. What we do not know is really anything concrete about how these signals interact to produce what we think of as intelligence.
This is not really surprising in itself. The brain is an incredibly complex and sensitive organ, and it is very hard to perform any sort of accurate experimentation on it. All of our current methodologies necessarily sacrifice either spatial or temporal resolution, and we really need both in order to say anything meaningful. Still, Hawkins would encourage boldness. We cannot spend all of our time simply collecting data without anything to use that data for, and our current theories are too low-level and low-risk to be of interest. We need a theory that can explain the phenomenon of intelligence as a whole, both in order to understand ourselves and in order to imbue this property on future generations of machines.
To this end, Hawkins suggests an overarching theory to describe the nature and purpose of brain function as it relates to intelligence. Such a theory, while likely flawed, would give direction to our research and change the way we think about our work, both in neuroscience and A.I.. Ultimately, On Intelligence is Hawkins’ attempt to lay out such a theory: the Memory Prediction Framework.
In short, the Memory Prediction Framework suggests this: The function of the brain, as leads to intelligence, is not simple computation. Computation would suggest nothing more than stimulus-response pairings. Instead, Hawkins claims that the human brain strives to use previously identified patterns in order to predict and change the future.
For Hawkins, the seat of what we think of as intelligence -- agency, intentionality, adaptability, even creativity and consciousness -- is the cortex. The cortex is not exclusive to humans, but the most notable difference in brain structure between us and other mammals is the sheer size of ours, thanks to the evolutionarily recent expansion known as the neocortex. Since humans are so far beyond other animals in our ability to understand and control our environments and pass on our knowledge to subsequent generations, this enlarged cortex ought to play the primary role.
Hawkins proposes that the chief function of the cortex is to contain a model of the world as we have experienced. At its most abstract, ignoring all of the biology involved, this is accomplished by forming what Hawkins calls invariant memories. We recognize patterns that occur together -- the shape of a hand, the sensation of heat on our skin, the sound of a musical interval -- and group the sensations together into a single mental concept, recognizable even when specifics -- the starting pitch, the orientation of the hand, the location of the burn -- change.
From these patterns, we construct composites of patterns. From letters we pick up words and phrases and stories. Eyes and nose and lips become a face. Intervals expand to phrases combine to make songs. Simple patterns, universally, become building blocks for more complicated concepts, both in the abstract or the specific. As we experience a pattern more, it becomes more accessible and more foundational to creating new patterns. In a world where we often encounter combinations of stimuli that seem completely unrelated, these models help us come to reasonable conclusions about our surroundings. If you hear an animal roar but look around and see that you are still in your kitchen, you don’t assume that a bear got into the house, but that somebody has the television on too loud.
The power of these models is apparent: if we know what follows from what we are experiencing right now, we know how to respond. We see a bottle teetering on a table and we can steady it before it falls. We see an old wounds threatening to reopen during an argument and we preemptively make peace. We build tools to initiate a chain of events that lead to a desired goal, whether that is an improved harvest or a man setting foot on the moon. By predicting how the environment will respond to our actions or what state naturally follows from the current one, we exhibit control over reality in a way that simple organisms simply cannot. We nudge the trajectory of the world to be more favorable for us.
What is even more remarkable is that we can create invariant “memories” about things we have never witnessed. We have developed tools, notably language, that allow us to pass on our experiences to others, providing them the ability to respond appropriately to situations before they have ever encountered them. Further, for lack of a better term, we have the ability to imagine. We can create mental worlds rooted in the model of reality that we have built but distinctly different. We can tweak parameters, ask ourselves how things would change if certain patterns coincided in a new way. We can test the outcome of a course of action without taking on the risk ourselves. This gives us our unique capacity for invention in both the practical and artistic senses.
All of this is well and good, and we can provide ample anecdotal evidence to convince ourselves of it based purely on reason. The question is, can it be supported by biology? Unfortunately, there is still so much about the brain, and especially the cortex, that is a mystery to us that little can be said conclusively. Hawkins does, however, offer up a description of what is known about the cortex and how this could endorse the Memory Prediction Framework. This is where my expertise wanes, but I will relay Hawkins’ lesson to the best of my ability:
Physically, the cortex is a thin coating of brain matter, consisting of “grey matter” (neurons) and “white matter” (axons connecting the neurons), surrounding the evolutionarily old brain. The cortex is divided physically into six layers of neurons. Each neuron is connected at many points to the neighbors in its own vertical column as well as to neurons in its neighboring columns and a plethora of other neurons distributed throughout the whole of the rest of the cortex. It is well known that, in the visual regions of the cortex, these connections form further topological hierarchies, and Hawkins thinks it natural that this should also be the case for other regions, as we will see. Signals begin in the lower levels of these hierarchies and then collect and travel upwards as more elaborate relationships between signals are processed.
These upwards connections have been studied widely, but the first important thing that Hawkins focuses on is the fact that there are actually more feedback connections traveling down the hierarchies than there are transmitting data forward. Most theories of the brain seem to discount the importance of these connections, but we will see shortly that they take on prominence in the Memory Prediction Framework.
There is one other important feature of cortical design that we have to discuss first. I already mentioned that the cortex is consistent in its physical makeup across its entirety. In the late 1970’s, this led researcher Vernon Mountcastle to put forth a theory that has largely been dismissed ever since: that there is no significant functional differentiation in the cortex. That is, all regions of the cortex, whether they process sight or language or movement follow some universal algorithm. This, in part, is why Hawkins assumes logical hierarchies within all regions of the cortex and not just the regions that process vision.
As evidence for this claim, Hawkins offers up two arguments. First, the extreme plasticity of the cortex. We know, for example, that violinists have larger areas of their cortex dedicated to controlling the movement of the fingers on their left hand. Individuals born deficient in one sense seem to have larger regions devoted to their others. In one particularly shocking experiment, a man turned blind was able to regain “sight” be having a camera send electrical impulses to his tongue. This somatosensory information was processed in the visual regions of the cortex.
Second, there is no reason to believe that the brain has any reason to process different senses differently. The brain itself is without any sense, after all, and all senses can ultimately be described in the same way. Whether it is a collection of light rays collecting on the retina or longitudinal wave colliding with the cochlea, all of our perception boils down to spatio-temporal patterns. If the brain is just processing spatio-temporal patterns at every turn (as the Memory Prediction Framework already suggests it does), then it should not matter what the input mechanism for these patterns is, just that the pattern is delivered for interpretation.
From here out, I will adopt this theory that all inputs to the cortex are equivalent. I may use language that seems pertinent to the way we perceive a particular sense, notably sight, but please understand that I am referring to any generic sense.
Most of what researchers have observed in the brain has been how these signals are propagated upward through the (logical) hierarchy. A particular image or impression is received from the sensory organs and is transmitted to the cortex. A subset of neurons immediately connected to that sense fires in accordance with the received image and, in firing, pass a signal upwards. The next layer receives this input pattern exactly as the lower layer did and again fires off a subset of its member neurons. This continues until the signal reaches layers high enough for us to make conscious sense of what is being perceived. These groupings of firing neurons, notably the particular groupings that result in recognition, embody the patterns that constitute our invariant memories.
What is interesting is that, while the lowest regions are rapidly modulating due to constantly changing input (because of the saccadic motion of the eye or the continuous flow of sound through the air or whatever else), higher regions -- regions where things in the world are recognized -- stay active for much longer. Obviously, then, these higher regions respond to increasingly general patterns. Hawkins also suggests that there are specific temporal patterns, that is, sequences of spatial patterns, to which these regions respond, and that they will remain active for as long as they receive the same expected repetition of spatial patterns.
Tying into the importance of these temporal patterns, Hawkins now comes back to the significance of the feedback connections, the ones connecting logically higher layers in the hierarchy to those closer to the actual perceived input. In short, Hawkins claim is that, once a pattern has been recognized at a higher level, it tells the levels below which input it is expecting to receive next. In effect, we prime ourselves to see what follows from what we are currently seeing, and we begin responding to it before it can even occur. This is the “prediction” element of the Memory Prediction Framework.
When the actual input defies the expected input, we are jarred out of the current pattern and the image once again propagates normally until we can replace our previous pattern one with one that more accurately represents what we are truly seeing and resume along a more appropriate course of action.
All of this feedback and neural priming predisposes us to see patterns with which we are already familiar. This can help us resolve ambiguities in the environment and perform on-the-fly error correction, but it can also cause us to gloss over seemingly unimportant distinctions. We see this any time we automatically correct a minor spelling or grammatical mistake or even when we try our hands at a “spot the differences” exercise. We are very good at seeing what we want to see because that makes our lives easier and our processing swifter.
So, we now understand how memories are activated and how they are used to predict the immediate future. But how are they formed? Hawkins endorses a simple mechanism known as Hebbian Learning. Simply put, when neurons fire at the same time, the synapses between them are strengthened. This means that each member of a neuron pattern firing increases the odds of the other neurons in the pattern firing as well. These patterns identify an element of a “memory”, and the patterns themselves are stored in the synapses. The more we see a pattern, the more likely we are to see it in the future, even if not all of the elements of that pattern are available in the immediately presented image.
Hawkins has a lot more to say about how the brain operates: the significance of the columnar alignment of neurons as a processing unit; the nature and method of inhibition between neurons and columns; role of the hippocampus as the topmost hierarchical element of the cortex; the importance of the thalamus as a gateway between cortical regions. This is where nuance that is beyond me comes into play, though, and I cannot hope to do all of it justice. I take Hawkins at his word in these arguments, in part because, he offers a number of testable hypotheses that need to be confirmed in order for the Memory Prediction Framework to stand up.
For instance, Hawkins suggests that we will eventually be able to identify the downwards cascades of “predictive” activity through neural hierarchies that should coincide with sudden understanding. Novel events, on the other hand, should be seen propagating upward toward the hippocampus. He identifies specific sorts of cells that should exist in particular cortical layers and which should show excitement in anticipation of an input or respond differentially depending on whether its input is expected or unexpected.
Either because of a lack of interest or continuing limitations in our monitoring technology, it would seem that none of Hawkins’ hypotheses have been either confirmed or refuted in the years since he wrote On Intelligence. If Hawkins really wants to promote his theories or provide some evidence of their correctness without waiting for technology to catch up, the best route might be through implementing them successfully in technology. To this end, Hawkins has already started a company called Numenta, with the goal of producing machine learning packages that model a variation on what he calls Hierarchical Temporal Memory -- a learning architecture that mimics his theories about the design and behavior of the cortex.
Hawkins talks about future generations of machines that utilize Hierarchical Temporal Memory to process patterns and make predictions about any phenomena imaginable. Computers are already better than we are at processing large amounts of data. The electrical signals that travel through a microprocessor are orders of magnitude faster than the electrochemical impulses that drive neural activity. There is no telling what we could discover with intelligent machines churning away and making “informed” predictions based on all of the information we could feed into them.
This is especially apparent when we consider the sorts of input such machines could process. We have already committed to the notion that senses are arbitrary and interchangeable, so why should machines be limited to making decisions based on our senses? Imagine computers that operate and make extrapolations based on input from novel senses, like sonar or barometric pressure, as easily as we do sight. Such computers would drastically increase our ability to, say, predict weather patterns or plan unmanned spaceflight. And that is just the beginning. Imagine how much more effective the approach could be if we actually had computer architectures that integrate memory and processing the way the brain seems to.
But one important question remains: would such a machine be intelligent? This is where we need to start holding Hawkins accountable for some of his early rhetoric. For all of his grand talk early going about the superiority of the brain, Hawkins still appears to believe that intelligence can be replicated by digital machines. This would not necessarily be human intelligence, complete with all of the intangible qualities that make life fascinating, but at least the pattern-driven, future predicting intelligence that he believes is the key to our success as a species. So, whether he commits to it directly or not, Hawkins ultimately believes after all that the brain is nothing more than an augmented Turing Machine; a Turing Machine optimized for the sorts of feedback driven, memory intensive algorithms he has described, but Turing Machine nonetheless.
Moreover, for all of his complaints about behaviorism, Hawkins is ultimately insisting that behavior is the ultimate benchmark of intelligence. He simply shifts the scale from the macro (measurable action) to the micro (method of signal processing). If prediction is the defining measure of intelligence, does the implementation matter? It seems unreasonable to suggest that it is, in which case Hawkins is not really offering us anything new. Prediction, of a sort, has been the goal of A.I. all along. Big Blue was able to “predict” the correct move at each juncture to beat Kasparov. In the abstract realm, the Chinese Room is able to “predict” an appropriate response to a written inquiry in an unknown language. We already have all kinds of learning systems, ranging from traditional Neural Networks to Bayesian modeling, that seek to make predictions based on previous observations without making any effort at faithfully modeling the cortex. Why are these implementations less capable of intelligence than Hawkins’?
Truly, as regards prediction, Hawkins’ solution is different only in approach. This approach may have advantages, it does not give us intelligence by itself. Since prediction alone would appear to be insufficient (unless we suddenly want to change our minds and ascribe intelligence to existing implementations (actual or theoretical) of A.I.), we need to look for another primary feature of intelligence and ask whether that is fundamental to the Memory Prediction Framework and Hierarchical Temporal Memory.
Off the cuff, I would argue this defining feature of intelligence is willfulness -- the ability to direct (if not select) our thoughts, to forge mental connections of our own volition rather than programmatically. Truth be told, I am not certain that the Hierarchical Temporal Memory can provide that or that the Memory Prediction Framework can explain it. Certainly, Hawkins does. He suggests that consciousness is simply “the feeling of having a [sufficient] cortex”. But, while his description of the brain and the neural connections within the cortex certainly provides mechanisms and conduits through which thought could be consciously directed, the driving force is nowhere to be found.
So, if Hawkins’ framework cannot successfully explain intelligence in individuals, we cannot expect it to imbue intelligence on machines. Hierarchical Temporal Memory does not create any stronger argument for understanding than passing the Turing Test. That does not mean his model is useless. The feedback and priming systems that Hierarchical Temporal Memory contains seem well suited to monitoring and making predictions about real time systems, whether it be vision or other crucial signal monitoring. Hawkins’ work may not be any less artificial than other routes that we have taken, but it is still a potentially beneficial supplement.
For the neuroscientist, Hawkins ideas would seem to posses more significant value. The importance of feedback, pattern recognition, and pre-conscious prediction in cognition make a world of sense at a cursory glance, even if the mechanisms that guide it may not uniform throughout the cortex. Even if we limit the Memory Prediction Framework to being a description of cortical function and not intelligence as a whole, though, Hawkins still runs into hot water. For one thing, he discounts the importance of the old brain in intelligence far more than can be acceptable. More than that, though, there has to be a reason that Mountcastle’s theories of universal cortical function have not gained more traction over the past 30-some years.
I may be willing to accept that uniformity could well be the norm for processing sensory input. Patterns in where we process the senses are easily explained by nerve connections from those senses, and all of the signals ultimately are translated into the same sorts neural firings. How does this explain issues like Brocha’s area, though? Why would virtually all humans process linguistic syntax, something with no direct connection to a single sense or the outside world at all, in the same region of the cortex? There does not seem to be an easy “path of least resistance” explination available here.
Since this, ultimately, is Hawkin’s goal, I have to end by giving him credit. His goal was never to provide a definitive answer to these problems, but a starting point. On Intelligence does provide and intriguing, if likely flawed, account of what intelligence could be. I suspect it will not stand the test of time, but it may well provide a useful stepping stone. By taking the risk of proposing not only a theory but standards by which it can be tested, Hawkins has left future researchers with the ammunition to tear apart his framework and iteratively replace it with one that lies closer to the truth. That risk can only lead us closer to the truth.
Bibliography
Hawkins, Jeff. On Intelligence (New York: St. Martin’s Press, 2004).
Hebb, D.O. The Organization of Behavior (New York: Wiley and Sons, 1949).
Mountcastle, Vernon B. “An Organizing Principle for Cerebral Function: The Unit Model and the Distributed System” in The Mindful Brain (Cambridge, Mass: MIT Press, 1978).
Legacy Content. Numenta. http://www.numenta.com/legacy.php. Accessed March 6, 2012.
Thursday, February 16, 2012
Classroom Participation
Sitting in class right now and wishing more people would follow these simple guidelines.
Good Reasons to Speak Up in Class:
You're confused or curious about something that has been said.
A question has been asked (by a teacher or classmate) AND you feel you can supply an adequate answer/clarification.
You have thorough experience with the topic from a perspective other than what's being taught AND that perspective might legitimately enhance the understanding or appreciation of the class.
You have an especially fantastic joke to make.
Bad Reasons to Speak Up in Class:
You thought of something mildly amusing
You think, based on incomplete knowledge of the material, that your understanding of what is being presented is superior to that of your teacher and/or whatever experts (s)he is citing.
You simply don't like the way the teacher phrased something (from a nitpicky semantic point of view)
You've gone longer than you like without hearing yourself talk.
Good Reasons to Speak Up in Class:
You're confused or curious about something that has been said.
A question has been asked (by a teacher or classmate) AND you feel you can supply an adequate answer/clarification.
You have thorough experience with the topic from a perspective other than what's being taught AND that perspective might legitimately enhance the understanding or appreciation of the class.
You have an especially fantastic joke to make.
Bad Reasons to Speak Up in Class:
You thought of something mildly amusing
You think, based on incomplete knowledge of the material, that your understanding of what is being presented is superior to that of your teacher and/or whatever experts (s)he is citing.
You simply don't like the way the teacher phrased something (from a nitpicky semantic point of view)
You've gone longer than you like without hearing yourself talk.
Wednesday, February 15, 2012
Machines, Intelligence, and Problem of Motivation
I had the luxury recently of re-reading the paper "Computing Machinery and Intelligence", in which Alan Turing first poses what has come to be known as the Turing Test. In short, he suggests that, if a machine can communicate sufficiently well that a person could not identify it as a machine, then that machine may as well be considered intelligent.
Of course, you can't read this paper without also reading "Minds, Brain, and Programs", where John Searle establishes that Chinese Room thought experiment. In effect, Searle describes how syntactic manipulation without any awareness of semantic content (effectively all any computer obeying a program is capable of) eliminates the potential for understanding. Thus, he claims that no digital computer could ever be considered intelligent.
These are points that have been debated to death. It want to take a second to focus on a point made by each thinker that I haven't seen discussed as widely.
First, Turing describes how a machine such as could pass his test might "evolve" its own intelligence. He suggests that we could create a program that effectively mimics the learning mechanisms of the brain (neural nets of a sort) and then train it through carrot and stick to respond appropriately. This, he thinks, sidesteps the question of a computer simply running a normal man-made algorithm.
Searle, for his part, claims that the separability of hardware and software prevents this from really being the case. Mental states and mental representations, he says, are at least partly the consequence of normal brain function. They are not something that the brain simply executes.
I really think that Searle is right in this instance, and I think it comes down to biology and necessity. Suppose that we could perfectly recreate the brain's ability to learn: to acquire language and to exhibit intentional, causal attitudes in interaction with the world around us. For what reason would a computer utilize these mechanisms?
Intelligent beings use our brains and our ability to create mental representations and constructs out of necessity for survival. It is the natural order of all organisms to grow, decay, consume, and excrete (and, ideally, multiply at some point). In order to do so, we need to interact with the world around us. When you're an infant, your choice (conscious or not) is clear: cry or die. As we age, we develop more elaborate and more efficient ways of getting what we need from the environment, utilizing our cognitive abilities to do so.
A machine has exactly what needs? Electricity? It certainly doesn't have a biological impetus for reproduction. A truly intelligent machine, then, would have no reason to communicate or engage in any sort of causal behavior. It would simply want to stay plugged in. Any motivation that a computer would have (the punishment and reward suggested by Turing) would have no intrinsic value to its survival. It would be wholly arbitrary and entirely external.
Does this mean that a machine *couldn't* be intelligent? No. But it does have consequences for what intelligence means. If we ever describe machines as intelligent, it means that we would need to accept that mental representations and language need to serve some purpose beyond the evolutionary. You cannot claim that we can create "life" in this way without admitting the possibility of our own created-ness.
Obviously, as a Christian, I have no problem with this (even if I disagree with the idea of machine intelligence), but it's a dilemma that I highly doubt many computer scientists ever consider.
Of course, you can't read this paper without also reading "Minds, Brain, and Programs", where John Searle establishes that Chinese Room thought experiment. In effect, Searle describes how syntactic manipulation without any awareness of semantic content (effectively all any computer obeying a program is capable of) eliminates the potential for understanding. Thus, he claims that no digital computer could ever be considered intelligent.
These are points that have been debated to death. It want to take a second to focus on a point made by each thinker that I haven't seen discussed as widely.
First, Turing describes how a machine such as could pass his test might "evolve" its own intelligence. He suggests that we could create a program that effectively mimics the learning mechanisms of the brain (neural nets of a sort) and then train it through carrot and stick to respond appropriately. This, he thinks, sidesteps the question of a computer simply running a normal man-made algorithm.
Searle, for his part, claims that the separability of hardware and software prevents this from really being the case. Mental states and mental representations, he says, are at least partly the consequence of normal brain function. They are not something that the brain simply executes.
I really think that Searle is right in this instance, and I think it comes down to biology and necessity. Suppose that we could perfectly recreate the brain's ability to learn: to acquire language and to exhibit intentional, causal attitudes in interaction with the world around us. For what reason would a computer utilize these mechanisms?
Intelligent beings use our brains and our ability to create mental representations and constructs out of necessity for survival. It is the natural order of all organisms to grow, decay, consume, and excrete (and, ideally, multiply at some point). In order to do so, we need to interact with the world around us. When you're an infant, your choice (conscious or not) is clear: cry or die. As we age, we develop more elaborate and more efficient ways of getting what we need from the environment, utilizing our cognitive abilities to do so.
A machine has exactly what needs? Electricity? It certainly doesn't have a biological impetus for reproduction. A truly intelligent machine, then, would have no reason to communicate or engage in any sort of causal behavior. It would simply want to stay plugged in. Any motivation that a computer would have (the punishment and reward suggested by Turing) would have no intrinsic value to its survival. It would be wholly arbitrary and entirely external.
Does this mean that a machine *couldn't* be intelligent? No. But it does have consequences for what intelligence means. If we ever describe machines as intelligent, it means that we would need to accept that mental representations and language need to serve some purpose beyond the evolutionary. You cannot claim that we can create "life" in this way without admitting the possibility of our own created-ness.
Obviously, as a Christian, I have no problem with this (even if I disagree with the idea of machine intelligence), but it's a dilemma that I highly doubt many computer scientists ever consider.
Subscribe to:
Posts (Atom)