The key, I think, is to avoid making the machines self replicating. Otherwise, any protections against them becoming selfish will eventually be for naught, as copying errors become subject to the principles of evolution.

I'd reverse the order of "ensure survival of individual sentients" and "ensure freedom of individual sentients". Leaving them in this order would result in a nanny society, where freedom was ruthlessly crushed, in favor of perfect safety (sort of like having Democrats in power, but worse).

"as natural intelligence appears to be in diminishing supply."

Glenn had just been dealing with a specific example of idiocy - Andrew Sullivan. Of course a single example doesn't prove a trend.

TJIC,

Freedom might be crushed in a political context where these were the goals, but only because survival/safety is adopted as the only actual value and freedom just becomes lip service.

I think it's telling that the Declaration of Independence talks about Life, Liberty, and the Pursuit of Happiness. In that order. Sheesh, what a New Dealer / Nanny-Stater that Thomas Jefferson was, eh? :-)

If we promote any one of those values to the exclusion of the others, we have a problem. But if we strive to define all three in such a way as to work together, maybe we'll start closing in on something.

A robot must obey orders given to it by human beings except where such orders would conflict with the First Law.

Many have pointed out that this law essentially enslaves the robots.

I find that statement to be ironic because as it turned out in the Asimov novels it was the humans that were being controlled by the robot R. Daneel Olivaw.

What would work better would be transferring over the moral complexity that you used to make up these goals in the first place.

Also, as you point out, these goals are vague. More specific and useful from a programmer's perspective would be some kind of algorithm that takes human preferences as inputs and outputs actions that practically everyone sees as reasonable and benevolent. Hard to do, obviously, but CEV (http://www.singinst.org/upload/CEV.html) is one attempt.

The "Higher intelligence, dumber world" comes from two things, I think. The first, as one columnist pointed out, comes from the increasing sophistication of the world. You need to know more to get by in the world than ever before, and each deficiency is a glaring flaw that makes you look "dumb."

The second "problem" is the ever expanding scope of conversation. We have more memes, more participating individuals than ever before. And any forumite can tell you that the intelligence of his preferred forum is inversley related to the number of people posting there, ie, the more people post, the more you're likely to run into a jackass. Apply this to the news and internet writ large.

Of course, for every moron, you can meet a genius, and as a result, our world is advancing rapidly and things like Yahoo Answers or Wikipedia are helping more people than ever "figure things out," but people tend to notice the stupidity more than the brilliance, so everything "looks dumb."

phil...the concept u are reaching for is called recursive self-improvement.
mathematically it is semi-rigorous, and there has been quite a body of wvrk done on it. ;)
heres a link.
http://quantumghosts.blogspot.com/2006/09/friendly-ai-possible-friendly.html

Phil,

I'm afraid I have to side with TJIC with regard to the priority structure embodied in the Goals as presented.

The sort of guidance we are envisioning must, at the most fundamental level, be logically exclusive and compelling or the AI in question could 'reason' its way around the strictures imposed (much like Asimov's "Zeroth Law" robots) and, eventually, the value of the Goals in limiting AI evolution to the high-power, high-controllability track is lost.

Each Goal must have a clear and unbreakable priority over the others that follow it and thus, in the order stated, collective continuity trumps individual safety ("The needs of the many outweigh the needs of the few, or the one."), individual safety (broadly construed, 'stasis') trumps individual liberty ('free will'), and happiness ('utility', a notoriously slippery concept for economists and philosophers to get a firm intellectual grip on) trumps both individual liberty and individual well-being (allowing potentially self-destructive behavior on the individual level insofar as that behavior doesn't exceed the standard established for 'safety' in Goal 2).

Whole series of books could be written whose plots hinge on the resolution of internal conflicts among the Goals and creative resolutions thereof, but, as currently constituted, they lead, logically, resolutely, and inescapably (if they have the 'teeth' necessary to prevent a sufficiently creative or evolutionary AI from eventually transforming into the powerful, uncontroallable, and ultimately selfish and dominating, type) to the ultimate expression of the 'precautionary principle' and an eventual 'lotus eater' static state for all sentients.

dumb, michael
phil's rulebased system is woefully obsolete already.
read my linkage.
it is like......how rulebased systems an table lookups were subsumed by genetic algorithms and relational databases in software.
welcome to the 21st century, dudes.

phil read the literature please before commenting on Friendly AI.
your rulebased system is already obsolete.

Well, class, I just don't know what I'm going to do with you. I'm afraid I'm going to have to keep you all after school so we can spend some time trying to understand the difference between rules and goals...not the same thing, I'm afraid, and I was about as adamant as I could possibly be as to which one I was writing about. Matoko, you have to stay later for your disrespectful attitude towards other students. But, hey, it could be worse -- at least I'm not counting off for spelling and punctuation!

BTW, not everyone agrees that recursive self-improvement alone will get us there. In the piece by Eliezer Yudkowsky that Michael Anissimov linked above, Eliezer argues that recursive self-improvement on its own might prove very dangerous.

Michael A. -

What would work better would be transferring over the moral complexity that you used to make up these goals in the first place.

Actually, that's kind of where I'm going with this. The more I look at these goals, the more I think they're really goals for us. If we play our cards right (and something like CEV might get us there) these might eventually be goals we share with these new intelligences. It makes me wonder...have we (humanity or individuals) had some version of CEV that we've been running all along?

Asimov's laws focus on individual encounters. Your goals are more like a constitution.

Since sentience is a matter of degree, at what point do the desires/needs of robots become of equal value to those of humans? I.e., at what point does society emancipate them? And after that point, does it become criminal to produce a less-than-maximally sentient robot in the same way that it would be to induce brain defects in a fetus?

how rulebased systems an table lookups were subsumed by genetic algorithms and relational databases in software.

matoko, relational databases are tabled databases by definition, hence, one does table lookups when performing a variety of relational database operations (or at least it appears that way to the user). I gather object oriented or functional databases (the latter is the current version of rules-based systems) are state of the art these days, but I haven't been keeping up.

Second, genetic algorithms (GAs) solve a sort of problem mostly orthogonal to anything else you mention. Ie, optimization problems. At its most related, I imagine one could use GAs to optimize queries and other high level operations on the database. Search optimization is a serious problem and poor searching methods can slow down database queries by a considerable amount.