[MURG] Neuropeptide Function?

[Eugen Leitl]

On Fri, Feb 06, 2004 at 05:18:01AM -0500, Randal A. Koene wrote:

> Certainly more than a year ago, there were long threads here (and on
> another mailing list) about the possible danger to MU if AI achieves
> sentience first. So, I have some sense of the possible danger perspective.

I'm not sure we should rewarm this thread, but then, the list has been
largely quiescent. 

> (See archives for more... although the search will be tedious. :)
> But, this is an interesting new question: Is AI likely to be sentient? And

I don't know what sentient means. It's sufficient that it can enhance itself,
and give rise to a fast-evolving, radiating machine-phase life. Some of them
will be intelligent, some of them won't.

> if so, how does that come about?

You can't design an adult AI by hand, but it appears doable to use evolutionary
algorithms to evolve an evolvable system. There are obviously several kinetic
bottlenecks which need very large amounts of computational work (because I
don't believe in being lucky) to be broken-through. Of course, we can start
with an educated guess: a finite state network with a high dimensionality. 
You can map this very well to 3d integrated cellular architectures.

> Eugene, I don't have the same AL (artificial life) experience as you do,

I'm really very ignorant about evolutionary computation. 

> all I have is some knowledge about the diversity and stages of evolution
> on Earth. If we equate the kind of AI that could be dangerous to MU with
> human-level intelligence and sentience (self-awareness), then the evidence
> around us points to two things: (1) Most life-forms with a brain don't
> meet those criteria, so perhaps it is NOT obvious that AI would become

I've postulated a machine which is roughly humanly intelligent, and can
improve itself. Clearly this is something designed, not emerged from a
mutated packet in a router somewhere.

> dangerous rapidly through an evolutionary a feedback loop. Sentience may
> not be the obvious development. (2) Yes, it seems there is only space for
> ONE species that meets the criteria. Where are those Neanderthals again

Let's say we have a large human city there, and put it on fast forward, say,
about 10^6 speedup. What will happen in an hour, day, week, year? A day is
worth about 3 kYrs at that speed.

Now people can't just improve themselves (though they'll learn how to, given
time -- kiloyears and megayears are cheap on fast-forward), because they're 
not designed to be that. A machine which is just a bit vector doesn't have 
that problem. You can have generations flashing by in seconds. Not
necessarily on commodity hardware of the near future, though I wouldn't bet
my life on it.

> these days? Some cause for worry if human-level AI does appear before MU.
> Now, I think it is quite possible that sentient AI comes about as a result
> of intentional human endeavours. Not everyone shares our prudence, and

Of course intentional, the design space is much too small to be hit by
accident. 

> someone is bound to purposely create sentient AI. And from there... the
> feedback loop is almost guaranteed (look at the feedback loops humans are
> working at... AI is one part of that). Yet, there remains the question how
> rapid that feedback loop can be as long as the AI still depends on human
> intervention for its resource demands (e.g. truck drivers delivering
> components, just to name one simple down-to-earth factor).

You underestimate the amount of automation already online. I'm also not at
all woried about today, but a world some thirty years downstream.

Even now an intelligent system can purchase resources and human cooperation
online. Current systems are really bad at impersonating people, but I don't
think a planetful of hardware which has just transcended would take longer
than a few minutes or hours to break out -- all 30 or more years hence, of
course.

-- Eugen* Leitl <a href="http://leitl.org">leitl</a>
______________________________________________________________
ICBM: 48.07078, 11.61144            http://www.leitl.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
http://moleculardevices.org         http://nanomachines.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://askja.bu.edu/pipermail/murg/attachments/20040206/2876b367/attachment.pgp