[MURG] The main problem...

[Eugen Leitl]

On Fri, Feb 06, 2004 at 08:24:28PM -0500, Yan King Yin wrote:

> Intention, or sentience (having human-like emotions), are not

Sentience has nothing to do with human-like emotions. Emotions per se, probably.
Human emotions are just introspection view of action scheduling. Planning needs
priority to decide what to do, hence a self and a world model. "If I go
there, I will fall down the stairs. Hence, I will not go there, but go over
here instead." 

Don't get too hung up about emotions. Insects have them, so do robots. People
are complex; judging from evolutonary structure conservation their emotions are less so. 

> necessarily required for an AI to function as a useful tool.
> An AI can simply have rigid goal structures.

It's doesn't matter what it is. Implementation is irrelevant. 
It matters what it does.
 
> The term "human-grade" AI is ill-defined and as such is not

It is defined well enough. It means human-grade performance across the board.
It doesn't have to pass the Turing test (for that's a speciality skill, and
can be addressed at a later stage of development, if required), but
it is definitely no idiot savant. It can do science and engineering
sufficiently well to improve itself in an open ended fashion, and a
motivation to do so by virtue of being embedded in a darwinian context.

This is something I *postulate*. I'm describing a threat model.

> very meaningful. A human-level AI can simply be a tool without
> sentience, and as such we're not afraid of them. As for

You can produce all kinds of synthetic scenarios which are safe.
I'm not interested in synthetic scenarios which are safe. Because we can
ignore these a priori. The dangerous scenarios are self-selection from a
threat analysis perspective.

> sentient AIs, I have already said this: "If AI can do X, AI
> can do X without sentience." It's redundant.

You have some pretty curious notions here.
 
> Facts:
> 1. Utility-type AIs will arrive before uploading.

Utility-type AIs are irrelevant. We already have them. Every game package has
behaviour AI. Any Roomba has AI, or any security robot. Only military robots
shoot with the intention to kill.

> 2. Sentient AIs will arrive about the same time as uploading.

People can't do sentient AIs. People can make seeds of such. It is
imperative to find out how hard it is, how much resources this will take,
and whether we should shoot those who try dead. 
 
> The discussion re whether AI will take over is very stupid.
> AI will not replace us unless we want that to happen. The

My point is that there are several individuals which consciously want it. But
it's not all that relevant -- collectively, we all want it. You can't afford
not to use a powerful tool if your competitor uses it. There are billions of
people on this planet, and a considerable fraction of those people is very
competent, and also very stupid.

> only way for it to happen is that a majoring of humans
> choose to opt for having sentient AI offsprings (themselves
> retiring / dying off); this will result in uploads being
> outnumbered and outcompeted. Otherwise this will not happen.
> 
> Case closed.

If it only was so easy. No wonder Eliezer is so frustrated.

> YKY

-- Eugen* Leitl <a href="http://leitl.org">leitl</a>
______________________________________________________________
ICBM: 48.07078, 11.61144            http://www.leitl.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
http://moleculardevices.org         http://nanomachines.net
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://askja.bu.edu/pipermail/murg/attachments/20040207/fb049176/attachment.pgp