[Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Here you can find side projects of the staff and great projects which we think should get extra support. (Note that featured projects are not projects by staff members of MaxCoderz)

Moderator: MaxCoderz Staff

Post Reply
User avatar
qarnos
Maxcoderz Staff
Posts: 227
Joined: Thu 01 Dec, 2005 9:04 am
Location: Melbourne, Australia

[Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Post by qarnos »

This is just a little something I have been piecing together over the last few months.

It's an attempt to evolve neural-network based artificial intelligence. I'm not really sure what I am trying to do with this program, it's mainly just a toy to play with, but it makes for some nice video:

http://www.youtube.com/watch?v=1iamM0SuPto

I hope to expand it a bit to give a bit more user control - something more like the NERO project. But in the meantime, enjoy the video! (yes - I know the text fades too quick... I'm too lazy to fix it).
"I don't know why a refrigerator is now involved, but put that aside for now". - Jim e on unitedti.org

avatar courtesy of driesguldolf.
User avatar
benryves
Maxcoderz Staff
Posts: 3087
Joined: Thu 16 Dec, 2004 10:06 pm
Location: Croydon, England
Contact:

Re: [Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Post by benryves »

Very nice video. :) Good work!
User avatar
kalan_vod
Calc King
Posts: 2932
Joined: Sat 18 Dec, 2004 6:46 am
Contact:

Re: [Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Post by kalan_vod »

Back in High School we worked with the fish program, written in java..Is this something you have cooked up in class? Looks really cool, hope you make something out of this bro!
King Harold
Calc King
Posts: 1513
Joined: Sat 05 Aug, 2006 7:22 am

Re: [Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Post by King Harold »

They look.. alive
How long did you run the sim before they were like this?

I still remember my AI assignment with more than a little horror.. We were supposed to use a neural network to play Othello, after several thousands of generations they still hadn't learned enough to beat me more than 50% (that was the goal) of the time (and I'm not even that good at it).

But these bots rule! How much longer till they start hunting in packs? :)
User avatar
qarnos
Maxcoderz Staff
Posts: 227
Joined: Thu 01 Dec, 2005 9:04 am
Location: Melbourne, Australia

Re: [Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Post by qarnos »

@kalan - I haven't been in class for 10 years, so that is a "no". It's just a little project I'm working on in my spare time.
King Harold wrote:They look.. alive
How long did you run the sim before they were like this?
I think this run was only an hour or so. I haven't done any super-long runs since the frame rate drops pretty quickly as the networks get more complicated and I'm just too impatient! I can't really measure it in generations, since reproduction is random and occurs when someone else dies. The longer you survive, the higher your chances of reproduction.
I still remember my AI assignment with more than a little horror.. We were supposed to use a neural network to play Othello, after several thousands of generations they still hadn't learned enough to beat me more than 50% (that was the goal) of the time (and I'm not even that good at it).
Yeah, it can take a long time - especially if there are lots of things they need to learn. It works best if you give them a single simple goal and reward them highly for it.
But these bots rule! How much longer till they start hunting in packs? :)
Mmm... pack hunting. I guess we can dream!

This network also evolves the structure of the network (fully recurrent with sigmoid and LSTM units) as well as the weights, so I don't know if that's a plus or a minus. It definitely increases the potential, but at the expense of a vastly increased search space which = slow evolving.

I have posted another video. There is an interesting point at about 2:40 where one of the bots sees some food right in front of a poisonous obstacle. Rather than go straight for the food and hit the obstacle, it steers just to the right then turns and swoops. It was probably an accident, but still - it looks cool!
"I don't know why a refrigerator is now involved, but put that aside for now". - Jim e on unitedti.org

avatar courtesy of driesguldolf.
User avatar
tr1p1ea
Maxcoderz Staff
Posts: 4141
Joined: Thu 16 Dec, 2004 10:06 pm
Location: I cant seem to get out of this cryogenic chamber!
Contact:

Re: [Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Post by tr1p1ea »

Oh it was no accident, soon they will take over the world!

LOVE the music btw!
"My world is Black & White. But if I blink fast enough, I see it in Grayscale."
Image
Image
King Harold
Calc King
Posts: 1513
Joined: Sat 05 Aug, 2006 7:22 am

Re: [Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Post by King Harold »

Yes yes, these bots will evolve self-awareness, if they don't have it already (they're planning ahead already!)
User avatar
qarnos
Maxcoderz Staff
Posts: 227
Joined: Thu 01 Dec, 2005 9:04 am
Location: Melbourne, Australia

Re: [Staff][Dev] Neural Bots - Evolving Artificial Intelligence

Post by qarnos »

I decided to try and implement a neural network training algorithm (namely, Back-Propagation Through Time) and use it to train the Bots via a human.

Basically, you get to control one of the bots with the mouse, and your mouse movements are translated into the corresponding network outputs. These outputs are then compared to the output the network is generating, and BPTT is used to propagate and error signal through the network which is then used to alter the connection weights. The weight adjustments are only done in small increments which allows the network to converge on a general solution - in this case, trying to mimic your actions.

I have encountered some difficulties when using sparse recurrent networks - after a while the something goes crazy (I'm not sure what, just yet) and the output neurons become fixed at either 0.0 or 1.0 - but the successful runs are encouraging.

There's a clip up on YouTube showing one of the better runs and also a short clip of training at the end. You'll notice that in training mode I have restricted the view to essentially the same as what the Bots "see". I have found I get better results this way, since I am not using any information in my decision making which the Bots aren't privy to.

I think it could make an interesting game to have a team of bots which you can train to perform a particular task, and then put them into action against teams trained by other players. I just have no idea what that task should be. Any ideas?
"I don't know why a refrigerator is now involved, but put that aside for now". - Jim e on unitedti.org

avatar courtesy of driesguldolf.
Post Reply