MaxCoderz

This is just a little something I have been piecing together over the last few months.

It's an attempt to evolve neural-network based artificial intelligence. I'm not really sure what I am trying to do with this program, it's mainly just a toy to play with, but it makes for some nice video:

http://www.youtube.com/watch?v=1iamM0SuPto

I hope to expand it a bit to give a bit more user control - something more like the NERO project. But in the meantime, enjoy the video! (yes - I know the text fades too quick... I'm too lazy to fix it).

_________________
"I don't know why a refrigerator is now involved, but put that aside for now". - Jim e on unitedti.org

avatar courtesy of driesguldolf.

Very nice video. Good work!

Back in High School we worked with the fish program, written in java..Is this something you have cooked up in class? Looks really cool, hope you make something out of this bro!

They look.. alive
How long did you run the sim before they were like this?

I still remember my AI assignment with more than a little horror.. We were supposed to use a neural network to play Othello, after several thousands of generations they still hadn't learned enough to beat me more than 50% (that was the goal) of the time (and I'm not even that good at it).

But these bots rule! How much longer till they start hunting in packs?

@kalan - I haven't been in class for 10 years, so that is a "no". It's just a little project I'm working on in my spare time.



I think this run was only an hour or so. I haven't done any super-long runs since the frame rate drops pretty quickly as the networks get more complicated and I'm just too impatient! I can't really measure it in generations, since reproduction is random and occurs when someone else dies. The longer you survive, the higher your chances of reproduction.



Yeah, it can take a long time - especially if there are lots of things they need to learn. It works best if you give them a single simple goal and reward them highly for it.



Mmm... pack hunting. I guess we can dream!

This network also evolves the structure of the network (fully recurrent with sigmoid and LSTM units) as well as the weights, so I don't know if that's a plus or a minus. It definitely increases the potential, but at the expense of a vastly increased search space which = slow evolving.

I have posted another video. There is an interesting point at about 2:40 where one of the bots sees some food right in front of a poisonous obstacle. Rather than go straight for the food and hit the obstacle, it steers just to the right then turns and swoops. It was probably an accident, but still - it looks cool!

_________________
"I don't know why a refrigerator is now involved, but put that aside for now". - Jim e on unitedti.org

avatar courtesy of driesguldolf.

Oh it was no accident, soon they will take over the world!

LOVE the music btw!

_________________
"My world is Black & White. But if I blink fast enough, I see it in Grayscale."

Yes yes, these bots will evolve self-awareness, if they don't have it already (they're planning ahead already!)

I decided to try and implement a neural network training algorithm (namely, Back-Propagation Through Time) and use it to train the Bots via a human.

Basically, you get to control one of the bots with the mouse, and your mouse movements are translated into the corresponding network outputs. These outputs are then compared to the output the network is generating, and BPTT is used to propagate and error signal through the network which is then used to alter the connection weights. The weight adjustments are only done in small increments which allows the network to converge on a general solution - in this case, trying to mimic your actions.

I have encountered some difficulties when using sparse recurrent networks - after a while the something goes crazy (I'm not sure what, just yet) and the output neurons become fixed at either 0.0 or 1.0 - but the successful runs are encouraging.

There's a clip up on YouTube showing one of the better runs and also a short clip of training at the end. You'll notice that in training mode I have restricted the view to essentially the same as what the Bots "see". I have found I get better results this way, since I am not using any information in my decision making which the Bots aren't privy to.

I think it could make an interesting game to have a team of bots which you can train to perform a particular task, and then put them into action against teams trained by other players. I just have no idea what that task should be. Any ideas?

_________________
"I don't know why a refrigerator is now involved, but put that aside for now". - Jim e on unitedti.org

avatar courtesy of driesguldolf.