Welcome, Guest: Join Nairaland / LOGIN! / Trending / Recent / New
Stats: 2,641,572 members, 6,183,635 topics. Date: Sunday, 07 March 2021 at 10:51 AM

A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. - Programming - Nairaland

Nairaland Forum / Science/Technology / Programming / A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. (2978 Views)

Are You Interested In Robotics And Artificial Intelligence? / A Thread For Tutorial On Python Programming / Artificial Intelligence And Machine Learning Group (2) (3) (4)

(1) (2) (3) (4) (Reply) (Go Down)

A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 2:41pm On Oct 24, 2020
TLDR: This writeup is for enthusiasts of AI, automated gaming, and application’s of machine (deep) learning in general.

3 Likes

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 2:41pm On Oct 24, 2020
Self-driving vehicles, Industrial scale automations, Operations in unsafe zones like oil rigs, Increased successes in surgical operations … these are part of the many promises of artificial intelligence.

Do you know that the highest grandmaster of the hardest board game in the world GO is an AI? 9 March 2016 – 15 March 2016 was the week AlphaGo broke the internet by beating Lee Sedol an 18-time world champion by 4 – 1. It was a feat considered impossible as the number of possible moves in the game of GO is more than the number of atoms in the universe. And that was the week I was seduced from the medical career I was pursuing and everything became history.

2 Likes 1 Share

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 2:42pm On Oct 24, 2020
During this past few weeks I have been able to explore another part of the AI promise, automated gaming.
Games are part entertainment and part logical and strategical so training an AI agent through reinforcement learning could help us develop other important agents like the case of an agent trained by google that was able to cut their electricity bill of a server building by more than 70%.

2 Likes

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 2:42pm On Oct 24, 2020
In this post, I will be showing the result of my experiment with an AI agent trained with Pytorch that plays the game snake. Here is the final result.

9 Likes

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:06pm On Oct 24, 2020
The training took about 26 hours on an hyper threaded Intel Xeon CPU (good reward aware policy was learned around 5 hours). Although I tried training with a free tesla K80 GPU from google but I was getting almost the same result from my benchmark because most of the work the agent is doing is on the CPU.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:17pm On Oct 24, 2020
And here is the agent in it’s infancy stage about 0 – 2 hours of training. It can be seen performing random actions.

3 Likes

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:31pm On Oct 24, 2020
The inner workings

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:31pm On Oct 24, 2020
The agent is trained using the principles of Q Learning, Bellman Equation and the Markov’s decision process. The above is used to exploit an environment with a number of states by an agent performing an action on the environment. In this experiment, the environment is the snake game.

The environment gives our agent a reward for each action performed. The reward is positive when the agent eats the apple, and negative when the agent crashes the snake on the wall or when the agent crashes the snake on itself. The purpose of this reward is to learn a good policy i.e. eating apples is good while crashing is bad.

2 Likes

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:31pm On Oct 24, 2020
For the training loop, the environment provides the agent with a state containing vital information the agent needs to navigate the board. The state includes the direction of the location of the apple (i.e. north, west, south, east), the current direction the snake is travelling, and details of the surrounding position of the snake’s head whether they are occupied or not.

The reward function for the training is the same as the one described above, but I added an extra negative -0.05 reward for each action the agent takes that doesn’t result to the snake eating the apple. This will tell the agent to eat the apple as soon as possible and not to roam around the board.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:32pm On Oct 24, 2020
The loss function is the hubber loss. A hybrid of the Mean square error loss and Mean absolute error loss function. Another viable loss function is the trusted Mean square error loss itself.

The agent itself is nothing but three layers of feed forward neural networks with 64 hidden layers, nonlinear activation functions, and an output of three choices LEFT, RIGHT or keep moving STRAIGHT.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:33pm On Oct 24, 2020
Conclusion

The experiment was a bit of success although the agent didn’t maximize its potential mainly because it can only sense the board only through the region around it’s head and a better option is the snake seeing the whole board as an image through a convolutional neural network. The agent was able to eat about 50 apples at max.

For the next experiment, I will be training an agent to play the old shooting game CONTRA and I am really excited about the training. I will also build upon the lesson learnt by this experiment by giving the agent more awareness of the environment by providing the agent the direct image render of the screen. IF the next experiment is successful, other prospective games for experimenting include Asphalt, Mortal Kombat etc. and also maybe soccer (but definitely the old versions like winning eleven 2008 and will probably need the help of a C++ dev to tune the game to a usable environment)
Adios guys.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Predstan: 5:30am On Oct 25, 2020
Interesting. I am also an AI enthusiast and my interest is on the Microcontroller programming (TinyML). Interestingly I don’t like Pytorch, I use Tensorflow for deep leaning and neural network. I’m currently working a traffic light(Computer Vision) model to be deployed to PyBoard and Arduino.
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 10:56am On Oct 25, 2020
Predstan:
Interesting. I am also an AI enthusiast and my interest is on the Microcontroller programming (TinyML). Interestingly I don’t like Pytorch, I use Tensorflow for deep leaning and neural network.

I started with Tensorflow actually, but it doesn't feel pythonic to me so I switched to pytorch but with the Keras api now merged with the new Tensorflow, I think I might start to use it again.

I’m currently working a traffic light(Computer Vision) model to be deployed to PyBoard and Arduino.


Sounds interesting, could you explain more about it. And why PyBoard and not raspberry pi.
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Marvel1206: 4:09pm On Oct 25, 2020
So how did you start learning this?
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Sirmwill: 4:40pm On Oct 25, 2020
Everything here is good beautiful... smiley

2 Likes

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 5:33pm On Oct 25, 2020
Marvel1206:
So how did you start learning this?

Learning Python was the first step. Microsoft did an online lecture about python around that time and I followed on Youtube. Once I was comfortable with Python I got the book Hands on machine learning with Scikit learn and Tensorflow which gave me good introduction to machine learning with each chapter about a different kind of ML algorithm/Model (I read many other books but none is as influential as this book). Subsequently i took a few courses on Udemy about Reinforcement learning that introduce me to the field.

3 Likes 1 Share

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Abcruz(m): 3:05pm On Oct 26, 2020
Interesting work you have there I'm also an AI enthusiast. With Tensorflow and keras I have been able to build deep learning models on breast cancer prediction, diabetes prediction and heart disease prediction among other works.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 4:33pm On Oct 26, 2020
Abcruz:
Interesting work you have there I'm also an AI enthusiast. With Tensorflow and keras I have been able to build deep learning models on breast cancer prediction, diabetes prediction and heart disease prediction among other works.

Sounds cool. These are very important areas that can benefit humans greatly.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Abcruz(m): 4:41pm On Oct 26, 2020
Enceladus:


Sounds cool. These are very important areas that can benefit humans greatly.

Thanks for the compliment.
AI is such an interesting field of practical knowledge because it can be applied to so many areas of our lives in automating our repetitive task with a measurable level of accuracy.
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Predstan: 9:53pm On Oct 26, 2020
Enceladus:


I started with Tensorflow actually, but it doesn't feel pythonic to me so I switched to pytorch but with the Keras api now merged with the new Tensorflow, I think I might start to use it again.

I’m currently working a traffic light(Computer Vision) model to be deployed to PyBoard and Arduino.


Sounds interesting, could you explain more about it. And why PyBoard and not raspberry pi.

It’s a neural Net model for object and signage detection. The agent basically will be able to detect the traffic signs and make decisions from there. Like self-driving car. The signs are just the usual American traffic signs; the stop signs, yield signs, traffic lights. The agent will control the speed of a motor to either stop, slow down or run based on its decision through the model from the camera and sensor.
The choice of Arduino and PyBoard is just because that’s what I have and I have been using for some electronic projects.

2 Likes

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by BelieverDE: 7:38am On Oct 27, 2020
Amazing!

I am a Data Science enthusiast and I really like everything you wrote. Before commencing any online course I usually check Quora and Reddit for reviews. I understand that there is no particular online course that will teach you everything about DS.

Could you please enumerate the courses you took while climbing the ladder of Data Science?

I'd really appreciate. Thanks in anticipation.
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:25pm On Oct 27, 2020
Predstan:


It’s a neural Net model for object and signage detection. The agent basically will be able to detect the traffic signs and make decisions from there. Like self-driving car. The signs are just the usual American traffic signs; the stop signs, yield signs, traffic lights. The agent will control the speed of a motor to either stop, slow down or run based on its decision through the model from the camera and sensor.
The choice of Arduino and PyBoard is just because that’s what I have and I have been using for some electronic projects.

I get now. Sounds very interesting. I think reinforcement learning could also be applied here.
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 3:32pm On Oct 27, 2020
BelieverDE:
Amazing!

I am a Data Science enthusiast and I really like everything you wrote. Before commencing any online course I usually check Quora and Reddit for reviews. I understand that there is no particular online course that will teach you everything about DS.

Could you please enumerate the courses you took while climbing the ladder of Data Science?

I'd really appreciate. Thanks in anticipation.

Sure. Data science is too broad for a course.

Courses from SuperDataScience on reinforcement learning (2) and one on machine learning helped a lot. The book hands on machine learning too was influential.

Medium.com and towardsdatascience.com blogpost also keeps you updated with new trends.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Abcruz(m): 12:21am On Oct 28, 2020
Enceladus:


Sure. Data science is too broad for a course.

Courses from SuperDataScience on reinforcement learning (2) and one on machine learning helped a lot. The book hands on machine learning too was influential.

Medium.com and towardsdatascience.com blogpost also keeps you updated with new trends.

How did you create those animated photos?
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 6:56am On Oct 28, 2020
Abcruz:


How did you create those animated photos?

I used obs studio to record the screen, premier pro to edit, then I used an online service to convert the final video to .gif animated image.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by BelieverDE: 9:14am On Oct 28, 2020
Enceladus:


Sure. Data science is too broad for a course.

Courses from SuperDataScience on reinforcement learning (2) and one on machine learning helped a lot. The book hands on machine learning too was influential.

Medium.com and towardsdatascience.com blogpost also keeps you updated with new trends.

The head of SuperDataScience is Kirill Eremenko. @Ejiod suggested his Machine Learning course to me on udemy. I suppose I'll have to follow his learning path, if any.

I never liked the idea of using books to learn Machine Learning, but now that you've recommended this book, I'll definitely download it!

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Kinglyte56(m): 12:32pm On Oct 28, 2020
Of late I had interest in machine learning and artificial intelligence ...
I am just learning the basics now .....
If possible I wanna code and build a personal ai assistant like that of the Jarvis and I greet everyone in this thread

2 Likes

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Abcruz(m): 12:33pm On Oct 28, 2020
Enceladus:


I used obs studio to record the screen, premier pro to edit, then I used an online service to convert the final video to .gif animated image.

Nice! thanks for sharing.

1 Like

Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 12:43pm On Oct 28, 2020
BelieverDE:


The head of SuperDataScience is Kirill Eremenko. @Ejiod suggested his Machine Learning course to me on udemy. I suppose I'll have to follow his learning path, if any.

I never liked the idea of using books to learn Machine Learning, but now that you've recommended this book, I'll definitely download it!

Kirill and his team are quite good. Same here too but that book is quite different.
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Enceladus(m): 12:53pm On Oct 28, 2020
Kinglyte56:
Of late I had interest in machine learning and artificial intelligence ...
I am just learning the basics now .....
If possible I wanna code and build a personal ai assistant like that of the Jarvis and I greet everyone in this thread

Quite possible. Mark Z built a Jarvis himself. His source code should be somewhere on github, that will give you some idea about ow to start. GL
Re: A Thread On My Experiment With Reinforcement Learning & Artificial Intelligence. by Drybones: 4:47pm On Oct 30, 2020
It would better if we can apply this reinforcement knowledge to games like Fifa and Pes.

Nice work op!!!

(1) (2) (3) (4) (Reply)

Anyone Interested In Having A Peer-to-peer Script Or Custom Made Website ? / wizGrade School Manager is Open Source - 100% free + source code / Has Anyone Tried Out Windows 10? Pls Share Your Experience

(Go Up)

Sections: politics (1) business autos (1) jobs (1) career education (1) romance computers phones travel sports fashion health
religion celebs tv-movies music-radio literature webmasters programming techmarket

Links: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

Nairaland - Copyright © 2005 - 2021 Oluwaseun Osewa. All rights reserved. See How To Advertise. 117
Disclaimer: Every Nairaland member is solely responsible for anything that he/she posts or uploads on Nairaland.