Welcome, Guest: Register On Nairaland / LOGIN! / Trending / Recent / New
Stats: 3,148,017 members, 7,799,473 topics. Date: Tuesday, 16 April 2024 at 10:04 PM

Artificial Intelligence And Machine Learning Group - Programming (10) - Nairaland

Nairaland Forum / Science/Technology / Programming / Artificial Intelligence And Machine Learning Group (39134 Views)

Are You Interested In Robotics And Artificial Intelligence? / Artificial Intelligence And Robotics In Africa. / Survey - People Interested In Artificial Intelligence And Machine Learning (2) (3) (4)

(1) (2) (3) ... (7) (8) (9) (10) (11) (12) (13) (Reply) (Go Down)

Re: Artificial Intelligence And Machine Learning Group by 4kings: 11:16am On Apr 05, 2018
Darivie04 welcome!!! smiley
It's very cool that you are in JSS3 and are tackling machine learning. That's dope. cool

Anyways, a while back i spoke with raymod170 offline about taking fundamentals of machine learning.
That is explaining the maths behind most algorithms in simple details instead of using libraries without understanding what's going on under the hood as that impedes the capability to understand and make research in this field.

I've been busy, but would put it in my schedule to explain the details behind most machine learning algorithms in simple details including reinforcement learning, occasionally on this thread. smiley

1 Like

Re: Artificial Intelligence And Machine Learning Group by 4kings: 11:44am On Apr 05, 2018
SoftEng:
Hi all,

Here's another competition that might be of interest to you.

Microsoft Cloud AI Research Challenge.
https://www.microsoft.com/en-us/research/academic-program/microsoft-cloud-ai-research-challenge/

The Challenge
Use Microsoft AI services to develop (breakthrough) applications or embed their AI (cognitive) services into your application.
Prize: 25,000 USD Grand Prize

My take on the challenge
In simple terms, Microsoft wants to encourage more developers to use their AI platform and also discover new use cases of AI from these challenge. I believe it is a fun and interesting challenge. For anyone who will be interested, you can think of wide variety of application areas, but if you can narrow it down to application areas that solves some of our local problems that would be a good thing.

Thanks.
9 days left.
Reminder.
smiley embarassed embarassed
Re: Artificial Intelligence And Machine Learning Group by osarenomaspecial: 2:31pm On Apr 05, 2018
4kings:
Darivie04 welcome!!! smiley
It's very cool that you are in JSS3 and are tackling machine learning. That's dope. cool

Anyways, a while back i spoke with raymod170 offline about taking fundamentals of machine learning.
That is explaining the maths behind most algorithms in simple details instead of using libraries without understanding what's going on under the hood as that impedes the capability to understand and make research in this field.

I've been busy, but would put it in my schedule to explain the details behind most machine learning algorithms in simple details including reinforcement learning, occasionally on this thread. smiley
taking fundamentals of machine learning will be great at this point is what we need now in the thread.

1 Like

Re: Artificial Intelligence And Machine Learning Group by 4kings: 2:41pm On Apr 05, 2018
osarenomaspecial:

taking fundamentals of machine learning will be great at this point is what we need now in the thread.
Great!!!
Re: Artificial Intelligence And Machine Learning Group by osarenomaspecial: 3:32pm On Apr 05, 2018
4kings:
Great!!!
once we have the foundation of machine learning then come together to build awesome project and solve problems with it. in less than a year we will have some newbies who will become proficient in ML.

1 Like 2 Shares

Re: Artificial Intelligence And Machine Learning Group by Nobody: 4:39am On Apr 06, 2018
Dear friends,

AI and Machine Learning are transforming multiple industries, and I want to help you become great at it. I have been writing a book, Machine Learning Yearning, to teach you strategies for structuring Machine Learning projects. If you would like to receive free draft chapters as they are released, please sign up at:

http://mlyearning.org

The ML MOOC and deeplearning.ai’s Deep Learning Specialization teach a lot of technical algorithms, but the best AI technologists also need to know how to structure an ML project.

This book is focused not on teaching you ML algorithms, but on how to use ML algorithms. If you aspire to be a technical leader in AI and want to learn how to set direction for your team's work, this book will help.

Most of this content is not taught in typical “academic” ML courses. It is drawn from my experience building and shipping many deep learning products. After reading this book, you will be able to:

Prioritize the most promising directions for an AI project

Diagnose errors in a machine learning system

Build ML in complex settings, such as mismatched training/test sets

Set up an ML project to compare to and/or surpass human-level performance

Know when and how to apply end-to-end learning, transfer learning, and multi-task learning

I've seen teams waste months or years because they don’t understand the principles taught in this book. I hope it will save you months of time.

I had started working on this book over a year ago, but I got distracted building the Deep Learning Specialization on Coursera. Now that the specialization is complete, I’m returning to this and hope you will benefit from it!

Sign up here: http://mlyearning.org

Andrew Ng
Re: Artificial Intelligence And Machine Learning Group by MYSELF2018: 11:18pm On Apr 06, 2018
4kings:

"Unifying theory of data mining"?
I don't see how that's possible for now.
At the end of the day it will still be selection of best-fit algorithm on some set of categories which might even have a high bias variance based on the dataset.

Besides you are talking of unified theory and also talking on comparing two algorithms, i'm lost.

You can make your research domain based instead of data mining in general or maybe derive hierarchical clusters as "unifying theory" for cluster selection in any dataset(which still isn't as efficient when comparing with Inpedndent component analysis(ICA) on specific problem set).
That's as close as i can think.

Anyways do tell about the outcome of your research or any progress you make.
Thanks a lot my brother
Re: Artificial Intelligence And Machine Learning Group by 4kings: 12:34pm On Apr 07, 2018
MYSELF2018:
Thanks a lot my brother
You welcome.
How far have you gone with the research or have you switched to another?
Re: Artificial Intelligence And Machine Learning Group by 4kings: 2:52pm On Apr 07, 2018
Hey guys, so i said i’ll be initiating explanatory tutorials occasionally on the details behind most machine learning algorithms in order for us all to understand and make research especially as we progress in this group.
I say *initiating* cos everyone should contribute.

We would of course be applying these machine learning algorithms to areas of Artificial intelligence like computer vision, natural language processing, bioinformatics, stock market prediction and many more.

Hope this would turn out fun and useful.
Contribution by all is encouraged. smiley
And remember to always ask questions if you not clear about a concept, no question is nonsense.



Well here i start smiley :

Look at this data:



It tells what wage a person recieves based on the time(in hours) put to work.
With this naive data we can by intuition after looking at it for a while see and predict what a person wage would be given a value of the total time he/she spent working.
We know this because after looking for a while we can see a pattern that the wage increase by 75000 as time increase by 10. wink

But how can a computer know this pattern and make prediction, without being explicitly programmed.

And this is where we look at Regression algorithms.
Regression algorithms: are used to study relationships between two or more continuous variables and generates a model used for prediction.
“Continous variables” in the sense that wages in our small dataset are of different values and cannot be represented in discrete form like 1 and 0 or True and False. As such the data we are exploring is a regression problem.

There are many algorithms for regression but we would look at linear regression for now.
Linear Regression:
Linear regression is divided into 1) SImple Linear Regression and 2) Multiple Linear regression.

Simple linear regression deals with one independent variable, like in our dataset.
Multiple linear regression deals with more than one independent variables.

For our dataset the dependent variable is the label or target(Wages) and the independent variable is the feature(which is Time[in hours]). So in any dataset the label is the dependent variable and the independent variable are the features. → Features determine what the label would be. Dig it?

So let’s take a look at simple linear regression;
Simple linear regression deals with the linear equation in mathematics were the features in a graph are connected or fitted by a line as much as possible.

To understand linear equation let’s examine this sample graph:



Things to note:
a) The obvious ones: there are two axis the vertical is Y-axis and the horizontal is the X-axis.

b) In the graph above the direction of the line at any two points is called the SLOPE or GRADIENT(actually in deeper details this two are a little bit different but we would get there).
The gradient is usually represented with the letter m in maths.

c) The INTERSECTION is where the line cuts or intersect on the y axis. This is usually represented as b or c.

The formula for the linear equation is Y = mx + b
m and b is explained above. Y and X are the points on the axis to represent the values of our dataset. So given a value x we would be able to predict y.

If you want to be able to proof or derive this formula yourself i’ll discuss it in the next post. smiley

So let’s plot the graph for our dataset:



From the images above. The first graph are the unconnected points(called scatter plot) of the values in our dataset, while the second graph is the line fitting the points and the linear equation is useful for fitting this line to our dataset points and for prediction.

Making Prediction:
So let’s write the code for the linear equation and also to predict a wage(y) given a time(x).
So given a time(x) as 70 we need to predict what y(the wages) would be.
Y = mx + b

The gradient m is denoted as rise/run. Meaning the difference of any two points on the y axis divided by the difference between the corresponding points on the x axis. (Like i said the proof and derivative of this formula would be discussed in the next post)
I usually take the last two points, therefore:
(450000 - 375000) / (60-50)
=> 75000/10 = 7500
Therefore the gradient of our small data points is 7500.


B is the intersection. SInce in our graph it’s not noticeable we can derive b using simple math.
If y = mx + b:
Then when y = 75000 and x = 10 then:
75000 = (7500 * 10) + b
75000 = 75000 + b
75000 - 75000 = b
Therefore b = 0. Meaning the points in the dataset are evenly distributed.

So for our prediction, when time(x) is 70 then wages(y) would be:
Y = mx + b
Y = (7500 * 70) + 0
Y = 525000 + 0
Y = 525000

So our algorithm(the linear equation) has predicted that when the time(x) is 70 then the wages(y) would be 525000.

Remember when looking at the dataset, our Wages increases by 75000 as time increase by 10. Therefore our algorithm has accurately derived the pattern and predicted the wage correctly. cool

This simple illustration is not all for linear regression yet, but a demonstration of how mathematical modelling is used to derive pattern and make a model for prediction on dataset

So here’s the code for linear equation and making prediction based on our dataset. (the code is well commented to follow through)
I’ll prefer you(that’s if you didn’t know this already) to not look at the code and write it yourself to cement your understanding of the equation.

#Code:
#This is a python library for plotting of graphs.
from matplotlib import pyplot as plt

#Make variables for our dataset with their values ordered appropriately
time = [10, 20, 30, 40, 50, 60]
wages = [75000, 150000, 225000, 300000, 375000, 450000]

#Code for plotting the graph of the dataset.
plt.plot(time, wages) #Use plt.scatter to see the scatter plot instead.
plt.yticks(wages)
plt.xlabel('Time(in Hours)')
plt.ylabel('Wages(Naira)')
plt.show()


#define function for the linear equation (y = mx + b)
#The linear equation function should take additional argument x to be used for prediction.
#data1 and data2 represents time and wages.
def linear_equation(data1, data2, x):
#remember how to derive m.
#here we used the last two points on the x and y axes.
# (y2 - y1)/(x2 - x1)
m = (data2[-1] - data2[-2]) / (data1[-1] - data1[-2])

#Remember how to get b, by substituting the corresponding values for a point(x,y).
#Use the last values for x and y also for simplicity sake.
#b = y - mx by substitution.
b = data2[-1] - (m * data1[-1])

#therefore to predict y given the argument x:
y = (m * x) + b
return y

#Like in the explanation:
#To predict wage(y) when time(x) is 70:
#Call the function with the argument for x also putting the two intial arguments for the dataset.
predict = 70
result = linear_equation(time, wages, predict)
print("The wage prediction when time is {} is:".format(70), result)

1 Like

Re: Artificial Intelligence And Machine Learning Group by 4kings: 2:56pm On Apr 07, 2018
Okay.
So we've used the linear equation to demonstrate how to generate a model to learn pattern in a dataset and make predictions.

But the dataset i used was too simple and just to give a simple introduction to linear regression.

Most dataset in the real world are not evenly distributed like our dataset, therefore we would encounter errors but how do we optimise our functions to deal with these errors?
Moreso, dataset usually have more than one independent variables.

We would deal with all these soon. And we would use stock market data for stock price prediction with linear regression. smiley



In the mean time, here's a 7 minutes simple video if you need to know how to proof y=mx + b.
I believe the video is simple to understand.
However if you don't understand any concept, you can always ask question for clarification.

https://www.youtube.com/watch?v=F4IOxTuVMgQ

1 Like

Re: Artificial Intelligence And Machine Learning Group by Samcent: 3:19pm On Apr 07, 2018
This is great! Following the lesson....

1 Like

Re: Artificial Intelligence And Machine Learning Group by Nobody: 4:29pm On Apr 07, 2018
@4kings what your doing is so informative I really appreciate it!!

1 Like

Re: Artificial Intelligence And Machine Learning Group by 4kings: 5:13pm On Apr 07, 2018
Samcent:
This is great! Following the lesson....
Darivie04:
@4kings what your doing is so informative I really appreciate it!!
Thanks guys. smiley
Re: Artificial Intelligence And Machine Learning Group by agwaisrael(m): 9:38pm On Apr 07, 2018
omohayek:

You do? Alright then, what is Q-learning? What is partial observability? Can you tell me what the Bellman equation is, and how it works? Please provide the answers to these questions in your own words, instead of regurgitating whatever you can find on Google.

If you have the book, I doubt you've even begun to read it, as otherwise you wouldn't be on here asking such rudimentary questions.

Typical Nigerian, scratch that. Typical Hater!

It doesn't promote progress trying to find flaws, but instead try to build.

What point are you trying to prove? If you are so much of an expert, why haven't you created your own algorithm or even a common ml library.

Try not to look for flaws in people and if you do find, don't be a hater. Find strengths

1 Like

Re: Artificial Intelligence And Machine Learning Group by omohayek: 10:40pm On Apr 07, 2018
agwaisrael:


Typical Nigerian, scratch that. Typical Hater!

It doesn't promote progress trying to find flaws, but instead try to build.
God, another mouth-breather! Didn't you read the part where I already said I get paid to build full time? And I earn far more doing so than a sub-literate bozo like you ever will, that much is certain.


What point are you trying to prove? If you are so much of an expert, why haven't you created your own algorithm or even a common ml library.
Except, to repeat myself, I have in fact done such things, not that you'd know the significance of any of them if I bothered to mention them to you. You wouldn't know the Bellman equation from Bell's inequality if your right to keep living depended on the answer, yet you dare to presume to know what I have or haven't done? By God you're dumb! Here, tell us all the right answer to this simple question and I'll deign to spend some time telling you something about what I've achieved and been well-rewarded for: how many neurons in a feed-forward architecture would it take to master pendulum-balancing in the OpenAI Gym environment, and what sort of non-linear activations would you use to accomplish it? Bonus points for sharing with all of Nairaland what optimization technique you'd use, and what hyperparameters you'd employ. Somehow, I suspect we'll see Halley's Comet return before you can answer even such elementary questions.


Try not to look for flaws in people and if you do find, don't be a hater. Find strengths
Save your utterly stupid, content-free pontifications for someone of your own dishwater-level of intellectual depth. It's not compulsory to comment when you have nothing of value to add to a discussion. I know it's hard for half-wits like you to understand, but not everyone in the world is on your low level of bare-survival doing whatever inconsequential menial tasks you do to survive (assuming you even have a job of any kind).

1 Like

Re: Artificial Intelligence And Machine Learning Group by agwaisrael(m): 12:44am On Apr 08, 2018
omohayek:

God, another mouth-breather! Didn't you read the part where I already said I get paid to build full time? And I earn far more doing so than a sub-literate bozo like you ever will, that much is certain.


Except, to repeat myself, I have in fact done such things, not that you'd know the significance of any of them if I bothered to mention them to you. You wouldn't know the Bellman equation from Bell's inequality if your right to keep living depended on the answer, yet you dare to presume to know what I have or haven't done? By God you're dumb! Here, tell us all the right answer to this simple question and I'll deign to spend some time telling you something about what I've achieved and been well-rewarded for: how many neurons in a feed-forward architecture would it take to master pendulum-balancing in the OpenAI Gym environment, and what sort of non-linear activations would you use to accomplish it? Bonus points for sharing with all of Nairaland what optimization technique you'd use, and what hyperparameters you'd employ. Somehow, I suspect we'll see Halley's Comet return before you can answer even such elementary questions.


Save your utterly stupid, content-free pontifications for someone of your own dishwater-level of intellectual depth. It's not compulsory to comment when you have nothing of value to add to a discussion. I know it's hard for half-wits like you to understand, but not everyone in the world is on your low level of bare-survival doing whatever inconsequential menial tasks you do to survive (assuming you even have a job of any kind).

Well, I guess you are over compensating for something. That's the only reason you would become so egoistic. Or you are plainly an unreasonable person, in which case #handsOff!

I'm glad to see someone so proud of being "employed".
As for your judgment of my intelligence, how'd you figure all that from a well worded English paragraph? Guess you are a general artificial intelligence hero.

1 Like

Re: Artificial Intelligence And Machine Learning Group by zamie(m): 1:56am On Apr 08, 2018
omohayek:

Save your utterly stupid, content-free pontifications for someone of your own dishwater-level of intellectual depth. It's not compulsory to comment when you have nothing of value to add to a discussion. I know it's hard for half-wits like you to understand, but not everyone in the world is on your low level of bare-survival doing whatever inconsequential menial tasks you do to survive (assuming you even have a job of any kind).

Mr Man, YOU ARE FOOLISH. Fools talk like this. Someone is trying to give you an advice and you are here spitting English. Because, it's u that started machine learning abi. Pride Comes before a Fall.

Just get off this Page, We don't need people like you who know nothing and yet make noise. Prove me wrong and tell me one achievement you have made. Instead, you will come up with more English. Incase u don't know, Machine learning is for smart people unlike people like u with an empty skull but a big mouth.

Let me give you a simple Question to even know ur level of intelligence sef.

What is 1 + 1?
(pls, no calculators allowed)

OLODO, ANSWER THIS QUESTION O, DON'T COME AND BE SPEAKING ENGLISH

2 Likes

Re: Artificial Intelligence And Machine Learning Group by Damoxy(m): 7:27am On Apr 08, 2018
Good Day everyone, just joined this Group via someone here. I really want to learn Data Science. I dont know how it connects to AI or ML. If it does. How then can I start learning it.

The softwares and books and books that I will need to kick start this learning process.
Re: Artificial Intelligence And Machine Learning Group by Nobody: 8:38am On Apr 08, 2018
agwaisrael:


Typical Nigerian, scratch that. Typical Hater!

It doesn't promote progress trying to find flaws, but instead try to build.

What point are you trying to prove? If you are so much of an expert, why haven't you created your own algorithm or even a common ml library.

Try not to look for flaws in people and if you do find, don't be a hater. Find strengths
Thank you my brother....

1 Like

Re: Artificial Intelligence And Machine Learning Group by Nobody: 8:46am On Apr 08, 2018
Damoxy:
Good Day everyone, just joined this Group via someone here. I really want to learn Data Science. I dont know how it connects to AI or ML. If it does. How then can I start learning it.

The softwares and books and books that I will need to kick start this learning process.

https://www.google.com.ng/url?q=http://bigdata-madesimple.com/8-best-python-data-science-books/&sa=U&ved=2ahUKEwid8Oa9lqraAhUDKFAKHUFrCsIQFjAAegQICRAB&usg=AOvVaw0kTU8N0d341QHj7JPB0GJw
I personally uses the book called Data Science From Scratch
And I guess Data Science is connected to ML and AI cause you have to prepare the Data you will use for training in ML
Re: Artificial Intelligence And Machine Learning Group by Damoxy(m): 11:06am On Apr 08, 2018
Darivie04:


https://www.google.com.ng/url?q=http://bigdata-madesimple.com/8-best-python-data-science-books/&sa=U&ved=2ahUKEwid8Oa9lqraAhUDKFAKHUFrCsIQFjAAegQICRAB&usg=AOvVaw0kTU8N0d341QHj7JPB0GJw
I personally uses the book called Data Science From Scratch
And I guess Data Science is connected to ML and AI cause you have to prepare the Data you will use for training in ML
Thanks
So I have to learn Data Science first before entering ML and AI properly
Re: Artificial Intelligence And Machine Learning Group by Nobody: 11:08am On Apr 08, 2018
zamie:


Mr Man, YOU ARE FOOLISH. Fools talk like this. Someone is trying to give you an advice and you are here spitting English. Because, it's u that started machine learning abi. Pride Comes before a Fall.

Just get off this Page, We don't need people like you who know nothing and yet make noise. Prove me wrong and tell me one achievement you have made. Instead, you will come up with more English. Incase u don't know, Machine learning is for smart people unlike people like u with an empty skull but a big mouth.

Let me give you a simple Question to even know ur level of intelligence sef.

What is 1 + 1?
(pls, no calculators allowed)

OLODO, ANSWER THIS QUESTION O, DON'T COME AND BE SPEAKING ENGLISH
The guy I utterly hopeless, that other guy gave him good advice and he starts insulting him.

Let's all be praying for him sha

1 Like

Re: Artificial Intelligence And Machine Learning Group by Nobody: 11:08am On Apr 08, 2018
_
Re: Artificial Intelligence And Machine Learning Group by Alejob: 4:00am On Apr 09, 2018
Are you in need of a portable energy solution that is efficient and can be carried around without stress ..... I recommend Mobile Power Solution for you


This setup can power your TV LED plus Decoder (5hours plus) , Laptop (10-24hrs+) , Phone/Tab (a week) Light Bulb (Days), Clipper (10hrs+), e.t.c. and the icing on the cake is that it is exceptionally mobile.....you easily charge at work and use at home or add solar panel to charge it during the day


If you are into freelancing, coding, programing, manuscript editing, data analysis or working with Autocad, Archicad or work for lengthy hours on laptops.... At the price of 35K.....this is a must have.....


It can be recharge back in about 4 -5 hours
Compatible with all laptop and many electronic device....

Extremely mobile and therefore
Improves productivity and entertaining hours everywhere and anywhere
Compact and easy to carry about
Quite affordable and no maintenance needed

Maximum load capacity is 100watt... So overloading is a no no

Price: N35, 000

For more info, contact: 08065316307
oladipoadeyeye@gmail.com
Cheers!

https:///Cf4dTuhBnLM5sBXo0hN17I


https://www.nairaland.com/1565780/power-bank-power-tv-decoder


https://www.youtube.com/watch?v=puMRpTOwnNM
Re: Artificial Intelligence And Machine Learning Group by 4kings: 7:11pm On Apr 09, 2018
Na wa oo
Una still dey fight.
And who is this Alejob sef.
Seun please we need an active moderator abeg.


Anyways Omohayek cool achivements smiley
Have you thought of interwining the bellman's equation with bayesian approach? what's your view?

And how do you expect one to just come up with an answer for "how many neurons in a feed-forward architecture would it take to master pendulum-balancing in the OpenAI Gym environment", that isn't intuitive but only derived from data na.

By the way, Let's not fight again ooo
Just ignore anyone you don't like, so as not to derail the thread. That shouldn't be so hard. smiley

1 Like

Re: Artificial Intelligence And Machine Learning Group by 4kings: 7:16pm On Apr 09, 2018
Hey guys, been busy would continue the tutorial maybe this night when i get home.

By the way my images are not showing again, abi is it my laptop.

Can anyone recommend a good image sharing site, i'm using imgur and apparently it's no good.

1 Like

Re: Artificial Intelligence And Machine Learning Group by Nobody: 8:52pm On Apr 09, 2018
4kings:
Hey guys, been busy would continue the tutorial maybe this night when i get home.

By the way my images are not showing again, abi is it my laptop.

Can anyone recommend a good image sharing site, i'm using imgur and apparently it's no good.
So whats today's topic?
Re: Artificial Intelligence And Machine Learning Group by 4kings: 7:56am On Apr 10, 2018
Darivie04:

So whats today's topic?
Today!!! wink
We would be looking at the inefficiency of the linear equation on a non evenly distributed dataset, therefore introducing the concept of optimisation or error functions and a look at multiple linear regression, then we'd write code for predicting stock market data.
Insha Allah... embarassed

2 Likes

Re: Artificial Intelligence And Machine Learning Group by Technobs(m): 8:37am On Apr 10, 2018
*ARTIFICIAL INTELLIGENCE.*

*LEARN THE SECRET BEHIND FLYING CARS USING PYTHON PROGRAMMING language AND C++*


If you are interested in flying cars, drones, autonomous systems, this program is for you. Join one of the world's largest online platform. Learn from professionals.

Click below to read.


https://technobs.com/learn-the-secret-behind-flying-cars-udacity/
Re: Artificial Intelligence And Machine Learning Group by 4kings: 7:03pm On Apr 10, 2018
4kings:

Today!!! wink
We would be looking at the inefficiency of the linear equation on a non evenly distributed dataset, therefore introducing the concept of optimisation or error functions and a look at multiple linear regression, then we'd write code for predicting stock market data.
Insha Allah... embarassed

Hey guys, with the look of things today, i won't be able to write and complete the next tutorial. embarassed embarassed

Can someone do this. I think i wanted to explain error functions using ordinary least squares(with little matrix explanation for the multivariable part) before moving to logistic regression while using another error function; gradient descent.

But i won't be able to cover this today again embarassed embarassed embarassed , maybe someone can help, or later when i'm free.
And my images ain't still showing. sad
Cheers.
Re: Artificial Intelligence And Machine Learning Group by 4kings: 2:23pm On Apr 16, 2018
Okay guys, here i go with the next tutorial. Finally!, right.
Would have done this yesterday, but got distracted watching Eye Candy. SMH


The previous tutorial we looked at how simple linear equation can solve a regression problem, however our dataset was evenly distributed on purpose for demonstration and we only dealt with one independent variable or feature.

So today we would explore two datasets to see the functions available for adjusting error for prediction accuracy in a dataset(that’s actually what machine learning is about, how well an algorithm predicts or generalise with error minimisations ) and also looking at multiple linear regression for handling more than one independent features.



But before we go on, it is important to have an understanding of Matrix as we are going to be using that in our calculations. Matrix helps to solve complex simultaneous equations with multiple variables faster and easier.

You can goto mathisfun website and get a simple and quick intro to matrix here: https://www.mathsisfun.com/algebra/matrix-introduction.html, matrix inverse here: https://www.mathsisfun.com/algebra/matrix-inverse.html and finally solving simultaneous equation with matrix here: https://www.mathsisfun.com/algebra/systems-linear-equations-matrices.html . That’s all about matrix that we would be using for now.
It’s really easy to understand especially with the elegant explanation in mathisfun website, however if you don’t understand any concept you can always ask. smiley



Let’s explore an unevenly distributed dataset.

Though this dataset is not as evenly distributed as the previous one, but you can still observe the pattern, where price increase by 100 after two intervals. I prepared it this way to demonstrate the example of the next tutorial in a simple manner.

So with the pattern in our dataset we can predict that at hour(16) price would be 800 and at hour(20) price would be 1000, right?.

Now let’s scatter plot our dataset:
 
from matplotlib import pyplot as plt
prices = [100, 100, 200, 200, 300, 300, 400, 400, 500, 500, 600, 600, 700, 700, 800]
hours = [1,2,3,4,5,6,7,8,9,10,11,12, 13, 14, 15]
plt.xticks(hours) #To indicate the numbers on x axis.
plt.scatter(hours, prices)




Now before we use our linear equation function to fit a line and make prediction let me make an important change.
-- Our previous dataset was evenly distributed, so when calculating the value for gradient(m) we used any two points in y and the corresponding points in x.
-- In that logic, if we get the average value of y and average value of x, we would arrive at the same value since the data is evenly distributed. (try it yourself)
-- So m = Average of Y / Average of X
-- This is what we would use for a normal dataset because there are not evenly distributed and we need to get the average as a randomly picked points my cause a high deviation from the original slope.
-- So our code would change by one line:


#Code:
def linear_equation(data1, data2, x):
#Average of y divided by Average of X
m = (sum(data2)/len(data2)) / (sum(data1)/len(data1))

#Remember how to get b, by substituting the corresponding values for a point(x,y).
#Use the last values for x and y also for simplicity sake.
#b = y - mx by substitution.
b = data2[-1] - (m * data1[-1])

#therefore to predict y given the argument x:
y = (m * x) + b
return y



-- Hope that’s clear…


Now let’s use the function to fit line into our scatter plot.
By the way, the function y=mx + b is what fits a line to our data and enable us to make prediction. It fits a line by calculating the points from when y = 0 to y = n. Get it?


#Code:
plt.xticks(hours)
plt.scatter(hours, prices)

line_y = [linear_equation(hours, prices, i) for i in hours] #Draw line by connecting all values for y.
plt.plot(hours, line_y)
plt.show()



[img]https://image.ibb.co/cNV7bn/regression_line.png[/img]

So you can see what i mean by fitting a line, if you had a graph and continue drawing the line, you could connect the points we want to predict 16 and 20 and see where there meet at y.

But we would predict that with code:

#Code:
predictions = [16, 20]
result = [linear_regression(hours, prices, i) for i in predictions]
print(result)

>>> [853.3333333333334, 1066.6666666666667]

The normal linear equation predicted that at point 16 and 20 the results would be 853.3 and 1066.6 respectively.
However, the prediction we expected is 800 and 1000.



So what is provided in linear regression to adjust for this error to come closer to accuracy.
Therein, lies the heart of what most machine learning algorithms do: adjusting the error or cost function for better accuracy.

For this problem, there are two solutions, either using statistical method called Ordinary Least Squares or using a Calculus method called gradient descent.

For this tutorial, we would be using the statistical method called Ordinary Least Squares Method and when we look at other machine learning algorithms we would switch to gradient descent.
The only difference between this two methods is that gradient descent is faster.



Least Squares Method:
So with the linear equation function, we can get it’s predicted value from when y = 1 to y = n and the true or correct value of y at each point.
The difference between this is the error.
Example:
Using the linear equation function; when we predict what y is at 1.
#Code:
print(linear_equation(hours, prices, 1))
>>> 53.333333333333336


It gives us 53.33 whereas the value of y at 1 is 100.
The difference 100 - 53.333 = 46.666666666666664
Is the error.

And also there would be some differences that would give negative values.
Example:
#Code:
print(linear_equation(hours, prices, 2))
>>> 106.66666666666667


Where the correct value is 100, making the difference -6.66 a negative value.

So to deal with this negative value we square them. -2 ^ 2 will give a positive value of 4.

So the goal of least square, is to get the total sum of the squares.
And then randomize the value of m(the slope) and consequently b in the equation y = mx + b until we get the LEAST TOTAL SUM OF SQUARED ERRORS
. Get it?
Actually this is the goal of gradient descent algorithms too… To get at what point in the slope or gradient(m) that reduces the error in prediction.

Anyways, you might be thinking of writing a code to randomize the value of m then calculate b and get the sum of squared errors until we find the lowest total sum. wink
Well that would be a very slow method especially when we have a large dataset, as m could be even in decimal like 0.1 so going through every number and decimal is just too much CPU processing, even at that m could be -0.25 a negative value so you have start as low as much as possible, which you might not get right eventually after much cpu processes.

So in Linear regression or statistics in general there’s a concept called Pearson’s correlation.
With the formula we can get at what point m describes the closeness or relationship between two variables the most in a dataset.
And this is the shortcut for finding the gradient(m).
When we get to other machine learning algorithms, m would be referred to as weights and finding m would be referred to as adjusting the weights, there mean the same thing, just using a different approach.(just incase i forget to add that).

This is the formula.

[img]https://image.ibb.co/dHrr2S/pearson_formula.png[/img]
I would provide a short video that proves this formula in a later post.

Therefore to get b using the formula y = mx + b,
b = y_mean - (m * x_mean)
Remember i talked about using the average of y and x in a non-evenly distributed dataset.



With the formula for m and b, it easy to write a code for our least square regression.

#Code:
def linear_regression(x,y, predict):
#Calcluating m
x_mean = sum(x)/len(x)
y_mean = sum(y)/len(y)


numerator = 0
for i in range(0, len(x)):
numerator += (x[i] - x_mean) * (y[i] - y_mean)

denominator = 0
for i in range(0, len(x)):
denominator += (x[i] - x_mean) ** 2

m = numerator/denominator

#Now let's calculate b
b = y_mean - (m * x_mean)

#So to predict y:
#y = mx + b
y = (m * predict) + b
return y



So to make prediction for our dataset at point 16 and 20 with the least square regression formula.

predictions = [16, 20]
result = [linear_regression(hours, prices, i) for i in predictions]
print(result)
>>> [826.6666666666667, 1026.6666666666667]

So you can see that the least square regression has increased in accuracy and is closer to the correct value of 800 and 1000 as opposed to the normal linear equation formula that gave [853.3333333333334, 1066.6666666666667]

This is Linear Regression using least square regression and as we look at other algorithms you would see how their accuracies compare, though the amount of data also determines how well the algorithm performs but that’s a topic for another day.



Before we proceed, let’s look at one last concept: MULTIPLE LINEAR REGRESSION.
So far we’ve been working with one independent variable or feature for prediction, but whenever a data has more than one feature that defines a label which is usually the case, we use Multiple linear regression instead.

Multiple linear regression is easy to grasp. It’s just an addition to the y=mx + b formula, that’s all.

So for y= mx + b
X is the feature while y is the label.

But when we have more than one feature say 3 features, the formula changes to:
Y = m1x1 + m2x2 + m3x3 + b
Or Y = b + m1x1 + m2x2 + m3x3
Where: each x [x1, x2, x3] are different axis.
And each m [m1, m2, m3] are the corresponding slopes.


And that’s it.
****

But it would be difficult to calculate this(every point of m and x) especially when there are many features.
So to do this easily, we need the help of MATRIX.

In the intro of this post i gave link to learn matrix, especially how to solve linear equations with matrices https://www.mathsisfun.com/algebra/systems-linear-equations-matrices.html. Check the tutorial if you don’t understand Matrix before now, mathisfun.com is a popular website that explains maths concept in a simple way, so it should be easy to follow along, again if you have any issues while learning it you can always ask.

Once you understand that:
Then the formula for multiple linear regression, can be transformed into this Matrix form.


Where in y = mx + b:
A is X transposed. X are the features
And B is Y.
And X represents m and b as the coeficients.


We would use the python library NUMPY which is a libary for matrix calculations, modelled from the popular MATLAB(matrix laboratory) software for complex mathematical calculations using matrix forms.
See MATRIX is important in Machine Learning, hmmm wonder about the MATRIX Trilogy movie on AI domination now. Lol grin grin grin

So if we had three features [2,4,6,8,10], [3,6,9,12,15] and [4,8,12,16,20].
With labels [1,2,3,4,5]
Using Numpy to calculate from our matrix understanding to get coefficients for prediction.

#Code:
import numpy
y = np.array([1,2,3,4,5])
X = np.array([[2,4,6,8,10], [3,6,9,12,15], [4,8,12,16,20]])

X = X.T #Get X transpose.
coef = np.linalg.lstsq(X,y)[0] #This function follows the linear matrix equation.



#This is the coeficient. In y=mx + b, coefficient equal m * b, again if you don’t understand matrix use the links given.

#Therefore if we want to predict the value of y the 3 X features are [12, 18 and 24]
#We would use dot multiplication with our coefficient to get the value.



features = [12, 18, 24]
prediction = np.dot(features, coef)
print(prediction)
>>> 6



#You can see it predicted it correctly, that was a simple demonstration on matrix calculations with Numpy.

So let’s build our final Linear Regression code using the Matrix solution.

#Code:
#Remember to convert the variales to numpy arrays instead of python lists.
def LinearRegression(X,y,predict):
#x and y would be in numpy arrays.

X = X.T
coef = np.linalg.lstsq(X,y)[0]

result = np.dot(predict, coef)
return result


#So we can solve the equation above as:
import numpy as np
y = np.array([1,2,3,4,5])
X = np.array([[2,4,6,8,10], [3,6,9,12,15], [4,8,12,16,20]])

prediction = LinearRegression(xd, yd, predict)
print(prediction)
>>> 6.0

1 Like

Re: Artificial Intelligence And Machine Learning Group by themonk(m): 3:41pm On Apr 16, 2018
I indicated to be part of the AI learning group in the thread that lead to this one but unfortunately my moniker wasnt mentioned properly when this thread was started so I didn't know you guys have proceeded. I will try to go through each of the pages one by one.

(1) (2) (3) ... (7) (8) (9) (10) (11) (12) (13) (Reply)

Post Ur Vb 6.0 Questions Here / Java Programming For Dummies / Nigerian Software Engineer given test to prove he is an engineer at JFK Airport

(Go Up)

Sections: politics (1) business autos (1) jobs (1) career education (1) romance computers phones travel sports fashion health
religion celebs tv-movies music-radio literature webmasters programming techmarket

Links: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

Nairaland - Copyright © 2005 - 2024 Oluwaseun Osewa. All rights reserved. See How To Advertise. 114
Disclaimer: Every Nairaland member is solely responsible for anything that he/she posts or uploads on Nairaland.