|Join Nairaland / LOGIN! / Trending / Recent / New|
Stats: 2,034,998 members, 4,355,283 topics. Date: Sunday, 15 July 2018 at 11:56 PM
|Re: The Future Of Machine Learning In Nigeria by Nobody: 11:45am On Dec 11, 2016|
FloatingPoint:That's exactly what I'm trying to do
I find linear algebra easy but statistics is a little challenging though.
I'm also working on becoming an intermediate pythonista.
|Re: The Future Of Machine Learning In Nigeria by neahyo(m): 12:30pm On Dec 11, 2016|
Artificial intelligence, or AI, is the field that studies the synthesis and analysis of computational agents that act intelligently. Let us examine each part of this definition. An agent is something that acts in an environment – it does something. Agents include worms, dogs, thermostats, airplanes, robots, humans, companies, and countries. We are interested in what an agent does; that is, how it acts. We judge an agent by its actions. An agent acts intelligently when what it does is appropriate for its circumstances and its goals,it is flexible to changing environments and changing goals,it learns from experience, andit makes appropriate choices given its perceptual and computational limitations.
An agent typically cannot observe the state of the world directly; it has only a finite memory and it does not have unlimited time to act. A computational agent is an agent whose decisions about its actions can be explained in terms of computation. That is, the decision can be broken down into primitive operation that can be implemented in a physical device. This computation can take many forms. In humans this computation is carried out in “wetware”; in computers it is carried out in “hardware.” Although there are some agents that are arguably not computational, such as the wind and rain eroding a landscape, it is an open question whether all intelligent agents are computational. The central scientific goal of AI is to understand the principles that make intelligent behavior possible in natural or artificial systems. This is done by
the analysis of natural and artificial agents;formulating and testing hypotheses about what it takes to construct intelligent agents; anddesigning, building, and experimenting with computational systems that perform tasks commonly viewed as requiring intelligence.
As part of science, researchers build empirical systems to test hypotheses or to explore the space of possibilities. These are quite distinct from applications that are built to be useful for an application domain. Note that the definition is not for intelligent thought.We are only interested in thinking intelligently insofar as it leads to better performance. The role of thought is to affect action. The central engineering goal of AI is the design and synthesis of useful, intelligent artifacts.We actually want to build agents that act intelligently. Such agents are useful in many applications.
|Re: The Future Of Machine Learning In Nigeria by neahyo(m): 12:32pm On Dec 11, 2016|
There is huge amount of data available in Information Industry. This data is of no use until converted into useful information. Analysing this huge amount of data and extracting useful information from it is necessary.
The extraction of information is not the only process we need to perform, it also involves other processes such as Data Cleaning, Data Integration, Data Transformation, Data Mining, Pattern Evaluation and Data Presentation. Once all these processes are over, we are now position to use this information in many applications such as Fraud Detection, Market Analysis, Production Control, Science Exploration etc.
What is Data Mining
Data Mining is defined as extracting the information from the huge set of data. In other words we can say that data mining is mining the knowledge from data. This information can be used for any of the following applications:
Need of Data Mining
Here are the reasons listed below:
In field of Information technology we have huge amount of data available that need to be turned into useful information.
This information further can be used for various applications such as market analysis, fraud detection, customer retention, production control, science exploration etc.
Data Mining Applications
Here is the list of applications of Data Mining:
Market Analysis and Management
Corporate Analysis & Risk Management
Market Analysis and Management
Following are the various fields of market where data mining is used:
Customer Profiling - Data Mining helps to determine what kind of people buy what kind of products.
Identifying Customer Requirements - Data Mining helps in identifying the best products for different customers. It uses prediction to find the factors that may attract new customers.
Cross Market Analysis - Data Mining performs Association/correlations between product sales.
Target Marketing - Data Mining helps to find clusters of model customers who share the same characteristics such as interest, spending habits, income etc.
Determining Customer purchasing pattern - Data mining helps in determining customer purchasing pattern.
Providing Summary Information - Data Mining provide us various multidimensional summary reports
Corporate Analysis & Risk Management
Following are the various fields of Corporate Sector where data mining is used:
Finance Planning and Asset Evaluation - It involves cash flow analysis and prediction, contingent claim analysis to evaluate assets.
Resource Planning - Resource Planning It involves summarizing and comparing the resources and spending.
Competition - It involves monitoring competitors and market directions.
Data Mining is also used in fields of credit card services and telecommunication to detect fraud. In fraud telephone call it helps to find destination of call, duration of call, time of day or week. It also analyse the patterns that deviate from an expected norms.
Data Mining also used in other fields such as sports, astrology and Internet Web Surf-Aid.
|Re: The Future Of Machine Learning In Nigeria by Nobody: 2:52pm On Dec 11, 2016|
Nice writeup bro.
I hope you continue.
Could you please talk more on data cleaning. Most sites I've visited tend to lay emphasis on it a lot.
|Re: The Future Of Machine Learning In Nigeria by FloatingPoint: 7:54am On Dec 14, 2016|
I think it all depends on the way we think. For me, statistics was challenging because of its relationship with probability. I was averse to randomness(probability) and continuity(calculus) in my school days but liked discrete mathematics because my thinking pattern was "tuned" to appreciate the deterministic. But all that as changed since I discovered the extreme usefulness of this fields in the real world.
|Re: The Future Of Machine Learning In Nigeria by anthropino: 9:24pm On Dec 19, 2016|
This seem an interesting question!
Well it depends on your definition of thinking and what thoughts are. If by thinking , you mean up to human capacity, then I will say no for now. The human thinking mechanism is incredibly complex.
What we can do and we do appreciably well, is to model the human brain and thought process. From this abstraction , we construct a computational model from which we get numbers that we can work with. Check the book by Prof Willian Gerstner "Spiking Neuron Models: Single Neurons, Populations, Plasticity" to get quick insight. Or the "Neural network a comprehensive foundation" by Haykins and Simon or "Theoretical Neuroscience: Computational and Mathematical Modelling of Neural Systems " by Peter Dayan. These are the classics in the field.
As an example you can start with the concept of receptive field from wikipedia . Thats what these models try to do.
Secondly, this question is also related to machine learning. I will say the relation is this:
[ [ (Convex optimization Vs Machine Learning Vs Deep Learning ) ]* Statistical Modelling ]*Algorithmic blackbox = Artificial Intelligence.
This is to illustrate overlapping fields that seeks to build intelligent machines. They share a lot in common but are quite different.!
In particular when you do machine learning , you also do convex optimisation, and when you do deep learning , you are also have to use machine learning. Statistical Modelling wraps everything up--and is the origin of the field. If you ve all these but you don't use the right algorithm then the model is a failure, hence the need for algorithm.
All these fields do seek to make machine make good Decisions-- Thinking. But as far as the *Thinking* concept you are interested in goes, it is the neural networks/deep learning models that are often used -- probably. You can search for Google AlphaGo , an AI machine to confirm this.
In short in the ML world, it is the deep learning/neural models that win. This is a complex topic to discuss here.
Start with Machine Learning then to deep learning, there are still lots to be learned. All these things have existed before but the difference is that today we have enough computational capabilities. In particular these complex models are built on High Performance Computing Machines (with GPU).
I hope this helps!
|Re: The Future Of Machine Learning In Nigeria by FloatingPoint: 6:47am On Dec 31, 2016|
@ anthropino, Very insightful thanks.
|Re: The Future Of Machine Learning In Nigeria by halmat(m): 7:41pm On Jan 01, 2017|
What help can you provide. I'm about handling an online tutor that uses AI. Please how can the AI part be achieved
|Re: The Future Of Machine Learning In Nigeria by neahyo(m): 12:11pm On Mar 24, 2017|
Since this thread has died, I guess i have to make it alive again. How? You may ask, I would review the jobs i receive from different clients with you and i hope someone somewhere will learn one or two things; what's the essence of living a life without impacting on anyone's life anyways?
Way to go
I would start with easy ones first before moving to more advanced ones, I hope it's okay by you. Lest i forget, our 'gurus' in the house, don't be offended oh [biko! I'm on my knees] i know this looks easy to you but guess what? You don't know who might be interested in these little things.
Yeah! Now that I've gotten your approval let's get the party started! Young John 'the wicked producer' please give me the beat, thank you sir!
EuroCom provides new fixed line, mobile phone and broadband services to customers in the Euro-zone. It uses the Euro-zone telecommunications network and competes for customers against other major telecommunications companies in the market. Currently the company has a 50,000 customer base and there are roughly 250,000 calls per day.
One of the biggest problems facing EuroCom is customer retention. Customers are free to move between telecommunications service providers and some regularly change service providers, a phenomenon known as churn in the telecommunications industry. Some companies have churn rates as high as 20% of their customers changing service providers per annum. EuroCom has placed a high emphasis on churn and are always trying to find new ways of reducing it. The directors of EuroCom are looking for the answer to the main question of how we detect customers who are going to churn
Overall, the directors are looking for answers to these types of questions.
1. What is it that makes a customer churn?
2. Are some customers more likely to churn than others?
3. How can we identify these customers before they churn?
The data was collected from 2071 subscribers, below is the screenshot
|Re: The Future Of Machine Learning In Nigeria by neahyo(m): 12:27pm On Mar 24, 2017|
Perform data processing on the data set, as required. Give evidence whether there are problems with data quality, duplicate data or missing data. Note: With respect to predictor variables, missing categorical values should be replaced by the mode value by gender for that predictor variable. e.g. if the voice_mail flag is missing for a record that has gender of male, the replacement value should be the most popular voice_mail flag value for males. Missing numeric values should be replaced by the median value for the variable. Comment on your findings as well as the actions you carried out.
Note:[/b]Before starting your analysis, please familiarise yourself with the churn dataset provided. You can assume the sample was randomly selected from a population that is normally distributed. Please refer to Appendix 1 for description of the predictor variables and response (target) variable.
As part of an appendix to your report provide clearly commented supporting R code you used to carry out your tasks).
From the data processing, it is evident that there are 10 missing values and 12 empty/blank cells. Below is the breakdown of the predictors with missing values and blank cells:
CUST_MOS (The number of continuous months the Customer is with the provider) had 3 missing values
MINUTES_3_MONTHS_AGO (Number of phone minutes used in the previous 3 months) had 3 missing values
TOT_MINUTES_USAGE (The total number of minutes used to date) had 4 missing values
PHONE_PLAN (The phone plan the customer has signed up for) had 4 empty cells
EDUCATION (Highest Level of education attainment the account holder has achieved) had 8 empty cells
With this, it is evident that there are missing values and blank cells in the dataset. We use the R command below to import the dataset into R and replace all blank cells with NA
# R command
eurocom <- read.csv("eurocom.csv", header = TRUE, na.strings=c("", "NA")
We use the command below to count the number of NA’s in the dataset which was 22
The command below tells us the predictors with NA’s
sapply(eurocom, function(eurocom) sum(is.na(eurocom)))
We also use the R command below to determine the customer with missing values
For the predictors that are numeric in nature (CUST_MOS, MINUTES_3_MONTHS_AGO, TOT_MINUTES_USAGE), we replace the missing values (NA’s) with the median value using the R command below
#Replaces NA's with median value of Number of minutes 3 months ago
#Replaces NA's with median value of months the Customer is with the provider
#Replaces NA's with median value The total number of minutes used to date
For the predictors that are categorical (PHONE_PLAN and EDUCATION), we replace the missing values by the mode value by gender for that predictor variable. To do this, we use the commands below to view the mode value by gender for phone plan and education.
PHONE_PLAN F M
Euro-Zone 20 39
International 449 618
National 261 410
Promo_plan 0 270
EDUCATION F M
Bachelors 270 130
High School 1 2
Masters 0 330
PhD 190 220
Post Primary 266 563
Primary 1 90
It is shown from the results above that the most used phone plan for both gender was “International”. The highest education level attained for most males was Post Primary education while for females it was Bachelors’ degree. So, we replace these according to the customer with blank cell. Recall, we use the sapply(eurocom, function(eurocom) sum(is.na(eurocom))) command to view the predictor variables with missing values (see appendix) and it was shown that customers’ 6, 10, 109, 233, 283, 339, 673, 979, 1191, 1366, 1389 and 1465 have missing values for PHONE_PLAN and EDUCATION [recall we have replace the rest by their median values since they were numeric variables but we can’t do that for this as it is categorical]. To do this, we use the R command below to store the new dataset (i.e. the one we removed the missing values by its median) and make the adjustment.
Now, the issue of missing values has been resolved, the next we check for is duplicate data, we use the R command below to do this
# REMOVING DUPLICATES
This tells us there is no duplicated data.
Discretise the Income predictor variable as follows;
Income >= 88,000 -> High Income
Income < 88,000 && Income >= 38,000 -> Medium Income
Income < 38,000 -> Low Income
We use the R commands below to discretize the Income into groups
Income <- data.frame(eurocom)
HighIncome <- subset(Income, INCOME>=88000)
MediumIncome <- subset(Income, INCOME<88,000 && Income >= 38,000)
LowIncome <- subset(Income, INCOME<38,000)
Please, if you want me to continue, you all have to cut short your sabbatical leave and drop your comments oh, ehn ehn!
|Re: The Future Of Machine Learning In Nigeria by anthropino: 7:19pm On Mar 25, 2017|
This is a nice job @neayho ---you are the big boss. If you permit me to humbly make few suggestions:
1 ) It is highly recommended to use Jupyter notebook and Anaconda: (to provide neat code and explanations)
One can install Anaconda as the package manager :
With the above installations one can use both python packages and R.
The R packages can be installed ib the conda environment from the link >>https://conda.io/docs/r-with-conda.html
But if you dont like python we can use RStudio the equivalent of jupyter notebook:
2) Provided here is a repository of examples of jupyter notebook: (link below) from online communities
3) Why R is very good and the work perfect , one can also try out python libraries:
>> Numpy http://www.numpy.org/
>> Pandas http://pandas.pydata.org/
>> Matplotlib http://matplotlib.org/
and some other ones.
4) Why not consider these libraries for visualisation in your preprocessing (python):
>> Seaborn (https://seaborn.pydata.org/)
>> D3JS (the best for interactive visualisation not pythonic https://d3js.org/)
These are just additional standard libraries common in every day job and these are the ones I could remember --accept my ignorance.
Lastly I admire the treatment of missing data. But missing data is a very heated topic among intellectuals. I humbly invite you to read this paper:
and read the book by Roderic and Rubin " Statistical Analysis with missing data"
I hope this helps!
|Re: The Future Of Machine Learning In Nigeria by neahyo(m): 11:27pm On Mar 27, 2017|
Thank you very much for this sir! To start with, I hope i can take my time to learn Python as i heard it's a great software though I know it won't be easy to venture outside my comfort zone (R), can you recommend a book for a beginner in data science using python?
Pertaining to the paper sir, i read it and was impressed with the application of various software to solve the problem of missing data though I'm familiar with virtually all the R packages listed there to solve the problem of missing data, I'm impressed with the application of Stata, I guess there's much for me to learn this year and I'm grateful you cared to contribute your quota.
Thank you very much for this boss, I hope you can shed more light on your research interests and provide journals for me to read so i don't become rusty. Again, I say THANK YOU.
I look forward to reading your reply soon.
|Re: The Future Of Machine Learning In Nigeria by anthropino: 7:49pm On Mar 28, 2017|
Hi @Mr. Neaho,
Thanks sir for the feed back.
I have attached here a pdf file (MCCExternalKeyCpncept.pdf) from Google Machine learning crash course conducted some months ago. In the file you have the required python programming paradigms and the links to the python doc explaining those. Though, I never looked at it then, it is about the same thing I would recommend-- And I guess I will also keep it as reference too. It also contains links to important python data science libraries.
I don't really have a particular recommended book on python. And maybe sir (and I might be wrong for this) it might be good to use the docs and stackoverflow and try working on small project.[/center]
For Pandas (after one completes the fundamentals), I also find these videos on youtube very useful:
But whenever one is in doubt it is good to use the doc always.
The other 2 Pdf files (as dropbox link) are on statistical Machine learning and Neural network. Since sir, you have a strong background in Statistics, I found both papers, (from the early contributors to the field) to be almost perfect for you. Moreover, both papers are far better than most textbooks on the subjects in terms of clarity and explanation. You will love them!
From these papers one can then start implementing those ideas using scikit or tensorflow.
Check this link (www.youtube.com/watch?v=n5NcCoa9dDU&list=PL6il2r9i3BqH9PmbOf5wA5E1wOG3FT22p) to get started. And in addition sir, after getting through the background concept, feel free to use the examples provided by others: (github.com/d3/d3/wiki/Gallery). At a point you will need a local web server to load your pages while using d3js --especially when you have data in a seperate JSON file . I found google chrome extension "webserver for chrome , 200 ok!" to be useful for this sir.
I hope this makes sense. Thanks sir.
dropbox links to papers:
|Re: The Future Of Machine Learning In Nigeria by neahyo(m): 10:12am On Mar 29, 2017|
Hello Mr. Anthropino,
Thanks for taking your time to reach out, I've been going through the articles on Statistical pattern recognition and Neural networks from a Statistical perspective; I've found it to be an interesting read though at a slower pace than I anticipated [I guess my mathematical knowledge are starting to betray me after a long time from academics] but I'm optimistic I would understand it hook line and sinker. I will get back to you as soon as I'm done with it. Cheers!
Have a beatific day!
|Re: The Future Of Machine Learning In Nigeria by NaijaTroops(m): 10:00am On Mar 31, 2017|
hello all, i also have interest in AI but all books i have seen do not show a way to translate the idea in to code, i personally know a bit of java and hoping to find something that implements the theory to practical. please any one know a good book to consult
|Re: The Future Of Machine Learning In Nigeria by anthropino: 7:08pm On Apr 01, 2017|
Hi @ Mr. NaijaTroops,
I don't really know your areas of research in Artificial Intelligence, so I hope this helps. First look at the quora discussion below :
Most researchers in AI focus on Machine Learning nowadays (as opposed to classic Artificial intelligence). In the last discussion with the early researchers in the field, I was told no useful research results has been produced in classic AI except for a renowned professor in MIT who died some years ago.
So that ,put simply, you will want to focus on Machine learning, Neural Nets and reinforcements learning ( and newly adversarial network).
Most Probably, you won't use JAVA. Among the Artificial Intelligence communities , JAVA is not that common, so far as I know--I might be wrong here. For example the recent NEON (https://www.nervanasys.com/technology/neon/) platform is pythonic. Others like Tensorflow,Caffe, Theano are definitely pythonic. Though there are Torch and CNTK for Microsoft which are not pythonic.
Therefore you need to refresh Python programming. A pdf file was provided above for python. You can follow this to refresh python.
There are other 2 pdf research papers attached in previous posst (kindly check the last 2 posts). You should definitely try to read those - even if you find them to be a heavy read, never mind just browse through to get the concepts.
Since you have a programming background,and you want to get some practice ,I humbly invite you to go through this github link and follow the materials:
I also attach the book that comes with the materials in the dropbox link just below.
You might also want to read previous post by @ Mr. Neahyo (just above).
dropbox link: https://www.dropbox.com/s/460joemh39xw7kf/Sebastian%20Raschka-Python%20Machine%20Learning-Packt%20Publishing%20%282015%29.pdf?dl=0
Hope this helps. Thanks sir.
|Re: The Future Of Machine Learning In Nigeria by uzoexcel(m): 12:02pm On Apr 22, 2017|
Words on marble
|Re: The Future Of Machine Learning In Nigeria by steinalb(m): 6:12pm On Apr 22, 2017|
I am really very interested in ML but have not seen any good java textbook on it.
Someone can please lend a helping hand.
|Re: The Future Of Machine Learning In Nigeria by HBola(m): 6:21am On Jun 11, 2017|
Hello. I'm interested in ML and I'm trying to put together a community around it. I'd like us to discuss further. Nairaland won't let me send you a message so Pls contact at firstname.lastname@example.org or telegram @SilverH I'll very much appreciate your time.
|Re: The Future Of Machine Learning In Nigeria by Ezechinwa(m): 8:59am On Jun 11, 2017|
Our Universities are still in the 17 century, cause machine learning should be something Africans are researching on, or are we waiting to do the usual? which is waiting for them whites folks to study, create machine learning devices snd then we import them!
we nees to up our game
|Re: The Future Of Machine Learning In Nigeria by calculator123(m): 11:51am On Jun 12, 2017|
neahyo:Boss please I would really like to tow this part...Show me the way sir
|Re: The Future Of Machine Learning In Nigeria by akalamoyo(m): 1:47pm On Oct 31, 2017|
Can Machines think?
I currently work as a Data Scientist with an insurance firm in UK just after completing my masters degree. I have a BSc in Mathematics and built up my programming skills while working as a Marketer in a Nigerian Bank. Machine Learning and AI has gained momentum in the UK and it cuts across many fields. I currently build algorithms from supervised to unsupervised in python. Algorithms that predicts claims, optimize price and all but there is much more out there.
The tech world as we know it is very dynamic. There are crazy developers and machine learning experts building 'badass' algorithms and selling to SMEs. From DataRobot to Google ML APIs, the available tools are increasing daily. As a beginner, I would advise you get familiar with basics and what Machine Learning entails bearing in mind you must be a lover of data, statistics and programming. But more importantly, it is important you begin to use the free tools to your advantage. Don't try to reinvent the wheel. There are probably a group of tech guys working on an algorithm you are thinking of building. If they've made it free or at a little cost, get it and use it; it most likely going to be better than yours even after all your sweat in building from scratch.
Unfortunately, there are not so much opportunities for machine learning experts in Nigeria at the moment and I'm not sure you get to study these courses in the University (I left OAU 2010 so things might have changed though). But I see it as an opportunity for the enthusiasts in this field. #TakeTomorrowToday
|Re: The Future Of Machine Learning In Nigeria by talk2hb1(m): 6:28am On Nov 01, 2017|
|Re: The Future Of Machine Learning In Nigeria by nijabazaar: 4:38pm On Nov 01, 2017|
After watching Sophia Demonstrate to Amina and thr Saudiz , I still think Artificial Intelligence is a phrase lifted from science fiction. Machine "learning" is a long way from anything that might be reasonably defined as intelligence. And there is no good reason to suppose that will ever change. Naming some interesting and potentially useful programming techniques AI is, in truth, nothing more than marketing.
|Re: The Future Of Machine Learning In Nigeria by nijabazaar: 4:44pm On Nov 01, 2017|
Nonetheless, I ve picked up Go...
A better description of the current state of artificial intelligence would be artificial decision making or, more accurate still, learned computer-aided decision making. The (pseudo-)sentient AI of science fiction is till a long way off.
|Re: The Future Of Machine Learning In Nigeria by nijabazaar: 4:46pm On Nov 01, 2017|
And again for Nigeria chances are that so many peeps will lose their jobs if Synthetic intelligence takes centre stage....pity
|Re: The Future Of Machine Learning In Nigeria by bukiii(f): 7:31am On Nov 04, 2017|
I am female who is interested in ml but getting started is my problem. Couldn't get any help from Google tho I I have been able to install few lib(numpy,pandas,ipython,and like other three).
|Re: The Future Of Machine Learning In Nigeria by 4kings: 8:18pm On Nov 04, 2017|
bukiii:You can check this or this(preferably if you want to go hardcore)
Just start if you have any issues you can google or ask here on Nairaland.
|Re: The Future Of Machine Learning In Nigeria by classicdude1(m): 8:59pm On Nov 04, 2017|
Check out this Machine Learning project that recognizes the color of objects.
|Re: The Future Of Machine Learning In Nigeria by bugzy84: 7:50pm On Jan 13|
|Re: The Future Of Machine Learning In Nigeria by bugzy84: 7:52pm On Jan 13|
PM me and I may be in a position to help with SAS 9.1 Base
|Sections: politics (1) business autos (1) jobs (1) career education (1) romance computers phones travel sports fashion health |
religion celebs tv-movies music-radio literature webmasters programming techmarket
Nairaland - Copyright © 2005 - 2018 Oluwaseun Osewa. All rights reserved. See How To Advertise. 287