Welcome, Guest: Register On Nairaland / LOGIN! / Trending / Recent / New
Stats: 3,143,325 members, 7,780,809 topics. Date: Thursday, 28 March 2024 at 10:58 PM

Artificial Intelligence And Machine Learning Group - Programming (4) - Nairaland

Nairaland Forum / Science/Technology / Programming / Artificial Intelligence And Machine Learning Group (38948 Views)

Are You Interested In Robotics And Artificial Intelligence? / Artificial Intelligence And Robotics In Africa. / Survey - People Interested In Artificial Intelligence And Machine Learning (2) (3) (4)

(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) ... (13) (Reply) (Go Down)

Re: Artificial Intelligence And Machine Learning Group by odizeey(m): 9:33am On Dec 28, 2017
Samcent:
From your r studio, enter the code: install.packages('rio'). After installing, load the package with the code: library ('rio'). You can also load your csv files using 'read.csv()' function. You don't need to load any package to use read.csv function. Make sure your data is in your working directory.
do in I Internet connection for this.
Re: Artificial Intelligence And Machine Learning Group by raymod170(m): 9:40am On Dec 28, 2017
Samcent:
Thanks for your contribution. Indeed a whole lot to do.

I just thought of one.... game developers would find this extremely useful as their work also would have been done 20% just by getting this dataset
Re: Artificial Intelligence And Machine Learning Group by raymod170(m): 9:48am On Dec 28, 2017
J0nyb0y:

None specifically, I guess I'll have to specify the various challenges and ways of Combating then in the course of the project work..

The issue am having is that i have researched online, didn't get much helpful materials. As I think the subject is not yet broad in Csc



hey I think your project isn't too technical....since you dealing with experts alone you can pick up for example your project supervisor as your case study and design a knowledge Engr product of him alone

I maybe wrong but if you said you can't see much information online then you can be the first to make ground breaking discovery in that field of ai
The field of Knowledge Engineering closely parallels some of the skills taught through NLP . There are obvious differences , but certainly at the core of what constitutes the NLP technology & skillset is a set of tools to model human knowledge & subjectivity . That very description would sound highly familiar to those who do professional Knowledge Engineering. In many ways they are two parallel fields . Knowledge Engineering is about building software systems to behave & ' think ' like human experts . In such a project , core software development team members would be tasked with acquiring knowledge from the experts , and then encoding that knowledge in a fashion that allows computers to reproduce the expert ' s behavior consistently and fast ! We can list more " similarities " between these two approaches to modeling than we can " differences . " But rest assured , both exist .
The Knowledge Engineer ( KE ) spends time interviewing & studying the experts , and learning directly from them. There are a variety of different methods which can be used to accomplish this , all grouped loosely under the term " Knowledge Acquisition . " Once a sufficient initial knowledge - base of modeled knowledge has been amassed , the KE then determines ways of storing /structuring / representing the expert ' s knowledge , and then proceeds to build a software application known as an expert system.
NLP practitioners & master practitioners may recognize the above process as being highly similar to what is known as Modeling in NLP . This goes beyond strategy elicitation and is used to logically reproduce behavior in a given domain of expertise . Occasionally we hear people thinking of ' reproducing a tennis swing ' or ' matching / mirroring ' someone else ' s behavior as ' modeling. ' It isn' t , don ' t let them oversimplify things . If they ' re suggesting such things , they categorically don ' t know what modeling is .
Re: Artificial Intelligence And Machine Learning Group by 4kings: 9:48am On Dec 28, 2017
J0nyb0y:

None specifically, I guess I'll have to specify the various challenges and ways of Combating then in the course of the project work..

The issue am having is that i have researched online, didn't get much helpful materials. As I think the subject is not yet broad in Csc
That might probably be because of the term "knowledge engineering" is not used very often.

SoftEng and Osarenomaspecial please come clarify this before i make any further suggestion.
What's on my mind is data engineering and I'm not so sure i'm correct.
Re: Artificial Intelligence And Machine Learning Group by J0nyb0y(m): 10:03am On Dec 28, 2017
raymod170:




hey I think your project isn't too technical....since you dealing with experts alone you can pick up for example your project supervisor as your case study and design a knowledge Engr product of him alone

I maybe wrong but if you said you can't see much information online then you can be the first to make ground breaking discovery in that field of ai

Thanks knowledge engineering has to do with experts in AI field..

The topic as about Combating Security Challenges in Knowledge Engineering Products...

So I guess I have to identify these knowledge engineering products, then the security challenges that they might face and then the ways of Combating them..

Any ideas would be great
Re: Artificial Intelligence And Machine Learning Group by raymod170(m): 10:09am On Dec 28, 2017
J0nyb0y:


Thanks knowledge engineering has to do with experts in AI field..

The topic as about Combating Security Challenges in Knowledge Engineering Products...

So I guess I have to identify these knowledge engineering products, then the security challenges that they might face and then the ways of Combating them..

Any ideas would be great

if you can identify them good else design one yourself using your lecturer or.... Bill Gates who is an expert in a certain field and amazed your lecturer by your progress while documenting your stages as this is all project is all about am preparing for mine too lol

for the security challenges I can't really say for now but maybe in the course of your working you might just find them out
Re: Artificial Intelligence And Machine Learning Group by J0nyb0y(m): 10:09am On Dec 28, 2017
4kings:

That might probably be because of the term "knowledge engineering" is not used very often.

SoftEng and Osarenomaspecial please come clarify this before i make any further suggestion.
What's on my mind is data engineering and I'm not so sure i'm correct.

Yeah, that's what am thinking too... The term is not really used in research works and journals so I've been looking into some AI research journals that might be helpful a lil..

Data engineering is a lil different form knowledge engineering... Knowledge has to do with expert AI systems.

But your suggestions would be appreciated tho... Please read the reply I gave above if youll have any ideas about it.


Cc
SoftEng and Osarenomaspecial
Re: Artificial Intelligence And Machine Learning Group by J0nyb0y(m): 10:11am On Dec 28, 2017
raymod170:


if you can identify them good else design one yourself using your lecturer or.... Bill Gates who is an expert in a certain field and amazed your lecturer by your progress while documenting your stages as this is all project is all about am preparing for mine too lol
Don't you think that would be like extra work load since I already have a goal I wanna reach... I wasn't asked to design a knowledge engineering system.

I was just asked to identify ways of securing the already designed knowledge systems
Re: Artificial Intelligence And Machine Learning Group by raymod170(m): 10:15am On Dec 28, 2017
J0nyb0y:

Don't you think that would be like extra work load since I already have a goal I wanna reach... I wasn't asked to design a knowledge engineering system.

I was just asked to identify ways of securing the already designed knowledge systems

you said you can't find resources am only suggesting that's all its difficult I know I myself would find it stressful but if at the end you find then discard else..... your choice mate
Re: Artificial Intelligence And Machine Learning Group by Samcent: 10:53am On Dec 28, 2017
odizeey:
do in I Internet connection for this.
To install a package you need Internet connection, but once installed, only have to load whenever you want to use which does not require Internet connection. But using the read.csv function doesn't not require any installing or loading, the function is available by default.
Re: Artificial Intelligence And Machine Learning Group by 4kings: 12:34pm On Dec 28, 2017
J0nyb0y:


Yeah, that's what am thinking too... The term is not really used in research works and journals so I've been looking into some AI research journals that might be helpful a lil..

Data engineering is a lil different form knowledge engineering... Knowledge has to do with expert AI systems.

But your suggestions would be appreciated tho... Please read the reply I gave above if youll have any ideas about it.
Your mention of "knowledge engineering product" is weird, i don't know what that means.

I'm not sure i quite understand raymod170 explanation, his explanation makes knowledge engineer seem like project managers for AI, again don't know if i'm right or wrong.

But what i had in mind was that knowledge engineering might have been used interchangeably for data engineering which is also a subset of data science.
So if you were trying to "combat security challenges with data engineering", then the next question which i asked would be what sort of security challenge.
It could be one that involves image processing were image data are extracted, processed(feature engineering) for stuff like weapon detection or suspicious movement alert. You can check this out
However it could be security challenges involving text data and bureacracy observation in the financial sector and so on.

So with data science you could combat some select security challenges.
But however i'm not sure if this is what your supervisor has in mind, that's why a good definition of Knowledge Engineering would be helpful.


However, Wikipedia says "Knowledge engineering (KE) refers to all technical, scientific and social aspects involved in building, maintaining and using knowledge-based systems."
and "A knowledge-based system (KBS) is a computer program that reasons and uses a knowledge base to solve complex problems."
I didn't really go over the wikipedia article to get a good idea, but i wonder how different this is from data science.
Re: Artificial Intelligence And Machine Learning Group by raymod170(m): 1:26pm On Dec 28, 2017
I just thought of two challenges but would wait for other contributions to see if am wrong or right before posting
Re: Artificial Intelligence And Machine Learning Group by odizeey(m): 1:28pm On Dec 28, 2017
Samcent:
To install a package you need Internet connection, but once installed, only have to load whenever you want to use which does not require Internet connection. But using the read.csv function doesn't not require any installing or loading, the function is available by default.
ok.
Re: Artificial Intelligence And Machine Learning Group by J0nyb0y(m): 1:48pm On Dec 28, 2017
raymod170:


you said you can't find resources am only suggesting that's all its difficult I know I myself would find it stressful but if at the end you find then discard else..... your choice mate
Thanks, I appreciate
Re: Artificial Intelligence And Machine Learning Group by J0nyb0y(m): 1:55pm On Dec 28, 2017
4kings:

Your mention of "knowledge engineering product" is weird, i don't know what that means.

I'm not sure i quite understand raymod170 explanation, his explanation makes knowledge engineer seem like project managers for AI, again don't know if i'm right or wrong.

But what i had in mind was that knowledge engineering might have been used interchangeably for data engineering which is also a subset of data science.
So if you were trying to "combat security challenges with data engineering", then the next question which i asked would be what sort of security challenge.
It could be one that involves image processing were image data are extracted, processed(feature engineering) for stuff like weapon detection or suspicious movement alert. You can check this out
However it could be security challenges involving text data and bureacracy observation in the financial sector and so on.

So with data science you could combat some select security challenges.
But however i'm not sure if this is what your supervisor has in mind, that's why a good definition of Knowledge Engineering would be helpful.


However, Wikipedia says "Knowledge engineering (KE) refers to all technical, scientific and social aspects involved in building, maintaining and using knowledge-based systems."
and "A knowledge-based system (KBS) is a computer program that reasons and uses a knowledge base to solve complex problems."
I didn't really go over the wikipedia article to get a good idea, but i wonder how different this is from data science.

I think knowledge engineering products has to do with these experts AI systems such as ATM etc... Haven't still gotten my head round the products tho, but that's what I think it will be.... That is examples of systems that make use of Knowledge Base... In different industries.

I have to identify these products before I can identify their security challenges, expect I wanna give general security relating to Artificial intelligence such as Cyber Security, data loss etc... So it all comes down to identifying the products and then taking a look on how they operate to identify their security challenges....

That's my general idea of the topic tho...

Thanks for your contributions so far
Re: Artificial Intelligence And Machine Learning Group by SoftEng: 4:03pm On Dec 28, 2017
4kings, raymod170, j0nyb0y
well done on the discussions.

I agree with some points raised by raymod170, however I'm not sure knowledge engineering can be viewed as a parallel of NLP (I may be wrong on this).

Also, I understand 4kings perspective as well as the perspective of raymod170. I'll try to bridge both by comparing / contrasting both. Please bear with me, may be lengthy.


TL; DR
Data Engineering is sub part of data Science with focus on acquiring data, as well as cleaning and analyzing data. The output of data engineering process is fed into a machine learning algorithm to produce a model (that contains implicit rules not directly programmed by humans) . It is part of modern AI toolset. Knowledge Engineering is a sub part of Knowledge Based System (a type of Classical AI) that focus on acquiring (and analyzing) and also manually encoding/programming knowledge from the data as rules (if statements). The rules are programmed directly by humans.
Re: Artificial Intelligence And Machine Learning Group by SoftEng: 4:15pm On Dec 28, 2017
Longer Version
The most common type of knowledge based system is an expert. Most times, this knowledge they talk about are encoded as rules (if... then... else... statements).

Note:
This rule-based structure of representing knowledge are similar to decision tree technique in Machine Learning. However, the main difference is that the rules in a decision tree (i.e. branches) are learned from your dataset, while the rules in the knowledge base of an expert system are explicitly design/encoded/programmed/specified by the programmer/knowledge-engineer.

The idea of knowledge engineering is part of Classical AI systems where Expert Systems (mostly rule based) was the most popular.

Knowledge Engineering Vs Data Engineering
Knowledge engineering is a sub part of Knowledge Based System just as Data Engineering is a part of Data Science. The both have some similarities and differences.

Similarities
1. They are both involved in data acquisition in the specific domain (e.g. health care, weather forecast, business etc) in their respective field. Knowledge engineer acquires data by: interviewing human expert with domain knowledge, (electronic) records and so on. Since the role of a data engineer (mostly data scientist as well) in exists more recently, they acquire data mostly from electronic records (especially from database and large data storage software tools like hardoop and spark) by data pipeline software built by themselves.


Differences
Once the data is acquired, their roles start differing.

1. Knowledge engineer starts analyzing the data, preprocesses it and then manually design the rules which will be programmed into the knowledge base. The knowledge engineer also has to think about how the knowledge is stored and represented in the knowledge base.

On the other hand, the data engineer (data scientist) "cleans" the data, analyze and visualise the data, removes unwanted features and then employs a machine learning algorithm that automatically produces a model (which can be viewed as implicitly defined rules) by learning from the data. I agree that some of the analysis process for data engineering is similar to knowledge engineering as well. However, I guess what differentiates them is the technologies/tools employed in the process. Software tools used at the time knowledge engineering was popular may not be same with what's currently being used in data engineering.
2. In recent times, there has been ongoing discussion about bias (e.g. race, ethnicity, gender, financial status etc) in AI algorithm. Therefore, in terms of biasness, here's what I can deduce (my personal opinion) :
Data Engineering mostly encodes bias from data (the dataset being learned from), however Knowledge Engineering encodes the bias of the data and that of the knowledge engineer.

Conclusion
To some extent, Knowledge Engineering is to Knowledge Base (and Expert Systems) as Data Engineering is to Models learned from data (Data Science / Machine Learning). Additionally, Knowledge Engineering is to Classic AI as Data Engineering is to Modern AI.

2 Likes

Re: Artificial Intelligence And Machine Learning Group by SoftEng: 4:27pm On Dec 28, 2017
I hope the above description clears the air with respect to knowledge engineering definition. If that's the case, then let's move on to the question by j0nyb0y.

The question is about Classical AI systems, which was specified as out of context of this thread based on the requirements defined at the beginning this thread. I have the impression that you did not know that discussions about knowledge engineering as a part of classical AI, therefore I'll let this one slide. We will continue the discussion to help you with your project.


One of the few examples of expert system we were taught in school is called MYCIN (for healthcare) you can look into security challenges in that area.

I'll try to do some online search about Security Knowledge Based systems later in the day (or tomorrow) and report back here.

Here's a interesting idea that comes to mind.
If you finally identify a security challenge, you can design another knowledge based system or machine learning system that combats the challenges. I don't know how it's going to work, but it may be interesting.

Thank you all.
Re: Artificial Intelligence And Machine Learning Group by wane01(m): 6:12pm On Dec 28, 2017
oh mehn ave never really played with AI just programming only...... please who can recommend sites and tools for AI

1 Like

Re: Artificial Intelligence And Machine Learning Group by J0nyb0y(m): 6:13pm On Dec 28, 2017
SoftEng:
I hope the above description clears the air with respect to knowledge engineering definition. If that's the case, then let's move on to the question by j0nyb0y.

The question is about Classical AI systems, which was specified as out of context of this thread based on the requirements defined at the beginning this thread. I have the impression that you did not know that discussions about knowledge engineering as a part of classical AI, therefore I'll let this one slide. We will continue the discussion to help you with your project.


One of the few examples of expert system we were taught in school is called MYCIN (for healthcare) you can look into security challenges in that area.

I'll try to do some online search about Security Knowledge Based systems later in the day (or tomorrow) and report back here.

Here's a interesting idea that comes to mind.
If you finally identify a security challenge, you can design another knowledge based system or machine learning system that combats the challenges. I don't know how it's going to work, but it may be interesting.

Thank you all.

I really appreciate your effort in breaking it down... Thanks for allowing the discussion continue..

Ive been able go grab some cogent and important points from your writeup... From the expert system example you gave, I did a lil more research and found out some other expert systems also...

Which I can also term as knowledge engineering products...I'll begin to look in to them individually so I can be able to identify some of their security challenges and ways the knowledge engineers could have done it better..

On designing a new expert systems for each security challenge might be cumbersome for different of these expert systems and I might begin to go off point from what my supervisor intended... I might just write another kA Stroud grin

What am thinking is I'll just propose a model of ways of Combating them.

I'll await the results of your research..

Thanks so much.
Re: Artificial Intelligence And Machine Learning Group by raymod170(m): 7:45pm On Dec 28, 2017
J0nyb0y:


I really appreciate your effort in breaking it down... Thanks for allowing the discussion continue..

Ive been able go grab some cogent and important points from your writeup... From the expert system example you gave, I did a lil more research and found out some other expert systems also...

Which I can also term as knowledge engineering products...I'll begin to look in to them individually so I can be able to identify some of their security challenges and ways the knowledge engineers could have done it better..

On designing a new expert systems for each security challenge might be cumbersome for different of these expert systems and I might begin to go off point from what my supervisor intended... I might just write another kA Stroud grin

What am thinking is I'll just propose a model of ways of Combating them.

I'll await the results of your research..

Thanks so much.

good am happy you have found some resources get busy and keep us updated overtime
Re: Artificial Intelligence And Machine Learning Group by 4kings: 8:05pm On Dec 28, 2017
Great explanation as usual from SoftEng. smiley
Just checked MYCIN out and it was interesting and the process also explained a lot.
Re: Artificial Intelligence And Machine Learning Group by SoftEng: 1:33am On Dec 29, 2017
4kings, j0nyb0y
Thanks.

Additionally, thanks 4kings as you have contributed a lot in the group already.
Thanks to raymod170 and Samcent as well.
I'm hoping that others in the group will become more active (maybe after Christmas and New Year celebration).

Please, we need more people to participate. It does not have to be questions and answers alone. If you have some interesting project/research, please feel free to share.

Thanks all.



Dmayor7, osarenomaspecial, unphilaz, shams040, lum1, kelvine95, rayval, Ibroh24, BunchObute, NaHiim, EbonyMira, themonk7, Unmarshalled, Omoadeola, lydia4211 and others

Hope you are all fine?
Re: Artificial Intelligence And Machine Learning Group by raymod170(m): 1:45am On Dec 29, 2017
stay blessed

am awake working on the FIFA data set to achieve one or two of the tasks I mentioned earlier today
Re: Artificial Intelligence And Machine Learning Group by Samcent: 8:03am On Dec 29, 2017
raymod170:
stay blessed

am awake working on the FIFA data set to achieve one or two of the tasks I mentioned earlier today
Please, do keep us updated. I will be glad if we can come up with a useful data product.
Re: Artificial Intelligence And Machine Learning Group by odizeey(m): 3:36pm On Dec 29, 2017
raymod170:
stay blessed

am awake working on the FIFA data set to achieve one or two of the tasks I mentioned earlier today
weh done sir, I anticipating your work. My problem now is my pc, anaconda is not loading, my pc is damn slow. I have minitab, but I'm not sure it can do the things R, Python, SAS, can do.
Re: Artificial Intelligence And Machine Learning Group by raymod170(m): 9:00pm On Dec 29, 2017
odizeey:
weh done sir, I anticipating your work. My problem now is my pc, anaconda is not loading, my pc is damn slow. I have minitab, but I'm not sure it can do the things R, Python, SAS, can do.

OK cool but I think I have a small problem

I can access the data but manipulating the csv data to achieve what I want to achieve is my problem

am thinking if I can convert the FIFA dataset to sql would solve the issue but then I tried using phpmyadmin to achieve this am the records in that file exceeds the required record that can be imported so if anyone can help me convert this FIFA dataser to an sql file then my problem would be solved thanks in advance

or better still offer a more suitable solution to manipulate the data to show record from a specific datarow when called randomly from the algorithm
Re: Artificial Intelligence And Machine Learning Group by odizeey(m): 9:17pm On Dec 29, 2017
raymod170:


OK cool but I think I have a small problem

I can access the data but manipulating the csv data to achieve what I want to achieve is my problem

am thinking if I can convert the FIFA dataset to sql would solve the issue but then I tried using phpmyadmin to achieve this am the records in that file exceeds the required record that can be imported so if anyone can help me convert this FIFA dataser to an sql file then my problem would be solved thanks in advance

or better still offer a more suitable solution to manipulate the data to show record from a specific datarow when called randomly from the algorithm
I don't know how to manipulate dataset o. I'm still amateur.
Re: Artificial Intelligence And Machine Learning Group by Samcent: 10:56pm On Dec 29, 2017
raymod170:


OK cool but I think I have a small problem

I can access the data but manipulating the csv data to achieve what I want to achieve is my problem

am thinking if I can convert the FIFA dataset to sql would solve the issue but then I tried using phpmyadmin to achieve this am the records in that file exceeds the required record that can be imported so if anyone can help me convert this FIFA dataser to an sql file then my problem would be solved thanks in advance

or better still offer a more suitable solution to manipulate the data to show record from a specific datarow when called randomly from the algorithm
I wish I could help, but I am only familiar with r. R has an excellent data wrangling package. Do you want to get a subset of the data?
Re: Artificial Intelligence And Machine Learning Group by SoftEng: 11:49pm On Dec 29, 2017
raymod170:


OK cool but I think I have a small problem

I can access the data but manipulating the csv data to achieve what I want to achieve is my problem

am thinking if I can convert the FIFA dataset to sql would solve the issue but then I tried using phpmyadmin to achieve this am the records in that file exceeds the required record that can be imported so if anyone can help me convert this FIFA dataser to an sql file then my problem would be solved thanks in advance

or better still offer a more suitable solution to manipulate the data to show record from a specific datarow when called randomly from the algorithm

You can try one of the following:

1. if you can use python, then I'll suggest you checkout pandas library. I believe it meets the requirements of what you are trying to achieve. Even though I haven't used extensively, I believe it's a mature and easy to use library.
2. check for online converter "csv to sql". (I don't know if such converter exist, but it's worth the try)
3. opening the csv file in excel, save to microsoft access format, and then check if you can convert the data to sql from microsoft access.
Re: Artificial Intelligence And Machine Learning Group by J0nyb0y(m): 11:59pm On Dec 29, 2017
raymod170:


good am happy you have found some resources get busy and keep us updated overtime
Yeah, sure. Thanks cheesy
Re: Artificial Intelligence And Machine Learning Group by J0nyb0y(m): 12:00am On Dec 30, 2017
4kings:
Great explanation as usual from SoftEng. smiley
Just checked MYCIN out and it was interesting and the process also explained a lot.
Really interesting.... I believe I'll be able to get something from it and others I found out too

(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) ... (13) (Reply)

Post Ur Vb 6.0 Questions Here / Java Programming For Dummies / Nigerian Software Engineer given test to prove he is an engineer at JFK Airport

(Go Up)

Sections: politics (1) business autos (1) jobs (1) career education (1) romance computers phones travel sports fashion health
religion celebs tv-movies music-radio literature webmasters programming techmarket

Links: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

Nairaland - Copyright © 2005 - 2024 Oluwaseun Osewa. All rights reserved. See How To Advertise. 102
Disclaimer: Every Nairaland member is solely responsible for anything that he/she posts or uploads on Nairaland.