₦airaland Forum

Data Analyst Project Example for Portfolio

This is provide ideas, guidance and inspiration on those hoping to embark on personal project for their Data analyst portfolio

Nairaland Politics project
Background of the project:
Nairaland is a Discussion forum where people come and share their thoughts on different topics.
The site is divided into categories and sections to guide the users.

Scope of the project:
This project shall focus on the first 300 pages of the politics section of nairaland.com

Project Questions:
1 What is the topic with the highest number of views
2 What is the topic and original Poster of the topic with the highest number view
3 What is the topic and original Poster of the topic with the highest number of comments or updates
4 Which post was last updated and when in the 300 pages scraped or oldest in terms of update
5 Who are the most active users in the politics section of Nairaland and how many topics did they create
6 Which month had the highest number of Headings
7 Any other question that can help us understand the data very accurately

Project Task:
task 1 : Write a python script to scrape 300 pages from politics category of nairaland.com
task 2 : using necessary libraries clean and analyze the data
task 3 : present your result using suitable visualization

Skills demonstrated in the Project :
1 Web scraping
2 Regular expression usage
3 Writing to a file
4 Data manipulation
5 Data cleaning
6 Data visualization
7 Research
8 Documentation

Column name definitions for the project :
1 Heading : The title of the thread
2 Poster : The person that created the thread
3 Updated : How many times contributions have been added to the thread
4 Viewed : How many people opened the thread to read the content without contribution
5 Time : The time and date the last contribution was made on the thread
6 Last_Updated : The lasted person to make contribution to the thread

https://github.com/brightonu/portfolio_projects/blob/main/politics_project_Final.ipynb

Part 9

Course 9 : Big Data, NoSQL, Big Data Ecosystem Tools and Frameworks and Cloud Services

Big Data: This is the term that describes the large volume of data and this data can be structured or unstructured. These type of data is been generated by businesses on daily basis. Big data can be analyzed for insights that lead to better business decisions and great business strategic moves.
The key concepts guiding Big data are
1 Volume
2 Velocity
3 Variety
4 Veracity
5 Value

NoSQL: [/b]This is database that provides a mechanism for storage and retrieval of data that is modelled in a different method other than the tabular relations used in relational databases. You do not need to define the structure of the database before its usage.
Example of NoSQL Databases are:
1 Google MongoDB
2 Apache Cassandra
3 Amazon DynamoDB
4 Apache CouchDB etc

[b]Big Data Ecosystem Tools and Frameworks: There are several tools and frameworks for handling and processing big data. It takes care of all the stages needed to effectively process and analyze your big data until desired result is achieved.
Examples are:
Apache Hadoop, Apache Spark, Apache Hive, Apache HBase, MapReduce, Apache Impala etc

Cloud Services and Computing: Considering the volume and requirement for handling and processing big data, Having the knowledge of cloud services and computing might be needed in your tool box to effectively navigate the big data field.

.......To be Continued.........

emperor03:
Bro, I can't find the links to the courses you spoke about. Can't you please address this. Thank you

Please use the Course Title and search for the material. Most of the materials are available in edx.org

THE BEGINNING OF DATA SCIENCE JOURNEY

Part 8

Course 8: - Mathematics, Statistics and Probability

Mathematics: Some people love it while others hate it with passion. It was taught in Nursery school, Primary school, Secondary school and University and yet most people just memorized formulas to pass their exams.
That narration is about to change because behind every mathematical formula or concept is a solution to the problems of humanity.
Do you know that linear regression is based on Gradient or Slope in mathematics while business model optimization is based on Differentiation in Calculus and many others like that. Your duty is to find out what models are related to what topics. You will study the principle of the topic and how it relates or affects your models.

Things to learn in Mathematics
1 Linear Algebra
2 Calculus
3 Discrete Mathematics
4 Graph Theory
5 Information Theory etc

Statistics: The field that deals with data and how they are used. This is one field you must be a master in to make head way in your career as a data scientist.

Things to Learn in Statistics

Everything is needed to be good at what you are doing, from inferential statistics to hypothesis formulation and testing, Principal component Analysis, Data collection methods, Population size, Interpretation of statistical results and many other things. You must be very good in statistics to be a good Data scientist.

Probability: the science of uncertainty. whenever there is any doubt about an event occurring, probability is in place. The concept of probability is involved to estimate the likelihood of an event occurring.
Example: It is 40% likely that it will rain tomorrow, if it rains tomorrow, you will be fine and if it did not rain you will still be fine since you have said the likelihood of raining is 40%.

Things to learn in Probability
1 Probability space
2 Random variable
3 Probability rules
4 Expectation
5 Variance and Covariance
6 Probability Distribution etc

Note: Mathematics, Statistics and Probability are the backbone of any model you hope to use. You must master them to be successful.

Key takeaways
You will be most effective working as Data analyst while studying to become a Data Scientist
Studying for Data science without working real time with data will not yield much for you.
70% to 80% of your role as Data scientist will involve data cleaning and preparation.
Know data cleaning, data wrangling and data preparation and know peace as a Data scientist

......To be Continued......

dfo12:
Thank you for this tread. I am a Chemical Engineer with no background knowledge of programming. I started learning Python on my own using some videos and printed text. I really want to learn programming to become a data scientist. Do I continue with the Python programming, or there's something fundamental I need to learn first before moving?

Please follow the instructions on this thread. Python is not the first course to start with.

Part 7
The Final part for data analyst

The Summary summary of Data Analyst journey

What to learn
1 The Data ecosystem
2 Spreadsheet Applications
3 SQL and Databases
4 Descriptive Statistics, BI and Visualization
5 Computational Thinking, Problem solving with computer and Programming Language
6 Personal Portfolio is a must

What you should be able to do
1 Ability to solve data related problems
2 Extract, Transform and Load data
3 Descriptive analysis
4 Diagnostic analysis

When to seek for help or give up
After you have studied all the content and you are given a dataset and you don't know what to do or how to start, please just humble yourself and seek for help. If the help did not work, then, the journey is not for you.

My final word to the Data Analyst
It is an interesting career path with plenty reward but there are challenges on the way. You must become a friend to constant study.

THIS IS THE END FOR THE DATA ANALYST

THE BEGINNING OF DATA SCIENCE JOURNEY


..... To be Continued...........

Part 6
Course 6 : The application of our Data analyst knowledge to a task

The Role, Task and activities of a Data Analyst using Nairaland personal account as case study 1

Client Question
I need the summary and detailed account analysis of lovelybobo on Nairaland

Data Analyst Approach to the problem

Let us assume we have the correct and right dataset in our hands to respond to the client

Data analyst personal Questions that will lead to insight

Question 1 - How long has lovelybobo spent on Nairaland
Question 2 - When did lovelybobo register on Nairaland
Question3 - The time he has spent, can it be arranged in hours, days, weeks, months, quarter etc
Question 4 - What section does he spend his time
Question 5 - What is the average time spent on each section
Question 6 - How many topics did he create
Question 7 - How many thread did he comment on
Question 8 - Which of his thread is the most popular
Question 9 - What is the total number of days he will spend at the end of 2022
Question 10 - Has be been banned before and for how long
The questions can go on and on

Answers to the Questions and the insight they provide

Answer 1 - This question will require you to aggregate a column using SUM FUNCTION and the answer will be place on the Summary page of the DASHBOARD. It is a main feature of the result

Answer 2 - This should be easily located on the Date column of your dataset since you must register to post a thread, this will also be on the DASHBOARD because it is major indicator

Answer 3 - This task requires you to perform a calculation that involves conversion, minutes to hours, hours to days, days to weeks depending on what you want your final output to be.

Answer 4 - this will require a BAR CHART OR PIE CHART and AGGREGATION OF Time GROUP BY Sections

Answer 5 - AVERAGE AGGREGATE will be performed on each section

Answer 6 - COUNT FUNCTION will be used and parameter to identify created topics figured out

Answer 7 - COUNT FUNCTION will be used and parameter to identify comments figured out

Answer 8 - SORTING OPERATION will be performed on the list of his thread

Answer 9 - Predictive analysis

Answer 10 - DOMAIN KNOWLEDGE about Nairaland will be needed to answer this question.

Note

Some times your task may require you to scrap a web page

Python Libraries for web scrapping are
Request
BeautifulSoup
Selenium
Scrapy
and many others

......To Be Continued........

munstaqq:
Between Power BI and Tableau, which is advisable to learn?

God bless you for starting this thread. I'm grateful.

Power Bi and Tableau are very good.
This is my personal recommendation, If you can, please learn the two. Once you get a job you focus on the one the company is using.
Learning the two will double your opportunities

Finally, You can conduct your personal research and decide on what to learn

Part 5
Course 5 - Computational thinking and Programming
Prerequisite or Optional
1 Computational Thinking
2 Problem Solving with Computer

The above two courses will bridge the gap for Non Technical, and other people who wants to go into programming. It will lay a solid foundation for you and open you up to begin to think problem solving and computationally.

Recommended Study material
CS50 from Harvard University and many others in that category

Main Course
1 Python
2 R

Major Difference between Python and R
R was developed by statisticians to help them in their statistical analysis and till date R is most suited statistical analysis.

Python was developed as an automation tool kit to help the developer and over time many people have contributed a lot of Library, packages and modules to make python great. Python is great for Data Science, Machine Learning, AI and Automation.

Details of What you will learn in Python
Python basics, Numpy, Scipy, Pandas, Matplotlib, seaborne etc

Personal project and Portfolio
You must build personal portfolio by solving problem with different datasets to showcase your skills. my personal recommendations are
2 projects on Excel
2 projects on SQL
2 projects on Power Bi and/or Tableau
2 projects on Python
Note: the more projects you have the better for your career

Major Job Titles
Data Analyst, Business data analyst, Finance data Analyst among others

Major certifications to consider
Google Data Analytics Professional Certificate
IBM DATA ANALYST PROFESSIONAL CERTIFICATE

Non technical Skills to have
1 Communication
2 Presentation
3 Curiosity
4 Business acumen
5 creative and critical thinking etc

...To be Continued.....

Part 4

Course 4 - Statistics, Business Intelligence and Visualization

Optional or prerequisite
1 Introduction to Statistics
2 Descriptive Statistical Analysis

The above is an optional course depending on your background and previous studies

Main Course
Power BI
Tableau
QlikView
Looker
Cognos

4 Categories of Data Analytics
1 Descriptive analysis - What happened in the past
2 Diagnostics analysis - Why did it happen
3 Predictive analysis - What is likely to happen in the future
4 Prescriptive analysis - Recommend actions we can take to affect the future outcome

Level of Statistics you will need now
You must understand descriptive statistics very well to be able to relate very effectively with your dataset. Things you must cover in your basic statistics knowledge are
measure of central tendency like mean, mode, median, Average, Variance, Range, Standard Deviation, Discrete and continuous variable, Qualitative and Quantitative variable, correlation etc

BI and Visualization Details
Microsoft Power Bi is the most popular, followed by Tableau and QlikView. Looker from Google and Cognos from IBM are in the emerging market based on Gartner 2021 report.
You decide what you want to learn and why

Power Bi can be subdivided into 3 - Visualization and Dashboard, M Language and DAX, It is powerful and can connect to different data sources.
Tableau is believed to be an intelligent BI and visualization tool with its grouping of variables into measures and dimensions. It has support for auto suggestion of charts and enforcing industry best practices.
QlikView is good and worth looking at.

What you will study
Measures, Calculated columns, data cleaning, data preparation, different visualizations and their use cases, merging datasets, building models etc.

Major Job Titles
Business Analyst
BI Analyst

Key Takeaways
You must understand all the 19 most common KPI for business
You must understand the most basic charts and their use cases.

....To be Continued.......

part 3

SQL and Databases

SQL is the language you use to communicate with databases and since most of the datasets you will be working with will either be file format which Excel can handle or reside in a database in which SQL is needed.

what to learn

SQL for Microsoft SQL Server
SQL for Any open source database server
SQL for any Big Data database

The SQL required for each database is slightly different but the concept is the same.

Note
The are four categories of programming skills available
Basic
Intermediate
Advanced
Expert

About 80% of the learning materials you will come across will provide you with the Basic knowledge you will need, while 10 to 15% others will give you some intermediate skills.

what you learn in Basics

SELECT, FROM, WHERE, GROUP BY, HAVING, ORDER BY, LIMIT and a lot of AGGREGATE FUNCTIONS.

How to progress
Once you have mastered the basics, you can join forum for SQL and Databases or register in an online practice site where you can begin to attempt real business questions and see different business scenarios and then continue with your learning by solving those problems.

Personal Portfolio
You must begin now to build personal portfolio to showcase your progress

Key takeaways
Repeat the task you did in Excel using SQL and see which one is easier
SQL is the second most important skill you will need and must have after COMMUNICATION

....To be Continued....

Recommended materials for learning spreadsheet

Any Good learning material for Excel must meet the following conditions
1 Create a business problem or scenario
2 Provide a dataset for the business problem
3 Guide you to solve that particular business problem

Can I get a Job with only Excel?

The answer is YES but you will need to be a master in Excel.
The available job titles are
Excel Expert adviser, Excel Trainer, Data entry operator, Data Validation operator, Data Correction and checking operator among others.
You can enhance your current role with advanced Excel skills.

Key Takeaways

You must pay close attention to whatever thing you are doing in Excel and observe their outcomes carefully because you are going to do the same thing in SQL, Python, R etc

Part 2
Course 2: Spreadsheet Application
1 Microsoft Excel
2 Google Sheet
The above are the most popular spreadsheet applications out there in the market. They are very similar and the knowledge of one can help you learn the other very easily.
If you plan to restrict your work to Nigeria and Nigerians, You don't need to learn Google sheet.

Details of the course

1 You will learn data entry and validation, sorting, filtering, conditional formatting among other things
2 You will see data in a very big flat table for the first time
3 You will learn functions for different categories, industries and domains and how they are used.
4 You will learn different techniques for data cleaning and preparation
5 You will learn how to prepare reports, charts, dashboards and many more
6 Excel is very wide and deep and can be enhanced further by the use of plugins
7 You will learn syntax for writing functions and codes with autocomplete on your side
8 You will learn how to nest functions, different lookups and their use cases
9 You will learn how to summarize data with pivot tables
10 Excel VBA is a course on it's own inside Excel
11 Statistical Analysis is a course on it's own inside Excel with it's own plugin and many more
12 The major limitation of Excel is that it can only accept 1,048,576 rows by 16,384 columns by default
13 Microsoft has broken that limitation by introducing Power Query and Power Pivot in Excel

Introduction
This thread is intended to provide a comprehensive guide and proper direction for anyone aspiring to pursue a career in the data analytics field. it will help you to understand where you are, how to start and where to start from.
Most of the things will be based on my personal research, teaching and work experience among others.
I will make time to be updating it until we are done.
Where to start
I will suggest you start from Course 1 : Let me call it
Understanding the Data Ecosystem
You should cover the following in this course
1 Definition of data related terms
2 Data sources and data format
3 Tools used in data ecosystem
4 Frameworks available in the data ecosystem
5 Assessment of where you are in relation to where you want to go
6 The most important skill sets you must have
7 Various domains and how they use data
8 Data related job Titles and their requirements
9 And many more like Business metrics, KPI etc
This is a purely theoretical class and will prepare your mind on what is ahead of you

Recommended Study Materials

1 https://www.coursera.org/learn/foundations-data?specialization=google-data-analytics
2 https://www.coursera.org/learn/introduction-to-data-analytics?specialization=ibm-data-analyst
3 Any other course that looks like any of the above listed courses will be a good starting point, It will help both technical and non technical people

Duration of Study
1 to 3 weeks depending on your speed

Key takeaways
1 Broad understanding of what you are going into
2 The relevance of your existing degrees and your domain knowledge as it relates to data
3 Early Decision on domain of interest
4 The exact requirement for your desired job title
5 And many more

...... To be Continued.......

wyt:
can you name where ones can get projects to work on?

You can get datasets from the links below depending on Industry of interest.

1 https://www.pewresearch.org/download-datasets/
2 https://www.makeovermonday.co.uk/data/

Always pick an industry, To gain domain knowledge

Ulunne777:
@ Kshe and all, I'm pleased to meet this thread.I'm a female and a working mom but I'm challenging myself to learn Data science. I study mostly between 12am to 4am when network is better and no distractions from world ppl.
Been brushing up with excel for a while and ready to delve into SQL.

I'm enthusiastic about this journey. Thanks for This thread .
Pls what's next after sql or what is the order?

Please note that practicing a particular skill is very important to your learning journey. It will give you the opportunity to rate yourself and skill level. they are four levels
1 Beginner
2 Intermediate
3 Advanced
4 Expert
The links below can help you practice and also solve real life data problems from different companies and industries.
Focus on one or two industries to move quicker in your journey.

1 SQL AND PYTHON
https://www.hackerrank.com

2 https://www.leetcode.com

3 https://www.stratascratch.com/

4 https://excel-practice-online.com/

These are rules that have helped me to learn new things very quickly and progress very fast

1 No single learning material will give you everything you want. I therefore recommend you get 3 to 4 materials on the same topic or subject matter from different sources and learn with all together at the same time.

2 Assuming you want to learn VLOOKUP in Excel, Get materials on the topic from 3 or 4 different people or sources and get their individual perspectives. People teach based on their knowledge of the subject matter and communication skills.

3 Any learning material that has no real life dataset is a NO NO for me.

4 Once you have mastered a topic or subject build something with it and add it to your portfolio.

5 Get a Hard cover notebook, write codes, steps, commands and functions used in building the portfolio or solving a particular problem, They will help you in real work and job interview preparation.

6 With this method I realized that no matter the dataset, while using python, you will need about 20 to 25 functions and attributes to understand a dataset. with time these functions and attributes become second nature to you and you can work very fast.

It has helped my own learning process

Uncharted56:
I don't want to use virtual services I need a physical server with RIAD cos I work with large files, but thanks for the input

Please check Ebay, i am sure you will get used servers that are very good and meet your specification also.

Uncharted56:
Hello fellow programmers of niaraland I'm looking to buy a fairly used server or used one in good condition at affordable prices because I have some extremely private projects that require server technology to run, is there anyone here who has legitimate information on where to buy one? Also i have already installed solar panels so electricity is not an issue neither is internet connectivity. Any input will be greatly appreciated.

Ps: my budget for the server is between 90k for one with at least 8gb ram, clock speed of 2.4ghz and core i5 consumer based server for small business

In this era and time, how come you did not consider cloud services
Amazon web services
Azure
Google Cloud and others
It is pay as you use

please check them out

Femmyfestus:
I am searching for a place or someone that can teach me programming around lugbe or wuse.
I'm particular interested in datascience / database management and website development.

mrLj:
I'm also interested in artificial intelligence and data science.
I stay in Abj also.

If you stay in Abuja and need a mentor, tutor or learning materials for the following courses listed below, please reach out to me.
1 Introduction to Microsoft Excel
2 Data Analytics and Visualization using Microsoft Excel
3 Data Analytics and Visualization using Microsoft Power BI
4 SQL for Data Analytics
5 Business Metrics for Data Analytics
6 Story Telling with Data
7 Statistical Data Analysis with Microsoft Excel
8 Python Programming for Data Science

Remote training and mentoring is available.
Contact on my signature

Gracehayford1:
Hello please I am interested but I stay in port harcourt quote author=lovelybobo post=101254118]

If you are ready and serious, I can mentor you remotely.
Get in touch with me, let me know your level

Contact is on my signature

Kaydon001:
I am interested..how do I get in touch

My contact is on my signature

Remilekun187:
Good day to you all. Please, I'm here to seek your candid advice. I recently became a data analyst enthusiastic. I have performed some tasks using some excel tools(Vlookup, pivot) and also did some udemy courses yet, I feel empty. I had my first and second degree in Physics Electronics and Communication Technology respectively. I've tried learning python via some YouTube videos, but I'm not satisfied watching videos without real life projects. Recently, I've been contemplating between going for another degree in data analytics and running some professional certification in data analytics. My challenge is, I really do not know what the best option is. All I want is to learn and be confident that I can handle tasks successfully. I'll appreciate all useful suggestions.

Data Analytics is a context based learning and any material that does not define its context will not fulfill your yearning. People from different industry use the same data differently depending on what they are looking for.
from my personal experience, if you have a learning material that does not include dataset and the context of that dataset, it will not achieve much for you.
Since you have your degrees, try and get skills needed by engaging in very good learning materials with dataset and context defined.
Google Data Analytics Professional Certificate and IBM Data Analytics Professional Certificate are offered by Coursera and I am sure they will be good for you. Udacity and DataCamp have similar courses.
Above all, get real dataset yourself and work on it while acquiring your skills.

All the best

Schoolhike:
Hello, I stays in Abuja, how do we link up??

I’ve sent you a dm

I have responded to your pm.
My contact is on my signature.

Igengo:
Hello, Please I would so much appreciate it if you could mentor me remotely.
Thanks

You can get in touch with me and please follow this procedure
1 Introduction
2 Your level of education
3 Course of study
4 ICT knowledge if any
5 Your motivation

Thanks

Reccoxxy:
Hello everyone, please anyone who can mentor here in Lagos ?

If you are ready and serious, I can mentor you remotely.
Get in touch with me, let me know your level

If you stay in Abuja and need a mentor, tutor or learning materials for the following courses listed below, please reach out to me.
1 Introduction to Microsoft Excel
2 Data Analytics and Visualization using Microsoft Excel
3 Data Analytics and Visualization using Microsoft Power BI
4 MYSQL for Data Analytics
5 Business Metrics for Data Analytics
6 Story Telling with Data
7 Statistical Data Analysis with Microsoft Excel
8 Python Programming for Data Science

Do you stay in Abuja and wants to acquire any of these Business Analytics skills like

1 Introduction to Microsoft Excel

2 DATA ANALYTICS AND VISUALIZATION USING MICROSOFT EXCEL

3 DATA ANALYTICS AND VISUALIZATION USING MICROSOFT POWER BI

4 SQL FOR DATA ANALYTICS

5 BUSINESS METRICS

6 STORY TELLING WITH DATA

7 STATISTICAL ANALYSIS WITH EXCEL

personal and corporate training is available

Reach out to us. Contact is on my signature

hexzelle:
i'm not sure if that pix is clear
so i have uploaded the file also attached

if one opens that file in excel, there are about 25 number sets there , if one observes carefully one would find out that each set contains a number between(30-32=100%) , i.e all 25 sets have either 30, 31, or 32
while 24 sets contains between (20-22=95%).
and also 24 sets contain between(21-23=95%).
all three are a range of 3 numbers.

but also all 25 contain(20-23=100%) which is a range of 4 numbers i.e 20, 21,22,23

my question now is if there is a function on excel or another software, where i can enter different sets like this, and choose either a 2 NO range, 3 NO range or 4 NO range and then it would give me the range of numbers that is present in all the sets i.e (100%) or a percentage basis of the ranges automatically.

If I understand your question clearly, In Microsoft Excel you can solve the problem using IFS or Nested IF depending on the version of Microsoft Excel you are using.
All the best

Viola2017:
Please I need a tutor that will put me through on mySql and tableau.
My location is Abuja

I can Tutor you on the following
1 Mysql
2 Excel
3 PowerBi
4 Statistics
As for Tableau, I can get you any material of your choice to study on your own from Coursera.
I stay in Abuja and my contact is on signature. If interested, please get in touch with me. Thanks