₦airaland Forum

I've been into analytics like for ever. I began analytics while in school when I started playing around with Matlab. My colleagues and lecturers started giving me their research work to analyze their data. My interest into data science got spiked after I adopted ANN to model the start-up phase of a gast turbine for my masters project using data from Afam power plant.

I began making meagre earnings for my self from students and lecturers who often gave me their project data to work on and analyze. Mind you, prior to this I was actually teaching and earned about 20,000 naira as monthly salary. Afterwards, I realized teaching in secondary school was a complete waste of my time.

I decided to pursue analytics fulltime and I must confess that the skill puts bread on my table.
Although, I'm yet to get a full time data analytics role I have worked on projects I would never have dreamt of working on and lots of opportunities have been coming my way

I have always wanted to become a data engineering expert because, I have been nursing the dream of establishing an enterprise analytics department for the government and developing a real time Plant condition monitoring tool. With time more ideas of projects/strat-ups would pop up in my head 😌😌

The decision to take the leap into data engineering came about when one of the clients I worked with in the past asked me if I could automate the movement of data from Azure active directory to snowflake and then to Tableau. During this period I
resigned from working with a colleague of mine whom we often do data science projects mostly for foreign Nigerian students. Doing the student projects was really time consuming as it was difficult balancing combining both. I even ran into problems trying to do it as the quality of the assignment I did for three students was really poor(6 assignments in total). This was because my attention span was divided

Connecting Snowflake to Tableau with the aid of an ODBC driver was hassle free but I couldnt figure out how to create a pipeline to connect Azure data logs to Snowflake

The time frame of the project my client asked me if I could do elapsed but this was what drew me in to taking DE serious.

Due to the restriction on Nigerian debit cards, I couldn't use it to secure the 200 dollar Azure free credit and my cousin residing abroad tried to help but pulled out after he became suspicious when I asked him to use his phone number to complete the Azure free trial registration process.

One thing led to the other and the client that previously reached out to me stepped in to support me with Azure standard monthly subscription of 100 dollars

Let's work and build gpt for nairaland with LLM


You'll be in charge of the data scraping

DyingFetus:
Let's work and build gpt for nairaland with LLM


You'll be in charge of the data scraping

Let's connect on WhatsApp
Zero eight 1 three six three six five 6 zero three

Nice one OP.Following

My client tasked me to familiarized myself with Azure and data governance using Azure Purview and I was required to create a data map in Azure Purview that maps azure SQL database(cloud) and SQL server database on premise with the default Adventure Works database.
Note this assignment was not for a project but with the intent to get acquainted with data governance.

Doing the task was really daunting because it was my first experience working on the cloud 😁
Well... I was still able to create a data map that automatically classify's columns using regex and Microsoft built in default classification techniques.

Still on still, assigning sensitivity labels was still another subject matter that was hard... to grind. I struggled to understand how to assign sensitivity labels to the scanned data because creating sensitivity labels was done on the Microsoft Information Protection Portal. Each time I scanned my database both on premise and on the cloud the data map was unable unable to pick it up.

I figured if I could create a database having sensitive data like social security number (SSN) the sensitivity labels I created in MIP Portal would be picked up in the data map.

So I asked chatgpt to generate a ficitiotius table having SSN and other details. I then used the fictitious table to create a database in SQL server. Viola!! After scanning in Azure Purview data map the SSN was flagged as a sensitive data.

Azure Purview can be used for data lineage because it automatically captures the lineage flow of data through different sources like data lakes and databases. For this reason, I wanted to work on writing an article on how I was able to do the task with data lineage inclusive but my subscription and the use of Azure Purview ALONE had consumed about 113 dollars.

I adviced myself to stopeet!!!😭

Well done OP. I'm following.
I'm also interested in data engineering. But I'm currently just doing some web scraping.

It’s been a while since I updated this thread. I have gone through so many highs and lows since I created this thread. my growth and progress has come with some sweat and tears. But my boss then was really supportive. He compensated me then even when my work didn’t immediately pay off. His support enabled allowed me to get deep into Azure’s ETL tools like Azure Data Factory (ADF).

I learnt the essentials of Azure Data Factory such as how to make dynamic, reusable pipelines that load multiple SQL Server tables to Azure Data Lake using parameters, ForEach, and Lookup activity

I also figured out how to load multiple CSVs from Blob to Azure SQL, with each file loading to separate tables based on the filename. I found the course on Udemy by Ramesh Retnesamy( Azure Data Factory- Covid 19 project ) incredibly helpful.
Infact I was switching between YouTube and Udemy so as to get a deeper understanding of the topic Ramesh didn't explain well and I found the YouTuber 'Wafastudies' does a have a good break down of activities in ADF.

Although I am yet to finish the CI/CD section of Ramesh course(knowledge no dey finish) 😁

Since I began to get really comfortable with Azure and ADF, thoughts of the DP-203 exam started to come to my head.

For the exam, I learnt Microsoft puts a big emphasis on Azure Synapse. Funny enough, Synapse doesn’t have the same street cred as Snowflake or Databricks, yet about 60% of the exam comes from Synapse.
After my findings from Reddit I got the course by
Alan Rodriguez’s from Udemy. One thing I like about Udemy courses is that their courses are usually niched down which you may not find in some instances with YouTube.
Alan course did help a lot. Especially with figuring out how to connect to blobs and data lake to through SAS, connection string, managed identities, and RBAC. Although when it comes to the granularity of the permissions and access the YouTuber 'Tybul on Azure' did a lot of justice to the topic. Tybul gave a good breakdown on how to use Azure storage explorer and Access Control List to enforce granular connections starting from the hierarchy of container down to the file.
Tybul’s teaching is a bit slow-paced so you have to be patient to get a grasp of what he is trying to say.

What stood out for me in Alan's course is that he did justice to topics on data security. I learnt how to connect blobs and data like safely to DataBricks without connection string by using app registration in Azure AD and storing the App credentials in Key Vault. I did find out the Key Vault access policy can be used to grant Databricks permission to the credentials as a principal because the RBAC doesn't work

The DP-203 exam covers so much ground that it wouldn't be wise to rely on a single source of resource. You also need to be aware of dp203 course outline which you can get from Microsoft learn

There’s still plenty in DP-203 I haven’t mentioned, but these resources I mentioned has been of great help.
I booked my DP-203 exam in July at Pearson Vue in Port Harcourt, but there has been distractions here and there. Each time I reschedule my exam the lady that booked me for the exam would charge me for rescheduling fee 😎

Around the same time, my boss asked me to figure out how to do failover clustering on a hyper V cluster( still haven’t figured it out till this day) I was in the process of learning advanced DBA by Raphael Asgar which talks about High availability solutions like log shipping, mirroring and failover clustering(I did implement logshipping though).
At this point my boss began to give me headsup that my time was up. It was at this point I decided to focus in creating video contents to talk about the things I know about data when it comes to DBA, data science, data governance, analytics and data engineering.

With the help of my boss I got a side gig of teaching someone DBA online. Funny thing is I have never really seen DBA as a role in my arsenal but since the task was urgent I had to doit. I was amazed I was able to teach DBA. One thing that amazed me was that I was able to pull knowledge from what I learnt in my course of learning data engineering. Another factor that really helped me as well was that my sql skill is really rock solid. I still teach this person till this day though, because the client loved how hands on I was and wanted the class to continue.

Since working relationship with my boss started getting edgy how to create video contents became my focus. The plan is to post short 5 minutes video about the things I know about data on LinkedIn for a start.

To my greatest surprise this was not as easy as I had hoped. Since I wanted to use Davinci Resolve to learn how to edit my own videos. My PC couldn’t even open the software. I then figured I needed to get a gpu PC of Nvidia graphics card. I got a second hand hp Zbook 3 and thankfully Davinci Resolve works with the PC. I was able to get it through a third party for 610,000 naira.

Because I wanted the cinematic appeal to my vidoes I knew my camera and lightning had to be lit👌.
Three lightining system would be used( key, fill and back light) . as at this point I have ordered the lightning system and microphone with noise cancellation from Jumia. The overall cost of the items cost me about 220,000 naira. Initially I shared the idea with my boss of the need to create video content on Linkedin and he agreed to take up 50 percent of the cost. I was really surprised that he backed out down the line( I believe things happen for reasons we may not be able to understand). What's left as this point is procuring an Iphone and turning my room into a workspace sutiable for shootimg videos(my boss did put that idea in my head though). I am still scratching my head trying to figure out how to hide my big bed furniture from the camera😭. I am trusting God for an Iphone 13 though🙏

I believe from the depth of my heart that I have a lot to share and talk about and the attention and respect as a data practitioner I have always craved for would come.

Since my boss stopped paying it did put a pause to the Video content project. Thank God for my mom. That woman after God is my backbone. She assisted me with 80 percent of the money I used in procuring the lighting and sound equipment

DataMina:
Since working relationship with my boss started getting edgy how to create video contents became my focus. The plan is to post short 5 minutes video about the things I know about data on LinkedIn for a start.

To my greatest surprise this was not as easy as I had hoped. Since I wanted to use Davinci Resolve to learn how to edit my own videos. My PC couldn’t even open the software. I then figured I needed to get a gpu PC of Nvidia graphics card. I got a second hand hp Zbook 3 and thankfully Davinci Resolve works with the PC. I was able to get it through a third party for 610,000 naira.

Because I wanted the cinematic appeal to my vidoes I knew my camera and lightning had to be lit👌.
Three lightining system would be used( key, fill and back light) . as at this point I have ordered the lightning system and microphone with noise cancellation from Jumia. The overall cost of the items cost me about 220,000 naira. Initially I shared the idea with my boss of the need to create video content on Linkedin and he agreed to take up 50 percent of the cost. I was really surprised that he backed out down the line( I believe things happen for reasons we may not be able to understand). What's left as this point is procuring an Iphone and turning my room into a workspace sutiable for shootimg videos(my boss did put that idea in my head though). I am still scratching my head trying to figure out how to hide my big bed furniture from the camera😭. I am trusting God for an Iphone 13 though🙏

I believe from the depth of my heart that I have a lot to share and talk about and the attention and respect as a data practitioner I have always craved for would come.

Continue to push and don't give up.
If success was Easy nobody will need Education.

GOSPELTRUTH31:
Continue to push and don't give up.
If success was Easy nobody will need Education.

Yea. hopeful the recognition I've always wanted would come

DataMina:
Yea. hopeful the recognition I've always wanted would come

Do u teach who want to learn ?

africanman85:
Do u teach who want to learn ?

It depends 😀

When I do my cost benefit analysis

But I'm always willing to mentor

Hi guys have got good news for you.
I have been able to pass my Azure data engineering exam.
Although I have been preparing for this for the past 9 months the way most of the questions were framed was not what I expected. What helped me was I was 100 percent certain I got all the case studies questions correct and my command of SQL is really sound
As usual data warehousing using Synapse formed a huge chunk of the question I was surprised that I had a lot of stream analytics question.

My next cause of action now is to start creating short content with target to LinkedIn recruiters

These are the sample of my exam score and my certificate 👇

DataMina:
These are the sample of my exam score and my certificate 👇

Congratulations, you really did it.

Thank you Alakara

I hope your dream is alive! Don’t stop

Please advise for someone that wants to start data analytics