E.g., pretend there are 6 people on NL, A, B, C, D, E, F.

A is friends with B and C.
B is friends with D.

And some more friendships encoded by the following graph:



What would you do with this data?

What cool things could you figure out, both with this data and a user's posting history and other information?

So Facebook has a friend graph too, and they do stuff internally with it (e.g., people you might know, or should be friends with).

But I am sure that there must be other interesting things you could figure out.

We can generate a crude dataset of the graph. Just select a sample of 100 popular posters(beaf, Idowuogbo, Onila, Musiwa etc) in an excel sheet and list out all the nodes.

of course there will be some people outside those links who will be part of the dataset by association.

just weight it and see who's where and with who.


But to make sense of it, I suspect you might need to add other data like the spread of the posts (where do they post more), who&who tend to post on the same thread and what is the post distance ( average distance between their posts<---how frequently they reply each other).

of course Seun can also add stuff like their locations as derived from their IP addresses (not always reliable). or their stated locations (also not reliable.

also biodata like age.

there's lots and lots you could do. ( my mind is already considering what nouns and verbs their posts tend to have in common e.g who's else has a similar amount of 'GEJ' scattered all over his posts like Beaf).

You could even use it for plagiarism research. Can we find out who's using duplicate accounts just by analysing their posts? etc.

Lots of cool stuff waiting for curious minds.

There are some other pretty big online forums that I visit. Based on phpbb3 and vbulletin. I would be nice if there was software that could automatically construct social network data like this.

Recommend other posters on NL that you should follow, threads that you'd like, etc.

Recommender system stuff I guess, essentially.

While I'm not really good at this stuff. I think collaborative filtering makes sense, u know inversion and all.

The design of our social graph is yet to be perfected. Still working on figuring out how to structure it.

To just design the graph should be pretty easy.

E.g.:

www.nairaland.com/userid/4

is a page for user #4 in your database's friend list. Say a tab, space separated text file or something, with the data pulled from the DB.

In fact you basically nearly have this already. Just not in a format useful for manipulation.

kodewrita: We can generate a crude dataset of the graph. Just select a sample of 100 popular posters(beaf, Idowuogbo, Onila, Musiwa etc) in an excel sheet and list out all the nodes.

...

there's lots and lots you could do. ( my mind is already considering what nouns and verbs their posts tend to have in common e.g who's else has a similar amount of 'GEJ' scattered all over his posts like Beaf).

...

Broda Okon, come see e!
Dem de kill me hia and I no know!

Lol! Anyways, the immediate thing that comes to mind for me with social graphs is degree of separation and how it can be harnessed to produce a really powerful community (using peoples likes, deslikes, common words, gender, age, temperament, needs, profession etc).
I'm in the beginning stages of building a graph database in C# (its tough! ), it should be able to do most of the sort of analylis we are talking about here quite naturally. The relational model is the wrong architecture for such.

Beaf:
I'm in the beginning stages of building a graph database in C# (its tough! ), it should be able to do most of the sort of analylis we are talking about here quite naturally. The relational model is the wrong architecture for such.

do you mean relational database? how can that be? since relational database is the most naturally suited for social analysis(IMO), with little effort you can easily summarize what data-structures(tables) are related, and the number & specific relationships.

There are two common choices for storing graphs.

Either adjacency lists (http://en.wikipedia.org/wiki/Adjacency_list), or adjacency matrices (http://en.wikipedia.org/wiki/Adjacency_matrix).

Adjacency lists save space and are better suited for sparse graphs (like the NL graph presumably is).

Beaf, may want to check out this: http://stackoverflow.com/questions/1578493/net-graph-library-around

Beaf:

Broda Okon, come see e!
Dem de kill me hia and I no know!

**Covers face**

hehe. Bros,you know naa.

What I meant is that we're still kinda trying to figure out whether to have a symmetric model like Facebook or an asymmetric model like Twitter and what privileges and features should be attached relationships on the forum. We currently have friending and member following as completely different features.

creating a social graph is not beans.
Facebook has 400 A-players programmers, what do you think they are doing?
A simple feature to show what your friend's liked would take a team amature programmers a century!
And yes, like @beaf said, it would take a little more than relational db, though still relational, this time your data HAS AND BELONGS TO very MANY crazy relationships!

The twitter model seems more user friendly though. And your followership might be a better measure of your usefulness/value than your number of friends. If I like Beaf's comments (psyching in progress), I can follow him without him necessarily knowing.

okeyxyz:

do you mean relational database? how can that be? since relational database is the most naturally suited for social analysis(IMO), with little effort you can easily summarize what data-structures(tables) are related, and the number & specific relationships.

No, it isn't. There are very serious problems with the relational model. Firstly, it doesn't sit well in todays OO World and secondly does not represent social graphs naturally or efficiently (the keyword here is efficiency). You will find that all the biggest websites have dumped relational databases for either key-value stores or graph dbs. I am confident that in the next ten years, the relational db would be either extinct (or highly deminished). Wikipedia keeps getting noticably slower every year for example, because they have stubbornly stuck with MysSql.

The main killer of the relational db is the sheer growth in the volume of data applications use, this is in turn driven by ever decreasing storage costs (growing at a much faster rate than processing power can keep up with). That brings in all sorts of scalability and complexity issues and the relational model is already creaking at the seams, long since dumped by the likes of Google and increasing dumped by realtime financial systems websites, games websites etc.
Lets also not forget the facebook, bebo style apps that are beginning to gain currency. There are so many relationships and connections involved that a DBA handling one of those sites would soon find himself recuperating in a mental home. Lol!

The advantages of key-value and graph db's are ease of use (no SQL, you simply write in your favourite language), flexibility, very high scalability, very high performance. They also faithfully represent your OO code structure.

Please folks, my apologies for the long digression.

kodewrita: The twitter model seems more user friendly though. And your followership might be a better measure of your usefulness/value than your number of friends. If I like Beaf's comments (psyching in progress), I can follow him without him necessarily knowing.

Chei! Ogini ka in ge me?
Ayam in trouble today!

ekt_bear: Beaf, may want to check out this: http://stackoverflow.com/questions/1578493/net-graph-library-around

Thanks, bro. Thats a great page.

We use MySQL for the social graph and it works just fine. Twitter does the same, and so does Quora. True story.

Seun: We use MySQL for the social graph and it works just fine. Twitter does the same, and so does Quora. True story.

This is a pretty informative overview of Quora's architecture (but they do not run graphical data in the real sense):
http://www.quora.com/Quora-Infrastructure/Why-does-Quora-use-MySQL-as-the-data-store-instead-of-NoSQLs-such-as-Cassandra-MongoDB-CouchDB-etc

Twitter on the other hand is different, they moved away from MySql due to scalability issues. They now run on a mix of nosql data stores:
http://nosql.mypopescu.com/tagged/twitter
Twitter has also developed their own inhouse graph database (FlockDb - you might want to check it out, its Apache licensed):
https://github.com/twitter/flockdb

Seun is not at the level in which scaling is an issue. People honestly jump into noSQL options too quickly. For most applications, SQL DB is sufficient.

@Beaf: Twitter's graph database, FlockDB, also runs on MySQL. It's basically a sharding layer over a cluster of MySQL databases.

Beaf:

No, it isn't. There are very serious problems with the relational model. Firstly, it doesn't sit well in todays OO World and secondly does not represent social graphs naturally or efficiently (the keyword here is efficiency). You will find that all the biggest websites have dumped relational databases for either key-value stores or graph dbs. I am confident that in the next ten years, the relational db would be either extinct (or highly deminished). Wikipedia keeps getting noticably slower every year for example, because they have stubbornly stuck with MysSql.

sorry, if wiki is getting slower, let them check their database design. or mail them some check so they can create more nodes to handle the traffic.

Seun: @Beaf: Twitter's graph database, FlockDB, also runs on MySQL. It's basically a sharding layer over a cluster of MySQL databases.

Really? I didn't know that. Perhaps thats why many feel its a weak graph db.
Interesting.

@beef modelling issues with relational databases can be avoided using a good O/RM library. I am currently working on a complete friend graph (for a site I can't give details of at the moment) using the object-oriented paradigm. This graph may end up sitting on top of a relational database or a key-value store like the AppEngine datastore (most likely) in production. I am using JPA so I don't have to worry much about how the the data will be physically persisted (whether graphDB, sql-based db, key-value store, blah blah so far the store is JPA-compliant).

Its usually not convinient for OO programmers to work (manipulate complex structures) in relational pattern which I think is what you are facing.
I'll advise that you design in OO and have all your relational stuff auto-generated from your design. I am talking from a Java EE perspective but I know such facilities should be available on c# too (not for the AppEngine datastore anyway).

SayoMarvel: @beef modelling issues with relational databases can be avoided using a good O/RM library. I am currently working on a complete friend graph (for a site I can't give details of at the moment) using the object-oriented paradigm. This graph may end up sitting on top of a relational database or a key-value store like the AppEngine datastore (most likely) in production. I am using JPA so I don't have to worry much about how the the data will be physically persisted (whether graphDB, sql-based db, key-value store, blah blah so far the store is JPA-compliant).

Its usually not convinient for OO programmers to work (manipulate complex structures) in relational pattern which I think is what you are facing.
I'll advise that you design in OO and have all your relational stuff auto-generated from your design. I am talking from a Java EE perspective but I know such facilities should be available on c# too (not for the AppEngine datastore anyway).

relational db remains the best approach. Mysql stands undefeated.

Beaf:

Really? I didn't know that. Perhaps thats why many feel its a weak graph db.
Interesting.

@beef
you know your political stunts cant work in programming section.
Who are the 'many' that 'feel' ?
Statistics please. You know twitter gave up switching from mysql after one year of trial. Facebook runs mysql except in its inbox search.
The future has not found a replacement for mysql.

It seems Oracle is preferred in the financial sector (I dont have any stat to back that up)

moderattor: @beef
you know your political stunts cant work in programming section.
Who are the 'many' that 'feel' ?
Statistics please. You know twitter gave up switching from mysql after one year of trial. Facebook runs mysql except in its inbox search.
The future has not found a replacement for mysql.

If you attempt to learn about graph db's, you would soon find out who the many are. I'm not even sure you know what a graph db is, so why waste my time?
You owe yourself a duty to learn, bro. I do not owe you any lesson or lectures. Thank you.

http://www.readwriteweb.com/cloud/2011/01/how-twitter-uses-nosql.php

@beef thanks for that info. I really enjoyed it.

Creating an efficient Friends Graph will require the use of a Map for languages that supports Map or IDictionary in C# or VB.Net and a Composite Object which can be used with an in memory Database or a flat file. A Composite Object will be good for storing and retrieving the different attributes you want for each user.

You guys might find this interesting:

I recently announced our "member following" feature, which lets you instantly follow any Nairaland member you fancy without waiting for her approval. I'm pleased to inform you that the "member following" feature has now replaced the former "friendship" feature. You can't add people as "friends" anymore, but you can follow them with or without their permission. When the people you're following post new topics on Nairaland, those topics will instantly be added to your list of "followed topics".

https://www.nairaland.com/952229/member-following-replaces-friending-nairaland

Can you expose this data a bit more cleanly?

E.g., if I want to see who user #5 is following, I visit:

www.nairaland.com/users/5

then see the id #s of who he/she is following

as well as say his username.