₦airaland Forum

Am very sorry if you can not actually find what you are searching for at the moment, but try again in a week time. As you know, this is new and the index data is not yet upto a million.
Please help submit relevant website that has useful information.

Cheers.

dhtml18:
Why you come dey swear for sach now? this is unfair!!

Bro, the girl is really mean.

Djangocode:
I tried using opera mini on Mobile and it couldnt even search a page... It's way too slow... It also shows no search results for data... Even Nairaland loads way faster...

Perhaps its your internet, you can try again.

Booyakasha:
Just as adesege will say.. why reinvent the wheel? is your search engine better than Google or even bing? why do you think people will use your search instead of the former which they're accustomed to and makes searching easier for them? think about this again..

Your journey is inevitably leading to a dead end... sorry.

Like seriously, are you saying their should be no more android fork because there is already an exisiting android used majorly.

babatope88:
Good work bro, making search engine is not easy. See you at the top.

Thanks, no be small thing i tell you.

DanielTheGeek:
Looks like you made your search engine crawl only Wikipedia and StackOverflow.

Think about adding autocomplete feature to your website, using either JQuery autocomplete.js, or TypeAhead.js you can download it at http://twitter.github.io/typeahead.js/

Thanks will implement that later on.

@kodewrita @Javanian please help me move it to the frontpage.
Cheers.

Hello, i will like to offically introduce the rave of the internet http://sachng.com
With sachng.com you can search for information around the web using our powerful search functionality that returns accurate results within the shortest possible time.
You can also add your website to our system, so that people can easily find your webpages on our platform.
We do not track you search history, so your search is safe with us.
Sachng.com na our own.

dhtml18:
@op, i hope you know that you are owing me a goat? a live goat at that (not an e-goat mind you).

No problem sir, your goat is still on the pipeline.

Check out http://sachng.com

Djade007:
For example, a keyword was searched, you check your database to see if there was any result.

If there is no result or if the number result is low, you can make an api request to Google search engine. You populate your database with the returned data from the api and give the user the result. Next time the same keyword was searched you would already have some content in your database.

Thanks I will look Into this.
We are still improving the system, so every idea is valuable.

Djade007:
Op why don't you use Google search engine api, for keywords that aren't found. You will also be updating your database with the result returned from Google

How do you mean?

dhtml18:
But wait, i search for:
goat - not found still to my greatest dissapointments
nairaland - not found
php - found (but wrong - the resulting page when opened does not contain anything like php)
php boys - not found (this is so wrong, if you cannot find php boys, you should search for php instead)

at this point, i abandon the search once again. but this version is slightly better than the previous.

Nairaland wasn't added that's why no result showed.
Soon eneough you get latest result.
Still working on fine tuning the result.

Hey guys, Sachng.com is back and better.
Thanks to all those that drop their little dislike and like.
@dhtml18 you goat search is now activated.
CHeers.

micodon:
Did he say anything against that? This guy sef... He only told you how the multiple connections are handled... IN A DARN CONNECTION POOL

Show my the doc that states that mysql uses one connection pool.

Please make una leave me let me use want am comfortable with, and you guys can you what you wish.
Like I stated early, the search engine is slow not because Php or mysql is bad or can handle the task, but due to error on my part.
I am coming out with a fix soon.

sleepingdemon:
mysql uses one connection pool, so if you have 10 users launching your app at the same time, and the 10 isers need to run 100 queries each, mysql lines up the 1000 queries from the 10 users and executes them one after the other.
that said, the 10th user gets his data last and slower. now apply that logic to 1 million users running a query that searches a mysql database of over 40tb of data....
why do you think even mysql enginners wont tell you its scalable, rather they would optimize indexes for faster search time and at the worst, they master - slave replication, which is almost the same thing as data sharding which i mentioned.
i hope this gives u a bit more insight to your understanding.

This is a fat lie, mysql can handle multiple connection to same data. Please read about mysql properly.
http://stackoverflow.com/questions/14901508/maximum-concurrent-connections-to-mysql

micodon:
You see, this is one problem with some people who know only 1 or 2 languages of the same type. They never listen. I was like this about 7 years ago.

The best PHP (or an interpreted language) code of an app will ALWAYS RUN TONS OF MAGNITUDE SLOWER than the best c++ (or any compiled language).

If you read my post you'd notice that my PHP framework clocks at about 1,300 rps which makes it one of the fastest PHP frameworks ever.

Why are Facebook transpiling their PHP. codes to intermediate byte code if running PHP is good enough.


Please write a simple application that prints the first 10000 numbers of the fibonacci series in PHP and any of the languages I listed before say JAVA and run a benchmark. The PHP script may even run out of memory .

You may be right according to your understanding, but I will like to get to that bridge where Php can not sustain the app.

sleepingdemon:
Using php and mysql isnt the best choice really because:
To scale PHP on a relational database, you need to shard your data. This means splitting the data by some reasonable key. This might mean East Coast customers go on one RDBMS, Midwest customers go on another, and the West Coast on a third. This is a lot of complexity to swallow when you chose PHP because it was "simple" and "free."
But like i said, till your app grows to when scaling is most needed, its not a bad idea using php alongside a no-sql database not a RDBMS database.

To my understanding, mysql can handle millions of data and is very much scalable.
The only limitation is the app logic, how you code.

micodon:
The very first rule in developing an app is: CHOOSE THE RIGHT TOOL(S) FOR THE JOB. In choosing "THE RIGHT TOOLS", you look at performance, scalabilty, ease of development and others

PHP has been my first choice language for close to a decade now, but TBT its only REALLY good at making websites and some web apps.
Last Year i wrote an eCommerce application with my HIGHLY DECOUPLED PHP framework (a "HELLO WORLD" benchmark gives about 1,300 rps). Benchmarks showed that it ran at about 250rps (Opencart runs at about 60rps). But it starts dropping requests when there are above 500 concurrent users.

I rewrote the same application with GOLANG and it handles over 5 million concurrent requests with breeze at about 6,500 requests per second with less than 15MB or RAM.

The Point? In a PERFORMANCE CRITICAL Application, PHP will fall your hands....It was designed for just short-lived webpages. Here are some reasons why PHP is not suitable for PERFORMANCE CRITICAL APPLICATIONS

1. Blocking I/O
2. No Parallel Processing out of the box
3. No Multi-threading out of the box.
4. Not Suitable for background Tasks (Remember your crawler)
5. Most probably used with Apache which spins up a thread for every request
6. Dies after every request
7. Not good for Computationally Intensive tasks (i.e Search engine analytic, ranking system, etc)
8. Not Memory Efficient
9. Extremely Poor Concurrency Model

In designing and developing your application, NEVER EVER LET THE DEVELOPMENT LANGUAGE IMPLEMENTATION BE YOUR BOTTLENECK...THAT PROBLEM SHOULD BE YOUR DB IMPLEMENTATION.

Should you use PHP now until you have performance or scalability issues? ABSOLUTELY NOT. By the time you get to that problem, you'll have had a huge PHP code-base which would make transitioning to another language NEAR IMPOSSIBLE. If you think i'm lying, ASK FACEBOOK.

JAVA is good (except for the JVM being resource-hungry)
GOLANG has its Limitations but its good
RUST is a new kid on the block, but is EXCEPTIONAL
JULIA,
SCALA (JVM Problem again)
HASKELL
C/C++

One of these could be used in combination with any of the following:
Python
Nodejs (Not a language, but an execution environment)
LUA, etc

My brother leave all these English for British.
The only reason your Php app is slow is due to your logic, and also because Php is not a strict language.
If you write Php the way you write your c++ you will notice improvement.
Beside this is just a start, there is always room for other languages.

sleepingdemon:
in all of this, that guy that asked you to use golang, i need to flog him very hard. ehen, that advice on java is still fair, on the other hand, what do i know, just in case you're more comfy with php, you can use mongodb as your database instead, when the app outgrows php, you can move to another lang. or if ure a javascript person, mean stack is actually useful.

Thanks, just wanted to kick off with my favourite language, looking at nodejs in the near future.
It seems like some people don't realise how much it cost to host on a dedicated server.

trytillmake:
OP Nice concept, na somewhere google take start, but seriously if u wanna get somewhere with this,
1.go with the above advice, the search is too slow,
2.yr ajax use is a non issue use "get" abeg,
3 yr green color boring and dull, yr favicon dull not catchy/sharp not standing out,
4 spacing of the search items too wide apart, no pagination to determine how many pages,
5 meta details of each search not provided(i cannot just click for clicking sake) and
6 please it is too early for advert you can say contact us here simple, wetin, na so u like advert, dn't give yrself away too fast na, money go come but do am well.

Nice job sha,

Thanks working already on some of these issues. The ads things dey make me laugh self, I no say people go talk about this.

Laolballs:
Hello search engine guy, check out this class i wrote sometimes ago, it would really help to improve your search engine.
www.phpclasses.org/package/8926-PHP-Perform-full-text-search-on-MySQL-with-autocorrect.html

Kudos and welldone

Thanks.

dhtml18:
^^^Santa maria! I actually sniffed his server (with my very long nose) and smelt out php/mysql. . . . .to my greatest surprise!!

Why are you surprised?

micodon:
My guess is that you are using a LAMP stack for this search NG. That's like swallowing a timed explosive.

You want to do it right, do It right.... You need a language that can handle concurrency with breeze. Someone said Nodejs. I wouldn't advice that. By default, it doesn't use all the cores of your server.

Spring up something with Golang (go routines and channels are the bomb), Rust, even JAVA.

UNLESS ITS A SCHOOL PROJECT, JUST DITCH PHP.

How do you know that Php is not powerful enough for this task?

Read the documentation of the class to learn how to use it.

dhtml18:
I know my own mouth is a bit sharp, but @op, i need test keywords that i can use to search to even see how the result page looks like.
I search for nairaland, go and brush my mouth come back (a little exagerrated sha), wait small before the result shows - nothing.

Imagine if this thread has hit FP, what would ave happened? I can answer that - a full scale war followed by relegation of the entire programming section for good (this section will never see FP ever ever again)

Sorry about that, soon enough you will get better experience. Will remove the ajax thing for now.

FincoApps:
COMMON GUYS.... Dhtml5 points are all valid.... however, the point most of us are giving that "the engine cannot find some keywords" is somehow unfair considering the age of this search engine.

This engine just started today right ? How on earth is it supposed to gather most of the internet in 1 day??

Even AI bots that use machine learning to get smarter always seem very dumb during the initial days of deployment so guys calm down

Thanks bro, so far the engine has already indexed 200k+ webpages.

DanielTheGeek:
Funny enough, [s]Your search engine can't even find your own site[/s] I'm not going to blast this one... With the big picture in mind and future possibilities I would like to index my page on your engine on one condition:

1. PURCHASE AND INSTALL A SSL CERTIFICATE.

I don't think that is a problem for now, all I want to focus improving user experience and search result.

dhtml18:
It is a good start, with proper work, it can be done. Keep up the good work dude. If you need my assistance, just PM me right away. I have a few libraries that can help you get started.

Thanks for all your input so far, at least they will help me improve the thing, you will be amaze at the next update.

Sambayour:
all complaint made by dhtml are all true, I was so exited when I saw Nigerian made search engine but was disappointed when words like Ikeja, even Lagos shows no result.

as if the database is empty or not querying any data.

Next version will be better, I am working on improving the user experience and result as we speak.

dhtml18:
You need to write a daemon that scrap sites, i too have a library somewhere that can do that.
Doing a search engine means you need to do an intensive research before implementation.

And do you know that if you search google for goaat - the first result will be goat. There is a term for that, which i don't fully remember now.
That is possible even in mysql database, but i dont recommend mysql for this kind of thing. This your search will be much better if you use NODE.JS.
Imagine a million people searching at the same time. . .how do you handle concurrency? that your ajax script will just die and rest in pieces.

kosytedy:
Great idea.
Please use <form method="get"...> and <input type="text" name="q"...>, the call the searched item with $_GET['q'].

Don't use caching, it's better to optimise your database and database queries to act faster (check stackoverflow).

I need to tell u many things but, don't have time now. Good work, and more grease to ur elbow. I have a robot that scrapes site and stores relevant keywords for the site in the database (that might be helpful too). Keep it up. I love ideas

Thanks guys for the info, I have an active engine that crawl and index webpage content as we speak.

dhtml18:
- That is another thing, regular search engines don't use ajax - they use <form method="get"> that way, a user can refresh search results, modify, bookmark etc
- As for my internet being slow, adonbilivit

- So you mean searching for goat is not realistic? Well i searched for fulani, php and even nairaland - no result!!!!

I believe we need other testers. . .

Alright, will work on speedy search and more accurate result.