₦airaland Forum

Hello Nairalanders,
I want to read the content of external sites dynamically and don't know which technology suitable and best for this.
I want to go into pages and search of its content divs, p, a , images and span tags and read their contents for onward manipulation.
Can JSON do it? I was thinking of using researching Java 8's Networking API for this purpose but would love a simply solution like PHP.

Hi Desyner, im not sure what you want to achieve but with php you can use 'file_get_contents', it'll fetch all the html code on the page and you can probably do your sorting from there

echo file_get_contents('https://www.google.com.ng');

@phainix,
I will try it out even though i found what looks like the java version of the solution. Thanks for the PHP solution, it uses less memory compared java. I will explore it also.

When you get the site contents, there are libraries to parse html as xml in php.
This has nothing to do with networking apis. You only need a simple ;
HttpURLConnection adam = (HttpURLConnection) URL.connect();
adam.getInputStream();... on java

Or

file_get_contents(url);... for php

Search for w3 html parsers or create your own library to parse what you need

Use simple html dom for php

After you may have used file get contents or curl

Desyner:
@phainix,
I will try it out even though i found what looks like the java version of the solution. Thanks for the PHP solution, it uses less memory compared java. I will explore it also.

I don't know exactly how you want to interact with the data, but check PhantomJs http://phantomjs.org

Sounds like you want to do web scraping, search for a web scraping library in your language of choice

Kindky pay a visit to php

Desyner:
Hello Nairalanders,
I want to read the content of external sites dynamically and don't know which technology suitable and best for this.
I want to go into pages and search of its content divs, p, a , images and span tags and read their contents for onward manipulation.
Can JSON do it? I was thinking of using researching Java 8's Networking API for this purpose but would love a simply solution like PHP.

Yes, JSON can do it. here is tool, which may help: https://jsonformatter.org