Welcome, Guest: Register On Nairaland / LOGIN! / Trending / Recent / New
Stats: 3,143,487 members, 7,781,461 topics. Date: Friday, 29 March 2024 at 02:50 PM

How Best Can I Read DOM Of External Pages? - Programming - Nairaland

Nairaland Forum / Science/Technology / Programming / How Best Can I Read DOM Of External Pages? (1206 Views)

Reasons Behind Of External Hard Drive Data Recovery / In Need Of External Hard Drive For Sale / NCDC Coronavirus Updates Brute, Efficient, Fast DOM Php Code (2) (3) (4)

(1) (Reply) (Go Down)

How Best Can I Read DOM Of External Pages? by Desyner: 1:25am On Aug 28, 2017
Hello Nairalanders,
I want to read the content of external sites dynamically and don't know which technology suitable and best for this.
I want to go into pages and search of its content divs, p, a , images and span tags and read their contents for onward manipulation.
Can JSON do it? I was thinking of using researching Java 8's Networking API for this purpose but would love a simply solution like PHP.
Re: How Best Can I Read DOM Of External Pages? by phainix: 9:17am On Aug 28, 2017
Hi Desyner, im not sure what you want to achieve but with php you can use 'file_get_contents', it'll fetch all the html code on the page and you can probably do your sorting from there

echo file_get_contents('https://www.google.com.ng');
Re: How Best Can I Read DOM Of External Pages? by Desyner: 10:27am On Aug 28, 2017
@phainix,
I will try it out even though i found what looks like the java version of the solution. Thanks for the PHP solution, it uses less memory compared java. I will explore it also.
Re: How Best Can I Read DOM Of External Pages? by agwaisrael(m): 10:50am On Aug 28, 2017
When you get the site contents, there are libraries to parse html as xml in php.
This has nothing to do with networking apis. You only need a simple ;
HttpURLConnection adam = (HttpURLConnection) URL.connect();
adam.getInputStream();... on java

Or

file_get_contents(url);... for php

Search for w3 html parsers or create your own library to parse what you need
Re: How Best Can I Read DOM Of External Pages? by yorex2011: 9:47pm On Aug 28, 2017
Use simple html dom for php

After you may have used file get contents or curl
Re: How Best Can I Read DOM Of External Pages? by Olyboy16(m): 12:22am On Aug 29, 2017
Desyner:
@phainix,
I will try it out even though i found what looks like the java version of the solution. Thanks for the PHP solution, it uses less memory compared java. I will explore it also.

I don't know exactly how you want to interact with the data, but check PhantomJs http://phantomjs.org
Re: How Best Can I Read DOM Of External Pages? by orimion(m): 4:12pm On Aug 29, 2017
Sounds like you want to do web scraping, search for a web scraping library in your language of choice
Re: How Best Can I Read DOM Of External Pages? by Nobody: 3:53pm On Aug 31, 2017
Kindky pay a visit to php
Re: How Best Can I Read DOM Of External Pages? by jamesmalvi(m): 10:35am On Sep 05, 2017
Desyner:
Hello Nairalanders,
I want to read the content of external sites dynamically and don't know which technology suitable and best for this.
I want to go into pages and search of its content divs, p, a , images and span tags and read their contents for onward manipulation.
Can JSON do it? I was thinking of using researching Java 8's Networking API for this purpose but would love a simply solution like PHP.


Yes, JSON can do it. here is tool, which may help: https://jsonformatter.org

(1) (Reply)

Pls I Need Someone That Can Teach Me How To Program Arduino Uno / Freelance: Programming with Microsoft Office Excel. / Do You Need Nigeria States And Local Governments?

(Go Up)

Sections: politics (1) business autos (1) jobs (1) career education (1) romance computers phones travel sports fashion health
religion celebs tv-movies music-radio literature webmasters programming techmarket

Links: (1) (2) (3) (4) (5) (6) (7) (8) (9) (10)

Nairaland - Copyright © 2005 - 2024 Oluwaseun Osewa. All rights reserved. See How To Advertise. 13
Disclaimer: Every Nairaland member is solely responsible for anything that he/she posts or uploads on Nairaland.