You can scrape the search results page or a web page containing list of items and get a list of URLs returned. You need to break up this page into separate sections for each item, then iterate over the sections and scrape the URL of each item. You can use explode function to break the results page and store the sections into array.

$url = ""; 
$result_page = curl($url);

$results_page = scrape_between($result_page, "", "">"); 

$separate_results = explode("", $results_page);

foreach ($separate_results as $separate_result) {
if ($separate_result != "") {
$results_urls[] = scrape_between($separate_result, "href="", """);
}
}

print_r($results_urls);

The steps are explained below:

  • Assigning the URL you want to scrape to the variable $url
  • Downloading the results page using curl() function
  • Scraping out only the middle section of the results page that contains results, while stripping away the header, footer, sidebar. Use scrape_between() function.
  • Exploding the results into separate parts into an array
  • Scrape the URL for each separate result using foreach loop
  • Adding this URL to our URL array
  • Printing out array of URLs just scraped