Crawl Web Loop – Intro to Computer Science


The next step is to write the loop that’s going to do the crawling. And we said the process we want to follow is to keep going as long as there are more pages to crawl. We can do that with a while loop, and we can use tocrawl like this in our test condition. If a list is empty that’s interpreted as false. If the list is not empty, that would be interpreted as true. So this means the same thing as testing if the length of the list is zero, it’s a cleaner way to write this by just doing while tocrawl. Inside the loop, well, we want to pick a page to crawl. We’ll store that in the variable page. And I’ll leave it as a quiz for you to figure out a way to pick the page to crawl. There’s certainly lots of different ways to do this. If you’re clever and can think about using all the things that we learned about, you’ll be able to have one line that both initializes page to the next page we want to crawl, and removes that page from the tocrawl list.

Add a Comment

Your email address will not be published. Required fields are marked *