Learn to Crawl: Customized Content & Pricing by Traffic Segment