Showing posts with label scrape. Show all posts
Showing posts with label scrape. Show all posts

Thursday, July 26, 2012

Scraping mobile phone number from web


web.search("td").each do |e|
e = e.inner_html
if e.length > 10
tmp = e.gsub(/(\r|\n|\t|\D)/, "")

tmp.split(/(010|012|013|016|014|017|018|019)/).each do |s|
if s =~ /(010|012|013|016|014|017|018|019)/
pos = tmp =~ /#{s}/
#puts pos
if pos
number = "#{s}#{tmp[pos+3..pos+9]}"
puts number
Contact.create(:group_id => 1, :phone => number) if number.length == 10
end
end
end

end
end

Tuesday, July 17, 2012

Scrape email from yellow pages and send them email with 300 second delay


em = []
(1..63).each do |i|
yp = Nokogiri::HTML(open("http://www.yellowpages.com.my/search.jsp?sfor=all&name=logistic&w=&p=#{i}"))
emails = yp.search("a.email").map{|x| x["onclick"].gsub("SqueezeBox.open('/plainmail.jsp?id=", "").split("',")[0] }
emails.each do |eid|
  t = Nokogiri::HTML(open("http://yellowpages.com.my/plainmail.jsp?id=#{eid}"))
  em.push(t.search("input").first["value"])
end
InvoiceMailer.inquiry(em).deliver
sleep 300
end