Mengikis www.mudah.my mengikut cara kita

Dalam tutorial ini Ya'kob akan menggunakan Mechanize didalam ruby environment untuk mengikis (dalam inggeris scrape) data dari website www.mudah.my. Dalam tutorial ini Ya'kob akan menggunakan os Ubuntu 11.04 64bit

Makluman:
Tutorial ini adalah untuk tujuan pembelajaran sahaja & bukan bertujuan untuk membahayakan mana-mana pihak.

Tujuan:

Teknik ini membenarkan kita untuk meng-Index semula informasi yang diperoleh mengikut keperluan kegunaan kita. Mungkin kita hendak mencari kereta di mudah.my yang mempunyai sekian2 mileage.. atau kereta yang transmisinya manual sahaja, tetapi mudah.my tidak menyediakan fungsi carian yang sedemikian. Dengan teknik ini, ia membolehkan kita boleh memprogramkan/mengindex informasi yang diperoleh, mengikut cara kita.

Prerequisite:
Ruby1.9 , Mechanize gem, Hpricot gem dan lain2 dependent lib eg: libxslt-ruby libxslt-ruby1.8 libxslt1.1 libxslt1-dev libxslt1-dbg serta juga gem nokogiri dan library2 dependency-nya

Kaedah:

1. Install  rvm (ruby version manager)

$ bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)

2. Install ruby

$ [[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm" 
$ rvm install 1.9.2
$ rvm --default use 1.9.2

3. Install gem2 keperluan

$ gem install mechanize
$ gem install hpricot

4. Masuk kedalam Interactive Ruby Console

akob@ubuntu:~$ irb
ruby-1.9.2-p290 :001 >

5. Paste code dibawah didalam console. oh.. gantikan /home/akob dengan home path anda.
6. Buka file log di directory home anda dengan web browser. Anda akan dapati hasil pengikisan anda disitu ;)


Code-nya:

require 'mechanize'
require 'hpricot'
agent = Mechanize.new
page = agent.get('http://www.mudah.my/Malaysia/Cars-for-sale-1020')
@response = page.content
doc = Hpricot(@response)
html = doc.search("a>img").search("..")[1,41].to_html
File.open("/home/akob/log",'w'){|f| f.write(html)}

Rujukan Hpricot selector:
https://github.com/hpricot/hpricot/wiki/hpricot-css-search
https://github.com/hpricot/hpricot/wiki/supported-css-selectors
anda juga boleh mencuba secara online pada url demo dibahagian Sumber dibawah

Hasilnya:

 

Sila tinggalkan komen dibawah jika anda mempunyai sebarang idea & pendapat.

Sumber:
http://hpricot.com/demonstrations
http://rorguide.blogspot.com/2011/04/simple-ruby-screen-scraper-using.html

DOS Batch RASDIAL script read from a txt list

Harini aku bangun lambat, kelas pun x pi. Apa mood aku pn taktau.. aku perlu keluar cari kawan2 nii.. tapi smlm aku ada buat coding untuk test account list yg aku dah scan dulu. sebenarnya objective aku adalah utk mendapatkan acc yang paling laju connectionya.. tapi sumber coding DOS Batch script ni agak sikit.

Aku akan buat coding batch script dalam linux plak.. boleh aku buat auto test download speed sekali nanti.

Anyway, kita kena ada acc list yang formatnya macam ni untuk batch script ni boleh baca list ni.

so script tu rupa dia cam ni

::==
@echo off
setLocal EnableDelayedExpansion

for /f "tokens=* delims= " %%a in (pass.txt) do (
echo connecting for %%a
rasdial s4 %%a
CHOICE /T 30 /C ync /CS /D y
)
::==

pendek jer an.... yang choice tu sebenarnya supaya ada timewait ja

so bila run script tu dekat CMD nampak la camni... (p/s: kalau nak run script tu just taip nama file yang kita save coding tu.. cth kita save as dial.bat dekat cmd just taip dial & enter)

Bus Ticketing

Backend

Ruby On Rails Active Record data format type :primary_key, :string, :text, :integer, :float, :decimal, :datetime, :timestamp, :time, :date, :binary, :boolean.

The types supported by Active Record are  
:primary_key,
:string,  
:text,  
:integer,  
:float,  
:decimal,  
:datetime,  
:timestamp,
:time,  
:date,  
:binary,  
:boolean

jz another helpful reference for a newbie like me ~.~

Installing Papperclip + Rmagick2 + ImageMagick ruby 1.9.2 & Ruby on Rails @ ubuntu.

I've already tried to implement this plugin last month or more, but it was a failure because i did not RTFM. I just get it to work few minutes ago & manage to wrote this down so that i'll not forget.

The key feature of successfully install those plugin, gem & image processor is as below :
1. Delegate libraries

• FreeType, Version 2.0 or above, to annotate with TrueType and Postscript Type 1 fonts.
• libjpeg to read and write JPEG v1 format images.
• The PNG library, to read and write PNG format images.
• libwmf 0.2.5 or later, to read and write Windows Meta File (WMF) format images.
• Ghostscript version 8.10, to read and write the PDF and PS document formats.

2. Decide the version of ImageMagick, Rmagick for your Ruby Version.

RMagick Version Feature Table

Feature	RMagick 1	RMagick 2
Works with Ruby < 1.8.5*	Yes	No
Works with ImageMagick 6.0.0-6.2.9	Yes	No
Works with GraphicsMagick	Yes	No
Fully supported	Yes	Yes
Works with ImageMagick 6.3.5 and later	Yes	Yes
Available as a Windows RubyGem?	No	Yes
Gets new features	No	Yes
Works with Ruby 1.9	No	Yes
Supports new ImageMagick features	No	Yes

* This is de facto that i did not concern during my previous installation which screws me.

3. Library dependencies for ImageMagick
just use the synaptic package manager to install ImageMagick and its dependent libraries. This is really time saving. I just use it and it's work!
* Note : I already do a tarball installation before i tried the synaptic package manager. After download & untar the ImageMagick, I configure it with this option below and make, then make install.

./configure --disable-static --with-modules --without-perl \
     --without-magick-plus-plus --with-quantum-depth=8

I don't know whether this configure do effect my ImageMagick synaptic package manager installation.

4. Install the Rmagick

gem install rmagick

this will automatically install me rmagick (2.13.1). The latest version.

5. Test Rmagick at ruby console
I use

rails c

to enter my ruby console. Enter commands below inside your ruby console. 

require 'RMagick'
Magick::Image.new(110, 30){self.background_color = 'white'}.write('/tmp/test.jpg')

Then check your image at /tmp/test.jpg

6. Install Papperclip
rails plugin install git://github.com/thoughtbot/paperclip.git
I'm not going to tell more about papperclip. Please read the ASCIIcasts

Yeay you are now capable of defeating facebook. haha

edited from the original source : http://rmagick.rubyforge.org/install-linux.html
Please add a comment if i am wrong in any part & we may share all here.

thx in advance
Ya'kob Ubaidullah