Smorgasbord - Politics, Lisp, Rails, Fencing, etc.

My musings on assortment of things ranging from politics, computer technology and programming to sports.

Tuesday, May 16, 2006


On this day:

acts_as_classifiable

Consider a scenario where you have a blog, and you want to prevent spam comments, but don't want to use captcha as they are bad from accessibility standpoint. What is the solution, in steps `acts_as_classifiable'. Use a Bayesian classifier to distinguish between spam and non-spam and if the comment is flagged as a spam, use a captcha based solution or reject it. Or maybe you want to track the preferences of each user and then based on that make suggestions to them. `acts_as_classifiable' can help in both scenarios and several other. Currently I use it for the web application at Kreeti.com.

To use this plugin, you need to have the gem `classifier' and its dependencies installed. The command below should do it.

gem install classifier --include-dependencies

The plugin itself can be downloaded from
http://opensvn.csie.org/sksinghi/acts_as_classifiable/

Next your database needs to have a table named `classifier_models'. This is used as a persitent store for the built classifier model.

create_table :classifier_models, :force => true do |t|
t.column :identifier, :int
t.column :classifiable_type, :string, :null => false
t.column :data, :blob
end

Now, to use this plugin in your model, put:

class Comment < ActiveRecords::Base
acts_as_classifiable :fields => ["text"], :categories => ["Spam", "Legit"]
end

Let us assume that we have an instance of the above model in '@comment'. Then the classifier can be trained by calling the method `train'

@comment.train :legit

or

@comment.train :spam

Better have some additional helper functions in the model which will do so.
You can also untrain (use it with care) an instance, by using

@comment.untrain :spam

Bulk training and untraining is also possible by

Comment.train @comments, @classifications

where both @comments and @classifications are arrays, such that @classifications contain categorization of each message in @comments.

To use, the classifier to make classification:

@comment.classify

this will return either "Spam" or "Legit". Bulk classification is also possible.

If you want the comment class to have multiple classifiers, one for each user, then all the above methods can be given an additional argument `identifier'.

@comment.train :legit, @user.id

This will create and store a classifier for that particular model, identified by the `identifier'. This can be used in scenarios when one wants to track preference of each user and then want to make suggestions.

The Bayesian classifier in the gem classifier needs some work, but more about it later.

Any questions/issues about this plugin, please post it as a comment or email me.

Update: New release Clusterer + other plugins

93 Comments:

At Fri May 19, 07:30:00 AM 2006, Anonymous Anonymous said...

Cool!

 
At Fri May 19, 09:58:00 AM 2006, Anonymous Anonymous said...

It would be really great if the community gets together to provide out of the box training for various classifiable models, spam being an obvious one...

 
At Sat Mar 10, 04:56:00 PM 2007, Blogger Graham Siener said...

Hi Surendra,

I'd like to use this plugin, unfortunately when I try to train something, I get the following error:

NoMethodError: private method 'word_hash' called for #String:0x48d7940, with the trace leading back to acts_as_classifiable.rb:111

This is Ruby 1.8.5 on XP. Any ideas?

 
At Sun Mar 11, 10:54:00 AM 2007, Blogger Graham Siener said...

Turns out there's a dependency on stemmer. So I added

require 'stemmer'

to the acts_as_classifiable.rb file and I'm all set.

 
At Tue May 08, 11:28:00 AM 2007, Blogger todd said...

Is it possible to have a variable number of categories to classify? In otherwords could I have a model declared such as,

class Content < ActiveRecord::Base
acts_as_taggable
acts_as_classifiable :fields => ["summary"], :categories => Tag.find_all

end

As the number of tags in my model grow, will the classifier reclassify?

Thanks for the great work!

 
At Wed May 09, 11:31:00 PM 2007, Blogger Surendra Singhi said...

Thanks todd. No, the classifier won't reclassify, so if you want to use variable number of categories, it won't work. In case you are using this plugin for something else, you may want to look at the new version, see the update above.

 
At Sat Aug 18, 12:50:00 AM 2007, Anonymous Anonymous said...

Great article! Thanks.

 
At Sat Aug 18, 06:42:00 AM 2007, Anonymous Anonymous said...

Thanks for interesting article.

 
At Sat Aug 18, 06:42:00 AM 2007, Anonymous Anonymous said...

Nice Blog!

 
At Tue Sep 04, 06:15:00 AM 2007, Anonymous Anonymous said...

Very interesting site. Blog is very good. I am happy that I think the same!

 
At Sun Sep 09, 12:08:00 PM 2007, Anonymous Anonymous said...

Nice! Nice site! Good resources here. I will bookmark!

 
At Sun Sep 09, 08:17:00 PM 2007, Anonymous Anonymous said...

Excellent website. Good work. Very useful. I will bookmark!

 
At Sun Sep 09, 10:11:00 PM 2007, Anonymous Anonymous said...

I see first time your site guys. I like you :)

 
At Tue Oct 09, 08:34:00 AM 2007, Anonymous Anonymous said...

Viagra
Generic Viagra
Google
Cialis
-------------------------------------------------------
Levitra
Propecia
Meridia
Yahoo!
-------------------------------------------------------
Zocor
Soma
Prozac

 
At Fri Oct 26, 07:45:00 AM 2007, Anonymous Anonymous said...

Indulge yourself and find partners for hot Sexual Encounters and Adult Dating at Adult friend finder free dating site!
If you are looking for a one night stand or a casual encounter, then the Adult Swingers Club is where the game is played.
If you practice a different range of sexual and sensual activities then adult personals has the Club for you.
Download over 2000 adult dvd movies, available formats: windows, mpeg, psp and ipod!
Shop for Adult Toys, DVDs and Lingerie and other Erotic Adult Products at adult sex toys shop and adult toys store.

 
At Tue Oct 30, 07:30:00 PM 2007, Anonymous Anonymous said...

qdf86V Thanks to author.

 
At Thu Nov 01, 07:23:00 PM 2007, Anonymous Anonymous said...

OIL4f5 The best blog you have!

 
At Fri Nov 02, 10:16:00 AM 2007, Anonymous Anonymous said...

a88Ykf Thanks to author.

 
At Fri Nov 02, 11:09:00 AM 2007, Anonymous Anonymous said...

Magnific!

 
At Fri Nov 02, 11:58:00 AM 2007, Anonymous Anonymous said...

Wonderful blog.

 
At Fri Nov 02, 12:49:00 PM 2007, Anonymous Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

 
At Fri Nov 02, 01:54:00 PM 2007, Anonymous Anonymous said...

Hello all!

 
At Sat Nov 03, 03:28:00 AM 2007, Anonymous Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

 
At Sat Nov 03, 08:05:00 AM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Sat Nov 03, 09:07:00 AM 2007, Anonymous Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

 
At Sat Nov 03, 10:10:00 AM 2007, Anonymous Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

 
At Sat Nov 03, 11:00:00 AM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Sat Nov 03, 11:05:00 PM 2007, Anonymous Anonymous said...

Q4C2Kc write more, thanks.

 
At Sun Nov 04, 08:35:00 PM 2007, Anonymous Anonymous said...

Good job!

 
At Sun Nov 04, 09:11:00 PM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Sun Nov 04, 11:02:00 PM 2007, Anonymous Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

 
At Sun Nov 04, 11:33:00 PM 2007, Anonymous Anonymous said...

Magnific!

 
At Mon Nov 05, 12:12:00 AM 2007, Anonymous Anonymous said...

Hello all!

 
At Mon Nov 05, 12:41:00 AM 2007, Anonymous Anonymous said...

Magnific!

 
At Mon Nov 05, 01:08:00 AM 2007, Anonymous Anonymous said...

Thanks to author.

 
At Mon Nov 05, 01:42:00 AM 2007, Anonymous Anonymous said...

Good job!

 
At Mon Nov 05, 02:13:00 AM 2007, Anonymous Anonymous said...

Hello all!

 
At Mon Nov 05, 02:54:00 AM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Mon Nov 05, 03:37:00 AM 2007, Anonymous Anonymous said...

Wonderful blog.

 
At Mon Nov 05, 04:11:00 AM 2007, Anonymous Anonymous said...

Hello all!

 
At Mon Nov 05, 04:41:00 AM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Mon Nov 05, 05:14:00 AM 2007, Anonymous Anonymous said...

Lottery: A tax on people who are bad at math.

 
At Mon Nov 05, 05:53:00 AM 2007, Anonymous Anonymous said...

When there's a will, I want to be in it.

 
At Mon Nov 05, 06:27:00 AM 2007, Anonymous Anonymous said...

Oops. My brain just hit a bad sector.

 
At Mon Nov 05, 06:54:00 AM 2007, Anonymous Anonymous said...

A lot of people mistake a short memory for a clear conscience.

 
At Mon Nov 05, 07:22:00 AM 2007, Anonymous Anonymous said...

Clap on! , Clap off! clap@#&$NO CARRIER

 
At Mon Nov 05, 07:59:00 AM 2007, Anonymous Anonymous said...

The gene pool could use a little chlorine.

 
At Mon Nov 05, 08:30:00 AM 2007, Anonymous Anonymous said...

Calvin, we will not have an anatomically correct snowman!

 
At Mon Nov 05, 09:04:00 AM 2007, Anonymous Anonymous said...

When there's a will, I want to be in it.

 
At Mon Nov 05, 09:34:00 AM 2007, Anonymous Anonymous said...

What is a free gift ? Aren't all gifts free?

 
At Mon Nov 05, 10:04:00 AM 2007, Anonymous Anonymous said...

Clap on! , Clap off! clap@#&$NO CARRIER

 
At Mon Nov 05, 10:37:00 AM 2007, Anonymous Anonymous said...

A lot of people mistake a short memory for a clear conscience.

 
At Mon Nov 05, 11:13:00 AM 2007, Anonymous Anonymous said...

Suicidal twin kills sister by mistake!

 
At Mon Nov 05, 12:00:00 PM 2007, Anonymous Anonymous said...

Energizer Bunny Arrested! Charged with battery.

 
At Mon Nov 05, 12:39:00 PM 2007, Anonymous Anonymous said...

Suicidal twin kills sister by mistake!

 
At Mon Nov 05, 01:09:00 PM 2007, Anonymous Anonymous said...

Ever notice how fast Windows runs? Neither did I.

 
At Mon Nov 05, 01:45:00 PM 2007, Anonymous Anonymous said...

Beam me aboard, Scotty..... Sure. Will a 2x10 do?

 
At Mon Nov 05, 02:16:00 PM 2007, Anonymous Anonymous said...

Save the whales, collect the whole set

 
At Mon Nov 05, 02:41:00 PM 2007, Anonymous Anonymous said...

I don't suffer from insanity. I enjoy every minute of it.

 
At Mon Nov 05, 03:07:00 PM 2007, Anonymous Anonymous said...

Oops. My brain just hit a bad sector.

 
At Mon Nov 05, 03:40:00 PM 2007, Anonymous Anonymous said...

Wonderful blog.

 
At Mon Nov 05, 04:06:00 PM 2007, Anonymous Anonymous said...

Save the whales, collect the whole set

 
At Mon Nov 05, 04:48:00 PM 2007, Anonymous Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

 
At Mon Nov 05, 05:16:00 PM 2007, Anonymous Anonymous said...

Beam me aboard, Scotty..... Sure. Will a 2x10 do?

 
At Mon Nov 05, 05:56:00 PM 2007, Anonymous Anonymous said...

Hello all!

 
At Mon Nov 05, 06:28:00 PM 2007, Anonymous Anonymous said...

A lot of people mistake a short memory for a clear conscience.

 
At Mon Nov 05, 06:57:00 PM 2007, Anonymous Anonymous said...

Calvin, we will not have an anatomically correct snowman!

 
At Mon Nov 05, 07:30:00 PM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Mon Nov 05, 08:51:00 PM 2007, Anonymous Anonymous said...

If ignorance is bliss, you must be orgasmic.

 
At Mon Nov 05, 09:29:00 PM 2007, Anonymous Anonymous said...

What is a free gift ? Aren't all gifts free?

 
At Mon Nov 05, 10:03:00 PM 2007, Anonymous Anonymous said...

Save the whales, collect the whole set

 
At Mon Nov 05, 10:46:00 PM 2007, Anonymous Anonymous said...

Oops. My brain just hit a bad sector.

 
At Mon Nov 05, 11:14:00 PM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Mon Nov 05, 11:51:00 PM 2007, Anonymous Anonymous said...

Beam me aboard, Scotty..... Sure. Will a 2x10 do?

 
At Tue Nov 06, 12:32:00 AM 2007, Anonymous Anonymous said...

What is a free gift ? Aren't all gifts free?

 
At Tue Nov 06, 01:14:00 AM 2007, Anonymous Anonymous said...

Oops. My brain just hit a bad sector.

 
At Tue Nov 06, 02:05:00 AM 2007, Anonymous Anonymous said...

Oops. My brain just hit a bad sector.

 
At Tue Nov 06, 02:49:00 AM 2007, Anonymous Anonymous said...

Lottery: A tax on people who are bad at math.

 
At Tue Nov 06, 03:28:00 AM 2007, Anonymous Anonymous said...

Build a watch in 179 easy steps - by C. Forsberg.

 
At Tue Nov 06, 04:12:00 AM 2007, Anonymous Anonymous said...

actually, that's brilliant. Thank you. I'm going to pass that on to a couple of people.

 
At Tue Nov 06, 05:07:00 AM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Tue Nov 06, 05:58:00 AM 2007, Anonymous Anonymous said...

What is a free gift ? Aren't all gifts free?

 
At Tue Nov 06, 06:45:00 AM 2007, Anonymous Anonymous said...

Lottery: A tax on people who are bad at math.

 
At Tue Nov 06, 07:23:00 AM 2007, Anonymous Anonymous said...

Ever notice how fast Windows runs? Neither did I.

 
At Tue Nov 06, 08:01:00 AM 2007, Anonymous Anonymous said...

Energizer Bunny Arrested! Charged with battery.

 
At Tue Nov 06, 08:41:00 AM 2007, Anonymous Anonymous said...

Lottery: A tax on people who are bad at math.

 
At Tue Nov 06, 09:12:00 AM 2007, Anonymous Anonymous said...

Hello all!

 
At Tue Nov 06, 09:57:00 AM 2007, Anonymous Anonymous said...

Clap on! , Clap off! clap@#&$NO CARRIER

 
At Tue Nov 06, 10:40:00 AM 2007, Anonymous Anonymous said...

Lottery: A tax on people who are bad at math.

 
At Tue Nov 06, 11:17:00 AM 2007, Anonymous Anonymous said...

Ever notice how fast Windows runs? Neither did I.

 
At Tue Nov 06, 12:07:00 PM 2007, Anonymous Anonymous said...

Please write anything else!

 
At Tue Nov 06, 12:59:00 PM 2007, Anonymous Anonymous said...

What is a free gift ? Aren't all gifts free?

 
At Tue Nov 06, 01:37:00 PM 2007, Anonymous Anonymous said...

Lottery: A tax on people who are bad at math.

 

<< Home