Ruby on Rails, Io, Lisp, JavaScript, Dynamic Languages, Prototype-based programming and more...

Technoblog reader special: $10 off web hosting by FatCow!

Wednesday, August 23, 2006

More Advanced Starfish Feature

I promised you in Dynamically Add Methods to Classes Through their Objects in Ruby that there was a good use for that idea coming up. The time has come to show you how to use it.

server do |map_reduce|
map_reduce.type = File
map_reduce.input = "some_file_name.txt"

map_reduce.process = lambda do |text|

map_reduce.finished = lambda do

client do |line|
if line =~ /some_regular_expression/

Notice how I am dynamically adding methods to map_reduce in the server declaration. I define the process and finished methods. The process method is called from the client via server.process and the finished method is called when the collection has been fully processed.

Astute readers will notice that being able to dynamically add server side helper methods does a non-distribtued version of reduce (from MapReduce), which is good enough for many real world situations. Enjoy!

You should follow me on twitter here.

Technoblog reader special: click here to get $10 off web hosting by FatCow!


Anonymous Jeff said...

This was asked in a previous post but never answered, so I'm going to ask it again.

In the example above, it appears to have a file as input to process.

Let's say that file contains a listing of unique e-mail addresses. The e-mail addresses are to persons that you want to send an e-mail message to.

How does Starfish prevent multi-processes sending multiple e-mail messages to the same unique e-mail address found in the input text file?

In other words, how does Starfish prevent an e-mail address found in the text input file from receiving more than one e-mail message?

1:21 PM, August 23, 2006

Blogger Lucas Carlson said...

It uses a queue system, so two clients can never grab the same line from a file because once grabbed, that line is not accessible to any other clients. It also has a simple mutex to prevent two calls at the exact same time.

1:25 PM, August 23, 2006

Anonymous Jeff said...


Does Starfish wait for the results of each line to process before dispatching the next "line" to be processed?

Or does it read in all the lines, queues them up and then dispatches each line to be processed. This is with the understanding that some "lines" might take a shorter amount of time to process than others and thus - the results are returned to the dispatcher out of order. If this is the case, would you mind explain how Starfish handles this case since sometime you might need to know the results of line 1 before line 2

1:38 PM, August 23, 2006

Anonymous Wayne said...


How is Starfish different than using Mongrel with regards to HTTP requests?

1:40 PM, August 23, 2006

Blogger Lucas Carlson said...

Starfish reads in all the lines, queues them up and then dispatches each line to be processed. In the case where you might need to know the results of line 1 before line 2, you would have to add a server helper method for that logic, since it is very possible line 200 might be processed before line 199.

1:51 PM, August 23, 2006

Anonymous KirinDave said...

What's so confusing about the reddit complaints is that they don't really make sense given the context. Ruby is all about using component libraries to speed up individual tasks through C. Ruby has a fantastic interface for this

Saying, "Abandon Ruby, use C for numerical computation" is basically saying, "Use Ruby and a good library like Narray or Ruby/GSL, but write 4x the code."

2:32 PM, August 25, 2006

Anonymous Anonymous said...

what do you think about backgroudrb compared with starfish?

1:10 AM, March 28, 2007

Blogger Lai Weizhou said...

chung cư tháp doanh nhân
tháp doanh nhân / tháp doanh nhân hà đông / dự án tháp doanh nhân hà đông / chung cư the legend / chung cư the legend 109 nguyễn tuân
chung cư 360 giải phóng / dự án 360 giải phóng / chung cư vietracimex / chung cư 201 minh khai / chung cư thanh xuân tower / chung cư 35 lê văn thiêm
chung cư eco green city / chung cu eco green city / chung cư eco green city nguyễn xiển / chung cu eco green city nguyen xien / eco green city / eco green city nguyễn xiển / dự án eco green city / toà ct4 eco green city / toà ct3 eco green city / toà ct1 eco green city

11:08 AM, May 21, 2016

Blogger Zhenhong Bao said...

hermes outlet
ferragamo outlet
swarovski outlet
coach outlet online
toms shoes
ray-ban sunglasses
beats headphones
coach outlet online
mulberry handbags
chaussure louboutin
tory burch outlet online
cheap ray ban sunglasses
ray ban sunglasses
louis vuitton neverfull
michael kors wholesale
true religion outlet
mont blanc pens
cheap football shirts
kate spade handbags
mulberry bags
gucci outlet online
cheap jordans
nfl jersey wholesale
ralph lauren outlet
rolex watches for sale
tory burch outlet online
christian louboutin shoes
mulberry handbags
true religion jeans sale
true religion uk outlet
christian louboutin outlet
lebron james shoes
coach outlet
nike blazer pas cher
longchamp pliage

6:18 PM, May 30, 2016

Blogger Hua Cai said...

ugg boots sale
coach outlet online
cheap replica watches
burberry outlet
hollister clothing store
adidas uk
kobe 9 elite
jordan shoes
herve leger outlet
lebron shoes
ferragamo shoes
lebron james shoes
nike roshe run
nike air max 90
michael kors wallet sale
louis vuitton bags
cheap jordan shoes
tiffany jewelry
fitflops uk
michael kors handbags
longchamp bag
cartier sunglasses for men
lululemon outlet
jordan pas cher
futbol baratas
oakley sunglasses
ugg outlet
celine outlet
swarovski crystal
ray ban sunglasses
true religion uk outlet
versace sunglasses on sale
true religion outlet
cheap oakley sunglasses

10:40 PM, August 03, 2016


Post a Comment

Subscribe to Post Comments [Atom]

<< Home


If you like this blog, you might also like top photography schools.