Wed, 28 Feb 2007
Yahoo Pipes
Yahoo Pipes allows you to manipulate rss
feeds. I built my first very simple pipe this morning. It takes the Amazon
Gold Box feed and strips out the Baby
Deals. You can also much fancier stuff like match geocoded addresses in
feed items and modify the content of individual items. Too bad they don't
provide a way to screen-scrape pages to build feeds.
If I build any more pipes, you'll be able to find them on my pipes page.
Sun, 04 Feb 2007
JSTOR Proxy Greasemonkey Script
I've written my first greasemonkey script, to rewrite links to JSTOR. JSTOR contains articles from tons of journals, but you need to have a subscription to access it. My university has a license, but in order to use the site remotely, you must go through the school's proxy server. So my script appends the name of your proxy server onto the jstor.org domain, in my case libaccess.sjlibrary.org.
Update: It now supports ieeexplore.ieee.org url's too. I knew the name would come back to bite me. I guess I'll wait to rename it until I rewrite to handle multiple domains more cleanly.
Another Update: Support for
Sat, 09 Sep 2006
Restricting Mail Delivery Times with Exim
Denisa decided she doesn't want to feel compelled to check her email all day long, and asked me if there was a way to restrict the hours during which she could receive email. Since I use the magical MTA that is Exim, I was sure this must be possible. While I couldn't find explicit support for such a feature, I was able to hack something out. Here's my new local_delivery transport:
local_delivery:
driver = appendfile
envelope_to_add
file = /var/spool/mail/${local_part}
group = mail
mode = 0660
no_mode_fail_narrower
return_path_add
# hack to queue messages during certain hours
message_size_limit = ${if ! and {\
{match_local_part{$local_part}{+time_restricted_users}} \
{or {{<{${substr_11_2:$tod_log}}{21}}{>={${substr_11_2:$tod_log}}{22}}}} \
}{0}fail}
The trick I used was to force expansion failure of the message_size_limit when delivering a message to her address and when the current time matches our constraints, in this case before 9pm or after 10pm. The expansion failure causes the message to be queued. To ensure that she actually gets her queued messages during that one hour window, I added a new retry rule for our domain that retries every 15 minutes for four days, rather than the default rule which increases the interval between delivery attempts as the time on the queue increases. In case I ever want to configure other accounts similarly, I set up a localpartlist named time_restricted_users.
tech » mail | Permanent Link
Mon, 26 Jun 2006
Blosxom Static Rending with Lighttpd
To get more familiar with lighttpd, I've switch over the web server on this machine from apache to lighty. The most difficult part has been replicating my apache setup to serve static blosxom files.
My goal is to serve a static html file if 1) a static version exists, 2) the query string is
empty, and 3) the request is not a POST. Because I've switched from using the
blosxom writeback plugin for comments to Haloscan, I don't have to worry about the third
condition, but it would be nice to figure out a way to replicate
RewriteCond %{REQUEST_METHOD} !^POST [NC] with lighty.
Here's what I've come up with in my lighttpd.conf:
$HTTP["host"] == "xn.pinkhamster.net" {
server.document-root = "/www/xn.pinkhamster.net"
$HTTP["url"] =~ "^/blog" {
cml.power-magnet = server.document-root + "/power-magnet.cml"
}
# There's no content at http://xn.pinkhamster.net/ so redirect to /blog/
url.redirect = ("^/$" => "/blog/")
# I'm not using the writeback plugin anymore so rewrite any
# requests for writeback pages to the corresponding html page.
url.rewrite-once = (
"^/blog/(.+)\.writeback(\?.*|$)" => "/blog/$1.html$2",
)
# If there's not a static file served by the power-magnet, have blosxom
# serve the page.
alias.url = (
"/blog" => "/usr/lib/cgi-bin/blosxom",
)
cgi.assign = ( "blosxom" => "" )
}
In order to serve the static pages, we have to use the power-magnet feature. The power-magnet is a CML (Cache Meta Language) script which runs for each request. In this case, we only run it for requests that start with "/blog". CML is designed to improve web serving performance by serving cached, static pages when possible.
CML scripts are written in Lua. My check for
an empty query string doesn't work yet; I'm hoping to get some help on the
lighty mailing list. Update: I've updated the check for an empty
query string so it actually works. There was a logical error in my first
attempt, but neither get.maxn ~= nil nor get.maxn ~=
0 works. So now I just loop through the query string values and update
a counter if there are any. It's not very efficient, but it works. The rest
of the script just checks for either a static index.html file if the request
was for a directory or the static version of the requested page, and returns it
if it exists. Update 2: I've updated the power magnet script to
not have to loop through the get array.
-- if query string is empty and static file exists, serve static file
dr = request["DOCUMENT_ROOT"]
-- local f=assert(io.open(dr.."/debug.out","a"))
-- this doesn't work how i think it should
-- if get.maxn ~= nil then return 1 end
i = 0
for k,v in pairs(get) do
i = i+1
end
-- f:write("i = ",i,"\n")
if i ~= 0 then return 1 end
sn = request["SCRIPT_NAME"]
static = string.gsub(sn, '^/blog/(.*)$', '/static/%1')
if (file_isdir(dr .. static) and file_isreg(dr .. static .. "/index.html")) then
-- f:write("cache hit on directory\n")
output_include = { dr .. static .. "/index.html" }
return 0
elseif file_isreg(dr .. static) then
-- f:write("cache hit on file\n")
output_include = { dr .. static }
return 0
end
-- f:write("no cache hit\n")
return 1
Tue, 20 Jun 2006
Debian Virtual Private Server
I got a new virtual private server (vps) from a company called TekTonic today. I've been meaning to replace the Pentium MMX 200 that hosted this blog and my email for a long time. The virtual server is running within Virtuozzo, a commercial virtualization product, though the underlying technology is also available as the open source OpenVZ.
For $8 per month, I've got a virtual server running on a Dual AMD Opteron 246 with 128MB of RAM, 5GB of disk space, and 512kbps out (with unmetered inbound traffic). Since I had been running on an old piece of crap, I've already got everything set up to run fine within a small amount of memory. Apache is server mostly static pages from blosxom; Postman is my lightweight webmail application for the rare occassion when I need it; visitors for web stats; uw-imapd and exim (though I am using the heavy build with exiscan-acl) for mail; and the good old jabberd in case I ever find someone who uses jabber/xmpp. If I were running a database or some java application server, I'd need more ram.
The bandwidth is capped by the virtualization software which means you can't spike, but it also means no overage charges. TekTonic charges $50/month for an additional megabit which is pretty good.
They offer CentOS 4, Fedora Core 4, Suse 10.0, and Debian 3.1, and they had my Debian server provisioned in about 10 minutes.
I've only been running on the new server for about 7 hours, but so far, I'm very impressed. If you're reading this, the server is still up, so that's good.
Mon, 01 May 2006
Google and A9
A few days ago, Amazon's A9 replaced their main web search results pane, previously filled with results from Google, with "Windows Live" results. So I changed back my default columns to Google and Wikipedia. Then again this morning, Google was gone and Windows Live was back. But now, the Google results are nowhere to be found.
Fri, 14 Apr 2006
Amazon's Photo Yellow Pages
Amazon's new Yellow Pages service with photos is the coolest thing I've seen in quite a while. There are a couple kinks to work out. I saw a couple pictures of buses, and they need to make sure not to take pictures of the rising or setting sun, but overall this is a fantastic service. It will be even better when it covers residential areas and is integrated with driving directions.
If you're coming to visit me, I'm just up the street from this Chinese restaurant.
Update: Russ isn't as impressed. Apparently, they've had something similar in Madrid for years.
Concatenate PDFs
How to Concatenate PDFs
- Install pdftk. I suggest
apt-get install pdftk. - Create a shell script call pdfcat:
- Run
pdfcat file1.pdf file2.pdf file3.pdf > files1-3.pdf
#!/bin/sh pdftk $* cat output -
Mon, 03 Apr 2006
kdissert
While working on a paper on immigration for a Model UN-type project, I discovered a great program called kdissert. It's a mind-mapping tool designed to help write papers and presentations. A mind map is similar to an outline, but ideas are laid out radially.
I usually use vimoutliner to organize my thoughts for a paper, but one problem I have when using vimoutliner is getting the text into OpenOffice for final editing. I generally convert to docbook, then to xhtml which I import into OpenOffice. Unfortunately, line breaks within the original text are preserved so the text doesn't wrap properly in OpenOffice. I'm sure I could fix this by either preprocessing the text prior to conversion to docbook or processing the docbook text, but I though I'd try something new.
Kdissert can export directly to OpenOffice and LaTeX. I found a small bug in
the OpenOffice generator in which URLs to linked documents are not properly
encoded, but the author of kdissert, Thomas Nagy, quickly addressed it. A PNG
of the entire mind map can also be generated.
Unfortunately, I never finished my immigration paper; I couldn't
figure out a way to reconcile my position with that of the US government which
I was supposed to be representing in the simulation.
Sun, 19 Feb 2006
iPod Shuffle Still Won't Shuffle
I thought that perhaps the reason why my iPod Shuffle wouldn't shuffle was that I had created the directory structure and loaded the files using gtkpod rather than iTunes. To test this theory, I wiped everything from the iPod's filesystem, started up VMware with Windows XP, installed iTunes, and loaded about three albums of music onto my iTunes-initialized iPod. I disconnected the Shuffle, switched to shuffle mode, and pressed play. Nothing. Back in linear mode, it works fine.
As a new consumer of Apple products, I'm not terribly impressed. The end cap on my shuffle doesn't quite fit correctly. When it's on, I can feel the raised edge where the cap meets the Shuffle. It's not a big deal, but it doesn't live up to the expectation that Apple products are engineered to perfection. It definitely falls short of the excellence achieved by the Tivo Remote, for example.
I'll probably stop into an Apple store to see if they'll replace my defective Shuffle. I'm not expecting much, though. My friend, Randy, is on his second defective iPod. His current one is having a battery problem and (from what I overheard in the cubicle farm), even though it is still under warranty, Apple wants him to pay $30 to ship it to them.
