Center for REALTOR® Technology Web Log

Warning: Totally geeky and non-iPhone related post!

With our move to VOIP phones last year, our e-mail and our voice mail became integrated. For most of NAR, this means in their Lotus Notes INBOX they see their voice mail as just another e-mail message with a wav attached. They can listen to the e-mail via their phone, or from their desktop computer.

For various reasons CRT runs its own mail server, and my e-mail is on that server. The nice part is I still get my voice mail as an e-mail with a wav attached. I just lose the ability to listen to my voice mail via a phone. With the way I am from a personality standpoint, even if I could do it via phone, I’d still listen via a computing device.

The wavs are uncompressed so they can get quit large depending on the length of the voice mail left. While disk space on the mail server isn’t really a concern, I’d still like the files to be smaller. The biggest win will be for when I’m on the road for NAR and in a hotel who’s net connection isn’t that hot. (It happens more often than not, sadly.) Also, when checking voice mail on a mobile device a smaller download in a well supported wouldn’t hurt either.

I decided that I’d write a script to convert the wavs in the e-mail to mp3s on the fly. Since I’m using procmail on our mail server to do server-side sorting, its really easy to write a program/script to act as a filter to work with procmail. With what I know about procmail, having done similar things in the past, I knew that I’d receive the mail in raw format via STDIN and that I’d want to write my filtered mail out STDOUT in a similar fashion. Since my current scripting love is ruby, I thought I’d start there.

The rubymail (or rmail as its called in the rubygems repository) is a library for easily parsing and editing MIME e-mail messages. This is exactly what I needed. What I end up doing for the given message is walking each part of the message and if its a wav I then extract the wav, encode it as an mp3, and then replace the wav in the mail message with the mp3.

I ran into a few interesting issues that made this more difficult than I thought it would be.

The first is that the wav file from our voice mail system is in a slightly uncommon encoding, namely ITU G.711 mu-law. The problem with that is that lame, the mp3 encoder I’m using, doesn’t know how to read it without being compiled specially. As I wanted to just use the lame rpm from one of our repositories, I’d need to find a work around. This is where the venerable swiss-army knife of sound, sox, comes in. Sox can quickly transcode from the uncommon encoding to a more common wav encoding that lame is happy with.

The next problem is that I would need to create temp files instead of what I would have like to do: just pipe from sox to lame in memory. I ran into some issues where a lot of a wav’s metadata is in its header, and just piping between programs will not easily work due to the length needing to be tweaked in the header after the file is processed. I decided to use ruby’s Tempfile class to create the tempfiles. Its thread-safe on the name creation, but its not set up to just deliver a filename. So in some cases I create the file and then close it right away. Its really icky. Since every ruby class is always open, I thought I might expose a private method it has that does the filename creation, but that seems icky too.

This could be easily adapted to connect via IMAP and do similar translation.

I’ve had this script in production for about a week and its rocking hard. 1MB voice mail has droped down to 100K. That’s fairly significant space savings. Not bad for a few hours fun.

Here’s the script, how to call it from procmail is in the comments. (Or you can click here to download it.)

#!/usr/bin/env ruby
#
# Written as procmail filter and requires sox and lame
#
# Sample procmail recipe
# :0
# * ^message-context: voice-message
# {
#     # Turn the wavs into mp3s
#     :0fc
#     |/usr/local/bin/convert.vm.to.mp3.rb
#
#     # Save the voice mail
#     :0
#     .VoiceMail/
# }

require 'rubygems'
require 'rmail'
require 'tempfile'

# This visits each part of a multi-part message and does our needed
# processes.  Will call itself recursively in the case of an attached
# mail message.
def process_message(message)
  message.each_part do |p|
    type = p.header['Content-type']

    # If the part is actually a mail message, we need to create a new
    # mail message and walk its parts.  The NAR VM system treats a
    # forwarded voice mail the same way as a mail message forwarded as
    # attachment.  Pretty smart, actually.
    if !(/^message/rfc822/.match(type).nil?)
      inner_msg = RMail::Parser.read(p.body)
      process_message(inner_msg)
      p.body = inner_msg.to_s
    end

    next if /^audio/wav/.match(type).nil?

    dispo = p.header['Content-Disposition']
    dispo = dispo.gsub('.wav', '.mp3')
    p.header.set('Content-Disposition', dispo)

    new_body = nil

    # Create our temp files.
    tempwav = Tempfile.new('wavconv')
    tempwav << p.decode
    tempwav.close(false)

    tempfixwav = Tempfile.new('wav2conf')
    tempfixwav.close(false)

    # I would have liked to do this all in-memory using IO.popen, but
    # i was having trouble sending stuff between the processes.  We'll
    # have to try it again later.  It looks like the leading cuase of
    # this problem is how "smart" the wav header is.

    # System out to sox to make it an easier to deal with wav
    system("sox -q #{tempwav.path} -u -twav #{tempfixwav.path}")

    # Run it through lame to turn it into an mp3.  Then read it
    # in and base64 encode it.
    #
    # I should add id3 tags to the voice mail.  Overkill, sure, but
    # what fun.
    IO.popen("lame --quiet -b16 #{tempfixwav.path} -") do |lame|
      new_body = [lame.read].pack("m")
    end

    p.body = new_body
    p.header.set('Content-Type', 'audio/mpeg')
  end
end

orig_mail = STDIN.read

message = RMail::Parser.read(orig_mail)

message.header['X-Converted'] = 'wavs converted to mp3 by ktg'

if message.multipart?
  process_message(message)
end

print message.to_s