Archive for the 'Tools' Category

I Hate You More Then Ever XML

I love Milkman Dan.Okay, so I don’t actually hate XML.

But recently I have been working on writing a syndication tool and I am beginning to agree with a lot of people that question the use of XML for simple data exchange. XML was originally supposed to be both machine and human readable, and in the case of using XML to create structured documents, like XHTML, it is. It was an offshoot of SGML but had much stricter and therefore simpler syntax rules. But then people started to try and use XML for any sort of communication over the network; CSV files got turned into XML (at no real gain other than it’s XML), protocols for method invocation over HTTP (SOAP), to defining the interface for those method invocations (WSDL) and now it seems, for any data exchange out there, a lot of people think that you need to do it in XML, and that you should define the XML via an XSD (XML Schema Definition).  Now XSDs I hate!  In defining the schema of an XML document using XML you are using an crude tool for the task of exchanging data by using a terrible tool for the task of defining a schema. XSD is painful unless you have some sort of tool to to help you.  Don’t believe me, here is the XSD for syndication. Maybe I am crazy but I think that a schema definition language should be human readable and I don’t think XSD is.  The arguments for XML are many, but mostly seem to revolve around it being a standard, and that there are a lot of tools that exist for it.  So XML has evolved from a simplification of SGML for the creation of structured documents, to a catch all hammer in the toolbox of many software designers. Soon people will start suggesting that we just write the programs that run XML based files in some sore of XML based programming language (oh wait, they did that already with XSL and XSLT). There has to be a better way.

Right now I have been looking at other data exchange formats and have been focusing on JSON and YAML. Both are more human readable (YAML even more so than JSON) and have less weight to them than XML for data exchange.   They are standards with decent library support and can cover any structured data format that XML can.  There is even a tool out there to create verifiable schemas for both JSON and YAML called Kwalify. I also am starting to think that there needs to be a language for defining schemas in a language/platform neutral way. This language could be used by tools to generate things like XSD (if you have to use XML), YAML for Kwalify, SQL etc.  This language becomes like a DSL (Domain Specific Language) for defining schemas.  I know there are a lot of people that think that creating a parser for a new language is hard, but using tools like ANTLR and yacc it’s fairly easy and a powerfull addition to your developers toolbox.  As Martin Fowler says, don’t be afraid of creating parsers! We need to start thinking about the proper use of XML as a tool. It has it’s place, but there are better tools out there for doing many of the things that XML is currently used for. Also, is the obsession with using XML for everything preventing us from creating even better tools?  It’s something we need to think about.

PS: Apologies to Max Cannon, and many thanks to folks that helped create Build Your Own Meat!

Fun with Visualization

In my reading this morning, Chicagoist had a post which alerted me to Wordle.  We’ve been using word clouds in some of the flyers we’ve been making lately.  (And by we, I mean Ana and her excellent work.) Wordle is a quick and effortless way to make clouds that you are then free to do anything you like with.

For some geeky fun, I took the C++ source code (and header file) to librets::RetsSession from the libRETS source code, and generated a could. Who woulda that that in C/C++ return was the most recurring word? Okay, yeah, EVERYBODY.  Click on the image for a larger view.

Save The Data

On Last weeks episode I spoke about an easy way to get a little better data piece of mind. This of course was after a horrible, data laziness accident.

My accident wasn’t catastrophic, it was some pictures of the my dog. But they were the only pictures, and of course I wanted salvage them. I tried hooking the hard drive up to an external enclosure and connecting via windows. Boy did that give all sorts of pain to Windows. It saw the drive, and either just said ‘Do you want to format’ NO!!!!! or it just locked up file explorer.

So, the easy way was out. I needed to come up with a better plan. I decided to download DSL Linux. DSL linux is a small linux distribution that can be booted off a CD/USB Stick. Once you boot into it, it has a bunch of tools and utilities for file system exploring, as well as Wifi Sniffers and other utilities that you may want to tinker with, but dont want to set-up configure a whole Linux roll-out.

After some experimentation with copying the files to a good drive, removing that drive, and checking to see that I didn’t copy a corrupted file, I came up with a very smooth and easy method.

Continue reading ‘Save The Data’

Is it a sine?

A few years back, I watched MLSNI turn off their consumer facing website because members took the stance that this was reducing broker/agent centric website exposure. Ironically, seemingly overnight, this almost perfectly coincided with the flash flood of Syndication Portals (Trulia, Zillow, GoogleBase, etc).

At first, the industry was very apprehensive to giving their listings away for free. Over time they eased up, and saw how they can leverage these advertising vehicles. In all essence, they are just a more refined online classifieds model (such as the newspapers, but focused). More and more of these advertising sites began to appear, somewhat diluting the benefits, increasing the work for exposure (see Andrews Syndication Article to battle this issue) and calling more attention to what was really going on. These sites were filling a critical void in the consumer marketplace.

Continue reading ‘Is it a sine?’

We Cannot Allow Ourselves To Have a Syndication Tool Gap!

George C. ScottAll the RETS buzz these days seems to be about the new RESO Syndication standard. It promises to make the lives of syndicators (Google, Zillow, Yahoo, Trulia etc), aggregators (ThreeWide, Point2), brokers and even MLSs easier.  With a common data format to use, the workload of everyone will go down significantly.  But right now there is one actor in that list that will be left with a syndication tool gap; the broker.  What is needed is a simple easy to use tool that can allow the broker to create a syndication file reliably, even if they don’t have their own listing database.

Syndicators, aggregators and MLSs are all technology companies that already share data online with various parties.  Moving to a new common standard is fairly easy and once done will help improve their efficiency.  However, many brokers are small operations with little or no IT staff.  What they need is a tool that can run without a database, on a desktop machine. It needs to be able to read from a RETS source, and output a syndication file.  That file can then be uploaded to a syndicator or aggregator, or put up on a web site to be pulled down by same.  At the April RETS meeting in Philadelphia I demoed a proof of concept tool at the RETS Exhibition that did just that, called the RETS Proxy (I even won a prize!).  While this tool is not ready to fill the tool gap, the design I showed has serious limitations, it is basically that idea that I am going for.

With such a tool, brokers can make effective use of the standard.  We at CRT will look into filling this tool gap, I for one plan on taking my proof of concept and expanding it to fill the above requirement.  We also look forward to others stepping up, maybe filling in the gaps that we missed.  The more tools the broker has, the easier it will be for them to use the standard; a standard that would be useless without their data.

Office 2007 Compatible?

zamzar_logoMy primary desktop is Linux, so I’m an OpenOffice user. At home, I run NeoOffice on the Mac. You can just imaging that special feeling I had when I started seeing e-mail attachments with file extensions like .docx, .pptx and .xlsx! These new XML-based file formats are associated with Microsoft Office 2007 and do not (yet) work with OpenOffice or NeoOffice.

I used a free on-line service called Zamzar to convert the Office files to the more familiar .ods (spreadsheet), .odp (presentation) and .odt (word processing) formats. It worked perfectly. Submit you file and you will be notified via e-mail when it is ready to pick up the converted form.

I’m a tweeter, he’s a tweeter, don’t you want to be a tweeter too?

While at the Technology Directors conference here in Atlanta, GA I was asked by a few people in short order, what the value is in using twitter. If you don’t know what twitter is, in a feat of amazingly good timing the commoncraft.com team has come out with Twitter in Plain English, which does a great job of explaining the social aspects of the tool.

However, as Keith pointed out in his introductory post, I am a passionate about not only social media, but about lifehacking and Getting Things Done, so I use twitter in a outside-of-the-box way to make my life more productive, and I’ll share some of those ways with you below the jump. Continue reading ‘I’m a tweeter, he’s a tweeter, don’t you want to be a tweeter too?’

We get signal. Main screen turn on.

aybabtu.pngMy travels for NAR are usually a day or two, and I’m back before I start to get missed.  However, there are a few times a year (*cough* mid-year *cough* annual) where I’m gone for a week or so.  With two young children at home, its a bit of a challenge not seeing them all week.  So, what’s a good way to see people when on the road?  Video conferencing to the rescue.

Strangely, this is one area of technology I haven’t ever really played with.  On the advice of Mark, I decided to look at SIP based solutions since anything that follows the standard should interact with each other regardless of platform.  Mark suggested ekiga for Windows and Linux.  For OS X, a buddy of mine had pointed me at XMeeting.

Now that I had the software, I needed to create an account for my wife and myself at a SIP registrar.  You can set up a free account at ekiga.net, and then you’re off to the conferencing races.

As to be expected, your results will vary with how fast your net connection at the time is.  The video from home was a bit blocky, but you could tell who the people were and read their facial expressions.  The limiting factor there is probably that I only have a 768kbit up channel from our cable provider.  ekiga is originally on Linux, but the win32 port works great, so the homeside on my wife’s computer was set.  XMeeting works well, but its echo cancellation could be much better.  Its only at version 0.3.4, so hopefully that will improve with future releases.

Anyway, Daddy was seen by the kids, and all were happy.  This is something I’ll be doing at every trip now that it works so well.

Online Advertising vNext.0hhhhh!

gadget.jpgThe talk of the blogoshpere today is Google Gadget Ads. Google has been quietly rolling out a new advertising platform for the last few months. They are taking their gadgets and adwords and mashing them together. What you get is interactive, data rich ads whose main limitation is the creativity of the campaign creator. These ads can be HTML, Flash, Javascript or a combination, all of which can access real-time data feeds, images, video, and other content. So what could this mean for the Real Estate ad market?

Continue reading ‘Online Advertising vNext.0hhhhh!’

On the fly wav to mp3 saving

images.jpegWarning: Totally geeky and non-iPhone related post!

With our move to VOIP phones last year, our e-mail and our voice mail became integrated. For most of NAR, this means in their Lotus Notes INBOX they see their voice mail as just another e-mail message with a wav attached. They can listen to the e-mail via their phone, or from their desktop computer.

For various reasons CRT runs its own mail server, and my e-mail is on that server. The nice part is I still get my voice mail as an e-mail with a wav attached. I just lose the ability to listen to my voice mail via a phone. With the way I am from a personality standpoint, even if I could do it via phone, I’d still listen via a computing device.

The wavs are uncompressed so they can get quit large depending on the length of the voice mail left. While disk space on the mail server isn’t really a concern, I’d still like the files to be smaller. The biggest win will be for when I’m on the road for NAR and in a hotel who’s net connection isn’t that hot. (It happens more often than not, sadly.) Also, when checking voice mail on a mobile device a smaller download in a well supported wouldn’t hurt either.

I decided that I’d write a script to convert the wavs in the e-mail to mp3s on the fly. Since I’m using procmail on our mail server to do server-side sorting, its really easy to write a program/script to act as a filter to work with procmail. With what I know about procmail, having done similar things in the past, I knew that I’d receive the mail in raw format via STDIN and that I’d want to write my filtered mail out STDOUT in a similar fashion. Since my current scripting love is ruby, I thought I’d start there.

Continue reading ‘On the fly wav to mp3 saving’