Musings of an anonymous geek

November 6, 2007

Deep Thoughts: Printing

Filed under: Big Ideas — m0j0 @ 1:52 pm

Some day, a language will exist that works as flawlessly across platforms as printing sucks across platforms. That will be a wonderful day (until and unless you have to print something).


November 1, 2007

Python Magazine Defies Skeptics

Filed under: Big Ideas,Productivity,Python,Scripting,Technology — m0j0 @ 8:40 am

I was informed today by the publisher that Python Magazine has been deemed “viable” using all of the important business metrics that they use to evaluate the magazine. This is fantastic news, and speaks volumes about the viability of the magazine in *non* business terms, as well as the model we’ve been employing at MTA since 2002.

We’ve never (yet) done anything to market the magazine. We didn’t really do a whole bunch of market analysis and research. We don’t pitch old guys in suits to convince them to fund our work. Not with php|architect, and not with Python Magazine. In each case, there was someone with a passion for the language, who was plugged into the community, who could see that a magazine would be valued as a tool by the community. In each case, we could see that there were people with great knowledge, and people with relatively little knowledge, and that those people didn’t often get around to finding each other to share that knowledge.

What we found with php|architect was that the magazine served as a bridge between those who have knowledge, and those who want it. We’re finding the same exact thing happening with Python Magazine.

There are millions of things we’ve discovered about how people consume documentation and think about languages and lots of other things along the way. It’s an immensely interesting business to be involved in. We’ve learned a whole lot about how publishing, distribution, translation, and even weird things like banking work in other countries all over the world. We’ve learned about how communities organize themselves into subcommunities in the digital realm and how different communication mechanisms affect how information is perceived and consumed and used. It’s fascinating stuff.

In the end, though, I think the success of the magazines is owed to the fact that the people producing them have a passion for the content. We’re still plugged into the respective communities – and not for the sake of the magazines. We’re plugged into the communities to help us perform at our day jobs, and we produce the magazine to help ourselves and our friends in the community get at the information they need.

October 23, 2007

I need a Google Apps Mashup

Filed under: Big Ideas,Productivity,Python,Scripting,Technology — m0j0 @ 7:31 am

Google Docs is nice. Calendar is really nice. Gmail is ok, too. The notion that you can more or less use any of the tools without going too far is pretty nice, and they’ve opened things up with the API just enough to get some useful plugin capabilities, *and* there’s a Python client available for the Google Data API, which is nice (my experience with Google Spreadsheets notwithstanding). The problem now is that I would like something that goes beyond a simple plugin.

Outside of my day job doing infrastructure architecture and sysadmin work (with some development thrown in for good measure), I run Python Magazine. I have a ton of communication and deadlines to track in working for the magazine; I get several article proposals per week (sometimes per day), I’m working with contract people, other editors, technical folks on the back end of things, layout folks, the people writing the checks and managing invoices, and whoever I need to talk to for business development tasks. I send emails to a great number of people every day, just for the magazine.

I use Gmail for my Python Magazine mail (my addy is forwarded to gmail. GMail also lets me send mail using my email address (otherwise, this would not be a usable solution for me).

I use Google Calendar to track deadlines. Each article deadline is a full day event in Google Calendar. I’d also *like* to use Google Calendar as something of a logging tool to track out-of-band conversations I have with people on IRC or (gasp!) in person.

The reason I haven’t gone this route yet is because there’s no interface where I can, say, search for a person’s name, and get a nice list of the things related to that person, grabbed from GMail *and* Google Calendar (not that you’d need to stop integration efforts at those two services – they’re just the two most useful to *me* right now).

For my purposes, it would even be OK if Google just added an “include calendar results” in the GMail search interface.  That would give me a list, ordered by date, of conversations via email, perhaps GTalk, out-of-band events logged with Calendar, and deadlines, also tracked via Google Calendar. It could essentially be a time line of my working relationship with a person, which can be very useful.

It might even be useful to get a time line of events and conversations related to a specific topic, rather than a specific person. If I could do a search for “contract request”, this hypothetical interface would actually spit out a time line showing all of the interactions between me and our contract person specifically in relation to commissioning articles, because I use the term “contract request” in the subject of all contract requests, and would naturally carry that consistency into notes I might take about contract requests in Google Calendar or other apps.

Well, that’s my latest idea. I’m not sure what form the app would take. Ideally it would be a web page I can get to from anywhere, but I have yet to do anything significant with Python as a web scripting language (though I’ve rewritten a whole lot of old Perl code in Python for sysadmin-ish stuff). A fat client application would inevitably *not* be useful to a whole lot of folks… I dunno. Thoughts hereby solicited on that.

Let me know if something already exists that does this, or if I just wrote this whole post for nothing because Google already does this somehow. I don’t think it does. It seems to treat applications as separate entities, and the same account using different apps are different entities as well. There needs to be a higher level vision of the user as a single object across all of the applications in order to get at the kinds of interesting uses of data that are possible and would add a lot of value to the individual services. My $.02.

October 21, 2007

“For the Community”

Filed under: Big Ideas,Technology — m0j0 @ 10:08 pm

Sometimes people claim they’re doing things for the good of the community, but I guess that doesn’t necessarily mean they intend to involve the community in the effort :-/

A group of open source/free software users in New Jersey (where I currently reside) learned that the hard way when the maintainer of a web site that advertised it was “For the Free and Open Source Software Communities of New Jersey” posted a shut down notice.

The biggest slap in the face to the community the site was allegedly for was the text of the shutdown notice itself. For example:

“Maintaining GnuJersey has been mostly fun, but I want to prune the list of blogs I read daily, and I can’t do that while I maintain a website featuring some blogs I don’t want to read.”

So… this is a site “for the community” whose shut down notice contains 5 instances of the word “I” in the single sentence that is supposed to give us some clue as to why this is happening.

But wait! There’s more!

“[The site being taken down] is not up for transfer and I will not use DNS to point to a successor blog aggregator.”

Sweet. Not only is he not entertaining the idea of maintaining the site himself, he’s also eliminating the possibility that the site will be maintained “for the community”, by anyone, at *all*.

He does offer to link to a successor site, but insists that we get permission from the syndicated bloggers (and presumably, that we prove that we have said permission), if we expect him to link to us. So, we shouldn’t have any expectations of him to live up to his word and maintain the site “for the community”, *or* to let the community maintain the site for the community, but we should honor his request to prove that we have permission from all of the authors involved to put a successor site in place.

Rich, ain’t it?

Well, I’ve cloned the site here for whoever wants to continue to keep up with their friends and colleagues involved in open source and free software in New Jersey.

October 10, 2007

The Technology Behind Python Magazine

Filed under: Big Ideas,Python,Technology — m0j0 @ 9:04 am

Hi all,

I mentioned to a buddy (who is also an editor) that we used subversion in our editorial process. He didn’t know what that was, and said that they used either this big nasty home grown system, or email attachments, to coordinate the editorial process.  He was incredibly curious about how we used subversion and what else we were using.

I started writing this kind of long email and then figured that others might be curious as well about the various technologies we use (or are moving to, etc) at Python Magazine, so here’s a quick list of tools we’re currently using:

  • Subversion – of course, I’ve mentioned this. We view every email attachment as a problem to be solved. Email is a communication tool. It is not a file transfer protocol (no, really – it isn’t), and it is certainly not a collaboration tool. We have a very simple directory hierarchy on the server representing the various stages in the editorial process, from the initial, original submission as received by the author, all the way to the final PDF rendering of the entire magazine, and all parts in between. The final review of the magazine even happens in SVN. We have a ‘corrections.txt’ file that we all add to as we review the PDF, and when that file is empty, the PDF is moved to the directory representing “go to press!”
  • Plain text – sometimes less is more. I’ve edited and authored using Word, OpenOffice, LaTeX, and a few other tools. In the end, plain text with extremely simple and minimalist formatting tags win the day by a long shot. Authors aren’t forced to use any particular tool or platform to write their articles, editors don’t have to wonder which version authors have, which language setting they were using, etc. We don’t have to wonder if our version control system will handle a binary format properly, and the files are smaller. It’s also easier to run scripts against them to do things like strip formatting, or selectively apply it given a regex or something.
  • Google Calendar – We are notified the night before any article deadline, and the calendar is shared among the editors. Theoretically, the same calendar could be used to indicate that an editor is going to be unavailable or a tech reviewer is going to receive an article, but so far, it mainly reminds us of upcoming deadlines.
  • IRC/Google Talk – We actually don’t send very much email to each other. Sometimes we talk on IRC about emails we received or need to be added to the cc list of, etc. Almost everything we do involves either IRC or Google Talk. Of the 50 or so people on the authors mailing list for Python Magazine, at least 40 have email addresses, and so do all of the editors here, so even some of the author/editor communication is email-free. In addition, the Python Magazine IRC channel is, and you can talk to editors and authors there. The only email that gets sent is:
    • subversion server updates,
    • users who need to mail info at pythonmagazine to ask subscription questions,
    • authors sending to editors at pythonmagazine to submit article ideas (we don’t take them on irc),
    • replies to threads, usually initiated by one of the above actions.
  • PHP – yes, believe it or not, the main site is written in PHP. The publishing company (MTA) was originally formed around php|architect Magazine, which is a magazine about PHP. That was in 2002. Today, there are two language-based magazines. Some day there may be five language-based magazines. Certainly, we’re not going to maintain websites using 5 different languages! O’Reilly doesn’t do it, and they publish entire *books* on different languages (and platforms! and databases!) I was impressed by the Python community’s understanding in this matter. Lesser communities would’ve sent lots of hate mail.
  • Python – Doug Hellmann (our tech editor) and myself (to a lesser degree, because Doug is far better at it) write any little tools and scripts we need using Python. Sometimes I think about writing Python scripts just to make Doug laugh. Don’t forget, I launched this magazine not because I professed any deep knowledge of Python. On the contrary – it was because I figured there were neophytes like myself who would like to know more, and advanced coders who would like to look into areas of Python outside their immediate area of expertise within the language.
  • Adobe InDesign – InDesign is the main layout tool. Layout is like some spooky ethereal realm to me. I imagine other tools are used during the layout/design process, but I don’t honestly know what they are. I’m sure the layout team prefer it that way. It’s probably better if I just say “I’d rather see the title moved up and to the right” than to start trying to tell them how to use their tools.

Those are the tools I can think of off the top of my head aside from back end things like a relatively standard LAMP stack that runs the web sites, and which I also don’t have much of a role in maintaining. Of course, there’s also one big element of all of this technology that blows them all away: the people. Every single person is technical in some way. Me, the layout folks, all of the editors for the whole company that I’m aware of, and even our fearless leader are all technical people. Technology is a common thread that runs through the entire organization, and ties all of us together. It makes an enormous difference, and I’m proud to be a part of the team.

October 8, 2007

With Great Funding Comes Great Responsibility

Filed under: Big Ideas,Sysadmin,Technology — m0j0 @ 9:28 pm

For the past ~6 weeks, I’ve been talking to people, getting buy-in, educating users and administrators, and generating copious amounts of project proposal and six sigma documentation presentig VMware Infrastructure as an infrastructure building and management tool.

There’s a whole manifesto behind this, but I’ll try to boil it down. Basically, this client has three sites, and the infrastructure needs to be consistent at all three sites. Also, ideally it would be overseen and generally managed from one site (there are obvious limits to this, but you get the idea). My thinking is that I have three choices:

  1. Order/rack/setup/test hardware and software, stage system, install stuff, ship to site, where someone else racks machine and turns it on.
  2. Assume and require that there is a senior enough admin at each site already to take care of all of that.
  3. Decouple the OS image from the hardware altogether and just build an infrastructure server “factory” at the main site, and ship (read, scp or similar) to the VMware servers at the other sites.

I chose option three – but this is oversimplification and doesn’t go into all of the benefits.

So, I just found out today that my bill has made it through Congress, and my project now has legs (read: funding)!! When the project is complete (the first phase is to migrate the main site using this methodology, and replication to other sites is a later phase of the project), I’ll try to give a talk on it or something.

In the meantime, if anyone has thoughts on virtualized infrastructure, or if you’re doing something cool with this technology, please post your comments. I value your insight!

July 19, 2007

Why you should write: common myths debunked

Filed under: Big Ideas,Productivity,Python,Technology — m0j0 @ 8:38 am

I give a talk that advocates writing. I do this because I owe my career to people who wrote down what they knew and made it available in one form or another; either free on a web site, or in a book that I bought.

When I started editing, one of my jobs was to encourage people to write for whatever publication I was working for at the time. I’m now Editor in Chief for Python Magazine, and I’m back in the role of encouraging people to make their knowledge available for others (including me!)

Python Magazine is going well in terms of authorship, but with all of the Python coders out there, I should be overwhelmed with article proposals! When I talk to people who clearly know what they’re doing, or are doing something cool, and ask them to write, the responses are, by now, pretty predictable – and mostly based on misunderstandings, myths, and other untruths. I thought I’d take a few minutes to address the most common ones here:

“I can’t write”

The king of reasons for not writing is this one. The problem isn’t really that the person literally cannot write. The problem is twofold: first, the person has absolutely zero confidence in their ability to write. Second, the person doesn’t understand what an editor is supposed to be doing to earn his keep.

I’ll tackle the second part of the problem first: an “editor” is not some gray haired guy with a cigar screaming about deadlines and rejecting 99% of everything that hits his desk. An editor is someone who works *with* writers. It’s not *supposed* to be an adversarial relationship – and it *is* supposed to be a relationship. It’s supposed to be a working relationship where the writer tries to relay concepts in a way that the *editor* can understand. In return, the editor asks questions, pokes, prods, and makes suggestions that help morph the article into something the *readers* will understand.

Further, the relationship may begin before a draft is ever submitted. I’ve helped lots of writers develop their *abstracts*, so that by the time they sit down to write the article, they have some clear idea where they’re headed. And the word is *helped*, not *dictated*. It’s not me passing back a marked-up copy and saying “fix this and do that”. It’s me writing back saying “I’m a little confused in this part, how does this work?” or “can you provide a use case in which someone would find this useful?”

Finally, this excuse may have foundations in a perceived language barrier. If English is not your first language, you may feel like you’ll never get published in an English publication. Not true. If you have the knowledge, and can write enough English to relay the concepts, a good editor will work with you to develop it into something suitable for publication. It may take a bit more time, but that’s usually not a problem. In technical publishing, knowledge trumps prose every day of the week.

“I don’t know what to write about”

One day while you’re coding, just stop. Stop and take stock of what you’re working on. In all likelihood, there’s an article in there. If you’re thinking “there are a thousand articles about this already, in addition to the online docs”, you need to know three words: “Fresh, relevant content”.

A thousand articles have been written about just about everything! Why do you think that is? In the tech world, it’s because technical people tend to pay close attention to the date of publication to figure out if the content is fresh and relevant to the current version of whatever they’re using. Because technology evolves, there is often a need to get updated information to the readers.

The other part of this objection is the idea that the reader can get the same information from online documentation, or that what you might write about is just too easy for anyone to find useful. This is usually based on an assumption that because the writer learned it in an hour reading online docs, that everyone can/will/wants to learn that way. I hate online documentation, especially for programming languages. So do lots of other people. That’s why people buy enough technical magazines for my publisher to let me head up another one!

Also, people read about things in magazines that they may otherwise never read about if it were online, and they get inspiration from these things on occasion. Why? Because it’s there. When I go to get my oil changed, I bring a magazine with me to read while I’m waiting, and I read stuff I wouldn’t normally seek out online because I have time to kill. When you see people at the DMV (or MVS, or wherever people go to get their licenses renewed or cars inspected), a lot of them have magazines with them. They have time to kill. Inspiration can come from the strangest places. So whatever it is you’re doing, write about it, and inspire someone to take whatever it is you’re doing and do something else cool with it. Even if it’s, say, the Python tempfile module or something mundane like that 🙂

There are three things I tell people about where to find article ideas:

  • Write about stuff you know/do – this is a gimme.
  • Write about stuff you *want* to know/do – You don’t have to be an expert on Twisted to write an article about it. If you’ve done some network programming with Python at all, and are wanting to get into Twisted, research doing what you want to do with Twisted, figure out if it’s a good choice, do a proof of concept, take some notes, and write an article based on what you’ve done!
  • Write about stuff that is not done, or often done wrong – Best practices documentation is severely lacking. There are a trillion articles that talk about different ways to do similar things, but very, very few articles that say “Here are three common ways this is done, and here’s why this fourth way might be better” or “Here are the pros and cons of doing this using 2 different methods”. Because best practices documentation is lacking, things are quite often done in a suboptimal way. Articles like “How not to do web programming with Python” might cover why you don’t want to use the split() function to parse URLs, for example 😛

More on the way

I’ll expand on these and address more common reasons for not writing in future posts. In the meantime, if you’d like to give writing a shot, and you write a decent bit of Python code, I invite you to come and write for Python Magazine. I look forward to working with you!

Technorati Tags: , , , , , , ,

Social Bookmarks:

July 6, 2007

On the demise of SysAdmin Magazine

Filed under: Big Ideas,Sysadmin,Technology — m0j0 @ 12:57 pm

CMP recently announced that they will cut 200 jobs, and shut down more than one magazine in the process, folding their content into other existing magazines. There has been a lot of buzz in the sysadmin community (which I know largely as a loose collection of people who belong to LOPSA, SAGE, USENIX, or local groups of various kinds) about how sad and unexpected this is and how it’s a sign of the times or something.

You Can’t Sell a Generalist’s Mag to a Market of Specialists

I actually think SysAdmin’s demise was a long time coming – in part because, well, I’m involved in the publication of a couple of magazines, and read lots of others, and know people who work in publishing on other magazines as well as large online media outlets (some of whom I also write/edit/consult with). The plain fact of the matter is that it is extremely difficult to cover a topic like system administration in a generalist sort of way when your audience no longer consists mostly of generalists.

For example, the last two issues of SysAdmin magazine I remember getting were about Database Management, and Linux, respectively. The database management issue talked about Oracle and MySQL, and then had 3 or 4 articles on things not really directly related to databases at all, if memory serves. The Linux issue is probably of no interest to the admin who was all revved up for the Oracle articles, because that admin is probably not so much a sysadmin as an “Oracle admin”. Meanwhile, the average Linux administrator is probably uninterested in the Oracle RAC Primer.

Most admins aren’t generalists anymore unless they work in academia, research, or a company small enough that there are only a couple of people to handle the entire infrastructure. Even people who would like to be more general aren’t doing generalist things in production. I know several people who work *only* on {Oracle, sendmail, websphere, whatever-other-service-you-like} at work, and nothing else, but they run Apache, Bind, Postfix, and a few other services at home. That knowledge is nice to have, but it’s hardly something you can use to market yourself as a production generalist administrator.

So the market is flooded with mail administrators, backup administrators, storage administrators, cluster administrators, network administrators, database administrators, websphere administrators, exchange administrators, desktop administrators… the list goes on and on and on. The magazine market has mostly followed suit. There are magazines about cluster computing, DB2, AIX, SQL Server, Linux servers, Windows servers, and lots of other specialized areas. Note in that list that there are two separate magazines for two separate database products, but no “DBA’s Journal”. In killing SysAdmin Magazine, CMP is just following along with market trends.

Programming has mostly gone in the same direction. Dr. Dobb’s Journal is no longer available on your local bookshelf, but you can find php|architect, a magazine about nothing but PHP, doing well. Others exist for .Net, Cold Fusion, C/C++, Java, and now even Python. I’m sure Ruby and Lua aren’t far behind.

“I know, let’s start our own magazine!”

There are lots of naysayers out there who seem to think you need a huge staff, tons of money, and loads of other resources to start a magazine. Not true. You need a few dedicated, motivated people, and a small amount of seed money, and some time and hard labor.

An example is php|architect. It started with two guys: the publisher (who was a software development consultant as well), and his business partner, who also knew Quark pretty well – plenty well enough to do the layout. Then I came on board. So after the first issue came out, the rest were all tech and copy edited by the publisher and I, and the layout was done by the other guy. That’s really minimizing the amount of work we did, but the point is you don’t need an army.

When you’re as big as CMP and you’re looking at cutting a million bucks or something from your operating expenses, you go ahead and cut away! A small outfit isn’t playing with these kinds of numbers. A full-time editor at a company like CMP probably makes a 6-figure salary. A guy like me who edits as a side job makes something like… far less than that. At that rate, you can add another editor, a tech editor, a couple of columnists, all part time, and the advertising revenue will still cover the costs.

So if you want to see a sysadmin magazine, get some dedicated people who have half a brain and go do it!

Good luck.

Technorati Tags: , ,

Social Bookmarks:

July 3, 2007

Python Magazine Lives

Filed under: Big Ideas,Me stuff,Python,Scripting,Technology — m0j0 @ 10:58 pm

I have a confession to make: For the past 6 weeks, I’ve been leading a secret double life. By day, I’m a mild mannered system/network/database admin in academia. I also write some PHP, Perl, and Python code. By night, however, I’m an author and editor. My latest project is bigger than most. In fact, it’s an entire magazine. Devoted to Python.

I am the Editor in Chief of the newly launched Python Magazine.

Why on Earth Are You Doing This?

Python Magazine was created as a result of some rather unfortunate events in my own early experiences with Python. Getting started, of course, couldn’t be easier. It was what happened after I had been coding for a while that I had issues with. Once you needed to do something a little out of the ordinary with the language, it was hard to feel confident that the way I was going was the right way.

For example, I decided to wrap up a bunch of SQL calls in Python and expose them as an API using Python’s built in SimpleXMLRPCServer. I thought this was great, because then I could maintain a single back end API, and any language that could make an xmlrpc call could use it without me having to maintain APIs in several languages. Nice in theory, but people smarter than I questioned my decision to use the built in SimpleXMLRPCServer. The right road to take, though, was completely unclear.

As another example, I needed to get up to speed on using the python-ldap module, but found that a lot of the documentation lacked anything but the most basic of features, but I was trying to write a full-fledged LDAP management API (and accompanying command line and GUI tools). Other articles I found were outdated enough that people warned me not to bother with them, pointing to glaring issues with the code samples (which turned out to be true – some of what was in the code samples turned out to be completely deprecated!).

When I wanted to write code against a PostgreSQL server, the correct module to use was also not immediately obvious, so I had to hunt down the sites of various modules, see which ones were maintained, search for articles that weren’t 5 years old on how to use them… Gah!

What I really wanted was a resource that fed me information in a way that my brain likes to feed on information. I really wanted to learn to do things with Python the way I learned to do things with Linux, Solaris, PHP, and even non-technical things like photography, billiards, and brewing beer. I wanted a magazine.

There was no magazine. I was bummed.

How did you finagle this one, jonesy?

I have a friend named Marco Tabini. He’s a publisher. He runs Marco Tabini Associates in Toronto. He is the publisher of php|architect Magazine. He’s also a total geek. For fun he does things like writing lexical parsers… in PHP. Nobody should ever do that. He thinks it’s fun. I say pass on that if you are given the chance.

Marco and I met via email. I wrote to tell him that I had received my first issue of php|architect, and would not recommend it to a friend. I had found something like 15 errors (typos and grammatical issues) in the first two pages of the magazine. Marco wrote back and said “hey, we’re a small outfit. We’re an Italian immigrant and an Iranian immigrant, living in Canada, trying to edit technical articles written by people from all over the world with varying levels of experience with English… all for a largely American audience. Come help us out!” So I did.

Shortly thereafter I became Editor in Chief of php|architect. Now there were three of us. Oh joyous day.

Those were great times, and the magazine has since spawned its own online and on-site training, its own line of books, its own series of conferences, and even a cruise! It probably has stuff I don’t even know about because I haven’t worked directly for that particular publication since 2004.

The success of that magazine gave me the courage to go to Marco about 6 weeks ago and ask about letting me head up another magazine, this time about a topic of *my* choosing. We chatted on IRC for several hours over the course of about a week, bought a couple of domain names, settled on budgets and team members and all that, and set out to make Python Magazine a reality.

So… How’s it going?

Things are *REALLY* rolling now. There are columnists, there are tech editors, there are authors. Articles have been commissioned. Logos and trademarks are in place. The design team is rocking, the contract team is rolling, and the emails are flying. In the background, the sound of constant typewriter activity can be heard, just like on those old newscasts from the old Cronkite days. Exciting times!

That said, we still need LOTS of content. The behind-the-scenes of a magazine is that you’d really like to have something like 4 months worth of content “in the can” before “Volume 1 Issue 1” is released. I’m convinced that this has never happened in the history of publishing, but it’s a great goal to have, and I’d be pleased as punch to be the first person ever to achieve it 😉

If you’re a writer who is doing or has done something interesting with Python, or can illustrate high level concepts from the fields of computer science, research computing, or IT, using Python, we’d love to hear your thoughts!

Import This

In the end, I hope I can be a good steward to the language and community. I’ve already been in touch with a lot of wonderful people – authors and others – who’ve helped out in some way, either with the magazine, with my own buggy Python code, or both. That’s all the news that’s fit to print for now, but keep an eye here and on the Python Magazine website for more updates as they happen.

Oh yeah – and if you subscribe now, you get a discount, and a chance to win a MacBook!

Technorati Tags: , , , , , , , , , , ,

Social Bookmarks:

May 14, 2007

Can Technology Kill the NAR?

Filed under: Big Ideas,Technology — m0j0 @ 1:26 pm

The NAR is the National Association of Realtors. They’re the main lobbying interest for pavement-pounding brick-and-mortar real estate agents. Of course, this is problematic for web-based real estate outfits like Redfin, because the NAR has the required influence to get legislation passed that can make life as a web-based real estate sales organization difficult, if not impossible.

NAR, Technology, and Legislation

The question is, at what point is the NAR going to step on its own toes? Does the NAR really believe that technology will play no part in the future of real estate? Well, of course not! In fact, the NAR is the keeper of the Multiple Listing System (MLS), which (when it became available) was a major technological advancement that greatly aided real estate agents in sharing data with other realtors. It made seller agents more productive because it provided a means of sharing new listings with an audience that basically encompassed the entire realty world. Since seller agents and buyer agents split commissions, it was now much easier to make your entire living based solely on getting listings. It made buyer agents more productive by enabling them to search listings on behalf of interested buyers, and being able to be kept up-to-date on new listings.

There’s also the website, which is an interface to the MLS and (according to the site itself) “the official site of the National Association of Realtors”. So if it’s not about doing without technology, then it must be about ownership of the data.

Of course! After all, the NAR is really only able to justify 6% commissions if it is the sole keeper of the *inventory* of things for sale, *and* it can influence legislation as it applies to how real estate transactions take place. For example, they’ve had some success in making it illegal for brokers to offer rebates. This makes things very hard for Redfin, specifically, because a part of their model refunds a part (actually, most) of the commission it splits with a third party agent back to their client.

What if there were no realtors? (aka Real Estate as Travel industry)

Of course, the NAR can only (currently) control legislation regarding real estate transactions that involve registered NAR realtors, so going forward, if there’s a compelling service that becomes a de facto standard marketplace for real estate (or at least, some subset of “real estate” proper), it would seem that the NAR would have two choices: find a way to justify their existence by representing a larger portion of those involved in the transactions (like the buyers and sellers themselves), or find a way to pass legislation that *requires* that realtors be involved in every transaction. Sounds impossible, but we’ve seen some pretty wacky legislation in the past, haven’t we?

I don’t really think they’ll pass the legislation needed to guarantee work for pavement-pounding realtors. I also don’t think the NAR is a breeding ground for the kind of progressive, independent thought required to take a new direction. In all likelihood, what you’re looking at when you look at the NAR and the real estate industry in general is somewhat similar to the travel industry in 1998. It’s an organization and an industry that doesn’t even know what’s about to happen. It’s an industry that believes, like the travel industry did, that “real estate is all local”. It’s an industry that, just like the travel industry, keeps its technology largely to itself. It’s an industry, just like travel, that used technology to empower agents, not customers. It’s an industry, like travel, that has largely failed to recognize the emergence of technology and the web as tools that (unbenownst to them) didn’t just make advancements *possible*, it *necessitated* a change in how they interfaced with customers, suppliers, and each other.

The young wippersnappers, with their new-fangled whirligigs, are going to change the real estate market, both for their own benefit, and the benefit of the customers.

The Customer Service Myth

And if you’re a realtor, you can save all the happy horse crap about customer service. I’ve heard it all before, believe me. I’ve purchased a couple of houses myself, and grew up in the business besides. I’ve had lots of friends and family who have worked in various parts of the industry, and while I understand that customer service is *supposed* to be the lifeblood of the industry, the reality is quite different. Customer service, and all of the things that an agent does for either side of the transaction, are largely “feelgood” services that are, at best, smoke and mirrors – and that assumes the realtor’s intent is good. In many cases, it’s just an outright scam.

How many realtors are “certified staging experts”? Ever look into all of those accreditations and certifications that started to magically appear something like 10 years ago? They’re all invented, and issued, by the very companies whose agents hold them. No conflict of interest there, eh?

And what about research? When’s the last time you went to a showing for a house only to find that it failed to meet most if not all of the criteria you set out when you first spoke to your realtor? Realtors *rarely* pre-screen houses by actually *going* to the houses anymore.

On the seller side, how do you think the realtor comes up with a price to sell your house at? They come to you with “comps”, which a lot of times amounts to houses that are similar to yours only in geographic location. In many markets, a couple of blocks difference is significant enough to render them useless. Real estate agents are not real estate appraisers, are not trained in the various appraisal methods practiced by appraisers. What’s more, they don’t really care about any deeper notion of “value”. What they really want to do, more times than not, is “price to move”. They want to price your house so that it looks really attractive compared to the rest of the market, so it will sell more quickly, so they can collect their commission more quickly.

“But doesn’t that mean a smaller payday for them?” Well, if we’re talking about the transaction in a vacuum, yeah, it does. But if a realtor can sell 8 houses per month by pricing them below the market, they’ll make far more money than if they sold 4 houses priced at or slightly above the market.

You can actually do your own comps if you take a little time to understand those things that are relevant to pricing your house and comparing it to another house. It’s not rocket science, but there are some quirks that you need to know about. Get to know those, and the rest of the research can be done using online tools, including

Going it alone

In the end, there are lots of online tools to do just about anything you want. You can research a community, research a school system, research the housing market, even answer questions like “are there any registered sex offenders in this area?” all online. Just about 100% of this information can be had for free.

Online virtual tours are commonplace. You can now see satellite images of the house, and the entire neighborhood, where you’re interested in buying. All of the tools to research any aspect of the house and neighborhood are available. The only missing piece is an organized way to bring the buyer and seller together to settle on terms and complete the transaction.

Well, it *was* missing. Redfin and other online tools like it are working to close the gap. Check out the story on TechCrunch, which links to a great 60 Minutes piece about all of this.

By the way – I don’t think realtors are going to disappear. Just like travel agents, there will always be some 80-year-olds who haven’t figured out the internet and need some local presence to get things done, but that kinda ties the lifespan of these places to the lifespan of… really old people.

Good luck with that.

Technorati Tags: , , , , , , ,

Social Bookmarks:

Next Page »

Create a free website or blog at