Posts tagged as: programming

Feeling the urge to code

I haven't been blogging much lately, because I've been spending my time reading Python books. That's one thing that separates Python from Ruby that people don't seem to mention when comparing the languages. There are tons of Python books, while there is basically just one good Ruby book. I'm going to spend a day or two more of just reading, and then I'll start doing some basic RSS coding to test out the Python libraries. Once I feel comfortable with the language, I'll get back to exploring the various forms of microcontent and their potential relationship to OPML. Don't expect much blogging here for a week or so, but I will report on my initial RSS library experiments on my code blog.

Hiring and retaining high quality programmers

I recently had to chastise one of the local startup guys here in Boston for mishandling the hiring of an old programmer of mine for some contract work. The exact details aren't important, but it reminded me how much difficulty most people have trying to hire and retain good coders. During the Nineties I managed to keep almost every programmer I hired in one of the craziest job markets for programmers we'll ever see. Out of a final team of about 20 people, I never had to fire any of them, and only had one quit over 6 years, and that was because his wife became ill. It wasn't that I paid the highest salaries. I paid well, but we were about 20% below the area's top salaries. It looks like we're moving into another programmer shortage as so many new startups get funding and try to fill out their staffs, so I thought it would be helpful to review some of the lessons I learned in Web 1.0. Some of the programming techniques and business models may have changed, but programmer psychology is still the same. These rules apply equally well to hiring consultants and full time staff, although I've usually had better results with full time employees. The pressures on consultants to juggle the maximum number of projects doesn't always allow them to focus enough on each job.

• Professional Student. The first mistake employers make is insisting on a programmer who is an expert on exactly the type of application they are being hired to write. While experience is obviously important, the best programmers are professional students. They move freely between different programming languages and application types. In many cases the hot new technology is really much easier to pick up than you think, so it isn't necessary to hire an Ajax "expert" even if you know Ajax will be used in the job. More importantly, there will always be a hot new language or technique, so I'd rather hire someone who can teach themselves new tools as they appear. Because of this I make sure during the interview to stress the need to keep learning on the job. I want someone who gets excited when they hear this. I'm also biased towards self-taught programmers. A computer science degree is useful, but it's what they have learned on their own since graduating that really matters. Speaking of graduating, I haven't found any difference between someone who drops out of college and someone who finishes. In fact, it's usually the ones who quit school because they were impatient to build things that have worked out best for me.
• Front or back end. One thing that is important to discover is whether a programmer prefers working on the user interface or internal engine of an app. This distinction seems to be fairly strong, and it's hard for most people to work equally well in both areas. User interface programming doesn't necessarily mean graphic design, which is another set of skills, but it does require focusing on lots of annoying details, none of which are particularly hard in isolation, but when combined they can be a nightmare. Engine programming, on the other hand, requires an ability to conceptualize a complex algorithm and break it down into a simple and elegant solution. Instead of asking this question directly, you can ask a potential hire to describe some of the most rewarding projects he or she has completed. These will generally be gathered in either the front or back end of an application.
• Let programmers pick their colleages. It's essential to have the other programmers meet all potential hires, both individually and in groups, and to have a say in whether they are hired. I want everyone to be able to work together, and to count on each other for different skills and areas of expertise. It takes a while to build this type of ego-less team structure, but once it's in place, they will look for people who can complement the team's skills. I learned the hard way to adopt a firm "no assholes" policy. No matter how badly I wanted someone who seemed to know exactly how to solve the problem they were being hired for, I refused to hire anyone if I and the others felt we couldn't be comfortable hanging out with them.
• Programmers don't work by the hour. The biggest mistake you can make in hiring a programmer is to think in terms of the amount of hours they will be working on a specific task. There is no way to know when someone is actually coding, even if they are sitting in front of you. The other variable is the wide range of productivity between coders. In my experience, it can be as much as 100 to 1, and 10 to 1 differences are common. The only way to measure the amount of work a programmer is doing is to set clear goals and then see how long it takes to accomplish them. Within that time period, I couldn't care less how many hours were actually spent coding. I also don't believe in holding programmers to 9 to 5 schedules. As long as the work gets done quickly and with a high level of quality, they can be as flexible with their hours as they choose. All of this is dependent on you or the person who manages the programmers actually knowing what they want to get done, and being able to tell when it is completed successfully. It's the managers who don't have a clue about what software actually is that depend on hours and timeclocks.
• Let them own it. Once a programmer completes a task, they should own it, and the rest of the team should know this. I don't mean legally, but within the structure of the development staff. Pride of authorship is the best way to get high quality work and to keep code free of bugs. When something breaks, everyone should be able to turn to the person who wrote it. Over time people will develop their own areas of expertise within the team, and others will know they can trust that person to handle a specific type of task when it comes up. This allows them to self-organize when a new project comes along that requires multiple skill sets. Eventually the number of modules or applications that a coder "owns" will grow, and you have to make sure to leave them time to take on new tasks as well. Nobody wants to be stuck doing nothing but maintenance, at least nobody you would want working for you. If there are too many maintenance jobs, then it is time to hire another programmer to take over some of this code. This new programmer should be allowed to rewrite the code he takes over if he wants. It seems like a waste of time, but you generally get new features that pay for the extra work. The important thing is to make sure the new owner also feels pride of authorship.
• Keep them learning new things. It's important to allow programmers time to experiment with new languages and tools. Sometimes the best new ideas come up when "playing" with a piece of code that has nothing to do with the normal team's work. You should be willing to pay for programming classes in the evening, and budget a certain amount for trips to developers conferences. One trick I adopted was to always have a bookcase full of the latest O'Reilly books. Programmers were encouraged to take these home and keep them. If they could get one good idea from a $30 book, it was money well spent.
• Keep the lines of command clear. Programmers have to know who is allowed to assign them tasks. The worst thing is when the VP of Sales and the CEO think they can wander up to any coder and start asking them for features. There should definitely be a way for the entire company to make suggestions for future development, but getting switched from one task to another by every executive who walks into a coder's office is terribly confusing. What coders need more than anything else is big blocks of time to work on a single task. They can have multiple projects going at the same time, but they need to manage their own time. If every angry call from a customer to sales results in a new rush project, their long-term productivity drops dramatically. There must be a buffer between sales and development to make sure these "emergencies" get prioritized and consolidated into the total work flow.
• Never cancel projects. The most important rule of managing programmers is to never, ever cancel a project. There is nothing that breaks a coder's heart like having several months or even years of work thrown away. This means that before you assign a task, you must be sure you will actually see it through to completion. That doesn't mean that requirements won't change and code won't get thrown away. Throwing away a first attempt at an application is fine, as long as the coder makes that decision, and it is a way of starting fresh once the parameters of the job are better understood. That is different from deciding that you didn't actually need that application after all, and all the work on it was a total waste of time.

Managers who think of coders as if they were hourly factory workers who put in their time and don't care what happens to the results of their work will end up with exactly that. If you give a programmer the chance to learn the skills needed, the time to build the code right, and then aknowledge their ownership, you'll get the best productivity and people who will work all night and over the weekend on their own if that is what is needed to get the job done.

Changing my Web programming approach

After thinking about it for a week, I've decided to start exploring other programming languages as a way of continuing my work with RSS, OPML and Web APIs. I still like Ruby, but there are enough limitations in it for me that I want to see if there are some languages that would be a better fit. I've changed the name of my Ruby blog to Code.Darwinianweb.com to reflect this shift in focus. It will still be the place where the source code for all of my Web programming experiments are posted.

My Web 2.0 stack

I'm not sure when "stack" came to mean a list of languages/technical standards used to build an app, but it is a useful description. It helps convey the logical architecture within a multi-layered development environment. The best example of a useful stack is LAMP (Linux, Apache, MySQL, Perl or Python or PHP), which summed up what most of us used to build Web 1.0. I've spent the last few months reading and skimming as many new technology books as possible, and I've narrowed down the list of things I need to become proficient in to understand how Web 2.0 works. What I still need is a catchy acronym. Here's the list:

• XHTML. This is basically HTML with some really prissy rules, like case sensitivity, and needing to close all tags. There are said to be tools that will make this conversion for you, but I haven't tried any.
• CSS. Once you understand the basic rules, CSS is a fun way to design a site, especially if you start with a pre-written stylesheet, so you can just change things like colors and spacing.
• XML. While XML itself can be understood in minutes, the many, many ancillary standards and protocols make it tough to find a real-world entry point. I've found RSS programming to be a good starting place.
• Ruby. I've been programming with Ruby for a month, and I'm getting to like it more and more. I think it may have the same level of ease and productivity that made the dBASE language so popular in its time.
• SQL. Yes, its still here, and its still the same, which is the problem. The issue will be fitting the object-oriented data structures of XML into the tables of SQL. The consultants will be paying their mortgages on this one for years.
• Javascript. I could say Ajax instead to assure a higher rating on the Web 2.0 Validator, but Ajax really means Javascript that maintains contact with a server without reloading a page.

Frankly, its not as much as I expected when I started researching Web 2.0 this summer. The good part is that it all fits together easily, and none of the parts are particularly challenging. That's when I am most productive. By the time a language gets as richly, and complexly supported as Java, for example, I get bored and confused and move on.

The urge to scale

I guess being a dot-com CTO is in my blood. I like to think through various architectures for managing groups of websites. You need to lock down a model for scaling early or you face big problems if you ever need to handle large amounts of traffic. The real key is a logical architecture for domain names. For example, if I thought I was going to serve a lot of podcasts, I would create something like data.darwinianweb.com or podcasts.darwinianweb.com. That would allow me to move that part of my content where it could be best and most cheaply served.

Right now I have darwinianweb.com to handle this main blog where I plan on covering general issues on the changing form of the Internet. I also have ruby.darwinianweb.com, which is a blog that allows me to go into as much depth as I want about learning the Ruby programming language.

I don't want to have too many subdomains, categorization can be handled more easily and on a larger scle with tags, which I am working on adding. At the same time, a separate domain creates more of a distinct place or channel of thought for the user. People automatically switch contexts when they change to a new site, just like a new TV channel.

I plan on having only a few more content subdomains, such as ajax.darwinianweb.com, and xml.darwinianweb.com. Programming languages or standards like XML are so broad and have so many supplementary tools and resources that they work better in their own site or subsite.

I'll also be creating separate domains for exchanging data with other servers. I don't know what will happen with my API experiment, or if that will become a target for abuse, so I'll also create api.darwinianweb.com to serve API calls. It isn't a matter of large amounts of traffic. I want to be able to shut down the API server easily. Of course, that brings up the issue of dependency on critical servers in a distributed environment called for by Web 2.0.

One solution, which also comes easily in an XML/RSS based communication model, is cache the most recent messages as text files, so the most recent result of an API call can be reused instead of calling the API again.

These issues will be played out on a much larger scale throughout the web. Chains of API dependencies will play interesting roles in the future.

My first Amazon API program

I now have a very simple program that queries the Amazon API for books on Ruby programming and displays the results as a list of titles linked to their product pages. The most interesting part wasn't the coding, which is pretty simple, but the incredible depth of Amazon's API. They make it possible to build a complete e-commerce site built on their engine. I now understand why Jeff Bezos was quoted on a financial program as saying that Amazon may eventually become a e-commerce systems provider instead of a retailer. You can learn a lot about a company's plans by studying the functionality they surface in their API. Even if I don't end up building a real product with anyone's API, I will get a better understanding of their strategy.

Reading Amazon RSS with Ruby

I'm having a ball with my Ruby project. I had the usual start-up issues, but once I found a webhost that knew what it was doing, and ironed out a few bugs in the database library for MySQL, things have gone great. I now have a simple script that will read the current Amazon RSS feed for computer books, and publish it as a list of linked titles on a web page. Now I'm ready to sign up for an Amazon API account and get started with web services programming.

Now I can get back to Ruby

With the site looking pretty good I can now indulge my inner coder and do some Ruby programming. I'm going to try and keep the details of this on my Ruby Development Site, so you can check in over there to see what I'm up to. I'll try to restrict my Ruby posts here to pointing out anything interesting that I've accomplished, or to new happenings in the Ruby world that will affect the general Internet's evolution.

Ruby.Darwinianweb is working

I think I've got everything working on ruby.darwinianweb.com at a pretty basic level. I wrote a simple Ruby script to retrieve an Amazon page with HTTP, just to get my feet wet. Considering the amount of time I've been working with it, Ruby looks like it will be a lot of fun. I've hardly been stuck at all during this process.

I think having a separate site to write about my experiments with Ruby is a good idea, because it will allow me to geek out there without turning this into a pure programming blog. The main Darwinianweb.com site will stay focused on my reactions to new events and technologies on the Internet. The only weird thing will be posting my code as I learn the language. In the past when I wrote about programming I always waited until I was proficient in the language before publishing my work. That way I could go back and fix any newbie mistakes I made early on. This new style of code blogging will take some getting used to.

Book Note: Programming Ruby

I'm not going to be reading this straight through, but since it seems to serve as the unofficial manual for Ruby, I'll be referring to it as I start doing my Ruby/Amazon programming. That really is my favorite way to learn a language, build something and look up what you need as you go. Of course, you need a forgiving language with a low initial learning curve, and that seems to fit the profile of Ruby.

The author, Dave Thomas, also wrote the first book on Ruby on Rails. A week or two ago I saw a post from him somewhere saying that his two Ruby books were the most popular on Amazon's computer book list. As an author, I'm always curious how you find a topic that is so hot. Unfortunately, the answer is that you have to start several years before things get hot, and then hope to get lucky. The lesson here is that you can't chase hotness, because you will already be too late. You have to be receptive to new things and follow your dreams when nobudy else is looking. My most succesful technology picks came from being an early adopter, not from being a follower. I don't plan on writing any books on the current crop of Web 2.0 technologies, it is already too late for that, but if I immerse myself in the current wave, something new will come along that I may be early enough with to make some money. In 1995 when I started with the Internet, my friends in the software business kept asking how anyone could make any money from that stuff. My answer was that I didn't know yet, but the best way to find out was to just do it, not to wait until the business models were obvious.

Making progress on Ruby project

I started writing a framework for my Ruby test site yesterday. All I need right now is a set of simple functions to display the source code of the programs I'm writing, and to execute them and display the results within a standard page format for the site. As is common with OOP languages, I'm just using the existing classes in a procedural manner for now. Once I start building up a larger codebase and have to deal with reuse issues, I will build some classes of my own. So far I'm finding Ruby to be a pleasant language to work with for what is now just text manipulation. The syntax stays out of your way, and it is extremely dynamic. So I can have code that constructs the test code I want to execute and then evaluates it and returns the result as a string. In this way it is reminiscient of the dBASE language, which is great for the "make it up as you go along" style I prefer. Today I hope to start building queries for Amazon in Ruby. I'll have more to say about that in later posts.

Interesting Ajax thread on Slashdot

As you would expect the Slashdotters view Ajax as a massive hack elevated by massive hype. In between the flames, however, there are some useful ideas.

Book Note: Ajax in Action

Ajax is so bleeding edge that I am reading a book on it from the future. I thought the rule in publishing was that you could use the next year for the copyright if it was printed in late November, but this book arrived November 2 with a copyright of 2006. After yesterday's Microsoft Live demonstration, it is clear that all forces are converging on client side programming with Javascript and DHTML. There have been many blog posts accusing Microsoft of being a follower rather than a leader in this area, but the irony is that Microsoft created the XMLHttpRequest functionality that is the heart of Ajax. Adam Bosworth has an interesting post on this history.

The idea of simulating a desktop app in a browser using DHTML and Javascript goes back further than the XMLHttpRequest. I designed just such a product called GifWorks in 1998. The goal was to create Photoshop in a web browser. It wasn't completely client-side though. The interface runs in the browser, but the image processing is done on a server.

I'm not sure what I will do first with Ajax, but the most likely candidate is some type of mashup with Google maps.

New Ruby site

I got my new site started to experiment with Ruby at ruby.darwinianweb.com. I decided to go with Textdrive.com, since they are recommended by the developers of Ruby on Rails. I went through the usual hassles of setting up the DNS. Darwinianweb.com is registered at Godaddy.com and its DNS is handled by Bluehost.com where this blog is hosted. So I had three different companies pointing me back to the others for a while.

Now I'm going through the standard "Why won't any of my scripts run?" that I experience on most new hosting services. Once I get everything placed right and working I can start setting up an Amazon web services development project.

Time to start Ruby programming

I've been reading Ruby books for a week now and I'm getting antsy to do some coding. My way of learning a language is to pick a real target and then push my way through all the obstacles. That forces me to do the hard tasks, but keeps me from wasting time on features I'll never need. I know I've been saying that I want to use Ruby to run the CMS for this blog, but I want to hold off on that until I add enough features. For example, I still need to add calls to ping servers and tagging to this blog. It will be easier to just do that in FoxPro first, and not worry about the programming language at the same time. Besides, from everything I've seen, Ruby on Rails will be the best way to build a CMS with Ruby, so I'll wait until later to start that project. I'll pick an easier target for my first Ruby project.

I've had an idea for Amazon web service programming for a while that might make a good candidate. I'd like to build a "pick the best book on a subject" app. I'm not sure exactly what that means, but the general idea would be to enter a search term and have the app rank available books based on a set of criteria I develop. That will let me explore the Amazon database from the inside.

My first step will be to build a development environment. I could start by doing Ruby programming on my own computer, but that won't let you look over my shoulder easily. I could do the Ruby project on the server where this blog is hosted, but that brings up the security issue. Programming on the web in a new language always leaves open the possibility of security holes. I'd rather do my quick and dirty development in a separate location and then move them to a new server for production use. I also like the idea of a distributed environment for web development. It allows for better scaling and gives me more control later for rearranging the architecture. So my first task will be to build ruby.darwinianweb.com at a new host.

When I first started looking at Ruby I saw that there were some hosting sites that provided Ruby and Ruby on Rails installations at extremely low rates. They billed themselves as Ruby playgrounds, which is a great idea. I've been a programming language junkie for over 25 years, and the idea of playing with a new language is always a thrill. It looks like these hosts are aimed directly at that market.

Book Note: The Ruby Way

I've already finished this book, so I'll just do more of a final review than a series of notes while reading. This is basically a cookbook aimed at an intermediate Ruby programmer who wants to see the most elegant way to accomplish both standard and fairly advanced techniques. Even so the first chapter is a superb review of the basic language features. I'd recommend it for any experienced programmer who wants an overview of Ruby's syntax and OOP capabilities. The rest of the book is well written, but has too much of a systems programming focus and too little information on application development for me. Hal Fulton's background as a CS professor certainly comes through. For example, there are 30 pages on threaded programming and 3 pages on using MySQL. The idea of controlling multiple threads is cool, but I'd ever do it. I could just launch multiple instances of the interpreter, or more likely use mod_ruby and let Apache deal with multiple threads. In the end, other than the first chapter review of Ruby syntax I found little of use to me, but anyone building a Ruby IDE in Ruby will certainly be pleased.

Plans for using Ruby

I guess I should explain what I want to do with Ruby. A major motivation for learning it is the amount of buzz surrounding it and the application framework Ruby on Rails. Language popularity comes in waves that usually last 3-4 years. In the 10 years I've been involved with the Internet I've seen Java, Perl, Javascript, PHP, Python and now Ruby each have their turn as the hot language.

Ultimately I want to write web 2.0 style mashups, but my first Ruby app will be the content management system that runs this blog. I know it sounds crazy to build yet another CMS, but that is the type of application I know best, so it makes a good testbed for new tools.Of course, I have no plans of creating a CMS that others can use, just something with enough functionality to serve my purposes. Right now I've written a system with Visual FoxPro, because I've been an xBASE programmer for 25 years and can do it in my sleep. I want to gradually rewrite it in Ruby and then rewrite it again with Rails.

One issue will be database access. Eventually I will use MySQL as the back-end, but at first I'll want to use Ruby with the .DBF files I am already using in the FoxPro version. Since Ruby has a much weaker set of available libraries than Python, I haven't been able to find a native Ruby module for DBF access. The path I'll probably go down will be to use a Python DBF library and a library that lets me call Python code from Ruby. It sounds like a horrible kludge, but it will let me learn Ruby while doing some practical work. I'll write the database routines so that they can easily be converted to MySQL later.

Book Note: Teach Yourself Ruby in 21 Days

I'm ready to give up on this one. As I expected, he dropped the pretense of writing for both novice and experienced programmers by the fifth chapter and is now in full-blown programmerspeak. That's fine with me, because I'd rather read something that assumes I know how to program, but doesn't assume I know Ruby. What is more frustrating is that he resisted explaining the OO features of Ruby early on, so now he is forced to introduce them as needed throughout unrelated chapters. For example, he explained how to create and use class methods as opposed to object methods in chapter six, which was supposed to be about I/O. He had no choice, because the File class has some methods that are used before a file object is created. Another problem I have is his use of examples. I believe in using either ultra simple examples, such as X = 1 and Y = 2, or useful examples that explain how a language feature can be applied in a practical way. This author is trying to stay somewhere in the middle by using examples that have real looking data but don't actually do anything useful. I'm not sure who would benefit from his approach. My next Ruby book will be Hal Fulton's The Ruby Way.

Book Note: Teach Yourself Ruby in 21 Days

The classic problem facing an author of any introduction to a programming language is whether to assume the reader has ever programmed in any language. This book adopts the approach of no prior programming, which in my experience always tends to break down quickly. It is easy to explain that a memory variable is like a cubbyhole with a name, but what happens when you reach topics like passing pointers to arrays of functions? The initial assumption is revealed as a farce. Taking this approach of no prior knowledge in a book on an object oriented language has the additional burden of explaining concepts such as encapsulation, polymorphism, and inheritance at the same time as saying a variable is like a shoebox that can hold any value. The mistake authors make is worrying that OO concepts are really difficult. The trouble isn't learning the concepts, it is understanding why they are necessary and justifying the extra work in using them. This book tries to avoid the pedantic style of explaining these concepts directly, by giving code examples first and then putting the names of the concepts in text box asides as if you didn't really have to know the names.

Instead of wrapping this term in all these qualifiers, I'd explain polymorphism by having a subheading called Polymorphism with examples of two different classes with a print method. A simple explanation of the consistency benefits of always calling this method print even if it does different things would explain the concept and the benefits at the same time.

Curiously, he even gets the description of inheritance wrong. He says that inheritance is when you add a method to a class and then call it in an object of that class. I always thought inheritance is when a subclass can call a method of the parent class.

This book is trying so hard to say "Don't worry, programming isn't really that hard" that it ends up confusing both new and experienced programmers. For example, it tries to explain that Ruby passes variables by reference through a strange example of changing part of a string in two variables pointing to the same value. To make it cute and more "accessible" he uses the name "Sandra" which he changes to "Sandra Dee." How old is this guy? I'd simply use an example like: After two chapters I'm getting pretty annoyed.

Book Note: Teach Yourself Ruby in 21 Days

I started learning the Ruby programming language today. Ruby and the associated framework product, Ruby on Rails, seem to be all the rage among web 2.0 developers, so I thought I'd give it a try. It seems to be a "real" object oriented language, so I'm looking forward to it. I've programmed in Perl, which can be forced to use OO techniques, and Python, which is much more OOish, but I've mostly used them in a procedural manner. This time I'm going to try and really work with classes as well as objects. Back in the early 90s, when OOP first appeared in popular use, Borland hired me to write a book about their future dBASE for Windows product. I spent a year doing nothing but studying OO ideas and really fell in love with the approach. Sadly, the product was a bust, and the book never saw the light of day.

I have a stack of Ruby books on my desk, and I'm going to start with this one from Sams. You can find the details at Amazon if you want to read along with me.