Category Archives: Philosophy

So You Wanna Try Agile?

Well, in a nutshelll, agile is a state of mind.
Agile is relative.
Agile is not dogmatic.
Agile is pragmatic.

Agile is designed to reduce the gap in time between doing something, and seeing a result that you can “measure.”

Blend in Lean concepts.
Read The Goal.
Don’t do giant monolithic things.
Learn how to decompose your “features/expected outcomes” into bite-size chunks.
Don’t plan across a far horizon to the same degree.
What folks are tasked with doing today better be clear and have unambiguous meaning of “DONE”
What folks are tasked with doing in 2 months better be rather nebulous and broad.

Don’t be complacent and relax.
Question the value of deliverables that are needed to fulfill some process step for some other team.
Ask any downstream recipient what they truly need — and why.
Try new processes.
Reflect.
Agile takes constant effort and constant partial attention if you are doing it right.
Be holistic.

A Fool with a Tool

One of my quips that people seem to get a kick out of is:

“A fool with a tool is still a fool”

My point is simple: woodworking tools in the hands of craftsmen can produce fine furniture or musical instruments. But merely the availability of great tools “doth not a masterpiece make.” Not to take the woodworking analogy too far, but power tools can also make it easier for the amateur to get hurt.

The reason a quality piece can be turned out more efficiently with a tool is the ability for the craftsmen to more quickly realize the design and architecture. In addition, tools can help make patterns easier to replicate. From actual patterns (like certain cross-section shapes on molding), to templates (like violin bodies), to connectors (like Kreg pocket hole jigs). A woodworking shop will also make good use of building components to put into larger assemblies.

So, tools in the hands of experts can help realize the design. Tools that are able to help apprentices produce parts designed by the masters, can help produce more output and help train the apprentices.

But put me in a room with the most awesome woodworking tools, and I will not be producing Stradivarius-esque violins any time soon (or ever, most likely).

A Java Developer’s Switch to Rails

As I read this account of transitioning from Java EE to Ruby on Rails,”Java Web Apps to Rails, A Developer’s Experience,” I thought the following:

Regarding the lack of a good Ruby/Rails IDE:

When it was time to look at the code, I was a bit disappointed.  TextMate looked like a popular choice but it fell short in many ways.  I struggled to navigate the code, link the methods across files and follow the code. Grep was my only savior.

I am surprised he did not find RubyMine. JetBrains makes awesome IDEs (IntelliJ IDEA for Java, too). I have to admit at being very impressed that the underlying model of the ruby code within the RubyMine IDE is amazingly capable of giving you the same sort of features that it can for a “compiled,” or more type-centric language like Java.

Ruby

Next he mentioned about learning the language…

My approach to learning Ruby was to start looking at code and figuring out pieces of syntax that did not make sense.  That turned out to be a bad idea. Constructs like “do |x| x+1” threw me off. … I found it very important to understand the motivation behind creating the new language.

There are some good books out there, and other resources…

Another comment rang true with me:

I found it very important to understand the motivation behind creating the new language.

More about the motivation behind Ruby… or at least, how I interpret things.

Ruby treats you like an adult. It doesn’t baby you, or hold your hand, or make you type all sorts of (obfuscating, in my opinion) fluff that a compiled language needs. Your code is more compact and obvious and “dense with meaning,” leading to less need for things like method return types and method parameter data types. Or semicolons.

However, I miss parameters list, curly braces, return statements, camel case naming conventions…

Be patient… soon you will grow weary of having to read all that extra stuff when you go back to Java. If you write well-formed classes, and very small, compact, and obvious methods, you don’t need the extra frills. You will run across the concept of “duck typing” which really means that classic C++/Java/C# style interfaces are not needed. If you create a method that expects a passed-in object to respond to a method call (e.g., “torque”), any object of any class will do — as long as it has that method call! And if you pass in an object that does not have that method, Rails will kindly throw an exception.

The concept of ‘open classes’ that allow developers to go in and change a base class sounds scary.

Go back to my opening statement. “Ruby treats you like an adult.” If you are naive, you can blow your code up and break stuff. If you do everything the Ruby way, you’ll be fine — because you will catch it with tests (right?). There are good rules of thumb for how and why and when you should consider adding to an existing class — or worse, monkey-patching. In Ruby you can actually alter the behavior of an existing method of an existing class. Do it poorly, and you will pay the price. And maybe not for a year or two. Mwahahahaha.

Ruby’s metaprogramming is incredibly powerful (even if the syntax takes a bit of getting used to).

Rails

Rails is incredibly opinionated about how the MVC code should look. It is basically a model-driven architecture approach. The framework expects the MVC stack to have a model, a view, and a controller for each domain concept. (As a long-time domain modeler, and UML guy, this is a real treat.) You can use the default rails generators (interesting when first starting, or for banging out a quick prototype), or write your own twist on a generator. Instead of focusing on the boring framework bits with repetitive, template-like code for each domain object, you can focus on quickly creating features that people want to use and pay for 🙂

Supporting Cast

Ruby on Rails has a great ecosystem of wonderfully talented folks providing 1000s of gems to do all sorts of useful things. For example:

The ecosystem of supporting tools is incredible. Some of my favorites:

These all make it so easy to “do the right thing” and write code with BDD/TDD to provide quality from the start.

Personally, my switch from J2EE to Rails (not to mention SQL to NoSQL/MongoDB) has been one of pure pleasure. The kinds of rich functionality and the ability to build stuff faster is imprerssive. I only wish I were better at writing kick-ass Ruby, Rails, CSS, Javascript, MongoDB…

Is it perfect? Of course not. There are other cool things to look at as well… Meteor, Node.js, MEAN, etc. So much to explore, so little time.

Back to my continuous learning!

 


Measuring Effectiveness: The Software Industry’s Conundrum

On “Measuring Effectiveness”

I’ve been saying for seemingly decades, this is (one of?) our nascent software industry’s biggest conundrums (maybe even an enigma?).

Just how do you prove a team is “doing better?” Or that a new set of processes are “more effective?”

My usual reaction when asked for such proof:

“Ok, show me what you are tracking today for the team’s performance over the past few years, so we have a baseline.”

Crickets…

But seriously, wouldn’t it be awesome if we could measure something? Anything?

For my money:

Delivering useful features on top of a reasonably quality architecture, with good acceptance and unit test coverage, for a good price, is success.

Every real engineering discipline has metrics (I think, anyway, it sounds good — my kids would say I am “Insta-facting™”).

If we were painting office building interiors, or paving a highway, we could certainly develop a new process or a new tool and quantitatively estimate the expected ROI, and then prove the actual ROI after the fact. All day long.

In engineering a new piece of hardware, we could use costing analysis, and MTBF to get an idea on the relative merits of one design over another.

We would even get a weird side benefit — being relatively capable at providing estimates.

In software, I posit this dilemma (it’s a story of two teams/processes):

Garden A:

  • Produces 15 bushels (on average) per month over the growing season
  • Is full of weeds
  • Does not have good soil management
  • Will experience exponential (or maybe not quite that dramatic) production drop off in ensuing years, requiring greater effort to keep the production going. Predictability will wane.
  • Costs $X per month to tend

Garden B:

  • Produces 15 bushels (on average) per month over the growing season
  • Is weed free and looks like a postcard
  • Uses raised bed techniques, compost, and has good soil management
  • Will experience consistent, predictable, production in ensuing years
  • Costs $Y per month to tend

I could make some assertions that $Y is a bit more costly than $X… Or not. Let’s assume more costly is the case for now.

To make it easier to grok, I am holding the output of the gardens constant. This is reflected by the exorbitant rise in cost in the weedy Garden A to keep producing the same bushels per month… (I could have held the team or expense constant, and allowed production to vary. Or, I could have tried to make it even more convoluted and let everything vary. Meh. Deal with this simple analogy!)

 

Year 1 2 3 4 5 6 7 8 9 10
Garden A 100 102 105 110 130 170 250 410 730 1600
Garden B 120 120 120 120 120 120 120 120 120 120

If we look at $X and $Y in years 1 through 10, we might see some numbers that would make us choose B over A.

But if we looked at just the current burn rate (or even through year 4), we might think Garden A is the one we want. (And we can hold our tongue about the weeds.)

But most of the people asking these questions are at year 5-10 of Garden A, looking over at their neighbor’s Garden B and wanting a magic wand. The developers are in the same boat… Wishing they could be working on the cooler, younger, plot of Garden B.

What’s a business person/gold owner to do? After all, they can’t really even see the quality of the garden, they just see output. And cost. Over time. Unless they get their bonus and move on to the next project before anyone finds out the mess in Garden A. Of course, the new person coming into Garden A knows no different (unless they were fools and used to work in Garden B, and got snookered into changing jobs).

Scenario #2

Maybe we abandon Garden A, and start anew in a different plot of land every few years? Then it is cheaper over the long haul.

Year 1 2 3 4 5 6 7 8 9 10
Garden A 100 102 105 100 102 105 100 102 105 100
Garden B 120 120 120 120 120 120 120 120 120 120

I think the reason it is so challenging to get all scientific about TQM, is that what we do is more along the lines of knowledge work and craftwork, compared to assembly line.

The missing piece is to quantify what we produce in software. Just how many features are in a bushel?

I submit: ask the customer what to measure. And maybe the best you can do is periodic surveys that measure satisfaction (sometimes known as revenue).

Matt Snyder (@msnyder) tweeted me a nice video: Metrics, Metrics, Everywhere – Coda Hale


Don’t be the Frog!

Ever hear the muse about the frog and the boiling water?

I use it to describe the behavior where we innocently allow otherwise really odd things to happen to software products over time. “Oh, it’s been like that for 10 years!” It is never a single, discreet moment in time; instead it kind of “sneaks up” on you if you are not paying attention.

For example:

  • Code that is hard to unit test without bringing in all sorts of dependencies (even mocking barely helps)
  • That javascript error that happens in (unsupported) Firefox, but not in (supported) IE or Chrome
    • Instead of being more compatible with all modern browsers via a simpler and more consistent UI framework.
  • Context menus (right-click) that have everything but the kitchen sink in them
    • Instead of more clearly articulating the commonly-used vs the rare menu choices
  • A UI that nearly defies Cucumber/Capybara testing
  • A “Save As” dialog that also offers up to the user the ability to control the next landing page, to deactivate the item, to create a new item under a new parent, and to generate the PDFs for the item All under Save As!
    • Instead of making the UX more clear and solving the real problem: undesirable workflow.
    • We have Save. We have Save As. But wait! There’s more! You can Save As, give a new name, and choose from 3 different pages to go to next!

I often think it stems from a few common shortfalls:

  • The real problem was not solved, but simply masked
  • Complexity begets more complexity
  • The domain is not understood, so incompatible fixes are tacked on in the wrong spots
  • We didn’t have time to do it right, so we just got something that would work shipped
  • The customer(s) wanted it built like this
  • There is no overriding vision for the type of user experience we want to convey
  • There is no hard-ass product owner putting their foot down

Don’t let this discourage you if you are still outside the pot looking in at friends and colleagues who are in the hot water. It’s an opportunity to improve, clean up, straighten things out, and start anew.

Don’t be the frog!

 


 

Best way to Learn Authentication?

This was a question posed on the Rails Studio mailing list…

My initial reply to a suggestion to check out Railscasts was…

And then, once you are happy with your learning, throw away all of your Auth code and get the Devise gem loaded and working 🙂

To which, the OP replied:

Hmm, I would think your own custom solution would always be smoother than using someone else’s code. Why use Devise?

Here is my reply:

One day, in a galaxy far away, I needed to have millisecond resolution timing on the PC to do some reaction time experiments with subjects as they were exposed to various shapes and colors (eventually used to improve HUD symbology for fighter aircraft display). Since the computer timer chip could only get down to 54ms, I was on the prowl for a better way. Lo and behold, I found some assembly code techniques I could use to over clock the timer chip. Problem solved. I hated writing assembly code… But I had to. Must. Not. Be. Blocked.

Product went into production, and was used by the researchers…

A month or two later, I found an awesome timer package for $50. It did what I needed and more, by a guy who specialized in this.

As fast as I could, I ripped out my code and dropped in the new library. I never looked back. You see, timing was not core to my research app, but rather contextual (like printing or graphing). (As an aside, the module grew and grew and kept up with hardcore CPU vagaries… it was able to tune itself in spite of processor logic becoming much less predictable as it went from 286 to 386 to 486 to 586. I was glad I didn’t have to keep up with the CPU technology! I used this same core functionality on apps that I wrote to control missiles in real-time.)

So, if your business is all about writing an authentication module, then that is what is “core” for your work, and you should craft your own and hone it to perfection.

Or, if you just want to do it as a learning exercise, that is great! Learn, and then pitch your code.

But for me, I am all too happy to drop in some other folks’ gem that gets a set of functionality up and running quickly, so I can get on with my core functionality and not be distracted.

YMMMV

The Bizarro Manifesto

Let’s try a little Bizarro1 test (if you agree to these, I’ll poke you with a hot krypton stick):

We are uncovering better ways to provide the illusion of developing software by listening to others talk about watching people try. Through this (dare I call it?) work, we have come to value:

  • Dogmatic process and CASE-tool-like automation over inspiring quality individuals to interact with the team and the clients
  • Sufficient up-front comprehensive design specifications over seeing frequent, tangible, working results.
  • Writing detailed Statements of Work and negotiating changes over collaborating to do our collective best with the time and money at hand
  • Driving toward the original project plan over accommodating the client changing their mind, or a path turning into a dead end

To elaborate:

  • We prefer to focus on building software with lock-step process and tools — and reduce our need to worry about quality individuals and having conversations amongst developers or with the client.
    • that way we don’t need to worry about people issues and effective communication.
    • That way we can hire any individual regardless of skill, and forgo all verbal/personal interactions in favor of solely written words. Even better if those written words are automatically transformed into code. Maybe we can get non-coder tools! After all, people are merely fungible assets/resources, and software is factory-work — with processes and tools, and a horde of low-paid serfs, we can crank it out!
  • We prefer to spend a lot of time up-front ensuring we have the requirements and design specs fully determined —  rather than have tangible, working results early on.
    • We start with complete requirements specifications (often 400 pages), that follow our company standard template.
    • Even our Use Cases follow a mandatory 3-level deep path, with proper exception and alternate paths worked out.
    • We link the requirements items into detailed design documents — which include system design diagrams, interface specifications, and detailed object models.
    • If we don’t write it all down now, we’re likely to forget what we wanted. And if we don’t do it to the n-th degree, the developers might screw it up.
    • Writing it all down up front allows us to go on vacation while the process and tools “write” the code from the detailed specs/diagrams. Sweet.
    • In addition, we love to be rewarded by reaching meaningless intermediate deadlines that we place on our 1500-node Gantt chart.
    • When we combine all of the upfront work with important deadlines, many of the senior managers can get promoted due to their great commitment to generating reams of cool-looking documents. By the time the sh!t hits the fan when folks realize the “ship it” deadline is missed, the senior managers are no longer involved.
    • Besides, if we actually built software instead of writing all sorts of documents thinking about building software, our little ruse would be exposed!
  • We prefer to work under rigid Statements of Work — rather than actually work towards a “best-fit” solution given changing conditions of understanding and needs.
    • The agreement is based on the 400-page, fully-specified requirements document, and we pad the cost estimate with a 400% profit margin.
    • We then hire dozens of people to argue during the Change Control Review Board monthly meetings about re-writing code to deliver what you wanted versus what you asked for when you thought you knew what you wanted (and wrote it down in that 400-page doc that was signed off by 6 execs).
    • Contract negotiation pissing matches are such great uses of our collective resources and always result in perfect software! We love our fine print 🙂
    • With a 400% padding, the projects are too big to fail.
    • Once we are in it for 1 or 2 million and 50% done and 2x schedule overrun, who would ever say “No” to a contract extension? Who better to get you to the goal line than the same folks who squandered away your treasure, pissed away the calendar, and delivered no working software yet?
    • We like to appear like we’re just about done… Asymptote? Never heard of one.
  • We prefer to be driven by our initial plan — rather than dealing with change and having to re-print the Gantt.
    • Especially a Gantt chart that has been built with tender loving care to include resource allocations, inter-project dependencies, and partial resource allocation assignments for matrix-style organizations.
    • We love hiring a small army to ensure that we drive the entire team to meet every micro-task deadline even when they no longer make any sense.
    • The real fun is collecting the “actuals” data from the developers assigned to each task so we can compare it to their estimated hours.
    • And nothing sweeter than seeing 90% of our tasks being started, and 75% of those being 67% resolved;  and 25% of the resolved actually being complete — the roll-up summary to management is such a compelling story of success.
    • Changing such a beautiful plan that took 4 man-years to develop, that incorporates all of the comprehensive non-code documents, and is an appendix in the contract, is no small feat!
    • Better to produce the software according to plan even if nobody wants it that way. That’s our motto, and we’re not going to change!
    • We love the illusion of activity over the truth of delivered features.

Feel free to sign the manifesto below. It’s free to be certified.


1

    Credit goes to Superman and Bizarro World.

Use Agile Wisely

For the past few years, I have been bothered by a nagging urge to write a pamphlet on software titled “Common Sense” — an homage to Thomas Paine’s great work.

This may be a stretch, and I may counter this point once I actually try and research and draw more parallels — or not. So here I am just thinking out loud, as it were. A risky means to formulate my ideas, I know.

The Agile Manifesto is akin to the Declaration of Independence and US Constitution. (Not in its impact on the world, and I make the comparison in complete deference to the greatness that were our Founders.)

Agile is about freedom and individual responsibility grounded in an agreed-upon framework of overarching ethics and morals. The beauty of our Founding Fathers, was that they got to the essence of human behavior — including the depravity and moral weakness, and corruption by power, that we humans suffer from. They designed a system of government that accounts for such weaknesses, and that provides a means of feedback and correction. Correcting in the small (local and state government), and in the large (amendments to the document itself).

The Agile Manifesto is similarly poised. The four main tenets of the Manifesto are irrefutable, getting to the essence of software development. The Manifesto sets the stage up for great success without the shackles of a tyrannical, command and control form of management. It works from the bottom up.

There are many in the software world that decry the freedom of agile developers practicing their craft. Some organizations still drive a tyrannical structure within a framework of bureaucratic control. While many “Freedom Fighters” see the errors in these organizations, those on the inside are somehow oblivious, and probably believe in the superiority of command-and-control.

The lure of the bureaucratic layers to many managers always confounded me. Is it a lack of education about what freedom means? Is it a lack of courage to be individually accountable? Is it an allegiance to a known and friendly tyrannical structure — versus the unknown and scary individualism? Is it a belief that somehow a complex human organization can be broken down into its constituent parts, such that if each cog does their small bit, then the whole achieves its ultimate goal? It is, undoubtedly, simply part of human nature. Some folks are comfortable being risk takers and sticking their necks out, while others enjoy the comforts of a more controlled system within which to toil.

Over the past few years, our community has been inundated with “enlightened” Scrum Masters. The Church of Scrum has literally popped up overnight, anointing new converts at an alarming pace. With a relatively trivial-to-acquire certification, many organizations seek out said experts to be their savior on the road to riches.

Scrum itself is not the issue, after all it is Agile. But much like freedom in the hands of the uneducated often has disastrous results, so too does being Scrumified in the absence of understanding Agile.

Our young democratic republic worked hard to teach children about the US form of government and the hard-fought liberties we were blessed to enjoy. The children all studied from primers that helped them be educated about just “how” our form of government works. And they learned about the context… the “why” the framers of the Constitution chose their forms of checks and balances to stem undersirable side effects of human nature.

In the absence of much formal education about what the intent of the Agile Manifesto means, Scrum filled the void. That’s the beauty of a free market. Ideas can compete.

Much like our venerable representative democracy, Agile isn’t the most perfect form of s/w development, but it’s the best that has ever been. There are no guarantees that being “agile” will result in fabulous wealth and riches through the killer app. Just like there are no guarantees of outcome in a free society (at least there shouldn’t be any).

As with the United States of America, with great freedom lies great responsibility. Use agile wisely.

The Agile Parallel

For nations that have known virtually nothing about individual freedom and only ever experienced tyranny, you cannot throw a switch and suddenly declare the citizenry ready for self-rule. Same goes for non-agile organizations.

The USA is probably the rarest of rare, in that we actually embarked on just such an experiment in 1776:

Can man rule themselves (without a strong tyrant*)?

* such as the Kings of Western Europe, the Czars of Russia, the Sultans of the Ottoman Empire, the Emperors of assorted Chinese Dynasties, etc?

Maybe the same applies to companies as to governments as to any large organization?

After reading “The 5000 Year Leap” and many other books about the founding of the USA, I have developed an opinion that our Founders distilled the essence of Man (and our flaws) in terms of governing. Around those inherent flaws, the Founders attempted to build an organizational structure that prevents these flaws from succeeding to take hold. After all, our Founders came from a experiential base of being under Kings… but they wisely studied many forms of government, all the way back to Cicero. The last thing the Founders wanted was a strong Federal Government — or they would be right back to where they started: a Ruling Class and Kings (just under a different name).

I (*very humbly*) think the Agile Manifesto distilled the essence of Man (and our flaws) in terms of software development. It will no sooner be foolproof at preventing failures than our own representative government. Without understanding the motive forces behind the tenets of the Agile Manifesto (or the US Constitution), there is little chance that merely acting out the parts will result in lasting success.

The Founders knew that the USA will only survive as long as we have an educated citizenry that participates in trying to maintain our collective goals as a society (primarily freedom and security), while being ever vigilant about not giving away the freedom of the individual.

At the close of the Constitutional Convention, a curious Philadelphian wondering “Well, Doctor, what have we got—a Republic or a Monarchy?” to which Ben Franklin quipped: “A Republic, if you can keep it.” He wisely also asserted the challenges behind keeping it:

“In these sentiments, sir, I agree to this Constitution with all its faults; if they are such; because I think a general government necessary for us, and there is no form of government but what may be a blessing to the people if well administered; and I believe, further, that this is likely to be well administered for a course of years, and can only end in despotism, as other forms have done before it, when the people shall become so corrupted as to need despotic government, being incapable of any other.”

I have always said, “Agile is Hard.” So is freedom, as pointed out by Thomas Paine:

“Those who expect to reap the blessings of freedom, must, like men, undergo the fatigues of supporting it.”

Similarly, an organization’s software development “freedom” will be limited by the understanding that the “citizenry” has of just what makes up quality software and quality processes and self-rule.

Much as it is the rare country that can pull off self-rule like the USA, it is probably the rare large organization that can achieve the same measure of success. In the business world, there is at least a chance that poor leadership in a large organization will be self-correcting via business failure (barring unnatural outside forces). What’s worse, in my opinion, is that there is virtually no corrective force at work in our Government (or military), where bureaucracy and inefficiency can grow un-checked (except by the power of the purse). In large organizations, it is is often easy for folks to “hide” in the comfortable confines of busying oneself with doing a prescribed function (or being in incessant meetings <g>)and passing it off to the next cog in the process, with nary a care if it is an effective use of your time, or not.

The Founders believed that our rights came from God (for if they came from a man — King or otherwise — they could be taken away at any time). The intent was to give the individual as much power as possible, delegating very limited powers to government. As the structure of “government” moved further from the individual, from the town to the township to the county to the state to the federal level, the powers over the individual were necessarily diminished. After all, who knows best about what the town square should have but the town folk? It’s kind of like high cohesion and low coupling…

Similarly, the Founders built in plenty of checks and balances and distributed “power.” Much like a well-oiled Agile team that embraces clients, testing, and the quality folks as checks and balances.

When a software organization is similarly structured to permit strong cohesion at the local level, and responsibility to stakeholders, and very little coupling through the levels of the organization, good things happen. Even a large organization that skillfully permits freedom at very low levels (recursive-like delegation) in the organization gains the benefits of Agile. When the power structure is turned upside down (like in large, command-and-control bureaucracies with prescriptive process, or in the US Gov’t dictating minutiae in our daily lives), less good things will happen. I might even suggest that this sort of system results in natural selection of followers/non entrepreneur types, further exacerbating the situation. But I digress…

If the percentages during the USA Revolution are any indication, one-third were for the old way, one-third were ambivalent, and only one-third were for the new way. What that bodes for Agile sweeping aside the old way, I am not sure.

Fortunately, we “Agile Founders” did not have to “pledge to each other our Lives, our Fortunes, and our sacred Honor.”

Carry the freedom to deliver quality software at your core. The revolution starts with you!