The Decline of Drupal, or How to Fix Drupal 8

Prologue

Almost 4 years ago I wrote a controversial post entitled "17 Reasons WordPress is a Better CMS than Drupal" that caused me to be persona non grata among some of my prior Drupal friends.

But while some of the issues I mentioned have been addressed by the Drupal community most of the issues remain in Drupal 7, and WordPress has continued to gain strength as a CMS.

Unlike almost 4 years ago, I’m now seeing many people replacing Drupal solutions with WordPress and the end users becoming happier. My team is even bidding on replacing a website so we can build a member’s-only private site to go with it after the Drupal developers have not been able to deliver on the private site for over 2 years.

The Impetus

What triggered me to write this post was I was composing a long reply to a comment on the other post and it became clear it would be better as a new post.

In the comment the commenter asserted:

With Drupal 8 coming up, I am sure the difference in number of users between Drupal and WordPress will come down.

However, I think that the commenter will find that the exact opposite happens. Why do I think this? I started college shortly before the IBM PC was released so I’ve seen enough computer industry history firsthand to know a bit about the patterns that repeat related to software platforms.

Will Drupal 8 grow Drupal’s User Base?

I highly doubt it, and I think there is strong evidence in the history of software platforms that would support my view. Those patterns I mentioned above indicate to me that the strategy of changing Drupal’s architecture in Drupal 8 will be a failing strategy.

Let me explain.

Fans Like Things As They Are

As with all software products and platforms that gain a notable level of success, Drupal 7 and earlier appealed to people who valued what Drupal had to offer. Some of those things include ease of end-user configuration and other things include hierarchical software architecture and the hook-based extensions mechanisms. Or at least those are what originally appealed to me in Drupal before I discovered all the downsides I explained in the prior post.

Now Drupal 8 promises to be a lot more "modern frameworks and platforms," adopting "modern PHP concepts and standards, object-oriented programming, and the Symfony framework." Now that sounds awesome, and on the surface should cause almost any Drupal fan to cheer.

But those stated aspects require a lot more programmer skill to work with yet one of the things that appealed to a lot of Drupal users-cum-developers is that they did not have to understand object-oriented programming, nor modern frameworks and techniques. To quote Jennifer Lea Lampton:

Back in the day, Drupal used to be hackable. And by "hackable" I mean that any semi-technical yahoo (that’s me, btw) who needed a website could get it up and running, and then poke around in the code to see how it all worked. he code was fairly uncomplicated, though often somewhat messy, and that was fine. At the end of the day, it did what you needed.

Given the fact far fewer people have a high-level of programming skill many of those who do NOT see themselves as professional programmers do not want to improve their coding ability, they would rather just focus on their chosen career where Drupal is only a tool to help them.

So Drupal 8 will be will be alienating all those users and they will feel abandoned. Or as Ms. Lampton says:

Today, the majority of the people in our Drupal Community aren’t CS engineers. They are self-taught Drupal experts, people less technical than myself, and people who can get by using this awesome software we’ve developed to help make their lives easier. What is the transition to Drupal 8 going to be like to them? Well, I asked some non-core developers, and I didn’t like what I heard.

A lot of professional Drupal developers already have exit strategies.

And my guess is that most of those alienated users and new users who would have otherwise chosen old Drupal will move to WordPress.

But Pros Want to be Pros

And on the other end of the spectrum are those who DO see themselves as professional programmers and those people (almost) always want to increase their coding skills. They will start asking themselves why they are working on a platform (Drupal) that still has lots of "impurities" when the could just more over to a "real" framework such as Symfony or even Rails or Node.js, and not have to deal with all the legacy issues of Drupal?

Or as Ms. Lampton continues (emphasis on WordPress mine):

They may even have a day job building or maintaining Drupal 6 and/or Drupal 7 sites, but they go home at night and study Ruby, Node.js, Angular.js, even some are looking into WordPress. They want to be "out" before they have to learn Drupal 8. These are smart, capable people, who I’m sure - if they wanted to - would be able to pick up Drupal 8. So, why are they leaving? Because Drupal 8 has become different enough that learning it feels like learning something new. If they are going to invest in learning something new, why not Ruby, or Node.js, or something else?

What Visual Basic’s History Can Teach Us

What makes me think the above scenario is likely? Because I saw it happen with Visual Basic and C#. Visual Basic pre .NET was easy to use and became arguably the world’s most widely used programming language for a time. But it was a ugly language with many inconsistencies and was very limited in what it could do compared to C++ so it was always looked down on by "real" programmers ignoring how Visual Basic empowered so many people who never would or could develop using C++.

So Microsoft envisioned a "better" way; a .NET platform on which both Visual Basic and a new language called C# would live making Visual Basic a "proper" programming language, almost on par with C+.

Fast forward to today and what happened was that those who valued Visual Basic’s simplicity continued to use the old Visual Basic (for a while), abandoned it for other tools that were easier, or just quit developing and focused on other parts of their career.

Those who wanted to become better professional programmers asked themselves "why stay with VB?" so most everyone just moved up and over to C#. This migration effectively killed off what 10 years ago was once the most popular programming language in was the world.

And I believe a pattern similar to the Visual Basic decline will occur with Drupal starting at version 8.

When Upgrades are Challenging People Evaluate Options

And then there are those who will stick with their current version of Drupal until they can no longer maintain the solution and still get the evolving solutions they need for web and mobile.

At which point these people will be forced with a choice; migrate to the newer Drupal, or migrate to a different platform? And given how little interest the Drupal core team places in 1.) "Being backward compatible" and 2.) "Creating an interface that is usable for end-users" the choice will often not be "Move to newer Drupal."

True Believers will be True Believers

Of course there will still be people who love Drupal 8. And unlike proprietary software like from Microsoft, Drupal 8+ will continue to exist as long as a group exists who are passionate enough to maintain it. But I am almost certain Drupal’s market share will drop significantly and lose most of it to WordPress (which BTW won’t make that much different to WordPress’ marketshare, by comparison.)

Don’t Mess With My Status Quo!

And this being the open-source world, Drupal has already been forked and the fork is called Backdrop from the same Ms. Lampton quoted above as well as Nate Haug. Assuming Ms. Lampton and Mr. Haug and team executes at least reasonably well then some of the more fervent believers in "Drupal Classic" will move over to Backdrop, and Drupal 8 will loose more marketshare from yet another source.

But Backdrop will almost assuredly never be more than a footnote because it won’t have the marketing muscle in IT shops that Acquia has, and IT shops have been the primary drivers of Drupal adoption from best I can tell looking in from the other side. And Backdrop being a fork won’t have the 10+ years of supporting organization that Drupal now has. Plus, Backdrop has an unknown brand at this time and building up that brand will take time.

Old Doesn’t Inspire, It Just Fades Away

Given that Backdrop is basically a stake in the ground to avoid evolving Backdrop is highly unlikely to become "the hot new thing" but will instead be like FoxPro that for years after Microsoft acquired it was "a user base Microsoft could not grow and Microsoft could not kill"; that’s a direct quote from a former marketing manager at Microsoft.

The Shrinking Girth: Traveling Up the Pyramid

So Drupal 8 will be pushed by Acquia into IT shops, but it will be used by an increasingly narrow user base until the user base becomes so small that Acquia can no longer survive.

This long tail may take a really long time, but I am certain it is inevitable, unless of course Drupal/Acquia/Dries change strategy.

What SHOULD Acquia/Drupal Do Instead?

So here’s where I’ll divert from my criticism of Drupal and advocacy of WordPress; I’ll actually recommend what I think Drupal/Acquia/Dries should do and how they could potentially grow their business even if they do not catch WordPress in marketshare.

Announce the Drupal 8 Will Be "Drupal 7 Enhanced"

Dries Buytaert should do an about-face and announce that Drupal 8 will NOT be based on a new architecture but will instead simply be an enhanced Drupal 7, much like the about-face Tim Berners-Lee famously did when he announced XHTML was no longer the future of the web.

Adopt the Backdrop Team for Drupal 8 and Beyond

Dries should then work the Backdrop team and any of the Drupal 8 team who want to continue the status quo albeit with evolutionary improvements, much like how Merb broke off from and was later merged back into Ruby on Rails.

Further, adopt a no-breakage policy for future Drupal releases and work to ensure backward compatibility so that people are not forced into painful upgrades if they do not want to invest a significant amount into redevelopment. Learn from WordPress how to evolve without introducing breaking changes.

Announce a New CMS Called "Acquia"

Then, take all the ideas and lessons learned with Drupal that were destined for Drupal 8 and create a clean from-the-ground-up implementation of a next generation CMS targeting those who work rather program at the level of a framework but prefer to have more of the features needs for content management ready-built and available so as not to require people to reinvent the wheel.

Launching an Acquia CMS would have the benefit of being new in a way that could appeal to more than just the existing Drupal user base that does want to level up but not abandon Drupal. And Acquia is already a very strong company that has a stellar enterprise sales and support team so they would be in a great position to market a new CMS, and launching it would give them a stronger offer to sell to and support for their customers.

Acquia CMS could become the better alternative to Symfony that offers more functionality without all the legacy cruft of Drupal instead of Symfony being viewed as the better alternative to the Drupal CMS that carries so much baggage which is where I think things are headed.

Give Developers Something NEW To Adopt

And this branding is not just for technical improvements, it’s more important for positioning reasons.

Acquia CMS could have none of the negative associations developed by prior users of Drupal. Acquia CMS would be free to address all the problems I outlined in my prior blog post. And Acquia could once again become the CMS mindshare leader, a position that Drupal previously held IMO.

But Wait, Don’t Listen to Me!

If Drupal/Acquia/Dries does follow my advice, it would probably mean that I’d loose opportunities to work on certain future projects. The type of work I do with WordPress is most often competitive with Drupal in the minds the stakeholders deciding the platform the project will use. So I really hope they do not listen. :)

But hell, if they do follow this advice I would evaluate Acquia CMS and might even consider using it instead of WordPress in the future.

But really Dries if you are listening, please don’t! I’m currently really happy with the progression of WordPress and doing this would just throw a monkey wrench into my future works.

So nothing to see here; just carry on as planned. Nothing to see. :)

UPDATE

In the first version of this post I incorrectly referred to the fork as "Backstory", not "Backdrop" and I did not include a link to the Backdrop website nor mentioned Nate Haug. I have corrected the post.

Thanks to commenters Doug Vann, Brian and Jen Lampton for pointing out my error.

The Character of People

Many years ago I learned of this pearl of wisdom shown below. I don’t know when or where I learned of it or who might have been known to say it, I don’t remember if I read it or was told of it by someone else, and I don’t remember it’s exact wording either.

But I do remember its essence. And I have found its essence to be a rather consistent evaluator of the character of people, so I’m paraphrasing it here for your consideration:

Notable people talk about ideas.
Regular people talk about events.
Trivial people talk about – other people.

A Crash Course in Software Quality

It occurred to me recently that while intuitively obvious to many the concept of “Software Quality” is probably something never actually studied by the average developer who doesn’t have a formal computer science education.

Yet understanding software quality beyond intuition has been extremely helpful to me over the years as a programmer and software architect. And while there are many amazing self-taught developers it’d be a shame for me not to blog about it since I can cover the essence of software quality so quickly.

A Bit of Personal History

It was back around 1988 I was first learning OOP, also known as “object-oriented programming”. One of the few books avaiable on the subject was “Object-Oriented Software Construction” by Bertrand Meyer, but of the books available I probably learned more from it than any other. I’ve learned an amazing amount since but it was from that book I learned many of the foundational principles I still use daily as a software architect.

I still think most of what was covered in the book’s 2nd edition is relevant. If you are interested in reviewing it to decide if you want to read it I found a copy online but you can buy a used one for US$0.01 on Amazon.com

The External Factors of Software Quality, Named

The first chapter in OOSC covers Software Quality. To explain the concept Meyer simply defined numerous related terms, which he called “External Factors”, and wrote a few paragraphs about each. I’ve edited his definitions slightly, for brevity, and I’ve included them here as your crash course in software quality. Enjoy!

  • Correctness - The ability of software to perform its exact tasks, as defined by its specification.
  • Robustness - The ability of software to react appropriately to abnormal conditions.
  • Extendibility - The ease of adapting software to changes of specification.
  • Reusability - The ability of software elements to be used to construct different applications.
  • Compatibility - The ease of combining software elements with others.
  • Efficiency - To place as few demands as possible on processor, memory and bandwidth.
  • Portability - The ease of transferring software to different computing environments.
  • Ease of use - The ease with which various people can learn to use software to solve problems.
  • Functionality - The extent of possibilities provided by a system.
  • Timeliness - The ability for software to be usable when or before its users need it.
  • Verifiability - The ability to detect failures and to trace errors during operation.
  • Integrity - The ability of software to protect code and data from unauthorized access/modification._
  • Repairability - The ability to facilitate the repair of defects.
  • Economy - The ability of a system to be completed on or below budget.

Take these terms and commit them to memory. And when building software ask yourself “How does my software fare with respect to these factors?” If you can answer that your software fares well then you can rest easy in knowing that you are probably a developer who builds quality software.

As a side-note, after having learned OOP in the late 80’s I’m still surprised when a professional WordPress developer says, in 2014, that “I don’t really understand OOP.” Time to get with the program, I say!

Why REST is More Like Religion than Most Technologies

As someone whose entire career has been involved with technology platforms, and specifically programming platforms in some form or another, it’s clear to me something which is obvious to most patient observers: that adherents of a particular technology platform tend to become very “religious” about it.

Advocates of specific a technology platform are known to rather vigoursly proselytize and defend their technology platform of choice, and they are also known to call out “blasphemy” (as they see it) against their technology platform. I guess it’s just human nature to gravitate to concepts and communities and to then defend them from perceived outside attackers. I myself have at times been among the technology platform devout over the years though I do try my best to keep it in check.

But I’ve noticed that the concept of RESTfulness in Web APIs has a religious tenor that is beyond what I’ve observed elsewhere. This post’s goal is to explain what I’ve perceived. As you read, note that I make several points along that way that seem to unrelated, but I bring them together at the end.

Well-Known Founders

Many technology platforms, while possibly having a single founder are promoted by companies and over time their marketing and promotion tend to minimize the founder’s visibility among its adherents, such as Windows, Java, .NET, Zend Framework, Sitecore and ExpressionEngine to name some commercial examples.

Yet other technology platforms have a single visible founder and they tend to be open source, for example: Linux, PHP, Python, Ruby on Rails, Drupal and WordPress to name just a few.

Like these mentioned open-source technology platforms REST also has a well-known founder Dr. Roy Fielding who named and defined REST in his chapter 5 of his doctoral thesis, titled Representational State Transfer (REST).

Architectural Style vs. Platform

Now if Dr. Fielding reads this post I’m sure he would first object to my associating REST with Platforms; he has made it clear on numerous occassions he considers REST to be an Architectural Style and not a Platform.

That’s fine and I don’t disagree in the least, but I’m associating them because they share at least one (1) attribute. Few (if any?) architecture styles have emerged that are the result of one man’s PhD definition, as I’m far as I am aware. And that has ramifications that caused REST to be treated by its adherents more like a software platform than a lower-level architectural style.

Requirements and Constraints

Unlike most technology platforms which are often not focused on the rules of how to use it properly, REST is instead a prescription for the requirements and constraints a system must follow. In other words its about both what you must and what you cannot do (in order to be considered RESTful).

Potential critics of this post might point out that that is the point of an architectural style. But this style has an engenered a level of religious fevor, similar to that seen around technology platforms that I’m not aware any of other architectural style receiving, at least not lately.

The Good Book

And this prescription for must and must not is where REST starts to look a lot more like a religion than most technology platforms. The Torah, the Bible and the Koran, for example, they are all written works that prescribe correct and incorrect behavior among their faithful. Similarly Roy’s thesis defines what is and what is not correct among the REST faithful.

God and the 10 Commandments

While most technology platforms that have a visible founder see the founder actively involved in evangelizing, writing about, and shepherding their platform on an ongoing basis, Dr. Fielding has pretty much been an absentee founder. In the earlier days of the web he was active on W3C and related mailing lists, and he wrote a seminal post clarifying (especially in his reply to comments) that REST APIs must be hypertext-driven But since then Dr. Fielding has been conspiculously absent when any of the debates regarding the application of REST have emerged.

In many ways Roy has been for REST like the God of the Old Testament; he spoke to the people in the early days and wrote his “commandments” in the form of his thesis, but since then the faithful have only had his thesis and that one blog post to clarify the meaning of REST.

Disagreement and Debate

Today, fourteen (14) years since Roy’s thesis and six (6) years after his seminal post on REST disagreement and debate rages on regarding RESTfulness and Web APIs, its relative usefulness, the level of RESTful purity required, and especially as it relates to one specific constraint; HATEOAS.

I’d link to specific debates but there are so many yet few epic or seminal debates so it’s hard to pick just one. But I can link to several conferences and mailing lists where you’ll find these debates and mentions of REST-related debates on blogs across the web:

The primary things you’ll find among these debates is disagreement on the role of hypermedia and an assertion that permeates much of the dialog among the most fervent being that most other people building APIs “don’t get it” and “are doing it wrong.” On the other hand there appears to be very little agreement on how to do it right, at least when it comes to specifics.

I will say that I do tend to agree with those debating that most people do not get it and that they are doing it wrong because parts of REST are not easy to fully understand so it’s very difficult to be sure of what exactly “right” is. And why is that?

Exegesis

It boils down to this. There’s little disagreement about who gets to define REST; everyone (I know of) points to Dr. Fielding as being authoritative and his writings canoncial. REST was defined by this one (1) man who wrote down it’s specification in an academically defined manner sans examples, and then briefly clarified it in one (1) blog post with follow up replies to questions for about two weeks after.

Since then the REST faithful have been left to interpret what REST means on their own much like the process of Exegesis related to religious texts.

And like religious movements, REST has a good many people who have taken it upon themselves to explain the meaning of The Good Book and the intentions of it’s founder. Without Fielding actively participating and making judgements on these debates who has the authority to declare who is right and who it wrong?

What if God was One of Us?

Imagine if God had decided to hang around all these years and intervene on the topic of religous debates? Imagine how much less contentious religion would be?

In that vein, I leave you with this joke from Emo Phillips as hopefully an appropriate analogy:

Once I saw this guy on a bridge about to jump.

I said, “Don’t do it!”
He said, “Nobody loves me.”

I said, “God loves you. Do you believe in God?”
He said, “Yes.”

I said, “Are you a Christian or a Jew?”
He said, “A Christian.”

I said, “Me, too! Protestant or Catholic?”
He said, “Protestant.”

I said, “Me, too! What franchise?”
He said, “Baptist.”

I said, “Me, too! Northern Baptist or Southern Baptist?”
He said, “Northern Baptist.”

I said, “Me, too! Northern Conservative Baptist or Northern Liberal Baptist?”
He said, “Northern Conservative Baptist.”

I said, “Me, too! Northern Conservative Baptist Great Lakes Region, or Northern Conservative Baptist Eastern Region?”
He said, “Northern Conservative Baptist Great Lakes Region.”

I said, “Me, too! Northern Conservative Baptist Great Lakes Region Council of 1879, or Northern Conservative Baptist Great Lakes Region Council of 1912?”
He said, “Northern Conservative Baptist Great Lakes Region Council of 1912.”

I said, “Die, heretic!” And I pushed him over.

P.S. Credit for Inspiration

This entire post was inspired by Nick Kallen’s comment on Roy’s blog post about REST and hypermedia. His comment starts with this (emphasis mine):

I had a hard time with the writing in this article; I don’t normally perform exegesis on blog posts. Am I interpreting this correctly?

Thanks for Offering to Guest Blog or Advertise, But…

Once again I’m blogging on a topic I’ve previously spent much time covering in email replies. So rather than compose that email reply yet again I’m blogging it for today and for future reference.

TL;DR

Thanks for offering to guest blog or advertise; forgive me though but I’ll have to decline.

Or, If You Want to Know Why:

First, thanks for offering to write a guest post here on my blog, or to pay me to advertise here. I’m honored. But pursuing either of those opportunies is not consistent with my purpose for this blog.

I Blog for Myself

While I do obviously blog, I don’t consider myself an active blogger and I intend for all posts on MikeSchinkel.com to be written by me. My blog is for:

  1. My opinions when I get inspired to attempt to persude on a topic,
  2. To write posts about information I want to remember or refer back to later, or
  3. As in the case of this post, to refer people to read instead of writing the same email reply repeatedly.

I’m Not Interesting in More Blog Traffic

I don’t add content to this blog with a goal of generating more traffic, except for when I’m trying to persude and then only from those I’m trying to persude.

I’d Prefer to Have Fewer Comments to Answer

Plus I’d prefer to get fewer comments rather than more because each comment I get generates a “todo” which if I’m going to answer should be answered in a timely fashion. Even though a guest post wouldn’t require writing on my part, I would have a follow-up obligation to answer comments on the post should the author not, since it’s my blog. These would all become “urgent” todos even though they would almost certainly not be important in the grand scheme, they would still compete for time with my “important” todos that I constantly struggle to find time for (see urgent vs. important.)

For Me, Integrity Trumps Ad Revenue

As for advertising, if I started including advertising then people would immediately assume my goals for this blog are to generate revenue and that might call into question any position I might take on the blog; i.e. was I being paid for my opinion? Since my integrity is far more important to me than any small amount of income I might get in ad revenue I say no to any and all advertising on the blog portion of MikeSchinkel.com.

Including One Means Others Will Expect It Too

Let’s say I allowed one person to guest post or allowed one advertisement on my blog. As soon as I did I’d open myself up to be asked constantly to allow it to happen again, and then I have to craft a personal reply explaining why I said “yes” before but “no” to them. Better to just always say “no” and then nobody’s feeling gets hurt and I don’t have to spend time explaining.

Guest Blogging Begets Confusion

Although this may not apply to other blogs, my blog doesn’t have a design that would make it clear that a guest post wasn’t written by me and people might assume I wrote it. Worse, if the guest post was controversial people think I wrote it could result in a negative fallout for me. But the most obvious thing likely to happen is simply that people will start contacting me in comments, on Twitter or via email for clarification or to offer more guest posts, and that would all translate into more unwanted “todos.”

Bottom Line: I Appreciate the Offer, But No Thanks

And yes I know I’ve had this domain for over a decade and yes it’s probably got traffic and Google visibility that a site launched on a new domain today would covet, and some people see that as a waste of opportunity. But it’s my blog and that gives me the option to choose what gets published here. And I’ve decided I want to keep it non-commercial and only include posts written by me.

I thank you in advance for your understanding.

The Web Needs You To… STOP BLOGGING!

Well, that was an incendiary title. On purpose.

Now that I have your attention, let me say that I don’t literally mean “stop blogging”, I mean to object to the meme that seems to have overtaken the zeitgeist of too many lately. The “You Should Blog Every Day” meme. Here are some of it’s advocates:

As an aside I think somehow the folks at WordPress.com must have used subliminal messaging on their platform to encourage blogging addiction and thus, for them, revenue growth. But I digress…

What’s Wrong with Daily Blogging?

If you’ve read (any) of the posts above you’ve read glowing prose about how and why you should post daily, but none of the counterpoints. Just as we wouldn’t appreciate our neighbors leaving their trash bags on their lawn why do so many people champion other people to churn out non-stop pablum? Where in anyone’s good book has this endeavor been canonized as a virtue?

Lacks Depth

Daily posts are rarely more than opinion, and from what I’ve seen are usually without significant research, or links to related information on the topic. After all, who has the time for any of that when posting daily?

Adds High Noise, Low Signal

Publishing daily adds to the total amount of information out of them web. Let’s just say only 1% of US Citizens, let alone the rest of the world followed the “Blog Every Day” meme; together they would produce 1 trillion posts/year! Do we really need that much more low-information content to contribute to overload on the web?!?

More is Not Better

Daily posts focus on volume and not excellence. Unless you are one of those extremely rare prolific individuals who can be blog daily and write high quality content (and those people are usually known as “professional jounralists”) then posting daily is just setting one’s self up for #fail.

Not that you won’t be able to successful at daily posting, you might, but is daily posting really more beneficial than writing a much higher quality post less frequently?

No Time for Greatness

Similar to the previous, people who blog daily set themeselves up on a treadmill after which they’ll likely never have time to write a truly epic post. Some of the best and most valuable posts I have read on the web have been long form posts that clearly took more than a day to write. Few daily post ever gain continous linkage, at least not from what I’ve seen.

Why Should You Not Want to Blog Daily?

Besides the reasons above not to blog daily, what about selfish reasons for not blogging daily?

Busy People Will Tune You Out

Although I can’t say it’s a completely valid indicator, busier people seem to be the ones who accomplish more. As such, they are a higher value audience most of the time. If you post daily you’ll likely overwhelm people who want to consume your content but simply cannot handle the volume.

Expect More of Yourself

The first question to ask yourself is “What else could you be doing with your time?” If you spend only an hour a day blogging that’s over 9 weeks full time for a year; could you not achieve something better than a slew of blog posts?

If you are a programmer, for example, why not build and release an open-source project? If you are venture capitalist, why not take more meetings with entrepreneurs or use your network to help your existing investments more? If you are a social media maven (really? seriously?) maybe you could use your time to research all the emerging tools for tracking and analysis.

In other words, envision a BHAG project you could complete rather than just writing a new blog post every day. If John F. Kennedy had rallied US citizens around all journaling daily instead, would we ever have ever made it to the moon?

You Are Competing with Millions

If you are blogging about anything that is not highly unique, you are competing with millions.

Consider if you were paid your effective hourly rate for the time you spend blogging; would you invest that money in lottery tickets if you had it in your bank account? If not, why would you compete for the attention of people against so many others who are blogging too?

Life is Too Short

One of the mantras regarding daily posting is that you have to train yourself and develop discipline, which means that for most people blogging daily is just not fun. Why put yourself through that unless you are really going to benefit greatly from it?

But What About the Benefits?

Reading through all of the posts listed above about why you should blog each day it seems that the primary benefits stated are these, paraphrasing of course:

  • Increasing your blog traffic
  • Readers expect it
  • Developing habits are a key to success
  • Establishing yourself as an expert
  • Exercising your “Writing Muscle”

Let me tackle these one-at-a-time, in reverse order:

Exercising your “Writing Muscle”

That’s probably the best reason listed, and I agree. Except.

If it is really important to you to become a better writer, then yes, write every day. But you don’t have to push the “Publish” button, at least not for your public blog. Here are strategies that you could use instead that would excerise that muscle just as much:

  • Write a portion of a longer blog post, and do it every day.
  • Create a Facebook page and write posts for it daily.
  • Write a private journal daily. If you really need/crave feedback, invite friends to access it. But publish your best work less frequently on your blog.

Establishing Yourself as an Expert

This is another good reason. But a weekly blog can be just as successful at establishing your expertise, if not more so.

Look around at some of the leading experts you know via their blogs; how many of them blog daily? I’ve listed at least one below, Mark Suster. His expertise is well established, but he doesn’t blog daily unless he has something to blog about.

Developing Habits are a Key to Success

Agreed. But do you really need to blog daily to develop habits? Aren’t there other habits that can generate a better return in your life? (exercise, to name one?)

Readers Expect It

Sorry, this is just rationalization as far as I’m concerned.

When I was young my father told me:

“Don’t worry what other’s think; they think about you about 1/1000th as often as you think about yourself.” Similarly, most people don’t wait with baited breath for you to generate yet another post.

As for the few people who do tell you they wanted you to blog more (search for “Loyal Readers Crave More Content”) they are just feeding your confirmation bias compared to the majority of your vistors.

Increasing your Blog Traffic

Really, it comes down to this.

Yes, blogging daily is about driving more traffic to your blog, nothing else. And sadly, it works.

If your primary goal is to drive more traffic to your blog but not necessarily higher quality traffic, and the other reasons for not blogging daily are of no concern to you then more power to you. Just please don’t try to fool yourself or convince others that your daily blog ritual is for any other person than yourself.

Sorry to be harsh, but I haven’t seen anything that has given me evidence to believe there’s any other reason for it than self-promotion, and in some cases even narcissism. Maybe you can convince me otherwise in the comments below?

Are There Exceptions?

Of course there are exceptions. The ones who should daily blog (well, actually “write daily”) are professional journalists, you know people who get paid to write daily, and especially those who write news stories, which by definition require daily writing.

Also, anyone who generates highly unique content, such as content about their own company’s products or services could possibly benefit from daily blogging, especially if they have internal content developed by others from which to draw upon.

Effectively anyone who is likely to write content that nobody else is going to write could be forgiven for blogging daily. But even then, less frequently per author is better because then they have the time to write higher quality posts.

Information Overload Redux

When I was in college and learning to love programming computers there was one (1) monthly magazine that covered the programming language I was learning at the time. Each month I would read it cover-to-cover several times, and anxiously await the next issue to start again.

Today, thinking of programming for WordPress, I could spend every waking minute reading good quality articles that would be somehow relevant and informative regarding my current chosen professional. But to find the good articles I’d have to sift through the other 90% of that were published just for publishing sake, the ones that are little more than noise.

Simply put, blogging daily exponentially increases the amount of noise on the web. And that’s really not good for (the users of) the web, or humanity.

Bloggers I Wish Didn’t Post Daily

Speaking of daily bloggers, here is a short list of people who I really respect and admire and who I would love to see write some truly epic posts inspired by their knowledge and experience. Unfortunately each day I find a typically a mediocre post albeit occasionally a few almost good ones. But (almost?) never do these people product really great posts.

Fred Wilson of AVC.com

Fred <@fredwilson> is known as the VC who has funded some of the most visibly successful Internet startups in the past decade. He blogs daily, and I get an email containing his daily posts. Given his daily blogging schedule and all of his other obligations and evident success, I am constantly amazed at how good his posts are.

But more than being amazed, I’m also disappointed because he never blogs long form or in-depth. I know he has great knowledge and experience to share, but I never really feel like I’ve learned something after reading Fred’s blog posts, I just feel like I’ve been kept up to date.

Now Fred’s blog has generated an incredibly active community of commentors. Many of the frequent commentors are well-known and successful people in the startup world, but I constantly amazed at how much time these people can spend commenting on Fred’s blog in a day. It flabbergast me, frankly.

Fred’s blog uses Disqus for commenting, a company he has invested in but one where I find product usability very lacking other than I do really like the ability to edit the typos in my comments, which you can’t do on most blogs. The reason I mention this is that as soon as I comment on Fred’s latest blog I am inundated with about 250 emails that day because Disqus emails every me comment, not just replies to my comments, and Fred’s blog overflows with comments. This is clearly good for Fred’s personal branding and provides him with a posse of people he can ask for help, but then if Fred wasn’t an uber-successful VC I doubt he would get the same following from his daily blog.

After all, people are attracted to where the money is…

David Cummings on Startups

David <@davidcummings> is an incredibly talented individual. He’s a “local boy done good”; he built and then in late 2012 sold Pardot for $90 million to ExactTarget, which was then sold to Saleforce.com. Almost immediately after be purchased a 100,000 SQFT building in Buckhead, Atlanta’s prime real estate market, a.k.a where old money lives and new money parties.

David christened his new Buckhead facility Atlanta Tech Village, now home to 100’s of startups. He has become a well-known ambassador in town for high tech startups and has receive the attention of the USA Today, the Mayor of Atlanta Kasim Reed, the Atlanta Journal/Constitution, Atlanta’s Creative Loafing, InvestAtlanta, the Metro Atlanta Chamber of Commerce and more.

I think 20 years from now Atlanta will look back at David as the father of Atlanta’s High Tech Industry Boom, or something similar. When I tell people about David who don’t know of him I say his is Atlanta’s future equivalent of Brad Feld re: Bolder, Colorado.

CLEARLY David is amazing, one of the best business thinkers in Atlanta. And he blogs daily, and yes I get an email for every one of his posts. Unfortunately David’s posts are short and rarely ever better than a high-level outline of some topic that he has clearly queued up for the day. I receive his post via email, like clockwork, around 10pm ever night.

David has so much value to share, and he obviously devotes the time to sharing. But unfortunately David’s choice of daily bloggingmeans that he rarely if ever (has the time to) write a really valuable, in-depth post that leverages his profound knowledge, experience and expertise. For example, David frequently mentions the importace of establishing a great company culture but he’s never blogged anything that helps a would-be entrepreneur know how to establish a great culture in a startup.

Such as shame, really.

Tom McFarlin

Tom <@tommcfarlin> is one of the sharpest guys in the WordPress space, if not the sharpest WordPress guy I know personally.

Tom is also incredibly prolific. He blogs nonstop it seems, constantly writes for Envato, he’s written numerous plugins listed on WordPress.org, he gets profiled frequently, he was a partner at 8Bit which used to sell the Standard Theme he helped develop, and he was a contributor to their WPDaily blog, which is no more.

Anyway, I tried to follow Tom’s blog for a while and periodically he had some incredibly valuable posts. Unfortunately they were interspersed with numerous opinion and/or low information posts many of which generated a lot of comments but few if any dispensed any real usable knowledge, at least for me. It’s as if he set himself a goal to write daily, so by gosh that’s what he is going to do.

Sadly, I had to unsubscribe because the noise-to-signal ratio was just so high. And that made me sad, but I had to do it.

Honorable Mention: Eric Mann

I got to know Eric <@ericmann> during the time I was actively involved in moderating and answering questions on WordPress Answers. While I learned that Eric is a very bright WordPress developer with lots of relevant experience and a great ability to explain answers to WordPress questions in writing, what I admired the most about him was how even-keel he was when interacting with others on the web. Never did I see Eric involved in a flameware (unlike me, unfortunately) nor have I ever witnessed him talking down to someone online, a behavior that otherwise runs rampant online, especially in certain open-source circles. Eric really has my respect.

And when Eric blogs a how-to article about WordPress, it’s usually well worth reading, at least for me. I know Eric has interest in writing a novel, and recently it seems Eric has decided to blog every day (if I misunderstood Eric, please forgive.) My criticisms are not of his posts so much as a knowledge he wants to blog daily and I fear we’ll see more quanitity and less quality.

As an aside, it was actually that comment exchange which triggered me to finally write this post rather than just repeatedly think this sadness I feel when faced with the daily blogging of others who blog posts I would love to read, albeit not daily.

Awesome Bloggers Who Don’t Post Daily

Conversely, here are a few of my (current) favorite bloggers. They seem to only post when they have time and inspiration, but their posts are almost uniformly excellent. When I think of these people I don’t think of how much pablum they have churned out, I only think of how much I am in awe of them.

Mark Suster of Both Sides of the Table

Mark’s <@msuster> blog tagline is “Entrepreneur turned VC” and his experience just exudes from his posts. From my perspective Mark is a VC on par with Fred Wilson albeit he’s not been a VC as long thus he hasn’t had the time to rack up an equivalent number of successes. But like Fred he writes for entrepreneur’s benefit and is clearly not a VC who thinks the best way to win is to take advantage of startup entrepreneurs. Both he and Fred write as if the best entrepreur-VC relationship is a win-win relationship, and that’s why I think both of them has gained so much attention.

Whenever one of Mark’s posts arrives in my email I think “I need to make time to read that, because I know it will be worth reading.” And 9 times out of 10, it is relevant to me and my interests in startups and provides me with significant insight or understanding that I would struggle to find published elsewhere.

Kudos to Mark, he’s my favorite blogger and I would love to have the opportunity to meet him in the future under circumstances where I have something of benefit to offer him.

Price Intelligently

I don’t know who actually blogs for Price Intelligently <@priceintel> but their posts are almost consistently excellent. They blog more frequently than Mark Suster, which surprises as they are able to keep up the quality, but if you care about optimizing pricing for a SaaS then this a must-read blog.

And I’m really glad they don’t try to do the daily thing.

April Dunford of RocketWatcher

Contrary to the rest of this post, I really wish that April <@aprildunford> would blog more. I think she’s like me; she’ll get a burst of inspiration and write a few posts, and then she’ll get busy and 6 months will have gone by with no new posts.

I don’t know April well but she’s got a startup marketing blog, and it’s excellent. When she posts.

Nuff said.

At Least I’m not the Only One

Finally it seems these people agree with me, at least somewhat:

One Final Takaway

So if you are contemplating the development of a daily blogging habit, please consider this the summarization of the above:

  • Choose Quality Over Quantity

 

-Mike

P.S. I do get the irony of my blogging on this topic. But since it’s contrarian in nature I think it counts as unique. One thing’s for sure; I could not write posts like this every day.

Tags: ,

Who Do You Recommend to Design a WordPress Website?

Here’s an email I got a few days ago from someone I met at a Meetup about 6 months ago:

Hi Mike,

I have a friend looking for setting up a non-profit website and they want to use WordPress. I was wondering if you’d be able to help with this, or if not if you know of someone you can recommend who does WordPress website design? My friend needs mostly graphics work, but also need help setting up their WordPress site. They already have the logo and the content. Their budget for this is around US$1000.

- Thanks!

Déjà Vu

I get an email similar to the above about once a week on average. It seems I’ve become branded in the eyes of many people in Atlanta as "The WordPress Guy" even though I don’t do what most people think of when they think of "People who do WordPress"; i.e. I don’t design nor do I build WordPress-based websites[1].

<sigh> :)

Pay it Forward

But when someone asks for help I really do want to help.

I’d never fault anyone for not knowing that I’m the wrong person to ask. And I also wouldn’t fault someone who doesn’t know how much it costs to hire a WordPress specialist; if someone is not a immersed in the web world how could they know?

No More 1-off Emails

Still, I’ve written a response to this type of email more times than I care to count. Speaking of, a few days ago a blog post by Eric Mann inspired me to stop replying long form to emails and to start writing blog posts instead.

So here goes.

A Custom Website Design for US$1000?

Let’s talk about that US$1000 budget. In the Atlanta area we have over ten (10) Fortune 500 companies and as a result we probably have over 100 digital agencies, all directly or indirectly serving those large companies as well most of the midsize companies in the area. Good graphic designers are in high demand here, and they are used to being very well paid. I’m not sure, but I expect the same is true in most major US cities as well.

For US$1000 it might be reasonable to expect to three (3) design comps for your future website’s home page. But it’s highly unlikely you’ll find a quality designer to generate a custom design for an entire site, encode the design into HTML+CSS, convert the code it into a WordPress theme, install WordPress at a hosting company, research, select and then install the various WordPress plugins needed for the features desired, and finally configure everything, all the while taking input from a client who is likely to constantly question aspects of the implementation. All for only US$1000.

And the previous paragraph assumes you can even find someone with the skill to do all those things rather than needing to find a team or to assemble a team of different skills to build the site.

Of course you might get really lucky and find a student from SCAD, Creative Circus, Art Institute, or Portfolio Center who would be willing to build your website for US$1000, assuming your student has already worked extensively with WordPress as a hobby.

So like I said, you might get really lucky…

What Should My Website Cost?

But what price is reasonable to expect?

Those who build WordPress websites know the "How Much?" question can be a landmine. Quoting a price too early can get a WordPress sitebuilder into hot water. I’ve seen WordPress websites cost between free – self-serve at WordPress.com – and US$500k or more. How can a site builder know how to price a website prior to fully understanding its requirements?

Price really depends on both what the client needs as well as what the client wants/expects. And the latter is rarely consistent with the former. For example, does a divorce attorney’s website really need a Flash-based header showing storm clouds, and lightning strikes on hover?!? (yes, that is an actual client request, no demand, that I heard from one of my friends who is a sitebuilder.)

WHICH IS WHY I LOVE this website price calculator:

It was built by Erik Wolf who runs ZeroG Creative. It walks a wannabe website client through a series of questions that help the prospect understand some of the things can affect price and by what magnitude.

Here are some of the questions:

  • "Are you planning on hiring a designer/firm?"
  • "How many people will be involved in the decision-making process?",
  • "Will your website require eCommerce?",
  • "Will your site require social media integration?", and
  • "Will you need a dynamic photo gallery that you can update yourself?".

Depending on the options selected Erik’s price calculator generates prices between US$500 and US$16,500, where for $500 you basically get a site and theme installed, nothing more. And for small business websites I’d say that’s pretty close although frankly I’d expect more like US$1000 and US$25,000.

What About Non-US People?

Note those prices above are for US-based WordPress developers.

And yes, you can pay significantly less to have a WordPress site built by someone outside the USA. But it’s also possible that you will pick the wrong person and that person will either not deliver or will deliver something that doesn’t match your expectation after your entire budget has been spent.

Offshoring can work great for certain type of projects if you have the luxury to pay to try numerous people to find the one that really meets your needs. But if you have only enough budget to try one person, your taking a big risk with your money. And you’ll have no recourse if they fail you.

So caveat emptor if you hire your web developer off Elance.

Agency Projects for Large Companies

By the way, if you are trying to determine the cost of a WordPress website for your Fortune 500 employer expect that your site will cost between US$100k and US$500k.

Why the huge difference in price between small business websites and large business websites? In a word, "Expectations."

More specifically, because of their collective need to see exhaustive design variations, their need to allow your numerous stakeholders to control and approve every detail, their insistence that unrealistic deadlines be met, their expectation that every aspect is perfect upon first preview of features, their desire for constantly scheduling unnecessary and unproductive meetings, and their IT department’s insistence upon using a hosting company that has no expertise in WordPress and no desire to learn it.

But I digress.

But Do You Really Need a Designer?

Considering the budget of US$1000, maybe a "Website Designer for graphics work" is not really what is needed. Maybe what they need is a what I like to call a "Site Builder."

A Site Builder is a jack-of-all-trades and master of none, with respect to WordPress. This is someone who can setup a web host, install and configure WordPress, select an off-the-shelf theme and tweak it to incorporate the logo, and finally add and configure various plugins to add functionality such as email signup forms, social media integration and optimization for SEO.

To further illustrate the difference between a Designer and a Site Builder I roughly categorize WordPress skills as one of these where many people having more than one (1), but rarely more than two or three:

  • User/Author (Content writer)
  • Layout/Graphic Designer (Photoshop)
  • HTML Coder (HTML+CSS)
  • Themer (WordPress Themes)
  • Front-End Developer (Javascript/AJAX Developer)
  • Back-End Developer (PHP/MySQL/Plugin Developer) <– This is me
  • and finally Site Builder (Installs/Configures/Adds a Theme and Plugins)

So for a US$1000 budget I think it’s realistic to find a US-based consultant who can deliver a basic website, assuming the client understands the limitations of using off-the-shelf themes and plugins, i.e. but no custom design and custom PHP or Javascript code. It’s a lot like buying a car; you might not like the wood-trim dash but if you chose a model that only comes with a wood-trim dash, that’s what you get.

Other WordPress Specialties?

I tend to want to make my lists exhaustive, so in that vein I might as well list these specialties too, for the record:

  • eCommerce Specialist - Expertise in online retail and payment processing
  • SEO Specialist - Optimizes for search engines
  • Security Specialist - Reviews code for security holes
  • Performance Specialist - Helps developer improve performance
  • Hosting Specialist - Configures servers for high scalability

What are Off-the-Shelf Themes?

Yeah, I threw that bit of jargon in there when I mentioned Off-The-Shelf Themes. If you are not familiar with this term it refers to packages of design and code that you can purchase from 3rd party vendors that, once installed will update the look and feel of your WordPress-based website.

In it’s 10 years WordPress has spawned a large number of commercial theme vendors, more than 100, although the vast majority of themes are probably sold by 10 or fewer vendors.

Look for an Existing Theme.

If you need a website and your budget is small I’d recommend you surf the main theme vendors websites to find a theme you could envision your future site using. If you can find one that meets all your needs, your low budget website might just be able to be reality.

The following list are the theme vendors I know the best, in alpha order (if you are a fan of another theme vendor feel free to list in the comments.):

But Don’t Expect Significant Changes

Please do realize though it is very difficult for a Site Builder, Graphic Designer or even HTML Coder to make more than trivial changes to the look-and-feel of an off-the-shelf theme without a huge expense. Themes can be very complex beasts and it often takes as long for someone to learn how to modify someone else’s theme than it does to create one from scratch.

So please don’t put your Site Builder in the position of having to explain to you why your "simple change" is really a very time-consuming and labor intensive task (that they will have to bill you for, if they will even agree to do it.) Instead, have them explain to you what is easy and what is hard and then only ask them to do the easy (and inexpensive) things. Your willingness to appreciate their efforts will keep them wanting to service you in the future when you need additional support.

Finally, Who Do I Recommend?

So who do I recommend to build your WordPress-based small business website here in Atlanta? Frankly in good faith I can’t recommend anyone. Why? Because I’ve never worked on a team building a small business website before so any recommendations I would make would really just be me telling you who I know.

That said, I can tell you who I know that specializes in building WordPress websites, in the Atlanta area. Here they are, listed in order of how well I know them (note: I believe all of these have minimum fees higher than US$1000):

There are a lot of others I know outside of Atlanta, but most of the requests I get are from people in Atlanta and thus I’m listing those I know who serve my local area.

And yes I know, it’s a potential faux pas for me to create this list as I’ve most certainly forgotten someone; if it was you who I have forgotten please accept my profuse apology and leave a comment with your contact information below.

Friends, Family or DIY?

Finally, if you or your friend cannot find a WordPress specialist to build your website within your budget, maybe you can find a friend or family member who can help? WordPress powers almost 20% of the web so that means you probably already know a friend of a friend at least who has set up their own WordPress website and can help you get your site going. Ask on Facebook, maybe?

If you are a non-profit, as above, maybe you can find someone passionate about your non-profit’s mission who knows how to set up a WordPress site; they might be willing to work for significantly below market rates.

And if all else fails, do it yourself. It really is possible for a reasonably intelligent person with moderate computer skills to install and configure WordPress; it just takes a bit of stick-to-it-iveness and lots of Google searches to figure out how to do it and launch it yourself.

Anyway, hope this helps.

-Mike Schinkel
WordPress Platform Architect

Footnotes:

  1. Instead of designing and building small business websites I architect and build products based on WordPress for software companies and agencies. I call myself a "WordPress Platform Architect." My focus is very narrow and as such I don’t develop the experience needed to build websites for small businesses. I can’t help select a theme and I don’t know which plugins work best. And I’m as far from being a designer as any web person has the potential to be. But if you want to use WordPress to implement a complex site and need a specialist to architect if for your team to then perfect, I’m your guy.

Presentation Notes on Sunrise for Metamorphosis Notes (WordPress Post Meta Team)

Post Meta Meeting

Tuesday December 3rd & 10th 2013

What

Where

Who

  • Mike Schinkel
  • Micah Wood (wpcholar)
  • Andrey (Rarst) Savchenko
  • Taras Mankovski

Why

  • We built a company’s products
    • Platform for building large lawfirm website
  • Client didn’t respond well to: "Well we can’t (easily) to do"

Architecture

  • Focused on programmer productivity

    • Source code solution for version control
    • Sunrise was on 2nd generation code
    • ASIDE: We are now on 5th generation code style
      • See: https://github.com/newclarity/exowp
      • Exo is an MVC Skeleton for WordPress, w/mixins
        • Not Forms/Fields, but lower level for MVC
        • Redesign of Sunrise would follow Exo architecture
  • Core OOP layer

    • Fields
    • Forms
    • etc.
  • Sitebuilder layer

    • register_form()
    • register_field()
    • Virtual Field Types
  • Themer layer

    • get_field()
    • the_field()

Admin User Interface

  • Should be able to completely replace metaboxes in admin UI
    • Especially Title, URL and TinyMCE editor
  • Most common use-case; a single metabox
    • Optional "modules" should be able to add to that metabox
      • vs. adding another metabox.
    • Adding to metabox is not hard with Sunrise architecture.

Storage Format

  • For Post Meta Key & Value
  • See SQL Query

Field Features Objects

Fields generate collection of HTML elements.

Made sense to create "field feature" class to subclass because all shared so much code.

  • Entry
  • Label
  • Help
  • InfoBox
  • Message

get_field()/the_field()

  • Simple naming
  • Filters:

    • 'pre_get_field'
    • 'get_field'
    • 'empty_field'
    • 'the_field'
  • Virtual fields

    • "nested dolls"
  • Issue to address with naming conventions or $args type: Return values of string/numeric vs. object/array vs. HTML

Virtual Fields

  • Especially useful to output Schema.org valid markup
  • Postal Address: ‘postal_address’
    • Stored as:
      • ’street’
      • ‘city’
      • ’state’
      • ‘zip’
      • ‘country’

Virtual Field Types

  • "Real" types defined by class
  • Virtual types based on existing type w/set of arguments.
    • Like cascading properties of CSS

Storage Types

Field storage defined by field, themer need not no storage location.

  • Meta storage
  • Core storage
  • Table storage (custom sql)
  • Taxonomy (term) storage
  • Future/Plugins
    • Pods
    • MongoDB
    • Redis
    • Amazon S3
    • Other…?

Dynamic Forms within Forms

  • Form Selector Field
  • One field could display a form within a field
  • Created a ‘form_type’ meta field to control which form type

Repeating Fields

  • Overuse leads to madness
  • Should support simpler use-cases
  • Repeating should be in core framework, not its own field type
  • Complex use-cases should use object/post relationships (hint/hint)

Other Considerations

  • Minimize nested arrays when initializing
  • Separate Declaration from Fixup
  • Multiple register_field() vs. one register_fields()
  • Then there is MVC + Mixins…

Demo Code

Proposal - Securing the WordPress JSON API

I was recently added to the WordPress API team and this post contains my thoughts about the recent authentication discussion.

WordPress have a reasonably robust authentication system built in, the username and password system and it would be possible to use it along with Basic Auth to allow for API authentication. Please forgive any typos in advance; this was long and I didn’t really have the time to fully proof it.

Authentication, Identity and Authorization

While Authentication is very important there is also Authorization to consider. Here’s a nice blog post from Apigee on the difference between three (3) terms: Identity, Authentication and Authorization (IMO Apigee are the leading experts on web API design at the moment). In a nutshell here’s what they terms mean:

The Term What it Means
Identity Who is making the request?
Authentication Are they really who they say they are?
Authorization Are they allowed to do what they are trying to do?

And as they point out we may not need them all but what we need is the point of this post.

As a side note they say "Take Twitter’s API; open for looking up public information about a user, but other operations require authentication." What this says to me is an API key would be ideal for most read activities but most write activities should require Authentication.

Authorization without Authentication

As much as we need Authentication I think we need Authorization even more. There are some API actions we’ll happily allow anyone to do such as download the list of our most popular posts and we don’t need to authenticate for that, we only need to authorize.

Why authorize? Why not just allow open access? So we can track who we authorized in case, for example we need to rate-limit their usage or even revoke their access.

About SSL

Let me get this out of the way sooner than later. Anything that requires SSL is a non-starter just as requiring PHP 5.3 for WordPress 3.7 is a non-starter. Need I say more on this point?

However we could allow support for SSL, assuming that for what we implement the SSL and non-SSL solutions are compatible.

Mainstream Options for API Security

Let’s discus the variety of methods for securing an API; some mainstream and some a bit esoteric. Bottom line is that most informed people seem to say "Don’t role your own." So with that in mind I believe we have these options:

Option Discussion
OAuth 2 Generally considered the best balanced security option for mainstream web apps where security and ease of interaction for users is balanced. But can be complex to implement, especially on the client end, and requires SSL to be secure.
Basic Auth Not as good as OAuth 2 but super easy for the client to implement OTOH it is not secure unless SSL is used.
Digest Auth More secure than Basic Auth but still not fully secure. Quite a pain for the client to implement..
Amazon Auth Well-tested and doesn’t require SSL but is non-standard (ignoring "defacto-" standards) and still requires an API key.
API Keys Very simple for the client to implement and as secure the Capabilities tied to the API key, i.e. if it can only see public data and not update then it’s "secure enough". Fully secure if used with SSL. Assuming users can’t change passwords with the API key then it’s more secure than Basic Auth because user credential are never in a position to be compromised.

(Did I miss anything?)

Given the available options it would seem to me that OAuth 2, Digest Auth and even Amazon Auth are non-starters as a requirement for use of a JSON API in WordPress core because of the complexity each of them heave onto the API client developer, at least if one of these is the option for accessing the JSON API.

Basic Auth vs. API Keys

Which leaves the unsecure Basic Auth and mildly secure API Keys. So review the pros and cons of using Basic Auth – which is tied to the WordPress user’s username and password in the current version of the JSON API – and API keys:

  Pros Cons
Basic Auth
  • Easy to implement
  • Insecure
  • If API login is compromised then user may loose their account or be made to go through the hassle of regaining access.
  • Since APIs access can be automated it’s much more likely that a hacker could capture a username/password on a non-SSL API call (calls might be made continuously) than for a user login (which comparatively happen very infrequently.)
  • Can only support one Authorization profile per user account.
  • To support multiple authorization profiles a user would need create multiple user accounts,
  • To allow another person API access they either need to share their username/password or create another user account for them.
  • If API access requires a user account some sites could go from 5-10 users to having 50,000+ users (think of smaller sites like Mashable.)
  • If multiple user accounts are required then we’ll need a way to relate user accounts and allow one user account to manage other user accounts.
API Keys
  • API authorization is decoupled from user accounts.
  • One or many API keys can be tied to a single user account.
  • If API Key is compromised user can login and deactivate it.
  • Plugins could easily deactivate API keys if they follow an abuse pattern.
  • API keys could be added with expiration dates.
  • Sites with a large number of API users do not gain an explosion of regular users.
  • Each API Key can potentially support a different Authorization Profile (example use-case: I provide on API key to a social network – the key has limited capability – and use another API key – one that can do anything my user account can do – for an official WordPress mobile app that I use to access my site.)
  • Requires what appears to be more architecture

It seems to me from this comparison that API keys are the only reasonable option for allowing JSON API access to much of WordPress. However they are only appropriate for some use-cases and not even as-is they are not as a complete solution. Let’s discuss the rest of the solution for the use-cases in which I think they apply.

It also seems to me that tying API access to users accounts could easily create an explosion of complexity and significant user experience problems as users see their logins hacked by unsecure usage and then are locked out of or even loose their blogs.

API Roles and Capabilities

One of the ways in which API Keys might be acceptable without Authentication is that some things can be made freely available holders of API keys if we add in "API Roles and Capabilities."

Just like User Roles that are assigned a collection of Capabilities we could add "API Roles" that also have "API Capabilities". These Capabilities could be used to determine the Authorization status for each (what I’ll name) an "API Service" when requested.

Note: I’m defining an "API Service" as a URL + an HTTP method (GET, POST, etc.) and I’m calling the collection of Authorizations for all API Services as a "Authorization Profile."

I’ve reviewed the code for the WP_Role, WP_Roles and WP_User classes and I think the first two could be used without modification. If so then we only introduce a WP_API_Request class. And depending on the opinion of others the WP_API_Request class could be standalone or the WP_User class could be refactored to extend from an abstract WP_Auth class thereby allowing the new WP_API_Request class to also extend from WP_Auth.

We could then decide on a convention that any Capability name prefixed with 'api_' is a capability for an API Service and we add a function current_api_request_can() or just api_request_can(). Armed with api_request_can() we could write code like the following (note that api_request_can() assumes 'api_' as a prefix and thus does not require it to be passed):

Source code example

Are We Adding Too Much Code?

Although a comment was made that "we don’t want a huge chunk of code just for authentication" I would suggest that even if it were to be a large amount of code, which I doubt there would be, it shouldn’t matter how much code we add as long as that code doesn’t require significant maintenance and more importantly does not impose significant complexity onto the admin user in terms of "more options."

  • Assume that in Settings > General we add only one (1) single checkbox with the label "Enable JSON API" which by default we leave unchecked.

  • Once the user has explicitly chosen to enable the API (the equivalent of activating the plugin we have today) a single "Tools > JSON API" option is added.

  • The Tools/JSON API admin page can use tabs to organize the information so it would not be overwhelming, if even needed.

  • To offer the user the list of API keys we can reuse/modify the Taxonomy add/edit functionality assuming we add a 'user_api_key' taxonomy to allow us to store, lookup and manage API keys related to Users who would "own" the API keys.

  • Another tab for the Tools/JSON API admin page could potentially offer the ability to add and manage API Roles and another tab for API Capabilities. Or not, we could require these be managed programmatically just like User roles currently are.

  • And finally a main tab that allows you to force SSL use, or not.

What I’ve describe above it really not that much code. Would it make sense to risk the potential downside of tying the API to username and password in order to simply avoid the code that the API keys management would require?

Handling Escalating Security Requirements

Consider the "API Services" discussed earlier; we could implement a mapping of authentication requirements to API services such that different services have different authentication/authorization requirements. Consider this table:

Requirement HTTP Methods API Services That Allows Example API Service
No API Key Required GET Access to public information with a low risk of needing a rate limiter. An API service that returns site name and other metadata. The metadata could also including a links to an API service to request an API key via API.
API Key GET Access to public information that might need to be rate limited. Return the current list of blog posts.
API Key + Nonce POST, PUT Add Content or Update Revertible Content Update of Posts, add Taxonomy Terms.
Nonce GET Add Content or Update Revertible Content Update of Posts, add Taxonomy Terms.
SSL+Basic Auth GET Returns secure information for client w/o API Key Retrieve an API key programatically.
SSL+API Key POST, PUT Updates secure information Modify User Profile, Deletes Posts.
SSL+Basic Auth POST, PUT Update highly sensitive information Change user password

API Keys + Nonces

Note that we combine nonces with API keys. One of the ways WordPress handles security is with nonces, and the API need be no different. Note that the nonce would be generated by WordPress core or a plugin for the logged in user to allow their browser’s to use the API via AJAX. These use-cases would authorize for the JSON API similar to how the current AJAX system in WordPress authorizes.

For mobile apps nonces could also be offered to last for longer, requiring a mobile device to retrieve a new nonce once every 15 minutes or so but then allowing them to just use the nonce + API key within those windows. Of course you wouldn’t want a 15 minute window for nonces used with AJAX apps

Using SSL

So if we follow the outlined approach we can provide a reasonably level of API access without requiring SSL but we can still enforce the benefit of SSL for those who are likely to have the where-with-all to upgrade to SSL.

Consider this, if they need their sensitive parts of their site updated via API then they are likely special enough that they can make sure that SSL happens. But if unexpected consequences occur and someone builds a SaaS that people want to use but that requires SSL then frankly it creates an opportunity for hosting companies to see a high level of demand for turnkey SSL setup.

And optionally we can add an 'WPAPI_ALLOW_NO_SSL' constant for those site builders and site owners with a "Devil May Care" attitude.

Summary

In summary I’m proposing for the JSON API for WordPress to:

  • Use API Keys for Authorization
    • (And if you are still not convinced, read this).
  • Incorporate API Roles and Capabilities
  • Support Escalating Authentication Requirements for API Services
  • Build Single Menu Item Admin UI for the admin to Manage the API.

Let me know your reactions in the comments below.

What the World Needs is a Markdown IDE for Programmer Documentation

As an active JetBrains’ PhpStorm user one of my feature requests was for First Class WYSIWYG Markdown/Markdown Extra Support. Unfortunately they told me (and others) to use a 3rd party plugin which given it’s lack of quality and features turned out to be a non-starter for me. So I continue to use Markdown Pro which I love for what it is but I really need an order of magnitude more features.

But today I was thinking hard about how I’m going to implement documentation for the project I’ve worked on over the past 3 months without killing myself. A sad realization came over me that using MarkDown Pro would be very painful to use because it’s really nothing more than a glorified Notepad with Markdown support and a preview window; it has nothing to support me in the developer of documentation projects.

Then it hit me; what I really need is not an ability for PhpStorm to edit markup but instead a full-featured documentation IDE targeting programmers. And frankly I think the company best positioned to offer this would be JetBrains but I’d be happy to see any company offer it, if someone just will (Maybe those Sublime guys could…?)

So if you are from JetBrains or from some other company please consider the following feature set:

Features

Here are just some of the features that I’d love to see a Documentation IDE support

  • Manage "Documentation Projects" vs. just individual markdown files.
  • Multi-pane editor like PhpStorm but with panes that support document creation.
    • Vertical or horizontal split edit and preview windows.
  • CSS-based Themes for preview.
  • File Watchers for post processing LESS, Saas another other features.
  • Version control support for Git, Mercurial, SVN, etc.
    • Support for docs in a subdir of a code repository or as independent repo.
  • Support for all major Markup/Markdown format including Different Dialects and HTML
    • Conversion between Markup/Markdown formats
    • Ability to configure all and/or specific documents to be edited in one format/dialect and saved in another.
  • Ability to Publish and Maintain Documentation Websites from DocStorm
    • Publish directly to GitHub Pages as well as maintain existing GitHub pages.
    • Publish to Evernote, DropBox, etc.
    • Offer "POST-To-Publish" feature that would allow us to publish and update using HTTP POSTs so that we could write our own server-side integrations to other locations besides GitHub such as CMS (WordPress, Ghost, etc), Wikis (Mediawiki, etc.), SaaS platforms and more using our own PHP, Ruby, Python, Node.js or other code server-side code.
  • Navigation Between Documents
    • Jump to Document via selected hyperlink
    • Jump to Section of Document via selected hyperlink+fragment
    • Jump to File by Name
    • Jump to Headings in Project (find by autocomplete)
  • Refactoring
    • Refactor Document Structure to change all affected links
      • If URL changes
      • If URL fragment changes
    • Move selected content into a new file and insert a link to the new file.
    • Provide a tree view of files and allow refactoring by drag-and-drop in tree view, with all necessary link fix-up.
  • Manage Images
    • As part of the project, relative to the project root
    • Enable images to be previewed inline
  • Search and Replace like the wonderful PhpStorm search & replace)
    • Regular expression search.
    • Highlight on up/down arrow of selected options.

Also potentially valuable would be integrations with existing documentation tools although I can’t yet envision exactly what that would look like:

Benefits to JetBrains

Back to JetBrains and why they should do this. Currently they have IDEs for Java, PHP, Ruby, Python and Javascript programmers and for the most part (I would assume) few of their customers cross over and purchase multiple tools. However if they were to offer a "DocStorm" then every one of their current IDE customers they might be able to up-sell a sizeable number of developers.

But even if they won’t do it, maybe someone else will? 

Update

If you like this idea please vote for it on the JetBrains tracker

Update 9-Oct-2013

So JetBrains considered the idea, but decided against it. :-(  But they have changed their mind in the past if they’ve had enough requests, so please vote for both these tickets, if you will:

Better yet, if you are in the editor space and looking to expand your market, please consider building this flavor of your product and I expect you will find many new customers.