Entries from Aug 2008 ↓

Event: Why you MUST have a Twitter Strategy

Just an announcement that we are going to be discussing Why you MUST have a Twitter Strategy at Atlanta Web Entrepreneurs on August 21, 2008.

I’m going to present a short intro/overview to Twitter and then, god willing and the creek don’t rise, we plan to have two (2) video conferences, one from Triangle Tweetup and the other from a soon-to-be-announced Industry luminary with over 25,000 Twitter followers!

After the 8pm break we’ll have a roundtable-less discussion and Q&A led by our featured participants:

Anyone that wants to attend should first be sure to have a Twitter account and to follow atlantaweb. We’ll use that list as a roll call for the meeting and we’ll announce our special guest on the atlantaweb Twitter account by 6pm Wednsday August 20th.

For more details and to RSVP see go here.

 

Is .COM Still Required for Mainstream Success?

delicious .US domain is now the Dunce

It’s been almost fifteen years now since the web first hit it’s tipping point and transitioned from an academic’s playground and a mere curiosity for the average person to the decidedly mainstream global change agent that now drives trillions of dollars in global value creation annually. During that time we’ve gone from asking "What’s this ‘World Wide Web’ thingy the geeks keep talking about?" to rapidly seeing Web-based services dominant the activities of practically every business person alive. In the days of the first Internet "gold rush" a.k.a. the "dotcom bubble" it seems that everyone and their brother grabbed a .COM domain, or ten, and set out to strike it rich.  Back then you really didn’t even consider getting anything besides a .COM for your website business but that was okay because many good brand names could be created from still available .COM domains.

Fast forward a decade and great .COM domain names have became scare and even good .COM domain names are hard to come by especially with all the domain squatters.  In addition a lot more top-level domains have opened up, and many small countries such as Tuvalu (.tv) have decided to cash in on their (un)natural resources. And just as fashions change, some creative types who needed a good domain name chose to forgo the .COM status quo and the "www." sub-domain convention and instead compose domain names from words by ignoring the domain level separators (i.e the periods ".") From this trend popular websites with domain names like http://del.icio.us (aka "delicious") were born.

If you are not familiar with delicious it is essentially a website to store your web bookmarks, but storing them "in the cloud" as opposed to in your browser. And one of it’s best innovations, since mimicked by thousands of other sites, it the ability to allow users to categorize with freeform "tags" and later recall their bookmarks by the tags they assigned. These tags are just words, any words your chose, such as "marketing", "video", "php", "bestpractices" or even "shoes."  What words you use to tag with is totally up to you. 

I’ve been using delicious for several years now and at this point not a day goes while surfing the web that I don’t tag at least one website for future reference.  You can even use it to create lists of sites groups by a tag and then send those links to others so they too can see your list of links. But I digress; there is a lot more to delicious but the subject of this post is the .COM domain so we’ll my detailed description of del.icio.us for another day.

As delicious got more popular with the many influencers on the web, Yahoo stepped in and bought them.  Since then delicious has languished for years, still there but never updated. Probably the best thing that has happened to delicious during that period was Firefox built delicious tagging into their browser as their favorites list — if you choose to let Firefox use delicious for you — and the fact that since it hasn’t changed it’s been a pretty stable target for people who wanted to use to delicious API to create add-on functionality and integrations.

delicious .COM domain is the new King

So, after years of languishing it turns out Yahoo has been paying attention to delicious behind the scenes, and behold; there is a new delicious! [1]  What’s more, Yahoo has redirected all attempts to access delicious at http://del.icio.us to instead find delicious at http://delicious.com/ and thus, in one fell swoop, have extinguished the quirky domain name that was in part why the web’s tastemakers first took note of delicious. The powers that be at Yahoo probably choose to do this because of usability data I expect they’ve collected that probably told them that the "in crowd" got the funny spelling but that the vast majority of users were simply confused.

Which brings me to the crux of my post where I posit the following:

Are .COM domains still required for commercial success in a mainstream website? Or or those rushing to get domain names with all the new top level domains simply exercising futility?  Were all these idiosyncratic domain names merely a fad and now we’re back to business with .COM, or did Yahoo jumped the shark on this one?[2] 

So what do you think? 

Footnotes

  1. Frankly I’ve long thought delicious to be Yahoo’s secret weapon with Yahoo just not yet realizing it. And it’s no wonder they haven’t realized it as anyone who’s paid attention what the critics have been saying about Yahoo being spread too thin. But that is all an entirely different topic and way beyond the scope of this post, yet again.
  2. While part of me feels the pang of loss from this change and what it may imply about the use of other non-.COM domains, I find that I quite prefer using http://delicious.com when compared to the original http://del.icio.us domain.

 

Anti-Follow Spam for Twitter

No Twitter Spam!
Wall of Spam courtesy of freezelight and enabled by Creative Commons

Damon Clinkscales blogged about Twitter Spam last month where he advocated proactively cleansing one follower’s list of "follow spammers" to help reduce the load on Twitter, improve Twitter’s reliability, and increase the value of the Twitter community in general.

I agree!

Still, I think Twitter could take a proactive step reasonably easy that would make it so we don’t have to. I think Twitter could reduce most of the type of Twitter follower spam I got today by applying two simple criteria (And I think Damon also got that same spam today. BTW, nice blog theme Damon! ;-)

I think a strong indication of Twitter follower spam is simply:

  1. Their following/follower ratio (or their ing/er ratio for short), and
  2. Their follow rate (i.e. how quickly they follow someone after that last time they followed someone.) 

This spammer I got today followed me with 4 different Twitter accounts within a few minutes and each account had around 2000 followings and just over 10 followers making their ing/er ratio about 20-to-1 and I’ll bet their followers were all auto-followed. It’s also clear from the fast & furious tweets that I was not their only mark.

I think it would be reasonable for Twitter to auto-block anyone with a ratio of greater than 15-to-1 ing/er ratio. Twitter could even remove the auto-followers from the calculation; those that follow within around 90 seconds of being followed wouldn’t count as a follower. Doing this Twitter would still give someone the ability to follow 15 people for every one that follows them, and heck they could give them their first 150 people[1] for "free" (i.e. not counting against the limit.) If someone really wants to follow 15,000 people they need to be interesting enough to have at least 1000 people follow them. Shouldn’t be that hard…

Also, Twitter could limit followings per day to, say, 75.  That should be enough for anyone, even the most hard-core twitter newbie (150 "free" + 75 more), and it’s not unreasonable to require a newbie to wait a few days to follow lots and lots of people. 

If I were in charge of setting these limits, I’d set the ing/er ratio to 5-to-1, give them only 25 "free" and then limit to 25 followings per 24 hour period, but I shot high because I was trying to be "reasonable." Of course, Twitter could allow for special cases by allowing people to request to have those limits manually raised if they provide a good justification for it.

What do you think?  Would this work to reduce most Twitter follow spam?  I think so.

Footnotes

RESTful Web Services in a WordPress Plugin?

UPDATE (2011-04-15):

Since I wrote this post I’ve learned a tremendous amount about WordPress plugin development; so much so that I can’t overstate how much more I know today than when I wrote this post years ago.  So, while the following post might be a novelty to read, I highly recommend that you don’t use this approach.

The only way that a RESTful approach to web services in WordPress would make sense to me is if a team of rockstar WordPress plugin developers were to create a fully fleshed-out extension to WordPress that offered a complete RESTful web service implementation including one that addressed all edge cases and security concerns; only then I would consider not defaulting to the non-RESTful approach WordPress uses for AJAX.  

In summary I recommend not trying to swim upstream today and instead use the approach provided by WordPress. Who knows, maybe in the future there will be a viable method of doing RESTful web services in WordPress.


So I’ve got a project where I need to have a Flash component built in Flex to call to a WordPress blog and get information about it’s latest post. Should be no problem right?  The easy way to do this would be to just write a "rest.php" file and brute-force all the setup that WordPress does but I thought it would be so much more valuable to implement this as a plug-in.

I figured that I’d just quickly learn how to build a WordPress plug-in and create one for exposing RESTful web services; after all with a year of programming Drupal modules WordPress’ plug-in API can’t be that hard, right?  Well turns out it wasn’t that easy and I think I have run into a design limitation with WordPress and I’m beginning to wish I’d just taken the brute-force approach and said to hell with writing a plug-in.

Although I am not 100% certain, and I hope someone can point out that I’m just doing something wrong, it seems sadly like I’m pushing the edges of the WordPress API and exposing where it’s design falls short. By the way, I’m working with WordPress v2.5 because why upgrade mid-project when god knows if WordPress will release another in the remaining days before this project is done and I’ll just have to do again before deployment?

Here’s the details.  I started writing a plug-in called "RESTful Services" with a goal of implementing URLs that behave in the following fashion; {format} could potentially be html, xhtml, json, xml, rss, atom, etc.:

http://example.com/services.{format}
Provide a list of RESTful services in specified {format}, defaults to html
http://example.com/services/{service}.{format}/{data}?{params
Provide a RESTful service in specified {format}, defaults to html, with optional provided data and parameters.

But before I got all those options working I just wanted to service a page from my RESTful Services plugin where Content-Type: text/plain. I found this page that professes to explain how to hook into the URL routing and after a few fits and starts I can came up with the following code for my plugin that would indeed response to my http://example.com/services URL:

wp-content/plugins/restful-services/restful-web-services.php:

1
2
3
4
5
6
7
8
9
10
11
12
13
add_action('init', 'restful_services_flush_rewrite_rules');
function restful_services_flush_rewrite_rules() {    
  global $wp_rewrite;
  $wp_rewrite->flush_rules();
}       
 
add_filter('generate_rewrite_rules', 'restful_services_add_rewrite_rules');
function restful_services_add_rewrite_rules( $wp_rewrite ) {      
  $new_rules = array(        
    'services' => 'wp-content/plugins/restful-services/rest.php',     
  );
  $wp_rewrite->rules = $new_rules + $wp_rewrite->rules;
}

The problem with the above was that it wouldn’t call "wp-content/plugins/restful-services/rest.php"; it would simply continued to call "index.php" and display the home page!  After literally hours and hours of debugging with my trusty PhpEd IDE & debugger I was able to find that the code on lines 737 & 738 of "wp-includes/query.php" told WordPress that my service was the home page! It is almost seems like the "generate_rewrite_rules" was implemented as "a good idea" yet no testing has ever been done on it because for the best I can tell it doesn’t work. (Note I’ve reformatted the code to multiple lines so that it is easier to read and does not extend past the right margin of my blog):

wp-includes/query.php:

if ( !(  $this->is_singular                           
      || $this->is_archive                           
      || $this->is_search                           
      || $this->is_feed                           
      || $this->is_trackback                           
      || $this->is_404                           
      || $this->is_admin                           
      || $this->is_comments_popup ) )   
       $this->is_home = true;

I could possibly hack it to get past this by setting one of those to "true", but none of them are really appropriate; there is nothing there quite like an "is_service" instance variable. Setting something like "this->is_singular" or "this->is_feed" might work but it could manifest incompatibility problems with other plugins or future versions of WordPress. Frankly it was rather disappointing to discover this because it tells me that WordPress has hard-coded all the potential scenarios and doesn’t really have a way around it. Seems to me there should really be a hook here and the type of pages should be allowed to be expanded by plugins rather than be hardcoded as only one of ’singular’, ‘archive’, ’search’, ‘feed’, … and ‘home.’

Anyway, where this manifests itself is "wp-includes/template-loader.php" file which I have included in it’s entirety below.  It is on lines 24 and 25 where the template loader loaded the home page because it’s not possible to specify otherwise:

wp-includes/template-loader.php:

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
/**
 * Loads the correct template based on the visitor's url
 * @package WordPress
 */
if ( defined('WP_USE_THEMES') && constant('WP_USE_THEMES') ) {
  do_action('template_redirect');
  $is_home = is_home() ;
  if ( is_robots() ) {
    do_action('do_robots');
    return;
  } else if ( is_feed() ) {
    do_feed();
    return;
  } else if ( is_trackback() ) {
    include(ABSPATH . 'wp-trackback.php');
    return;
  } else if ( is_404() && $template = get_404_template() ) {
    include($template);
    return;
  } else if ( is_search() && $template = get_search_template() ) {
    include($template);
    return;
  } else if ( is_home() && $template = get_home_template() ) {
    include($template);
    return;
  } else if ( is_attachment() && $template = get_attachment_template() ) {
    remove_filter('the_content', 'prepend_attachment');
    include($template);
    return;
  } else if ( is_single() && $template = get_single_template() ) {
    include($template);
    return;
  } else if ( is_page() && $template = get_page_template() ) {
    include($template);
    return;
  } else if ( is_category() && $template = get_category_template()) {
    include($template);
    return;
  } else if ( is_tag() && $template = get_tag_template()) {
    include($template);
    return;
  } else if ( is_tax() && $template = get_taxonomy_template()) {
    include($template);
    return;
  } else if ( is_author() && $template = get_author_template() ) {
    include($template);
    return;
  } else if ( is_date() && $template = get_date_template() ) {
    include($template);
    return;
  } else if ( is_archive() && $template = get_archive_template() ) {
    include($template);
    return;
  } else if ( is_comments_popup() && $template = get_comments_popup_template() ) {
    include($template);
    return;
  } else if ( is_paged() && $template = get_paged_template() ) {
    include($template);
    return;
  } else if ( file_exists(TEMPLATEPATH . "/index.php") ) {
    include(TEMPLATEPATH . "/index.php");
    return;
  }
} else {
  // Process feeds and trackbacks even if not using themes.
  if ( is_robots() ) {
    do_action('do_robots');
    return;
  } else if ( is_feed() ) {
    do_feed();
    return;
  } else if ( is_trackback() ) {
    include(ABSPATH . 'wp-trackback.php');
    return;
  }
}

Still another problem in this puzzle is the $wp->send_headers() method shown being called here on line 293 of "wp-includes/classes.php":

wp-includes/classes.php:

290
291
292
293
294
295
296
297
298
function main($query_args = '') {   
  $this->init();   
  $this->parse_request($query_args);   
  $this->send_headers(); 
  $this->query_posts();  
  $this->handle_404();   
  $this->register_globals();   
  do_action_ref_array('wp', array(&$this));   
}

The problem with the $wp->send_headers(), also from "wp-includes/classes.php", is that it seems to have the option of either serving an HTML content type on line 183 and 185, or a content type based on a feed (the content types for the feeds are set in their respective "wp-includes/feed-*.php" files) but no custom content types as far as I can determine as there seems to be no way to override calling this function or the logic path contained within:

wp-includes/classes.php:

175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
function send_headers() {
  @header('X-Pingback: '. get_bloginfo('pingback_url'));
  if ( is_user_logged_in() )
    nocache_headers();
  if ( !empty($this->query_vars['error']) && '404' == $this->query_vars['error'] ) {
    status_header( 404 );
    if ( !is_user_logged_in() )
      nocache_headers();
    @header('Content-Type: ' . get_option('html_type') . '; charset=' . get_option('blog_charset'));
  } else if ( empty($this->query_vars['feed']) ) {
    @header('Content-Type: ' . get_option('html_type') . '; charset=' . get_option('blog_charset'));
  } else {
    // We're showing a feed, so WP is indeed the only thing that last changed
    if ( !empty($this->query_vars['withcomments'])
      || ( empty($this->query_vars['withoutcomments'])
        && ( !empty($this->query_vars['p'])
          || !empty($this->query_vars['name'])
          || !empty($this->query_vars['page_id'])
          || !empty($this->query_vars['pagename'])
          || !empty($this->query_vars['attachment'])
          || !empty($this->query_vars['attachment_id'])
        )
      )
    )
      $wp_last_modified = mysql2date('D, d M Y H:i:s', get_lastcommentmodified('GMT'), 0).' GMT';
    else
      $wp_last_modified = mysql2date('D, d M Y H:i:s', get_lastpostmodified('GMT'), 0).' GMT';
    $wp_etag = '"' . md5($wp_last_modified) . '"';
    @header("Last-Modified: $wp_last_modified");
    @header("ETag: $wp_etag");
 
    // Support for Conditional GET
    if (isset($_SERVER['HTTP_IF_NONE_MATCH']))
      $client_etag = stripslashes(stripslashes($_SERVER['HTTP_IF_NONE_MATCH']));
    else $client_etag = false;
 
    $client_last_modified = empty($_SERVER['HTTP_IF_MODIFIED_SINCE']) ? '' : trim($_SERVER['HTTP_IF_MODIFIED_SINCE']);
    // If string is empty, return 0. If not, attempt to parse into a timestamp
    $client_modified_timestamp = $client_last_modified ? strtotime($client_last_modified) : 0;
 
    // Make a timestamp for our most recent modification...
    $wp_modified_timestamp = strtotime($wp_last_modified);
 
    if ( ($client_last_modified && $client_etag) ?
         (($client_modified_timestamp >= $wp_modified_timestamp) && ($client_etag == $wp_etag)) :
         (($client_modified_timestamp >= $wp_modified_timestamp) || ($client_etag == $wp_etag)) ) {
      status_header( 304 );
      exit;
    }
  }
 
  do_action_ref_array('send_headers', array(&$this));
}

Still, I was able to come up with a solution although it is so very hackish.  My solution was to hook the "template_redirect" action on line 7 of "wp-includes/template-loader.php" (see code from that file above.) Though it seems to works thus far, my solution just feels wrong for the following reasons:

  1. It ignores the fact that WordPress continues to think that my web service URL is the home page,
  2. It first lets "$wp->send_headers()" set the content type before it overrides it,
  3. It uses an "exit" rather than a return to keep WordPress from serving up the home page template, and
  4. It doesn’t use the routing mechanism apparent built into WordPress (see "null" on line 30 of "wp-content/plugins/restful-services/restful-web-services.php" below, I assume it should have been the URL of the .php file I plan to execute but WordPress doesn’t see to use what I put there.)

The function "restful_web_services_exec_service()" is what is called to execute the appropriate web service:

wp-content/plugins/restful-services/restful-web-services.php:

2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
/*
Plugin Name: RESTful Web Services
Plugin URI: http://mikeschinkel.com/wordpress/restful-web-services/
Description: This plugin enables REST-based API web services for WordPress.
Author: Mike Schinkel
Version: 0.1
Author URI: http://mikeschinkel.com/
*/
 
define('RESTFUL_WEB_SERVICES_DIR', dirname(__FILE__));
define('RESTFUL_WEB_SERVICES_URL_PATTERN','services(/?.*)?');
$abspath = trim(str_replace('\\','/',ABSPATH),'/');
$rest_services_dir = str_replace('\\','/',RESTFUL_WEB_SERVICES_DIR);
$rest_services_path = trim(str_replace($abspath,'',$rest_services_dir),'/');
define('RESTFUL_WEB_SERVICES_PATH', $rest_services_path);
 
// NOTE, See: http://codex.wordpress.org/Custom_Queries#Permalinks_for_Custom_Archives
add_action('init', 'restful_web_services_flush_rewrite_rules');
add_filter('generate_rewrite_rules', 'restful_web_services_add_rewrite_rules');
add_action('template_redirect', 'restful_web_services_exec_service');
 
add_action('init', 'restful_web_services_flush_rewrite_rules');
function restful_web_services_flush_rewrite_rules() {
 global $wp_rewrite;
 $wp_rewrite->flush_rules();
}
 
add_filter('generate_rewrite_rules', 'restful_web_services_add_rewrite_rules');
function restful_web_services_add_rewrite_rules( $wp_rewrite ) {
  $new_rules = array(
    RESTFUL_WEB_SERVICES_URL_PATTERN => null,
  );
  $wp_rewrite->rules = $new_rules + $wp_rewrite->rules;
}
 
function restful_web_services_exec_service() {
  global $wp;
  if ($wp->matched_rule==RESTFUL_WEB_SERVICES_URL_PATTERN) {
    if ($wp->request == 'services') {
      header('Content-Type: text/plain');
      print 'TODO: Generate a list of RESTful Web Services here for this WordPress Blog.';
    } else {
      list($dummy,$service_name) = explode('/',$wp->request);
      if (file_exists($service_php = (RESTFUL_WEB_SERVICES_DIR . '/services/' . $service_name . '.php'))) {
        include_once $service_php;
      } else {
        header('Content-Type: text/plain');
        status_header(404);
        print '404 - Service not found.';
      }
    }
    exit;
  }
}

You’ll note that my function "restful_web_services_exec_service()" is very bare-bones at the moment serving only a plain text message "TODO:" for the path http://example.com/services, and assuming that any path http://example.com/services/{service} will execute a same-named .php file in the services subdirectory, i.e. for http://example.com/services/latest-post it will look for "wp-content/plugins/restful-web-services/services/lastest-post.php" and then delegate all the work to that .php file.

wp-content/plugins/restful-services/services/latest-vidclip.php:

2
3
4
5
6
7
8
9
10
11
12
13
14
15
/*
Filename: latest-post.php
Service Name: Latest Post
*/
global $wp_query;
$post = $wp_query->post;
$file = get_attached_file($post->ID);
if (empty($file)) {
  list($file)= get_enclosed($post->ID);
}
$charset = get_option('blog_charset');
header('Content-Type: text/xml; charset=' . get_option('blog_charset'), true);
$link = get_permalink($post->ID);
$html = &lt; &lt;<post></post><post id="{$post-&gt;ID}">      <video>$file</video>   <link />$link  </post>  POST; print $html;

Here is an example output returned by calling http://example.com/services/latest-post:

1
2
&lt; ?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt; 
<post id="2">      <video>http://videos example.org/video1.flv</video>   <link />http://example.org/sample-post-1/  </post>

While bare-bones, that will meet my needs for the moment since I really only have the need for one service that responds to the HTTP GET verb. However, I can see where I might end up fleshing this out and create lots of helper functions that would streamline the creation of web services for RESTful access to the entirety of the WordPress database. If I do so I’ll be happy to donate this plugin to the community.  In the mean time if you’d like to use this for your own use feel free but caveat emptor. However, if you’d like to use this code to create a plugin to contribute to the community, please contact me for collaboration.  Of course if you’d like to retain me to fully flesh it out for you, I’m always open to that too! :-)

Finally, if anyone who knows the WordPress API better than I do can tell me where I erred in my analysis, and I really hope I did, please let me know so I can architect this thing better. On the other hand, if I was spot-on in my analysis maybe this will help the WordPress team understand what needs to be done so they can empower WordPress to generate any arbitrary content type without having to resort to hacks or bypassing the WordPress index.php startup code. 

Finding a WiFi Hot Spot in Atlanta

Recently on Twitter I’ve had several people ask where they could find another good place for free WiFi in Atlanta.  Each time I answered so I decided to maintain a page on my site with a list of the Atlanta WiFi Hot Spots I go to frequently or at least know about.

Enjoy.