I was interviewed (in my home office) and featured in a news segment on KTVU-2 news last night; you can view the segment here (but you’ll need to wait through a 15-second commercial before seeing the 2-minute, 15 second segment).

I was also interviewed today (June 30) by KPIX-5 for broadcast on tonight’s news.

Google always strives to deliver relevant and useful links in its search results.  All search engines must deal with lots of unethical people who seek an unfair advantage. One specific strategy that has exploded in recent years is called “content farming,” which means creating web pages that include text content which superficially appears relevant and useful for specific search terms — but which isn’t useful to anyone.

We’ve all had the experience of searching for a specific phrase on Google, and clicking on a link that appears to provide the exact information we wanted — only to end up at a web page that doesn’t provide any information, but instead urges us to click on paid links and advertisements.  Many “content farmers” simply merge the relevant keywords together with snippets of text copied from other web sites which rank well for that search phrase.

Several years ago, Google’s algorithms added penalties for “duplicate content,” so that “copied” web sites would be downgraded or excluded from search results. Many “content farmers” responded by using a variety of automated “translation” strategies so that the text wasn’t identical to the source which was being copied (most simply, by substituting synonyms for randomly-selected words in an article, or by automatically translating words, phrases, or entire articles into another language and then translating them back).  When Google tweaked its algorithms to counter this strategy, spammers began hiring writers in third-world countries to write “original articles” that would appear relevant to Google’s algorithms (these articles are often incomprehensible to humans, and never useful to anyone).

It appears that for the Panda (Farmer) update, Google created a list of factors or elements which were commonly found on unsatisfactory pages in search results (and not on useful pages), but which weren’t already recognized by its algorithms.  These are measurable, quantitative factors that correspond to users’ perceptions of quality.  Google also manually downgraded some specific “content farmers” whose web sites provided little or no value for consumers.  After the upgrade, Google also made adjustments to restore the rankings of a few publishers who had been wrongly downgraded by the new algorithms).

The new algorithm doesn’t seem to have affected my sites; traffic from Google to my newest site has continued to grow by about 30% per week.  (Any increased traffic after the upgrade wouldn’t mean that Google “likes” my web pages more than last week.  In amusement-park terms, nobody gets to skip ahead in the line, but instead some naughty people were removed from the line and ejected from the park, though we expect that they’ll soon sneak back in.)

Ironically, one “side effect” from Google’s update (for me) has been an increase in “blog comment spam” (spammers post fake comments with links to unrelated web sites).  My blog sites probably rank better, this week, for the search phrases that spammers use to identify “blogs which accept comments.”  (These spam comments are not visible to my site’s visitors.)

Google’s official blog post about the “Farmer”/”Panda” updates:

• Google’s Blog
• Matt Cutts’ blog

Other discussion threads and articles about the Google “Panda” algorithm update:

• ABestWeb-1 and ABestWeb-2
• WebProNews
• TechMeme
• SearchEngineLand-1 and SearchEngineLand-2
• SearchEngineJournal
• Wired

Some lists of impacted sites : Sistrix and SearchEngineLand and Quora

Added March 6: Two more articles about the Farmer Update:

• SearchMetrics: “[B]ounce rates, visit duration, and social reach [are obviously factors]…. [P]ages that are genuinely visually attractive to a user … will be spared by the Farmer Update. Meaning that ranking is going to come down to how a user values a page, as opposed to just what content is on it.”
• ReveNews: Noting the impact on User-Generated Content (UGC) sites: “In the more extreme cases, [the downgraded sites] actually incentivized users to generate this content. And in doing so, they’ve essentially eroded the value that content is supposed have because it’s user-generated.”

Some background: From mid-2002 through late 2004, I explored teaching as a career, but eventually decided that it just wasn’t a good fit for me. As a new teacher, I was  starved for resources, and couldn’t easily identify and obtain relevant and useful lesson plans or teaching guides that could reduce my workload.  (I ended up spending much of my time “re-inventing the wheel,” duplicating the work of thousands of other teachers.)

After I quit teaching in 2004, I created a “literature lesson plan” directory web site, as a part-time effort; after several hundred hours spent designing and implementing the site, I spent five to ten hours per month updating the site.  In July 2007, I sold that web site (and agreed not to compete for two years).  After my interest was revived in October, I learned that the earlier web site has stagnated; that discovery led me to do more research.

By mid-November, I was confident that I could create a new lesson-plan directory, this time for a full range of K-12 teaching and learning materials, in all content areas (integrated with state-specific content standards).

I spent some time designing an ontology and a preliminary database implementation, and soon I began work in earnest.  On January 1, 2011, I “launched” LessonIndex.com — which I believe is already the most useful and relevant directory of “Literature Lesson Plans” for K-12 teachers.

After gathering data from many sources and launching this version of the site as a “proof of concept,” I’m turning back to refine the ontology, database structure, and import scripts so I can expand the site to include resources for all K-12 content areas.

Some relevant links for my new site:

• Short Index page (with links to directory pages for about 200 literary works);
• Big Index page (with links to directory pages for each of 2,034 literary works);
• blog.LessonIndex.com, which provides update information, suggestions for publishers and merchants, and a list of the “Most Frequently-Taught Works” (works with the most resource links).

While my attention was focused on this new project, of course, I stopped posting to this blog.

A privacy policy is one of the most basic requirements for any professional web site. If a web site doesn’t have a conspicuous link to a privacy policy, it simply isn’t very professional, and is less trustworthy.

Consumers constantly evaluate the trustworthiness of web sites they visit, and they recognize that most professional, credible web sites include certain important links, including “privacy policy,” “about us,” and “contact us.” The presence of those links only provides a very subtle reassurance, but the absence of any of those links is a huge “red flag.”

Note that the actual content of the privacy policy isn’t nearly as significant as its presence. A web site could conceivably have a privacy policy that says, “we will track everything you do, and gather as much information as possible, and then sell your information to the worst people on the planet to use for any nefarious purpose.” While that would be quite unprofessional, very few consumers ever click to view a site’s privacy policy.

In March 2002, I posted information about the impact of privacy policies on visitors to an e-commerce merchant, in the I-Privacy Digest email discussion list. The most significant observation, drawn from a brief “slice” of one merchant’s log-file data:

“Of the 359 customers who actually placed orders, only 10 had reviewed the privacy/security policy during the same visit.” (Who Reads Privacy Policies?, in I-Privacy Digest #3, March 28, 2002.)

As an iPhone user, I sometimes find it annoying to navigate through web sites (including my own blog) which were designed with a standard desktop screen in mind.  On the other hand, I also sometimes find it annoying to experience a “crippled” version of a web site just because I’m using an iPhone.

This plug-in also gives users the option to disable the “mobile view” (just scroll to the footer and click on the toggle switch to disable (or re-enable) the mobile view).

There are many “settings options” for this plug-in, and I’m still experimenting with some. I’m currently using the free version, but will likely upgrade to the Pro version ($29) once I figure out how to use some of the features.

Let me know what you think.

Since this blog draws very little traffic, I doubt I’ll see any meaningful revenue, but I decided it would be interesting to see what ads are displayed by Google on my pages.

Please let me know if you see any objectionable ads (either post a comment, or email to Mark Welch at Mark Welch dot com).

The idea of adding a Google AdSense unit came to me in a very roundabout way: I was restoring some very old articles (see the “Web Advertising in 1997″ category), and I was amused to read the 1999 web page announcing the planned sale of the Adbility.com web site.  Reading that page, I noticed that I actually owned and sold the domain AdCents.com in 1999 (before Google created the AdWords or AdSense programs).  Somehow, thinking about that made me wonder which ads Google AdSense would display on my blog pages.

When I’d used AdSense in the past, I often found that it took a while to start inserting relevant ads onto content pages — and sometimes it would display ridiculous ad choices, even after many adviews for a particular content page. However, so far this afternoon, I am seeing highly relevant ads displayed adjacent to category and article pages.

Website tools

The biggest task this week was migrating my “free affiliate-program advice for merchants” from static pages over to the blog; if you’ve bookmarked or posted links to any of the old pages, they should redirect to the same exact article on the blog site.

Yes, I know my content will lose PageRank (and will likely impact Google’s perception of the “age” of the content, even though I’ve tagged each blog entry/article with the date it was originally written). And it is definitely a PITA to manually cut and paste content and then set up a redirect (for every static page) so they point to the correct blog entry.

But once I’m done, I’ll finally be able to make site-wide changes that actually affect my entire web site (without needing to manually edit a bunch of files that are five, ten, or even fifteen years old, dating back through and beyond several earlier content-management strategies I used). And I’m also inviting comments on the old articles (for example, I hope some folks might inform me of the new location of the many links within these articles which come up ’404 not found’ or worse).

Every time I do a “site-wide update,” I’m surprised to see how many hundreds of articles I’ve posted on my site over the years. Many of the older articles are deleted with redirects to my home page (most of these are articles about companies that are long defunct). (I’m actually considering “restoring” a few of those older articles, from backup CDs.)

I’m also surprised to see that my earlier migration of my “perspectives” content (the “opinion & commentary” essays which I first moved over to the blog) wasn’t quite complete, as I hadn’t replaced some of the static pages with redirects to the corresponding blog pages (I assume Google’s algorithms probably pounded those pages with “duplicate content” penalties).

Lots and lots of work, most of it very mundane, but it’s also a trip down memory lane. And ultimately, this work should make my life much simpler in future years, if I ever seek to profit from this content. It’s ironic that after 15 years mostly making a living from internet advertising, affiliate programs, and Google PPC Search, there are currently no ads anywhere on my site!

The migration is definitely not complete yet, and I’m still debating how to deal with some content which doesn’t seem appropriate for the blog site.

While I’d love to join the chorus criticizing both Apple and law enforcement, I simply don’t have enough information to make a reasonable decision about the conduct of various parties. Some key questions: did the “finder” of the phone make a reasonable effort to return it; was it legal to sell or buy the “found” item; was it legal to open and dissect the “found” phone; did Apple act reasonably to find and reclaim the phone; did Apple accurately describe the facts to law enforcement; did law enforcement properly investigate; did law enforcement properly describe the situation when seeking the search warrant; did the judge properly apply the law when issuing the search warrant; did law enforcement act reasonably in conducting the search after recognizing that the “shield law” might be applicable; did law enforcement act properly under the “shield law” both before and after the blogger and Gizmodo expressly asserted its applicability; and did law enforcement allow Apple agents to participate in the residential search?

One key issue, of course, is whether or not a “blogger” is protected under California’s laws protecting “journalists.”  While many people have strong opinions, the issue is one of interpreting a statute. (There’s a helpful column by Gillian Reagan at BusinessInsider.com discussing the California “reporter’s shield law” and how it includes bloggers.)  In short, there is no doubt that the “shield law” applies in this case — but it’s not an absolute protection.

The shield law protects journalists against being compelled to identify their sources or to reveal “unpublished information obtained or prepared in the course of newsgathering activities.” The “shield law” is actually part of the California Constitution (Article I, section 2, subdivision (b)), and in a 2006  decision by the California Court of Appeals (also involving an alleged theft of confidential information from Apple) the court wrote:

“We decline the implicit invitation to embroil ourselves in questions of what constitutes ‘legitimate journalis[m].’ The shield law is intended to protect the gathering and dissemination of news, and that is what petitioners did here. We can think of no workable test or principle that would distinguish ‘legitimate’ from ‘illegitimate’ news.”  O’Grady v. Superior Court (2006) (emphasis added).

The O’Grady case didn’t reach the California Supreme Court, and the ruling doesn’t hold that all bloggers are protected by the shield law. But the facts in O’Grady were remarkably similar to the current Gizmodo situation: both involved disclosure of Apple’s “confidential” materials.

After O’Grady, Apple knew it couldn’t compel disclosure of the iPhone-prototype “source,” and so it chose instead to report the phone as “stolen property” and encourage a law-enforcement task force to instead search the blogger’s home to try to identify the source, while also “fishing for” any other confidential Apple property that might be there.

What stinks here is that Apple knew that it was actively seeking to circumvent the protections of the “shield law.”  By conducting the search without warning while the blogger was not home, law enforcement sought to capture the critical information (the identity of the source) before the blogger could assert the privilege.  As hoped, the search disclosed the identity of the source.

It also seems quite likely that Apple already knew the identity of the source (who found the phone), if he actually did call Apple to report finding it.  If so, then did Apple tell law enforcement?

One key issue is whether the purchase of the “lost” iPhone prototype was legal; was this “stolen property”?  (If it was “stolen property,” then an effort to recover the stolen property, or to identify the thief, might reasonably appear to law enforcement to justify a search warrant, and law enforcement might also reasonably worry that if the search were delayed, the blogger might destroy evidence.  Note that days before the search, the iPhone prototype had already been returned to Apple.)

Gizmodo reported that the person who found the phone:

“… called a lot of Apple numbers and tried to find someone who was at least willing to transfer his call to the right person, but no luck. No one took him seriously and all he got for his troubles was a ticket number.”

Although the phone worked when found, it was completely disabled the next morning; apparently, Apple remotely “wiped” the phone to prevent its prototype software from being disclosed.

At that time, it seemed a distinct possibility (to both the “finder” and to Gizmodo) that the phone might be a hoax, intended to be “found” and reported about.  But the Gizmodo blogger paid $5,000 for the iPhone prototype; it’s hard to imagine anyone paying that much without the belief that it was “more likely than not” that it was genuine, and of course it was the hope that it was a genuine phone that led the blogger to pay for the phone. Once Gizmodo recognized that the phone was genuine, it knew it did not legally own the phone (and Gizmodo asserts that it always intended to return the iPhone prototype to Apple, if Apple would accept it).

Regardless of the “genuineness” of the phone, it clearly wasn’t the legal property of the person who found it.  (When I was a young teen, I found a nice watch in the snow one day, and after failing to find the owner at the scene, I dutifully turned it over the local police department; after the rightful owner failed to claim it within 90 days, the police gave me the watch to keep as my own.)

I started this column by writing that “I simply don’t have enough information to make a reasonable decision about the conduct of various parties.”  The situation certainly “stinks” — it appears highly likely that Apple deliberately sought to circumvent the protection of the “shield law” which is part of the California constitution, and it appears highly likely that law enforcement exercised very poor judgment in obtaining and executing the search warrant.  But all the facts aren’t in yet.

I would prefer to leave the blog “open” so that anyone can post a comment without “registering” here, with the comments hidden until I approve them, but unfortunately there is so much “comment spam” that this may become unworkable.

If you don’t know, “comment spam” refers to posted comments which are intended solely to publicize another web site. They come in two flavors: one is relatively subtle, including a vague comment like “I agree with you entirely, thanks for sharing this!” or “Thanks for writing this, it helped me with my homework assignment,” together with a URL linking to a spam site (mostly “phishing” sites).  The other “flavor” is a lengthy string of linked keywords, often arranged to appear to be sentences, but always total gibberish. These generally include at least a dozen links to site claiming to offer Viagra and such.

I have three options:

1. Leave things as they are, and just review the comments once a week or so, approving any that are legitimate and deleting the rest.
2. Limit comments only to people who have “registered” here (making it more cumbersome and time-consuming for “first-timers” to post a comment).
3. Disable comments completely.

I’d appreciate any feedback or suggestions, either via a comment below or via email.

The first is an editorial in the New York Times criticizing Supreme Court “Justices” Scalia and Thomas for their dissenting opinion in the case In Re Troy Anthony Davis, arguing that not only is it permissible, but legally required, that we execute an innocent person. (There is also a NY Times news article about the case.)  These two “justices” argued, in part, that a federal law should preclude an innocent death-row inmate from being allowed to present proof of innocence — and that it is constitutionally permissible to execute an innocent person.

The second issue was also widely reported (in many media outlets, including CNN, NPR, CNET, and the New York Post): New York Judge Joan Madden ordered Google to reveal the identity of an anonymous blogger who offended a model by describing her as “whoring,” “lying,” and “skanky.” (You might also wish to read a vivid account of the March court hearing here.)

This is a fascinating case because it illustrates some mistakes that lawyers and judges make.

• First, of course, model Liskula Cohen made a huge mistake when she hired Steven Wagner as her attorney. It’s hard to imagine any competent attorney advising a client to file suit against an anonymous blogger whose insults were visible to only a handful of people on a single day, knowing that the litigation would result in vast re-publication of the insults, and guarantee that news headlines would (quite properly) refer to Ms. Cohen as the “Skank Model.”  This case should never have been filed; the only “justification” for litigating this is to generate publicity for Ms. Cohen’s lagging modeling career (which would be an improper abuse of the courts).
• Second, the judge made a mistake (perhaps acting in a “legally correct, but unjust” way) by concluding that obviously-absurd comments about the sexual experience of a 37-year-old fashion model could potentially create a valid claim for defamation.  I can’t find a complete copy of the offensive post, but news accounts report that the blog described Ms. Cohen as the “Skankiest in NYC” and as a “psychotic, lying, whoring … skank.”  What I find absurd is that Ms. Cohen (or anyone else) could believe that the blogger’s statements were “factual” in nature. They are insults, and nothing more; nobody would believe that the anonymous blogger actually knew anything about Ms. Cohen’s sexual history.  Even if Ms. Cohen asserted that she was a virgin desparately concerned about her moral reputation, there would be no defamation case here; since she has now publicly described herself as a “serial monogamist,” it’s even harder to understand her objection to the hyperbolae in the anonymous blog posts.

If I focused exclusively on the legal issue here, I might conclude that Judge Madden’s ruling was correct, but Judge Madden clearly understood that this case raised serious First Amendment issues, and that anonymous speech is actually constitutionally protected.  Accordingly, I believe that her ruling was not just “wrong,” but “unjust and immoral.” I hope that the anonymous blogger finds support to appeal this incorrect ruling.