While evaluating (and now using) SEM platforms, I have been impressed by:

1) how well-suited the tools are for the day-to-day activities of the paid search marketer
2) the dramatic improvements in process and time-savings realized with having reporting and intelligence integrated with the ability to act

We are able to find outliers and immediately cut or increase spend. Tasks that previously used to take several hours can now be completed in minutes.

Demand-side platforms (DSPs) for display advertising are poised to deliver similar operational improvements to display marketing.

Abstracting up one level, I find these tools fascinating in that they’ve been able to productize (and consequently scale) the integration of BI and business processes. What is it about digital marketing that allows this approach to work, as opposed to say a tool built on top of Salesforce? The data sources are fairly homogenous (the three major search engines) and the data itself is even more so (campaign structure, bids, clicks, and CPC’s constitute the majority of the data).

The major difference between users is how they define conversion and revenue; however, even here, every marketer has a revenue number and a fairly standard set of conversion events (registration, revenue event, login, etc). You specify the definition of conversion and then upload one or more related metrics to support analysis and decision-making in the tool. Bidding is driven by revenue and cost and improving CTR and revenue/click allows marketers to focus on improving ad copy and landing pages.

Thus, in this case, limited data and a fairly homogenous cross-vertical definition of success metrics allows a scalable, tight integration of business process and business intelligence.

With regards to the platforms, you do start hitting limits once you start getting very sophisticated e.g. custom bidding algorithms, complex keyword expansion, etc. But even in this area, the vendors are increasingly offering solutions that are more sophisticated than what most in-house digital marketing and analytics operations will develop e.g. attribution models for bidding, automated SQR-driven keyword expansion, etc.

I’m looking forward to watching these products expand over the next year and simultaneously watching how digital marketing processes evolve.

In most cases, the vendors have taken a horizontal platform (whether Quickbooks or Salesforce) and developed a reporting / analytics offering on top of it. The real question for me goes back to some of the posts that I wrote when I started this blog – these plays are interesting, but they don’t get to the heart of the highest value problems:

• understanding a business (i.e. vertical) and offering BI / reporting that factors in specific challenges
• integrating BI as part of the business process of an organization (e.g. within Quickbooks usage or within Salesforce)

Some of the major players (e.g. Siebel Analytics) have offered a “matrix” (i.e. functional area x vertical) suite of applications for several years (e.g. sales for manufacturing or finance for retail). More accurately, at least they offered one several years back; however, it doesn’t seem like these got much traction in their target market as I don’t hear about them any more. I also haven’t seen other enterprise software vendors that sell to large companies universally adopt this packaging for their go-to-market. Presumably, they would have if the market were responding strongly.

- Summing the 5th column of numbers:

cat tmp.txt | awk '{t+=$5}END{print t}'

You may also find the above command useful for processing the output of ls -l

- cut, uniq, sort, grep, and wc are great for filtering and computing aggregates for a column of values e.g. for filtering a list of session ids in the 17th field, looking only for the duplicate ones:

cut -f17 | sort -n | uniq -c | grep -v ' 1 '

- And finally, a random aside that I learned about last week – bash treats the output of the following two commands differently:

echo $output

echo “$output”

If $output has no line breaks, then the output will be identical.  If it does have line breaks, then the former will flatten the multiple lines into a single line, while the latter will print out $output exactly as it was captured.

Added bonus that it easily integrates all the traditional data sources as well e.g. it’s easy to join a MySQL table against a “table” in Hadoop / Hive.  Would be cool if they could automatically discover the schema of your Hive tables and your traditional DB tables. They may already be able to; I didn’t see anything specific on their website indicating that though.

Tools like Datameer are a great addition to any BI practitioner’s toolset.  As datasets get larger, Hadoop allows ready access to large amounts of data at a reasonable price.  Datameer will now allow us to make that data more broadly accessible to a larger group of analysts.

1) a decent quality CLI (command line interface) for Hive.  Editing of a query in Hive is very limited. You can’t use custom keybindings – ideally, you’d want the CLI to get editing mode (emacs or vi) from your inputrc.  History and history search is poor. My workaround to date has been to instead use the shell’s command line and execute each query using the command line

hive -e "... query ..."'

Another benefit of using the Bash shell as your Hive CLI is that you can include variables in your SQL statement e.g. in my script, I can do something like:

TS=$1

hive -e "select * from foo where ts='$TS'"

2) Hive’s inability to map columns correctly.  I am unable to systematically reproduce this behavior, but when you do complex queries with multiple selects, Hive gets confused about the order of columns.  In particular, if the order of the columns in your hive query does not match the order of the columns on disk in the file backing the table, then you get lots of junk back.

3) Lack of documentation and confusing documentation.  The Hive Language Manual looks to be the definitive reference for Hive; however, it is missing quite a bit of documentation and mixes documentation from various versions of Hive, including the most recent, unreleased version, v0.6.0.

Gripes aside, Hadoop / Hive have been a great platform, and we’re continuing to expand our usage of them daily.

In the context of digital marketing, I’ve spent additional time looking into BI and analytics tools, so more on that shortly.

• Financial capital: traditional capital falls into this category e.g. How much money will you make? How much equity will you get? What will the equity be worth?
• Knowledge capital: what you are going to learn / what will you know when you leave this position e.g. People management? Project management? Ruby on Rails? Running a consumer website? Consumer marketing?
• Social capital: who are you going to build relationships as a result of your work e.g. Do you meet other marketing folks? Strong technologists? CIO’s in financial services companies? VC’s who could be potential future investors?
• Brand capital: how does this opportunity fit into your larger narrative and how does it develop your personal brand e.g. I have built deep infrastructure companies all my life and with this opportunity, I did it in the consumer space. I had spent a lot of time selling traditional enterprise software and now moved over to selling SaaS.
• Happiness: No capital suffix attached to this one e.g. Will you be happy when you come home everyday? Will you like the people you work with? Will you find what you do fulfilling? etc.

Rating my various opportunities against these categories and how much I would grow in each has been helpful as I think about what’s next for me.

• build a great product that people want
• tell the world about it

If you do the former and not the latter, you have the “tree falling in a forest” problem – no one knows you exist, so who really cares what you have built. If you do the latter without the former, people will come and will leave just
as quickly. For those who buy traffic, the former is what will ultimately get the company out of the business of buying traffic.

I didn’t mention making money in the above. My general take is that if you build something people want and you have a bunch of people using it, you’ll find a way to make money from it.