Divide and conquer?

Currently search engines are centralised in a fairly small number of very large data centres.  This means that the a search engine’s indexing – which records the location and provides a relative importance to information on the web (and thus also to your search string) – is replicated within these data centres across several locations.  This also means that the results you get from a search in the UK won’t differ so very much from the results received by someone carrying out the same search in another country.

Replication is inefficient

Researchers at Yahoo have proposed that web searching could become much faster and more efficient if the search index and additional data were distributed around the world across a larger number of smaller data centres.  The authors suggest that overall search engine operating costs could be reduced by up to 15% without compromising the quality of the search results and should also speed up the search.

However, it would also mean that searches would become more localised so that the same search carried out in  different countries would yield a less duplicated set of results.  This is because most of your results would be coming from your friendly neighbourhood data centre.  To resolve this, the authors say that the data centres must be able to communicate with each other.  It is more complicated than that…of course.  If your search needs to go to more than one data centre, query processing will suffer from “latency”, i.e. a slight delay.

“Distributed architecture” for search engines has met with resistance in the past because people, on the whole, don’t like inconsistency and do like having just one place from which to search all the info. on the web.  Not that that happens in reality course, it’s just an illusion (ref. deep web/invisible web).

So that’s all very well, but the researchers are concerned about the future health of the mammoth data centres.  It looks as if data centres are moving towards a bursting-at-the-seams scenario.

The researchers presented their paper at the The 18th ACM Conference on Information and Knowledge Management where, I gather, they were also awarded “best paper”.  Well done to them, after all, this approach won’t be implemented tomorrow, it’s far from simple, they’ve done a lot of mathematics and acknowledge that further work is needed, but as we’ll all notice if a data centre suddenly goes “pop!”, it’s good to know that the experts are working on some alternative options.

Fierce news is free news

It’s all down to Churchill evidently – the origin of the “fierce” name, that is.

Fierce markets has been providing sector-specific news for several years now, and they’re well worth signing up to if your interests lie in the medical/healthcare, telecom, IT or finance sectors.

They also offer events and webinars, but I know them best from their free email newsletters which have proved popular with my users.

A quick and cost-effective way to keep yourself up-to-speed across the above sectors in these straitened times.

And so it came to pass…into print

Having talked about the apparent efforts of digital information to look as much like print as possible in the last post, a satirical French website has decided to stop just pretending to look like print and actually launch a really-made-of-paper edition.  The Financial Times article described this move as “making a mockery of the newspaper industry’s dash for digital”.

I don’t really agree with that, I would like to think it will demonstrate that neither format has made the other obsolete and that as readers, we would still like to have the choice.

That’s my take on the customers’ view, but what’s the business case for doing this?

Unsurprisingly, the main reason seems to be financial.  It’s a familiar story with many online news sites: Bakchich has not been able to find a successful online business model.  Add to that the huge drop-off in advertising (which is often relied upon as the main revenue-earner) and you end up struggling.

By contrast, the business model for print is well-established: sell copies and make money.  However, the point that interested me most was the expectation that the quality of content will be higher due to lack of (digital) space – the print version will be only 20 pages long.  The editor, Nicolas Beau is quoted as suggesting that there will be a “qualitative leap” in the standard of writing and editing as they will need to be more selective.

Quality, or the lack of it, has been a constant criticism of web-based information.

Is it accurate?  All information literacy training urges students to check the source, the date, the author, apply common sense – are the facts corroborated anywhere else?

The other criticism I’ve seen, albeit less often, is slightly different.  The suggestion is that “more is less” – the sheer quantity means lower quality.  Anyone can publish now and “play at journalism”, and the web is becoming full of endless drivel as people spout on about this and that, just as they feel and no one’s bothering to edit themselves or be selective…and that it’s boring.  Twitter has been a prime candidate for this type of criticism.

Of course this is sometimes the case, but print isn’t immune from producing drivel either and I don’t think digital publishing, by default, should (or does) mean no editing or selectivity, often rather the reverse!  Surely editing is exactly what you have to do to compose those brief sentences on Twitter?  And the respected blogs tend to be subject-specific, which means being selective with the content.

I’d rather see the ability for everyone to have a go as a positive disruption within the world of publishing.  And it’s a great opportunity for all non-journalists to learn at least two new writing-skills: editing and selecting.

Flippin’ fast news

There’s a new service from Google called Fast Flip.  The aim is to more closely replicate how people flip through the news, so that we can “consume” news more quickly.   I feel a bout of indigestion coming on.

However, as we all know, delivering information effectively isn’t just about speed, it’s just as much about content and searchability.  It has Google’s search technology behind it and Fast Flip has some interesting content coverage – 30 providers so far ranging from the BBC, New York Times, Washington Post, Popular Mechanics and Technology Review to Elle, Fast Company and Goodhousekeeping.  Yes, well there should be something for everyone there!

Oh my, I’ve started laughing, why is this so funny?  Perhaps it’s the somewhat over-the-top,  over-keen description.  Hey folks, this is  “a new reading experience”! Is it?  Really?  The reading bit actually felt just the same to me, what’s new(-ish) is the presentation format.  I say “ish” because it’s not as if the screenshot format is a first-ever, check out Newseum.

Don’t get me wrong, it’s an interesting idea and progresses the move towards a more browsable interface – the screenshots of front pages give the user that “as originally printed” feel.  It will no doubt develop over the coming months and I expect it will be popular.  If it improves users’ ability to browse around or search and find specific news items that they want then it should be successful.   It may even help “boost the flagging fortunes of the news industry” as the BBC’s article notes.

The thing that strikes me most though,  is why this is not really a new reading experience.   It is the paradox of what is defined as innovative and progressive in electronic information provision.   I quote from the BBC article: “Fast Flip imitates a conventional print publication by offering screenshots of the web pages containing relevant articles.”

So, electronic information is shamelessly striving to become more and more like its print predecessor?  Interesting.  It’s been said that imitation is the sincerest form of flattery, will it also turn out to be disruptive?

The 3rd-party purchaser as 2nd-class citizen?

Strange, but true: if you buy market data from a third party aggregator, almost all publishers will refuse to answer any queries you may have on their data you have bought.  However, if you buy data direct from the publisher, the publishers will then be happy to answer any questions you have about the data and will often let you speak to the report author too.

Why?  I spoke to Frost & Sullivan, a major player in the market research sector.

The short answer is: total loss of customer-interaction.  They don’t get to know who is buying their material.

The aggregator pays the publisher an upfront fee for permission to sell their data (and takes a percentage of this).  The publisher doesn’t like this lack of visibility.   Contrarily, you might think that the publishers are missing a trick here – why not then encourage these “remote purchasing” customers to ask questions as a neat way of finding out who has bought their data?

Well, no, it seems that the dislike of third-party purchase goes deeper.  In fact, ideally the publisher would rather you couldn’t buy any of their data through a third party.  It is seen as “detrimental to our whole interaction”.

So they turn their backs on you and sulk.  Or maybe they do have a case.  Let’s be balanced here.  Frost & Sullivan pointed out that you may well miss out on special offers and discounts that they could offer, suggestions of alternative reports, as well as advice and further explanation of the content.  As is so often said in the world of search skills (and it’s all too easily forgotten): there’s nothing like actually speaking to someone who knows.

Maybe the role of the third party aggregator will gradually change into being more of a search portal, from which the user then jumps direct to the publisher for purchases.   I’d certainly recommend that if you want to purchase a complete report, go direct to the publisher and get that after-sales advice…oh, and a discount of course!

Here are some market research publishers that allow you to buy their reports by-the-slice direct from them.  This isn’t an exhaustive list, and many other publishers are open to selling sections if you ask nicely.

Business Communications Company: Coverage: multi-sector

Freedonia: Coverage: multi-sector

Global Industry Analysts: Coverage: multi-sector

Kalorama: Coverage: multi-sector, but especially healthcare. Only older reports are available to buy by section.

Packaged Facts: Coverage: consumer markets

SBI Reports: Coverage: food service equipment industry

Aggregators:

Marketresearch.com, Mindbranch.com, ResearchandMarkets.com, MarketsandMarkets.com and Profound
MarketResearch’s database contains summaries and tables of contents and a “search inside the report” option with some of the reports. They bought Profound back in June 2007.
Mindbranch and ResearchandMarkets are two other major aggregators. The content on all three will overlap, but cross-searching can still usefully reveal different reports.
MarketsandMarkets.com seems to be the newest kid on the block and (I absolutely love these urls!) just sneaks my Market Reports Website Naming Competition prize, can’t you just hear them? “Oh for heavens sake, I give up let’s just call it…”

It’s trivial, but it’s making an important point

I’ve gained a new habit.  It’s ok, this one’s fine to share – I’ve taken to visiting Microsoft’s Bing just to have a look at the daily picture.

In addition to the scenic photo, you can click underneath and within the picture to get pop-up snippets of information about it, which also give you the option to click onwards for further information.

Perhaps it’s trivial, little more than window-dressing, but to me this pleasant offering is more than that.  It is an illustration of the ongoing struggle to create something new for the search engine to do.

To me it really does seem to be a bit of a struggle, because despite the myriad ways we have available to us by which to search for the information we want, in truth, the effectiveness and success of any search is still almost completely dependent on the skills of searcher.

Call it what you will: “information literacy” or being search-savvy, but it amounts to the same thing.  If you don’t use the right keywords, or don’t know where best to start your search, or  how to evaluate the results you are given, it is unlikely that you’ll be very happy with your findings.  Unless perhaps, you have hours and hours to spend trawling or unless – of course – you’re lucky.

And yes, there is an element of luck behind many a successful search.  Any search professional worth their salt should be humble enough to admit that.  I think Google understands this too, why else would they have their  “I’m feeling lucky” button?

Semantic searching, fuzzy logic, Bayesian analysis, natural language, voice, images and – my goodness – “ambient findability” have all entered the fray, but for the time being it still boils down to the same requirement:

Searcher know thy searching
It takes time to learn to search well, and you then need to continue teaching yourself.

Simple, but handy – new Google search options

Google recently announced some new search options in the form of some side-menus.

To try it out, click the “Show options…” link just above your search results to display the
Search Options panel.

That’s all – just a handy little addition.  Well, I like it anyway!

Getting to know you – it’s worth the time

Since it officially launched, I’ve been asked quite a number of times about Wolfram Alpha.   Questions such as what do I make of it?  Why doesn’t it answer my question on…?  What sort of questions can it answer?  How does it work?  What type of content does it deal with?

I like Wolfram Alpha, it introduces a different angle on finding things – which is probably why some people find it a little tricky or unsatisfactory to use.  The answer is to invest a bit of time in finding out exactly what Wolfram aims to do and how it does it. This doesn’t have to take hours, just a bit of your time, please.

The point is, it is worth doing this for all the search resources you use.  Gaining a reasonable understanding of the tool you want to use is the way to improve both how you use a resource and the results it brings back for you.

I’m constantly amazed at how little time people are willing to put aside in order to train themselves to use an unfamiliar data source, they just want to dive straight in (“too busy, no time”), but I’m always depressed at how frustrated they then get with the source and their lack of ability to get the best out of it.  Not to mention the time they waste.  Why are they so indignant?  What did they expect?

In the trade it’s known as “Information Literacy” – the ability to search well or learn to search well.  It isn’t magic , it is usually just a matter of troubling yourself to invest a bit of time, be it taking that 30 minute online training session, or reading up on the source, and then exploring it further.

So go on, if want to know how to search Wolfram Alpha, the first step is to take a little time read up on it:  Search Me: Inside the launch of Stephen Wolfram’s new computational knowledge engine”

Next step:  happier searching.

Print only once, then eat it with your lunch

The Economist is currently running an interesting debate on copyright: “This house believes that existing copyright laws do more harm than good“. (this link will work despite the small window saying “error 404″)

Copyright affects people in both their work and home lives whether it be the ability to access academic papers or download films and music.  It certainly has a big impact upon the usefulness of search results as regards how tantalisingly close you can get to the full work and what you can legally do with it, once you have full access.

This is quite a comprehensive debate, but I think it’s a good thing that both sides seem to agree with the basic reason for the existence of copyright, in short: “ample payments to creator”.  They also agree that current copyright laws are “over inclusive” by which they mean that current laws exert too much control and contrive to cover types of information that it doesn’t make economic sense to cover.

The rest is up for grabs -  a multitude of issues that are quite complex and fragmented (yes, that’s part of the problem!).  However, please don’t be put off.   This debate is well worth a look as the cases For and Against present one of the more readable summaries to the rights and wrongs of copyright, plus you can also dip in and out of the many points of view…oh, and you can also place your vote.

Up and coming: 2 new search engines

The first one, Wolfram Alpha due to launch later this month, is getting a fair amount of publicity at present on various sites including the BBC.  It wants to answer your questions, but instead of presenting a results list of sites where your question should be answered as with AskJeeves (yes, the “Jeeves” bit is coming back, hurrah!  Why is that so pleasing?), Wolfram aims to present the actual answers to your questions.

This is nicely demonstrated by CNET news which has taken a preview tour and tried Wolfram out on some questions so you can see it in action at Screenshots: a tour of Wolfram Alpha

The second brand new search engine that’s appeared on the horizon is Oparla.  This one differentiates itself by allowing users to register and then rate the sites they visit for their usefulness.  If you do this you also qualify to enter a cash-prize draw – yes, really and you can try this today!

Oparla says it will be exploring other ways of “rewarding” its users in the future, as advertising revenues increase.  And how does it intend to raise advertising revenues?  The plan is to charge advertisers a set monthly fee, rather than use the more established pay-per-click method favoured by the likes of Google.

Paying your users?  How interesting…and shurely somewhat disruptive!