WOW! Searching DU for a topic just completely outperformed a Google web search

[Phoebe Loosinhouse]

Let me just say that if we do things right, and link, link, link to published material in order to disseminate information or to support the views we are promoting, we are doing a great public service.

I was just going to discuss the Zelikow/Rice connection in the current thread about the 9/11 commission and wanted to toss in how the commission completely overlooked or was hoodwinked into completing ignoring the July 10th 2001 meeting between Rice, Black and Tenet. I plugged in Rice Tenet Black july meeting and variations thereof and got almost nothing except blog references, nothing mainstream, which puzzled me, since I knew this had been widely reported in 2006 by the Washington Post, NYT, was in Woodward's book, etc. Almost nada. Do it yourself if you want to check me. Then I plugged in the same search on just DU and came up with gobs of great threads, mostly from 2006 that had in depth discussion, research, multiple links to every source imaginable.

Please keep up the good work and link whenever you possibly can to support your positions. I also want to say that particular posters like Kpete, babylonsister and so many others, some of whom specialize in specific topics and forums, you are so fantastic and the work you do is really important. Kudos to you!


edit for spelling.

[sutz12]

1. Disseminate...not decimate...

:)

[Phoebe Loosinhouse]

3. Thanks, caught it already.

[rucky]

8. When it comes to information, we do plenty of both around here.

[annabanana]

2. I'll guess we'll just keep DUing the VooDU that We DU so well!

No place like it.

[Phoebe Loosinhouse]

4. kick!

[Raine1967]

5. Interesting that you say this becuase

in the past few months quite often when I am doing research for things that were never really mainstean -- My google searches are -- more and more -- coming up with DU links.

[pacalo]

6. I've noticed that, too -- in the google images, also.

[Jamastiene]

7. That really does work.

There have been times when I have been searching for things totally unrelated to DU and end up with lots of DU hits close to the top. Half the time, what I was looking for was on DU to begin with. Now, I just search DU for things and marvel at the depth and wealth of information here. :thumbsup:

[seemslikeadream]

9. Thanks I needed that

You have no idea how much that meant to me

[hootinholler]

10. Not surprising

You are searching a more focused corpus.

It would be an interesting experiment to catalog the results orderings of some standard queries over time that represent issues. I wonder if blog postings are being used to dilute the usefulness of searches on certain topics. I'm sure the phenomenon happens somehat naturally as topics are discussed, but loosing the press reporting on a subject like that is troubling.

-Hoot

[Phoebe Loosinhouse]

11. Very true - that is part of what I was trying to say.

I was SURPRISED that the results came up mostly blog entries, and not even DU, mostly blogs I had never heard of. Do some searches of topics that you think should be easily researched and see what you find. I was extrmemly surprised with the results of my search.
This is a topic that bears investigating.

[hootinholler]

13. There's probably a PHD in that study.

To do the study repressively, one would need snapshots of the state of Google's corpus over time (that's a lot of data). To do it progressively, one would need to continuously define searches as issues are brought to light, and run them daily, fetching the entire results set. One would also need control topics as well.

I wonder what he nominal decay curve looks like? I imagine one schooled in statistics, set theory and search calculus could shed some light on the phenomenon. I'm certain that corpus manipulation could be detected, but I suspect it would take some sort of specialized geometry to trace it to a meaningful origin.

Definitely worth some serious chin-stroking.

-Hoot

[babylonsister]

12. Wow. Who knew? Not me. Thanks! And I'm glad to know

info posted goes beyond here. That's awesome.

[Mabus]

14. I hate to rain on your parade but...

Google uses cookies to track users preferences. That's how they target their ads but it also targets searches. As an anecdotal experience, a couple of weeks ago Sallie Mander was doing some research and was very excited when her website was the top hit in her google search. She asked several of us on facebook to run a search on a key term to see if her website was also at the top of our searches. To her disappointment she didn't even make the first page on the google results.

In other words, "As it works right now, if you use a Google product (Gmail, Google toolbar, AdWords, etc.), Google is keeping track of what you search for and what websites you visit, and it's then tailoring your results appropriately. Search for "bass," and Google will know whether you mean the fish or the instrument." http://searchenginewatch.com/2189531

So, if your google cookie knows you use DU it will search through DU and give you those results first. Your search results are weighted toward DU. If you go to another computer, one that you don't use frequently or hasn't cruised to DU lately, you'll get different results.

[Phoebe Loosinhouse]

16. The sites it threw up first were for the most part completely unfamiliar to me

whereas I vist mainstream newspapers all the time. Also, even though I am definitely a voracious user of DU, Du was also NOT among the first results, they were in there, but way down a ways. I also am NOT a user of Gmail, Google toolbar, or Adwords.

However, after doing a few more experiments I discovered that complete names made a big difference in my Google search and did finally throw up the MSM material I was looking for.

BUT, I stand with my statement that searching DU got me more info, better info and more complete info than my Google search and did it faster without chaff mixed in with the wheat.

[Spazito]

15. I much prefer using the DU search when attempting to find articles from previous years...

on issues being currently discussed re the Bush administration for the very reason you have pointed out. The DU search gives thread links which in turn have multiple links within. This makes the search so much easier than trying to do the same using Google.

[Phoebe Loosinhouse]

17. That is EXACTLY the point I was trying to make. nt.