On Mon, 24 Feb 2003, Alex Holden wrote:

*>> experience with htdig and a complex full text search over 2GB of text
*>> files should be well under 10 seconds in any case on a 233MHz machine
*>
*>TBH I don't know how they do it, but Google index vastly more than 2GB
*>and it still only takes a fraction of a second to perform a query.

Well that one is easy, they run acres of computers and dynamic aliasing
and routing to spread the load, and they probably optimize the database
chunk size with the hardware memory size. This should cause most queries
to run in ram only. These are basic search database setup optimizations,
aka 'book moves', and they probably use more than this. The ten second
figure I quoted was just an upper bound. Usual quesries are 2 seconds or
so.

Wrt this or that organization tagging people by their preferences, hey,
you said that, I did not say that. But I know for sure that some of my
postings are not in the Google database while others are. Maybe this is
due to the way the piclist feed to usenet was turned on and off repeatedly
in the past. It is none of my business to know this, and I do not care.
It's just something I have to live with. And I believe that for that
reason it is not good to feed piclist messages anywhere else than to
piclist archives. Because an incomplete archive is worse than useless, it
is misleading imho.

Peter

--
http://www.piclist.com#nomail Going offline? Don't AutoReply us!
email listserv@mitvma.mit.edu with SET PICList DIGEST in the body