Microsoft putting the beats on Google

[iago]

Well it depends if they do kick everyone elses ass, it wouldn't be wise to dismiss it either. Google doesn't have some magical undiscovered technology, it's just been the boldest corporation to implement search on such a widescale. Sure MS may want some of googles techniques but the technology they use isn't anything drasticly different from anything else.

I think it does.  It searches through ~7 Billion websites (with hundreds, sometimes even thousands) of pages in less than a second.

Microsoft isn't going to beat Google.

By the way, if you read that article I linked you, you'll know that Ballmer threw a chair across the room when he found Ralmulk (sp?) was leaving Microsoft for a job at Google.

It doesn't display them all on the same page..it isn't hard to get a number of pages meeting the criteria. They just display them as they get them back.

So what you're saying is that they also sort the millions/billions of pages based on what you typed.  Why are you trying to strengthen Sidoh's point?

[Warrior]

No, I'm saying it reduces the load of fetching them..you know unless you forgot how SQL works ..you can provide a limit. It will stop fetching after it reaches that limit..the speed is impressive but nothing no one else can't beat withought trying some.

[iago]

No, I'm saying it reduces the load of fetching them..you know unless you forgot how SQL works ..you can provide a limit. It will stop fetching after it reaches that limit..the speed is impressive but nothing no one else can't beat withought trying some.

It still has to sort it all, based on your search criteria, before it can limit them.  Sorting is generally pretty fast, with a decent amount of hardware memory.  Sorting a database that doesn't fit into main memory takes something like:

2(p) * (1 + log<b>(p/b))

i/o operations.  Where b is the number of pages that can fit into active memory and p is the number of pages that are being sorted.  The result is the number of disk i/o's required, which is the bottleneck operation. 

Since Google doesn't know what you're going to type before you type it, that means that this has to be done every time somebody types something.  Unless they've come up with a different way of indexing, but that's the way it's done in the popular relational databases (Oracle, MSSql, and MySQL for sure). 

For more information on how data is sorted in a database, Google (ha) for "external mergesort". 

[GameSnake]

Google doesnt seem to be as great as it used to be for me. I think other engines have learned from Google though.

[Warrior]

No, I'm saying it reduces the load of fetching them..you know unless you forgot how SQL works ..you can provide a limit. It will stop fetching after it reaches that limit..the speed is impressive but nothing no one else can't beat withought trying some.

It still has to sort it all, based on your search criteria, before it can limit them.  Sorting is generally pretty fast, with a decent amount of hardware memory.  Sorting a database that doesn't fit into main memory takes something like:

2(p) * (1 + log(p/b))

i/o operations.  Where b is the number of pages that can fit into active memory and p is the number of pages that are being sorted.  The result is the number of disk i/o's required, which is the bottleneck operation. 

Since Google doesn't know what you're going to type before you type it, that means that this has to be done every time somebody types something.  Unless they've come up with a different way of indexing, but that's the way it's done in the popular relational databases (Oracle, MSSql, and MySQL for sure). 

For more information on how data is sorted in a database, Google (ha) for "external mergesort". 

That's nice an all but if they use an SQL variant then they really are just using the technology that already exists. Still havn't found anything unique that they do thus far.

[MyndFyre]

That's nice an all but if they use an SQL variant then they really are just using the technology that already exists. Still havn't found anything unique that they do thus far.

You lost this argument, man, give it a rest!

[iago]

No, I'm saying it reduces the load of fetching them..you know unless you forgot how SQL works ..you can provide a limit. It will stop fetching after it reaches that limit..the speed is impressive but nothing no one else can't beat withought trying some.

It still has to sort it all, based on your search criteria, before it can limit them.  Sorting is generally pretty fast, with a decent amount of hardware memory.  Sorting a database that doesn't fit into main memory takes something like:

2(p) * (1 + log(p/b))

i/o operations.  Where b is the number of pages that can fit into active memory and p is the number of pages that are being sorted.  The result is the number of disk i/o's required, which is the bottleneck operation. 

Since Google doesn't know what you're going to type before you type it, that means that this has to be done every time somebody types something.  Unless they've come up with a different way of indexing, but that's the way it's done in the popular relational databases (Oracle, MSSql, and MySQL for sure). 

For more information on how data is sorted in a database, Google (ha) for "external mergesort". 

That's nice an all but if they use an SQL variant then they really are just using the technology that already exists. Still havn't found anything unique that they do thus far.

What the hell are you talking about?  YOU are the one who brought up SQL, and I was explaining to you why normal SQL would probably be too slow. 

[Sidoh]

* Sidoh bashes his head on his desk repeatedly as he gives up hope on Warrior.

[Warrior]

No, I'm saying it reduces the load of fetching them..you know unless you forgot how SQL works ..you can provide a limit. It will stop fetching after it reaches that limit..the speed is impressive but nothing no one else can't beat withought trying some.

It still has to sort it all, based on your search criteria, before it can limit them.  Sorting is generally pretty fast, with a decent amount of hardware memory.  Sorting a database that doesn't fit into main memory takes something like:

2(p) * (1 + log(p/b))

i/o operations.  Where b is the number of pages that can fit into active memory and p is the number of pages that are being sorted.  The result is the number of disk i/o's required, which is the bottleneck operation. 

Since Google doesn't know what you're going to type before you type it, that means that this has to be done every time somebody types something.  Unless they've come up with a different way of indexing, but that's the way it's done in the popular relational databases (Oracle, MSSql, and MySQL for sure). 

For more information on how data is sorted in a database, Google (ha) for "external mergesort". 

That's nice an all but if they use an SQL variant then they really are just using the technology that already exists. Still havn't found anything unique that they do thus far.

What the hell are you talking about?  YOU are the one who brought up SQL, and I was explaining to you why normal SQL would probably be too slow. 

Okay so tell me, what other technology can they possible utilize to sort through the billions of entries?

Additionally:
How would fetching entries with a limit possibly be slow? A technology which specializes in reading large amounts of data combined with a speedy server can indeed get the job done, you're just thinking unrealisticly.

[Sidoh]

Okay so tell me, what other technology can they possible utilize to sort through the billions of entries?

Are you seriously implying its impossible to use other database/search routines when SQL possibilities are excluded?  Hahahahah....

It's obviously something else, though.  Look:

Results 1 - 10 of about 6,980,000 for what's your favorite band right now. (0.22 seconds)

I just copied/pasted a random line of text from one of the IM windows I had open.  0.22 seconds to estimate that there are 6,980,000 articles related to my query?  Sure sounds like SQL to me... *cough*

Additionally:
How would fetching entries with a limit possibly be slow? A technology which specializes in reading large amounts of data combined with a speedy server can indeed get the job done, you're just thinking unrealisticly.

For the third time: because it doesn't pre-order the searches for every possible query.  It searches and then sorts.

you're just thinking unrealisticly.

You're not thinking at all.

[iago]

For the third time: because it doesn't pre-order the searches for every possible query.  It searches and then sorts.

Exactly.

you're just thinking unrealisticly.

You're not thinking at all.

DINGDINGDING winner!

[Sidoh]

I R WEENAR.

[Warrior]

Okay so tell me, what other technology can they possible utilize to sort through the billions of entries?

Are you seriously implying its impossible to use other database/search routines when SQL possibilities are excluded?  Hahahahah....

It's obviously something else, though.  Look:

Results 1 - 10 of about 6,980,000 for what's your favorite band right now. (0.22 seconds)

I just copied/pasted a random line of text from one of the IM windows I had open.  0.22 seconds to estimate that there are 6,980,000 articles related to my query?  Sure sounds like SQL to me... *cough*

With fast enough servers getting a ballpark figure isn't too hard. Please.

Additionally:
How would fetching entries with a limit possibly be slow? A technology which specializes in reading large amounts of data combined with a speedy server can indeed get the job done, you're just thinking unrealisticly.

For the third time: because it doesn't pre-order the searches for every possible query.  It searches and then sorts.

How would you know this? Link to an article, doc, specification, anything.

you're just thinking unrealisticly.

You're not thinking at all.

[/quote]

Cmon Sidoh, this is your usual insult the person instead of come up with something smart. Get over it.

[Sidoh]

With fast enough servers getting a ballpark figure isn't too hard. Please.

Find me a fast enough server then, please.  Additionally, it's not SQL.  SQL doesn't "estimate" things.  It's a query language.  It gives what you ask for.  Google does not use SQL, it uses something better.  Get over it.

How would you know this? Link to an article, doc, specification, anything.

Because it's COMMON SENSE.  How could they possibly construct a list of every possible query that people are going to use?  They're not going to.  They're going to develop algorithms to sort search results dynamically.  Since they update their possible results with every web crawl they do, they'd constantly be re-sorting the results.  The results would be less relevant and Google would suck more than it does now.

Warrior, use your fucking head.  Open your eyes.

Cmon Sidoh, this is your usual insult the person instead of come up with something smart. Get over it.

It's not an insult, Warrior.  You can take it as such, but if its the case, it was a secondary cause.  It's a retort.  You're not thinking -- it's the truth.

[Warrior]

With fast enough servers getting a ballpark figure isn't too hard. Please.

Find me a fast enough server then, please.  Additionally, it's not SQL.  SQL doesn't "estimate" things.  It's a query language.  It gives what you ask for.  Google does not use SQL, it uses something better.  Get over it.

SQL organizes how it reads the data,  anything else would do something extremely similiar.
Additionally, how can it use something "better" if by what you say "it's not SQL"..?

How would you know this? Link to an article, doc, specification, anything.

Because it's COMMON SENSE.  How could they possibly construct a list of every possible query that people are going to use?  They're not going to.  They're going to develop algorithms to sort search results dynamically.  Since they update their possible results with every web crawl they do, they'd constantly be re-sorting the results.  The results would be less relevant and Google would suck more than it does now.

Warrior, use your fucking head.  Open your eyes.

It isn't very hard actually. You can get a ballpark figure easily "on the fly" and it isn't like their webcrawls are done everytime you search either. It's using old data it collects.

Cmon Sidoh, this is your usual insult the person instead of come up with something smart. Get over it.

It's not an insult, Warrior.  You can take it as such, but if its the case, it was a secondary cause.  It's a retort.  You're not thinking -- it's the truth.
[/quote]

It's impossible to not be thinking unless you're dead so I'll take it you're either indeed insulting or just posting random bull which wouldn't be out of character for you.