The Bridge

tzenes

03:00

still 401

TomWij

Hit Recommendations at the top.

tzenes

Here is what it recommended for me:

TomWij

Sorting by recommendation match in search is impossible, then it would have to check each movie in the entire database.

But to go back to zero.

tzenes

Indiana Jones Temple of Doom (the worst indi movie), Jurassic park 3 (the worst jp movie), Godfather part 3 (which I shouldn't have to say how bad that was), Dail M for Murder (which I've never seen), Decent (which was terrible)

TomWij

It's not hard to suggest a set of games based on some keywords.

tzenes

03:04

sorting by match is what google does with every query

with a much larger database

TomWij

No they don't.

tzenes

they most certainly do

now they do some smart cacheing and prefetching

TomWij

They have nothing to base their recommendation on.

tzenes

but they do

they have an algorithm which does just that

its called the Page Rank algorithm

and as internet search algorithms go, its one of the best

TomWij

And how does that relate to Personal Recommendations? :S

tzenes

03:06

"Sorting by recommendation match in search is impossible, then it would have to check each movie in the entire database."

I was saying that sorting by recommendation is very possible

TomWij

As in...

Different for each person entering the same query?

As in...

tzenes

Different every time the page changes?

trillions of webpages, constantly changing

some with procedural content

TomWij

If I like Synonyms.com and you like Wiktionary, why does it give Wiktionary first for me?

tzenes

compared to having to run an algorithm on only a few thousand movies

it is very possible to sort by recommendation on match

TomWij

Having to run the algorithm for each user.

tzenes

03:09

remember, I only have to run the algorithm every time I learn something new about you

TomWij

Google only does it once for all users.

tzenes

not on every query

I can cache the data

TomWij

For Jinni it does it once for each user.

tzenes

Google has a larger dataset

TomWij

Caching is out of the picture...

tzenes

03:09

how so?

very easily

TomWij

What don't you understand about "personalized results"?

tzenes

every time you vote on a movie I hit my data base, calculate all your recommendation number an cache them

I wasn't saying google was personalizing, just that they have a much larger dataset to consider

about 10 orders of magnitude larger

TomWij

But they have to go through the dataset once for all users, then cache it.

Jinni would have to go through the dataset once for each user, then cache it.

tzenes

you'd have to have a billion people use Jinni before its data set was nearly as large

1 billion people * 1,000 movies = 10^12, about the number of webpages google has to rank

fortunately for Jinni they don't have to have a billion people

TomWij

You're just talking about a scalability difference...

How hard is it to see that.

tzenes

03:12

actually I think you're talking about a scalability difference

I'm talking about a number of calculations

since my calculations aren't going up

TomWij

Y times something VS X times Y times something is a huge difference?

tzenes

I have to make 1000 calculations per user to get personal recommendatiosn

well, my old P4 could do a billion calculations per second

so it could handle 1 million users per second

of course disk writes are more expensive

it could only handle about 10,000 users

TomWij

:107731 so in case of Google personalized results that would be 1 billion people * 10^12 ;-)

Oh look, Google can't handle that... :-S

tzenes

while I agree, that's not really relavent

TomWij

Nor can Jinni.

tzenes

03:14

still not relavent

google can handle 10^12 calculations, which they do daily

TomWij

well, my old P4 could do a billion calculations per second

what about Memory, I/O, overhead, ...? ;-)

tzenes

did you not read the lines below that?

where I talked about disk writes?

Grace Note

At this point I've lost track... what are the two sides of this argument? Is it as stupid as it is sounding? Because I'm seeing an awful lot of "nowhere" going.

tzenes

He said you can't calculate personal recommendations for every user based on a search

and I said, that given jinni's data set, you most certainly can

Grace Note

I meant from the beginning

tzenes

03:16

Oh, I said calculating recommendations is hard, and he said it was easy

so I gave a bunch of case example, and he mostly dismissed it

TomWij

tzenes not understanding that "doing something complex" once for all users (X movies) would not work out well if it was done once for each user (X movies * Y users).

Grace Note

And Tom? What's your interpretation of this argument?

tzenes

Tom, that's not the point, its an issue of scale, doing something once for a trillion things, is the same as doing its a billion times for a thousand things

so doing something for a trillion webpages is tantamount to doing personal recommendations for a billion users on a thousand movies

TomWij

Again

Where is your "trillion" in the end of your sentence?

tzenes

1 billion * 1000 = 1 trillion?

TomWij

03:19

Magically, some movies disappear?

X and X * Y

not X and Z * Y

tzenes

1 billion users * 1 thousand movies vs 1 user on 1 trillion webpages

TomWij

Your movies (X) don't magically turn into another smaller count (Z)

tzenes

I wasn't comparing movies to movies

I was comparing movies to webpages

TomWij

Cool, you started multiplying users with each other.

still wrong

tzenes

really? you think one of those two things is more work?

TomWij

03:21

Sure, Google obviously needs more servers for it, hence a Scalability issue.

McKay

What do you mean he's multiplying users with each other? 1 billion isn't a square

Grace Note

@McKay pre-edit was 1 bil users * 1 thou users

tzenes

@McKay there was a typo, I fixed it

TomWij

He has edited that out already.

Grace Note

It's a typo, honest enough

tzenes

03:22

:107785 Jinni also doesn't have a billion users...

TomWij

Comparing Jinni to Google is like comparing apples to oranges...

McKay

or apples to Orange Orchards

tzenes

Except they both do the same thing

they're both search engines

they both need to make calculations

TomWij

Not this again... >_<

They do the same thing now.

now

tzenes

and you said it was impossible to do personal recommendations on every movie for every user

TomWij

03:24

But with personalized results that wouldn't be the same anymore.

tzenes

and since they both need to do calculations, I have disproven this statement

I'm not saying they're the same, only that both need to do calculations and that the number of calculations is tractable.

TomWij

If you wanted Google to be doing the same as what I said was impossible for Jinni.

tzenes

I didn't say I did

I said the number of calculations was tractable

TomWij

You would have to do an algorithm for each web page for each user.

tzenes

you're now talking about something different

to do personalized recommendations for every user, Jinni has to do number of movies (about 10^3) times number of users (I dont' know how many but let's say 10^4)

this is 10^7 calculations

TomWij

03:26

There's is a difference in complexity (remember the Big Oh?), what do calculations and scalability have to do with that?

tzenes

I am demonstrating

so, Jinni has to do 10^7 calculations

TomWij

while non-personalized recommendations are just 10^3

tzenes

by comparison google has to do 10^12 calculations (once for every webpage)

10^7 < 10^12... this is the basis for my argument

since 10^12 is tractable, 10^7 must also be

TomWij

Yes

tzenes

thus for Jinni to do personalized recommendations for 10,000 users is tractable

TomWij

03:29

No

It's an inefficient solution because of the different complexity.

tzenes

I'm not sure how you can make that argument

how many calculations do you think 10,000 users requires?

TomWij

Let Google give personalized recommendations, can their servers handle that? No.

tzenes

I didn't say they had to

TomWij

But that would make the comparison fair.

tzenes

I was merely establishing an upper bound on the number of tractable calculations

I'm not trying to make a comparison

TomWij

03:31

Otherwise you're comparing an inefficient solution to an efficient solution.

You might as well introducte infinite resources.

tzenes

I'm trying to establish how many calculations Jinni can reasonably preform

McKay

tractably perform,

tzenes

how many calculations do you think Jinni can reasonably preform?

well I'm saying a tractable number is a reasonable number

TomWij

Jinni ain't Google.

tzenes

that's true

McKay

03:32

Yeah, I wouldn't say Google is "reasonable"

TomWij

So can Jinni perform what Google can?

tzenes

which is why I asked how many calculations you though jinni could reasonably preform.

TomWij

No.

tzenes

I was trying to establish an upper bounds

if you feel you have a more accurate estimate please bring it forth

TomWij

Jinni calculating their personalized search results is like Google calculating the Traveling Salesman Problem... ;-)

McKay

03:35

?

tzenes

I don't think that's an apt analogy as that's a 2^n*n^2 as compared to n^2

how many calculations do you think jinni can reasonably preform

TomWij

Not enough.

tzenes

how many?

ball park it

TomWij

123456

tzenes

so 10^6, you think that's all they're capable of?

McKay

03:37

My computer can do 10^6?

Heck, my phone can do 10^6?

tzenes

your computer can do a lot more than 10^6

TomWij

Ofc

tzenes

my phone does 7*10^6

its like 7 Jinni's

TomWij

xD

tzenes

maybe they should invest in the same technology as Motorola

TomWij

03:38

But your phone can't handle a big database o' movies and users. :P

tzenes

o contrare

my phone does handle a big old database of movies and users

TomWij

Nor can Jinni afford to shut down their site because they are doing a recalculation.

tzenes

but that's hardly relavent to the conversation

TomWij

Then why did you ask? :P

tzenes

that's true, we're only talking about spare cycles

TomWij

03:40

The funny thing actually is...

You're talking about operations.

But it's about movies and users.

tzenes

if I had to estimate how many DB calls Jinni can afford, I'd say probably 10^8 maybe 10^9th

TomWij

The algorithm has to do a lot more than 1 operation for a movie.

tzenes

remember operations aren't our bottleneck, DB calls are

TomWij

It has to check the movie itself, the user and the movies the user have.

And then use an algorithm of a certain complexity.

And all-together it can handle that for the movies.

tzenes

Actually we just need to do SVD on the matrix

which is very cheap by comparison to DB calls

TomWij

03:42

But it won't be able to do that whole bunch of processing for all the users.

tzenes

the expensive part is getting all the user ratings out of the database

and the truth is there just aren't that many movies

TomWij

Let's say we have one server which just can handle that.

tzenes

Jinni can probably handle about 10^5 users with their current load no problem

TomWij

Then we would have to place a server for each user, worst case.

tzenes

I don't see where you're getting one server per user

TomWij

03:44

No, they can't.

It would take 10^5 as long to do.

tzenes

at Amazon calculating ratings is slow low on our operational budget that we don't even consider it

TomWij

or it would require 10^5 as much CPU usage.

But this is leading nowhere...

tzenes

we essentially get it for free

because we have to have enough hard ware for stress periods

TomWij

I've shown (not proofed) my point, you've showed (not proofed) your point... It's a subjective matter anyway.

tzenes

actually that's not true

whether or not they can handle that load is not subjective

what's more there are companies which do handle that load

(Netflix, Amazon, etc)

the truth is you can go on about Big O notation, but unless you know how big N is, its purely theorhetical

the reality is, that N is small enough that we can afford N^2

or N*M as this case maybe

there are just not enough movies to make this a hard problem

turns out there isn't enough stuff people buy to make this a hard problem

N is just too small

TomWij

03:49

Well

Reality speaks otherwise again...

tzenes

really? Because at Amazon we have no problem doing it

TomWij

Call me when Jinni, IMDB, Hunch, Google and other sites have this implemented.

tzenes

Netflix doesn't seem to have a problem doing it either

TomWij

Netflix isn't doing it, and you stated that you're not doing it at Amazon either.

tzenes

I said we are doing it

amazon calculates a personal recommendation for every item we have in the system for each user

TomWij

03:51

:107916 Then this be a big typo you can't edit?

tzenes

calculating rating is so low on our opperatioanl budget

as in it costs us almost nothing

which is why my next lines says "we essentially get it for free"

TomWij

:107928 goes off to sleep now...

tzenes

no matter how hard you try you cannot make this a subjective matter

TomWij

And it's low on your buget, but you can't determine if it will be on Jinni's.

tzenes

Low compared to the budget of running the website

TomWij

03:53

It is one by essence unless you come with facts and proofs.

tzenes

which Jinni also has to do

TomWij

Running the important core algorithm once for each user is not really low compared to what the server is already doing...

tzenes

jinni.com/discovery.html?query=crime&sizeby=Rating

game set and match

TomWij

Where are the personalized results?

tzenes

they're sorted by my personalized rating

or what they believe it to be

TomWij

03:56

Say a title and amaze yourself.

tzenes

frankly their algorithm needs work

they said the wire was a good match for me

TomWij

Top left

Zatoichi: The Last to it's right

tzenes

not on my screen

TomWij

Older than America below it

tzenes

I got Dirty Harry

its the biggest one

which actually is a good match for me

TomWij

04:01

But not every movie in there is matched.

As clicking on the movie doesn't say a match rating.

So indeed, the top left movie is unmatched and is just based on IMDB or Jinni rating.

The top big movie is indeed personalized.

So they do it, but only to a small extent.

Good night... The sandman is after me.

Mana

04:57

man

that was a long conversation

McKay

yeah

tzenes

05:32

I need to find better shows to watch

Arda Xi

11:34

It seems SE supports polls after all

C. Ross

12:19

:108374 Huh? What are you referring to?

Arda Xi

the poll Jeff set up at Unix and Ubuntu

C. Ross

oh, but it's an actual poll ...

This is referring to the blog for future reference

and they added an actual poll tool, not using a question as a poll.

Arda Xi

exactly

C. Ross

I figure they'll need it for the odd thing on the SE multiverse

Arda Xi

13:05

damn, Valve is really intent on giving away Portal

Mana

13:33

Why, what are they doing now?

alexanderpas

13:46

new type of prepaid card accepted on steam, free portal when you buy enough other stuff with it. store.steampowered.com/paysafecard I say meh.

Mana

hm.

1

Q: How can I tell if I have the Team Fortress 2 update that supports Snow Leopard 10.6.4?

I recently saw that tf2 has updated its code so it doesn't run slow for the mac osx 10.6.4 update(which i was holding off on doing since i wanted to play tf2) just saw that i need to restart steam because of an update which included this: "OSX: Fix several memory leaks OSX: Fix copy/paste resul...

man, it's like he wrote that question as a stream of conciousness

C. Ross

yeah ... too localized?

Mana

No, the update will last

by which my tired brain means

can't you download older updates instead of the latest?

in which case it wouldn't be too localized if someone wanted to know what this specific update does

alexanderpas

did they include that update in their last update >> when did they include this in their updates.

Mana

Yeah.

Arda Xi

14:02

I think that question should be closed for 'not a real question', It's difficult to tell what is being asked here.

first they ask for a confirmation link, then instructions, then affirmation

C. Ross

15:04

Anyone else seeing strange things on the Tools:Close page? Things like questions listed as 7 close votes?

Mana

Can you link it?

C. Ross

Link (2k+ and Diamonds only, sorry)

Mana

oh

that would explain it

McKay

I think that might mean having a second round of close votes?

C. Ross

maybe ... I think it's a bug

McKay

15:07

Well, the one with 5? I happen to know has already been closed once.

So it would theoretically take 5 more to close it again?

Maybe the nethack one has already been closed once, reopened, then 2 more close votes?

Arda Xi

I think it's 5 close votes, then re-opened, then 2 more close votes

C. Ross

perhaps ...

Arda Xi

or well, what @McKay said

Mana

Correct on the nethack one

Arda Xi

I'm pretty sure that's what it is

Mana

15:08

Definitely. It was reopened with five votes before and has two now

Arda Xi

Answers like this make me think we really need better <kbd> css

7

A: What is the single tile trash can trick in Dwarf Fortress?

You designate a garbage dump zone in only one tile using i-g. Then you give the order to dump the stones you want to dump, or use mass dump: d-b-d. They will all be stored and forbidden on that one tile. To use them you'll have to unforbid them or use mass reclaim: d-b-c.

C. Ross

shakes head who keeps re-opening this crap

Arda Xi

:108751 I wondered that too.

Mana

Wait

are you talking about the nethack question?

McKay

:108751 what crap?

Arda Xi

15:12

:108757 I am..

C. Ross

As Arda says, the nethack question

Mana

I see. I was the fifth reopen vote. It isn't subjective in its current form; is it too localized?

C. Ross

I would say so

Arda Xi

:108763 It's another List of X question

and particularly useless since there's an official website listing them

Mana

hmm, okay.

I don't know, it's just that it reminds me slightly of "What is the easiest boss path through all the Megaman games"

as more Megaman games come out the answer will have to be updated

McKay

15:26

I think an update every couple of years (especially on a CW answer) isn't that big a deal?

Mana

I dunno. It's not really the effort involved that bothers me, more the fact that the answer has to be updated.

Arda Xi

Oh wow, somehow my tileset got messed up. Phoebus is actually much more beautiful than I gave it credit for :O

antony.trupe

indeed

Mana

maaan, that's it

I'm totally gonna learn how to play Dwarf Fortress

antony.trupe

6

A: Dwarf Fortress Character and Graphics Sets

Phoebus' Graphic Set comes both packaged with DF and separate. Legend Dwarves

Arda Xi

15:39

the point is, somehow it didn't look like that at all in my client :/

damn lazy newb pack

I just embarked near a volcano with no river, what have I done D:

Mana

:< I actually made that pack and would hope that you should retract that comment

Arda Xi

:108934 well, it's a good pack, but it doesn't work properly OOTB

I had to mess with the raws myself to get it to show the phoebus pack properly

Mana

Thanks. I'm very appreciative.

Arda Xi

wait.. what

Mana

:P

Gnoupi

15:41

The gaming site front page, with starcraft 2 and dwarf fortress in ignored tags. A grey ghost town.

Arda Xi

:108899 this is very

:108934 inconsistent

Mana

Yes, it should be, since I didn't make that pack.

I'm just bored.

Arda Xi

:108968 I figured

I just dislike having to start all over again every time I start having Fun :(

Mana

ah yes, Fun

All my experiences with DF have involved me having lots of Fun.

antony.trupe

but you don't know how to play.

Mana

15:52

Yes, that is a slight drawback :<

I need to learn how

oh well I'll look up some guides

C. Ross

The wiki is your friend

Arda Xi

I'm probably going to flood my fortress with magma pretty soon

Gnoupi

I have a dream. A game with the depth of DF, with pleasant graphics, and most of all an intuitive interface which wouldn't require you to read 100 pages of wiki to finally have fun

Arda Xi

DW? you mean DF?

Gnoupi

yes, DF, i'm tired

Arda Xi

16:01

DF in a 2.5 or 3D environment would be pretty awesome

but DF is already quite taxing

Gnoupi

although I was an avid player of X3-TC, so I guess complicated interfaces are not as much of a problem. But at least it was pleasant, inviting to dive into for hours

Arda Xi

:109096 good tileset can do wonders though

Gnoupi

:109101 yes, but the interface is still hard to get along with, as well as the clarity of what is happening

Gnoupi

it helps, though

Arda Xi

:109106 It's a learning curve I guess

Gnoupi

16:09

@Arda - probably

Ivo Flipse

I just got SC2 too

McKay

nice

Ivo Flipse

have to make sure it will run on my laptop though

so I can play during the vacation

so you'll guys have a quiet week next week :P

Mana

:<

McKay

16:26

Unless you have questions about the game :D

Ivo Flipse

well not sure if we have internet

it's France after all :P

McKay

SC2 requires internet?

Mana

I think he meant to--wait what?

does it?

McKay

Kind of.

Some stuff can be done as a "guest" without Internet

tzenes

16:44

@Mana yes it requires a connection to battle.net

Arda Xi

quick DF question, is there any way to channel the magma from this situation somehow? (cross-section)

antony.trupe

not without loosing a dwarf

build an up/down stairs, then dig an up/down stairs on top of the built stairs, until you're not against warm stone

then dig over the magma

alexanderpas

I wonder, would an artificial hill work to stop the magma, and save the dwarf.

Arda Xi

Hmm, while StrixVaria has a point ("@StrixVaria you should honestly be prohibited from posting answers to StarCraft questions.", that's totally uncalled for) I'm not sure a meta discussion is the way to go

alexanderpas

invite to chat?

Arda Xi

16:56

:109373 The magma is about 33 z-levels high

alexanderpas

:109358 what did you use to create that picture?

Arda Xi

:109401 Paint. I took an existing picture and just copied parts :)

alexanderpas

17:36

From Caravan to Happy Dwarves

2

antony.trupe

:109527 neat

Arda Xi

18:58

Who's Lu Bu

Transcript for