Root Access

Bob

3:00 PM

Then click Create new bookmark

select the first message and last message, and you'll end up with a page with every message between (inclusive)

Journeyman Geek

;p

Bob

Alternatively, you can click the hover-arrow to the left of any chat message, click permalink, and bookmark that link in your browser.

allquixotic

@Riju I'm just looking at some GPUs to see what your options are... it looks like the AMD FirePro S10000 is about $3000, which is way out of your price range...

the S9000 is still $2400 or so... still too high

the S7000 is down to about $1000-$1100, but it is probably slower than the equivalent desktop card

honestly I would suggest you just use a AMD Radeon R9 290X or an Nvidia GeForce GTX 980... these are very recent, high-end desktop graphics cards, which cost under $1000, and the equivalent performance on the workstation/server side is going to run you $2k - $3k

DemCodeLines

3:17 PM

Error 809: The network connection between your computer and the VPN server could not be established because the remote server is not responding. This could be because one of the network devices (e.g, firewalls, NAT, routers, etc) between your computer and the remote server is not configured to allow VPN connections. Please contact your Administrator or your service provider to determine which device may be causing the problem.

ouch, any solution out there?

Riju

I underestimated the price of gpgpu's. You know, I could probably convince my boss to spend a little extra and get a Firepro or Tesla around $3K. I looked at this pcmag link for Radeon: http://www.pcmag.com/article2/0%2c2817%2c2453389%2c00.asp?tab=Specs
I wonder why they advertise great gaming experience but nothing about, say, GFLOPS.

allquixotic

@Riju it's called targeted advertising

the average guy playing Call of Duty 4 doesn't care about GFLOPS

they just want their game to run fast

just keep in mind that I personally don't recommend buying a dedicated server-grade GPU like the FirePro S10000 or a Nvidia Tesla, unless you are sure that the GFLOPS of a desktop card won't be sufficient

the FirePro S10000, the highest-end server-grade dedicated GPGPU from AMD, has a SP GFLOPS of 5913.6 and a DP GFLOPS of 1478.4, for $3000; compared to the Radeon R9 290X, with a SP GFLOPS of 5632 and a DP GFLOPS of 704 for $549! I can't understand why you would want to buy the server chip, unless you are planning on putting it ... IN a server

Riju

@allquixotic Got it.

allquixotic

or unless you really need double precision GFLOPS

Riju

nope. I can start simple.

allquixotic

3:30 PM

heck, you could buy two Radeon R9 290X (or a single R9 295X, which is two GPUs on one card) for around your original price of $1000; and get way more SP GFLOPS than a FirePro S10000

Bob

There's a cat outside my window... I think

Riju

@allquixotic Makes perfect sense. I wasn't looking at the right place for the specs of Radeon

allquixotic

the main advantage of the FirePro cards is that they have more DP GFLOPS, and also, their cooling solution and size profile is optimized for server chassis (like 1U, 2U, etc) rather than desktop chassis

if you're putting this card in a desktop chassis, definitely having a dual slot GPU with a fan should be fine

but if your desktop is very old you might need a new PSU and/or motherboard and/or CPU

(if you upgrade the motherboard, you probably have to upgrade the CPU too)

Bob

You'll also be putting out a lot of heat. Make sure you have a good case and plenty of clearance around it.

Riju

No, I am going to get a new desktop for this project too.

allquixotic

3:32 PM

I don't know about your workplace, but at my workplace, they really scrimp and save on the quality of power supplies that come with "typical business desktops"

the reason is that it's very rare for the power consumption of a typical business desktop to increase much beyond what it ships with

HackToHell

Bleh the idea behind gradle is nice but it really really sucks if you have a slow internet

allquixotic

so why waste money on a higher quality PSU

Riju

hehe...it sounds about the same in my workplace

allquixotic

so trying to stick a R9 290X (or two) in, say, a HP Compaq 8200 Elite Compact Minitower, is a good way to blow out your PSU

HackToHell

Bleh it's downloading every single java library under the sun :|

Riju

3:34 PM

I will keep that in mind.

allquixotic

also, once you get to the point where you're deciding whether to buy red or green colored ice cream (AMD vs Nvidia), you should probably talk to @JourneymanGeek to get a balanced viewpoint

I'm very much a red-flavored ice cream kinda guy

my desktop at home has two Radeon R9 280X in it

Riju

'Good PSU for fast computing'

allquixotic

@JourneymanGeek is a green flavored ice cream kinda guy

HackToHell

Boo nvidia ;p

allquixotic

@Riju for one GPU, I recommend at least 600 Watt PSU. for two or more GPUs, I recommend at least 1000 Watt PSU

HackToHell

3:34 PM

AMD ftw

(cause they are cheaper :P )

allquixotic

@HackToHell yes, they are cheaper, but interestingly, the Maxwell/2 GTX 980 has the highest GFLOPS per Watt (so, a long tail of slightly lower power consumption for the same workload) of any GPU released to date... the R9 290X is at 19.4 GFLOPS/W, while the GTX 980 is at 28 GFLOPS/w

so if you had it for 5 years and maxed it out 24/7/365 (as in bitcoin mining) you'd save significant electricity using the GTX 980

and/or mine more bitcoins for the same amount of electricity

HackToHell

But isn't bitcoin mining extremely inefficient in nvidia cards ?

allquixotic

@HackToHell @Bob thought it was Nvidia that was better at integer GPGPU than AMD, but the general wisdom has been (at least historically, no clue about Maxwell/2) that AMD is better, yes.

and AFAIK there's no floating point in SHA

HackToHell

@allquixotic Nope, just int stuff

Someone did the algorithm on paper

It was quite simple :P

allquixotic

@Riju as for your new desktop, I think things are pretty simple: if you need to run certain non-parallelizable algorithms on a serial processor, definitely consider a Haswell or Devil's Canyon era Core i7 (so, i7-4xxx, as much as you can afford), and fast DDR3 memory -- clock speed is important too

@HackToHell OH YEA I linked to that lolz why don't I remember that

allq says X; doesn't remember X; asks someone else about X; someone else points out allq's earlier thing he said

!!meme okay

ChatBot John Cavil

3:42 PM

HackToHell

LOL

This Apache Software Foundation must receive a lot of funding, they have built a lot of stuff :O

allquixotic

@HackToHell the ASF is more of a hosting company; you don't have ASF employees who are paid to write all that code... there might be a few, but nowhere near as many as there are contributors who just contribute on the dime of their company's payroll or in their spare time

HackToHell

@allquixotic So basically ASF is an idea factory ?

allquixotic

the ASF's goals include making sure that their stuff continues to remain open source; that it's properly hosted and mirrored and secure (i.e. that unauthorized people can't inject binaries into the FTP download stream); that there's enough money and servers for this to continue existing; and to provide a general direction for what kinds of projects can be hosted there

HackToHell

And people are forced to contribute to it cause there's nothing better out there ?

allquixotic

3:56 PM

@HackToHell the actual developer-contributors of the code do so because they either believe in the ASF's ideals, and/or they need ASF's code for their day job (their company might use it in production, for example), so they want to make it better, and their company is willing to pay their salary to work part-time or full-time on that project

people.apache.org

> the ASF is the sum of its committers.

there have also been multiple instances of open source projects, whose original developers didn't have time or money to continue hosting the code / website, so they donate the project to Apache; and as long as Apache agrees with the license and thinks the code is at least okay, they'll take it

like this

HackToHell

Oh, I suppose that's how it gets a lot of projects.

allquixotic

as part of giving the code to Apache, Apache hosts the version control for it (SVN/Git/Mercurial/CVS), the bug tracker, the mailing lists, and the website

it's almost like Google Code, except that you can't just create a new Apache project on your own; they have to review it

so generally projects that suck don't make it to Apache, or sit in their "Incubator"

Riju

@allquixotic , @Bob , Thanks for all the help with my gpu-cpu confusion! I think I have enough good information to get started on the project. I will be back to let you know how things went in a few weeks I suppose :)

allquixotic

it's actually very similar to how the GNU Project works, except that the GNU Project requires copyright attribution to GNU

ASF projects' copyright is held by the committers; so if there are 10 committers with different bits of code in an ASF project, each of them has a partial copyright on the source file(s) they've modified

with the GNU project, GNU has copyright over all the code, and their executives can change the license at will without contributors' permission

(which is good if you believe the GPLv3 is better than the GPLv2, since they changed a lot of their licenses to GPLv3 when it was released)

if the ASF ever released an Apache License version 3.0, and it was better than 2.0, 99% of ASF projects could never get updated to version 3.0, because the probability of getting all committers to agree on a new license is quite low (as evidenced by proposals to change the Linux kernel to GPLv3)

you're always going to have that one guy who steadfastly believes the old license is better

HackToHell

Ah the super free GPLv3

allquixotic

4:05 PM

so you either have to nuke all of his commits and then upgrade to the new license (and clean-room reimplement anything important if you need it) or suffer with the old license

HackToHell

Wasn't it written to cover a loop hole in GPLv2

allquixotic

...which can be nearly impossible if they are a major contributor

@HackToHell several loopholes; and also to clarify the language

HackToHell

GPLv2 is the reason my phone has Cyanogenmod

If it were v3 I would have had Video Recording :/

allquixotic

if what were v3?

HackToHell

That's what some guy at xda told

Apparently instead of adding the Camera's interface code to the kernel they made it into a binary blob or something.

And added it to the Android Source Tree

allquixotic

4:09 PM

binary blobs are illegal even when linked to the kernel under the GPLv2 license

there are some people who (wrongly) believe that binary blobs are fine with GPLv2, but they aren't

there are unfortunately enough people (and companies) who believe that the GPLv2 wouldn't stand up in court, that they ignore the fact that they are basically violating the GPLv2 by shipping blobs in the kernel

basically AMD and Nvidia's binary blobs violate the GPLv2, as do binary WiFi drivers, binary camera drivers, etc

HackToHell

And no one does anything about it :(

allquixotic

Linux-libre

Linux-libre (/ˈlɪnəks ˈliːbrə/) is an operating system kernel and a GNU package that is maintained from modified versions of the Linux kernel. The aim of the project is to remove any software that does not include its source code, has its source code obfuscated or released under proprietary licenses from the Linux kernel. The parts that have no source code are called binary blobs and are generally proprietary firmware which, while generally redistributable, do not give the user the freedom to modify or study them. Linux-libre is a prominent example of free software. == History == Linux started...

HackToHell

@allquixotic Yeah but getting anything working in it is a pain in the ass

You will have to use the proprietary blobs at some point

if it's a well know chip someone might have a foss reverse engineered version of the driver

But that's rare :S

allquixotic

linux.slashdot.org/story/12/10/11/1918251/…

HackToHell

And since it's highly unlikely nvidia will give in, it's us the end users with poor performance :(

allquixotic

4:18 PM

that's Torvalds giving the finger to Nvidia

phi

When is Windows 10 available via MSDN?

allquixotic

@phi ask @Sathya; he's the CEO of Microsoft

!!wiki Satya_Nadella

ChatBot John Cavil

Satya Nadella

Satya Nadella is an Indian-American business executive. He is the current Chief Executive Officer of Microsoft. He was appointed as CEO on 4 February 2014, succeeding Steve Ballmer. Before becoming CEO of Microsoft, he was Executive Vice President of Microsoft's Cloud and Enterprise group, responsible for building and running the company's Computing Platforms, Developer Tools and Cloud Computing Services. == Early life == Satya Nadella was born in Hyderabad, Telangana, India in a Telugu family from Anantapur district in Andhra Pradesh, India. His father was a civil servant in the Indian A...

Bob

@phi The tech preview is already available at preview.windows.com

allquixotic

he drops in from time to time to be a moderator on SupserUser

Bob

4:21 PM

Consumer preview and RTM are still at least a year away

allquixotic

@Bob which means it's 5+ years away from getting it at work :/

HackToHell

Will there be a Win 10 contest like the Win 8 contest ?

Hopefully I can win stuff this time

@allquixotic :D

allquixotic

@HackToHell Yes, and they're giving away an exaflop IBM Mainframe as the grand prize

128x 6-core CPUs @ 5.5 GHz

HackToHell

@allquixotic Consumes too much power, I'll settle for a Mac Book Pro :P

phi

@Bob thanks. I went there, and I thought I had to create another account, but my existing msdn account also worked.

Bob

4:25 PM

@HackToHell !!no

HackToHell

@Bob or an ultrabook ;p

allquixotic

!!learn torvalds '<>http://i.stack.imgur.com/gdmU9.png'

ChatBot John Cavil

@allquixotic Command torvalds learned

Bob

@HackToHell wait for the consumer preview

HackToHell

I am trying to write a prototype for a backend that will capable of handling 1-2k simultaneous requests .

allquixotic

4:27 PM

seriously, Quora?

HackToHell

And I have no idea what to use

allquixotic

out of all the interesting questions you could have emailed me in the weekly digest...

quora.com/…

HackToHell

@Bob okie

allquixotic

you send me that?

Bob

@allquixotic eww

traitor!

@HackToHell multiple endpoints.

and stay the fuck away from php

@allquixotic ...it's like workplace.se, but worse

HackToHell

4:30 PM

@Bob Does that mean multiple servers with a load balancer ?

I could throw the entire thing at AWS or something

But no foreign servers :?

@Bob I'm thinking Go

Java is too slow

DemCodeLines

Windows 10 Technical Preview download page says:

We're not kidding about the expert thing. So if you think BIOS is a new plant-based fuel, Tech Preview may not be right for you.

...

allquixotic

@Bob I don't participate, just read, for shock value :P

@HackToHell HTTP? or?

HackToHell

@allquixotic HTTP

allquixotic

@HackToHell ...use nginx + cloudflare?

won't really matter what the backend is then unless you have to serve a separate page for each user

Bob

23 hours ago, by Bob

HackToHell

4:34 PM

@allquixotic Well it collects data from all the requests and then it has to store them somehwere

allquixotic

@HackToHell oh like a webservice?

HackToHell

yeah

I want it to be scalable from the ground up

allquixotic

well you could still use nginx as your frontend, with FastCGI serving ~~bash~~ your favorite language (even C/C++ if you wanted) on the backend

HackToHell

I have 40+ tabs open and I'm going no where

@allquixotic Aren't there any uhm frame works or such that make this easier

allquixotic

@HackToHell FastCGI is your framework

Bob

4:36 PM

step 1: have load balancer

step 2: store your data in a DB that's good with high concurrency

allquixotic

oh hey, it's @ThatStrangerGuy who never comes around here anymore!

are you here to !!hv, @ThatBrazilianGuy? or no? (anyway, hi!)

Bob

step 3: spin up more servers as necessary and tell the load balancer to spread across them

allquixotic

mmmmm AWS spot instances

HackToHell

@Bob no AWS :(

That Brazilian Guy

@allquixotic HI I NEED HELP HOW DO I MAKE DAYS HAVE 48HRS!!!111

HackToHell

4:38 PM

@allquixotic Reading about it

allquixotic

@ThatBrazilianGuy just change the gravitational constant

Bob

@allquixotic FCGI (or any other backend) on a single server probably won't be enough to serve >1k r/s

That Brazilian Guy

@allquixotic No, I'm here because I miss you guys (and I have near-zero social life). .____.

@allquixotic Someone's been playing too much Kerbal Space Program? ;P

allquixotic

@Bob really? that isn't all that many I thought. I mean with GbE you could probably do it, depending on the data volume

Bob

laptop manufs pls stop putting 4 GB RAM in a laptop with an i7 kthxbai

@allquixotic you're more likely to be limited by CPU & RAM

That Brazilian Guy

4:39 PM

(don't even know if one can change the gravitational constant it that game, just had to say it)

allquixotic

@Bob IKR? if they'd stop offering models with that low RAM, then my company couldn't buy them :D

Bob

I'm talking requests with real processing involved, not static files

HackToHell

wow, this is getting more complex that I imagined it to be

allquixotic

well if there's lots of DB ops that have to be done per request, you'll need something like Postgres at a minimum, on a dedicated (separate) box

most of the CPU should hopefully be offloaded to the DB

Bob

@allquixotic depends what you're actually doing, I guess

allquixotic

4:41 PM

wellya

HackToHell

It basically collects data from 1-2k hosts and stores them indexed

Bob

anyway, 1k r/s is very high already

allquixotic

the raw I/O of 1000 krs is probably not bad enough to require more than two (large) VPSes, like two EC2 xlarges

Bob

I don't think you even need to consider that, yet

just make it scalable

allquixotic

but throw in processing and you'll need several frontends and maybe several backends

depends on the details tho. SE serves probably 1krs (likely more...) on a handful of high-end boxen (dual proc, 176 GB RAM or so)

Bob

4:43 PM

@HackToHell it's not 1000 simultaneous requests per second, right?

if it's just bursts, say once per minute, that's perfectly manageable

allquixotic

yeah

sustained 1krs vs burst

HackToHell

@Bob It is.

allquixotic

@HackToHell sustained?

Bob

@HackToHell what on earth are you doing?

HackToHell

Though I could make it discrete

allquixotic

4:44 PM

@Bob :O he's a spammer!!! kill him with fire!

3

Bob

> collects data from 1-2k hosts

???

HackToHell

Considerably reducing the concurrent requests

Bob

XY problem. why do you need this?

there's probably a far better solution

like, yknow, not sending 1k requests per second

allquixotic

give us the elevator pitch

HackToHell

@Bob Uhm trying to implement a device that basically collects data from vehicles

allquixotic

4:45 PM

what data?

HackToHell

And then stores their movement history,accelerometer data

Bob

@HackToHell and they're transmitting every second?

allquixotic

@Bob I think he just wants to design it so it can handle potentially millions of vehicles, in case he manages to win a huge contract with Chevrolet or something :p

Bob

...also, why would you do this over HTTP? unnecessary overhead

HackToHell

@Bob That's probably a bad idea

@allquixotic Uhm no ?

allquixotic

4:47 PM

it's probably millions of vehicles sending a request once per 5 minutes or something, but who knows

HackToHell

@Bob What's better ?

Googling turns up a lot of protocols o0

Bob

@allquixotic yea, great, now stop asking for advice in a chatroom and hire someone to manage the infrastructure you'll need to handle something of that magnitude -_-

allquixotic

@HackToHell you could write your own, or use google protobuf or something, your call depending on your needs

Bob

millions of vehicles => who the hell is paying you?

allquixotic

@HackToHell so why do you think it needs to accept 1k requests per second, sustained? are you planning to own 1,000 vehicles yourself?

HackToHell

4:49 PM

@allquixotic no, basically I want to write something that is capable of handing such requests ;p

Bob

@HackToHell pretty much send the data over TCP in whatever protocol you want/can make up

HTTP has far too much overhead

HackToHell

For the prototype I am going to show them with about 100 virtual devices connected

Real ones are too expensive ;p

Bob

especially if you're sending just an id, location and accel. that should fit under 128 bytes.

response, if any, should be even smaller.

HackToHell

@Bob Okay, is UDP a bad idea ?

Bob

though ideally you'd only send a burst every minute or something

allquixotic

4:51 PM

@Bob developers.google.com/protocol-buffers

Bob

sending every second wastes power as well

allquixotic

protobuf is designed to be very low overhead

HackToHell

Or I could dynamically set the burst rate based on the speed of the bus

Bob

@allquixotic yes, I know about protobuf :P

I was looking at protobuf-net for serialisation at one point

allquixotic

@HackToHell I'd recommend TCP, since you want every transmission to succeed if it can, and there's no better algorithm for guaranteed delivery than TCP

Bob

4:52 PM

decided on json to make errors more recoverable (autosave, possible corruption)

allquixotic

I'd do protobuf over TCP, personally

Bob

there should be minimal processing involved, right?

accept data, fire off a database insert, NEXT, etc.?

allquixotic

just make the TCP socket fault-tolerant, since cars on the road will be connected via 2G/3G/4G, which is pretty unreliable, so failures will need to result in the client simply restarting the socket and trying again

HackToHell

protobuf is for serializing data, since I'll be sending just 3 data elements(lat,long,accel) is it really needed ?

@Bob That's the plan yes

allquixotic

@HackToHell it takes care of separating and distinguishing data from one another, so I'd say yes

Bob

4:54 PM

@HackToHell you forgot id :P

allquixotic

it's very low overhead for what it does, and it eliminates things like dealing with variable length strings in raw structures

HackToHell

Hmm it would be a good idea as it will be easier to add more fields later

@Bob Oh noes

allquixotic

that too

Bob

and you really should be sending this data as some numeric format, not strings

allquixotic

@Bob depends on whether the id is numeric or a GUID or what

HackToHell

4:55 PM

With some kind of error control mechanism perhaps

Bob

@allquixotic I meant loc and accel

should fit within three 32-bit floats

allquixotic

well yeah, but even with "numeric" data formats, you'd have to manually take care of things like host to network order if you don't use protobuf

protobuf's just a good idea imo... make it too low level and maintenance becomes a headache

we're not talking about overhead on the scale of HTTP, here

Bob

@allquixotic ya, I'm just reminding him not to transmit strings, protobuf or not :P

allquixotic

the biggest overhead will be TCP, actually

but it's good overhead (generally speaking)

Bob

hm

UDP would be doable, but more work to manage the resend

allquixotic

4:57 PM

or you could do unreliable UDP and just not care if the datagrams don't land

HackToHell

@Bob Okay all the values are in numbers so it's not problem

allquixotic

"packet didn't get through the cellular baseband? IDFC"

HackToHell

But it's over a GSM network and it's pretty unreliable

allquixotic

sucks for links with high packet loss tho

at least with TCP you get resends, etc. automatically implemented in an algorithmically sound manner

if you're going to even attempt to do resends, just use TCP

if you don't care if a packet is dropped, use UDP

you can tweak the TCP parameters to give it a lower timeout, or clean out packets if you get a queue of sends that start to stack up because the network layer is dropping your packets

HackToHell

So I could use Protobuf+TCP(C++ (?) program) to send data

@allquixotic I'll keep that in mind until we get a working hardware prototype

allquixotic

4:59 PM

protobuf's available in all kinds of languages, but your client language of choice will depend on what kind of device the client is... is it a chip on the car's CAN bus? that'll have very different operating environment than, say, an Arduino or a laptop

Transcript for