Charcoal HQ

@hichris123 Well... What are you wanting to run? I always use Ubuntu, because it can run just about every open-source thing out there, but if you want to use an MS language, you probably need Windows.

@hichris123 I think they're all in the free tier. The reason for regions is durability, I.e. if a tornado knocks out one datacenter you can have another one on the other side of the world that's probably safe. I know that N. Virginia is on the free tier (and it's cheapest when not on the free tier), so best stick with that.

Undo

1:40 PM

Got bored, so made this:

side-by-side activity history!

hichris123

3:12 PM

@Undo What do you mean by an MS language?

Undo

@hichris123 MSSQL, Visual Basic, that kind of stuff.

hichris123

@Undo Hmm, interesting, so I'll probably just use N. Virginia. It's also closer to where I live then Oregon, so maybe that'll help? I don't know.

Undo

@hichris123 Meh, it's the internet. Distance doesn't really matter much.

hichris123

@Undo Oh, well I don't care about those then. Also it looks like I can use two servers with the free tier, one Linux and one Windows.

Undo

@hichris123 I think you can. Yeah

hichris123

3:14 PM

@Undo Just me or do those look almost the same?

Undo

@hichris123 There's differences, but yeah. Almost the same.

Scary.

hichris123

Heh, then look on the Tavern. Me is curious... :P

in Tavern on the Meta on Meta Stack Overflow Chat, 4 hours ago, by Nick Craver

@hichris123 @Undo I will be working on a data dump some today/tomorrow (just did one, but it needs another run for updated ReadMe, we're including PostLinks now)

Undo

Yay!

Updated data!

hichris123

Yep! :P

Me is going to setup RDS first, then EC2.

Undo

@hichris123 What are you doing with RDS again?

hichris123

3:20 PM

Heh:

> No, this instance is intended for use outside of production or under the RDS Free Usage Tier

Undo

Yup

hichris123

@Undo Whole buncha random stuff. Probably put in some GTFS train stuff, I don't know.

Undo

@hichris123 Ah, ok.

@hichris123 One other thing you can do: If you plan on running a server 24x7 (like mine), you want to buy one of those reserved instances right at the end of the free tier.

Big (comparatively) up-front cost, but like 1/4 the hourly price.

hichris123

Oh, okay.

First thing I'm doing is also going to be a credit when the 12 months are over.

@Undo Oh gosh: docs.aws.amazon.com/AmazonVPC/latest/UserGuide/…

What should I do for my security group?

Undo

@hichris123 Hehe. You don't need to worry about those :P

Just a minute.

hichris123

3:26 PM

Why don't they just make an easy to read tutorial?

Undo

@hichris123 Because it would be too long ;) It'll all make sense eventually.

hichris123

So don't set up a security group?

Undo

@hichris123 Depends on what you're doing with the instance (Sec groups are only for EC2, methinks).

hichris123

So... that means what?

Undo

For EC2 you have to have a sec group. All they are are big firewalls that by default let absolutely nothing in. Your job is to open the ports for inbound traffic depending on what you're doing. MySQL is 3306, Apache is 80 & 443, SSH is 22, etc.

console.aws.amazon.com/ec2/…

hichris123

3:31 PM

Huh. I'll have to look at the docs for RDS.

Undo

RDS doesn't have sec groups, I thought.

hichris123

Read this: docs.aws.amazon.com/AmazonRDS/latest/UserGuide/…

It claims I have to.

Undo

Oooooh, yeah!

I've got to go now, be back in about half an hour.

Cya!

hichris123

Okay, bye!

Undo

3:59 PM

I return

hichris123

@Undo Hello again!

Enable Auto Backups on RDS?

Undo

@hichris123 Only if the data is really important.

hichris123

It isn't really... :P

Undo

Then don't.

hichris123

Why?

Undo

4:02 PM

Because I think it isn't free :P

hichris123

Aha!

> 20 GB of backup storage for free per month

Well, I'm not enabeling it right now anyway.

There's nothing in the database, so no need to have it backup.

Undo

Ah, ok

They've really made the free tier better since a year or so ago.

hichris123

Yeah, I was looking, it looks like they didn't have RDS or some services.

> Your DB instance is being created.

@Undo

I feel like cloud-databases are so cool now!

Undo

4:38 PM

They are :D

hichris123

@Undo I have succesfully connected to my MySQL database!

Undo

Yay!

hichris123

@Undo You know of any tool to import txt files into MySQL?

hichris123

5:04 PM

Oh, @Undo, why do you do like Int(11)'s?

And also, what collation do you use?

Undo

5:40 PM

@hichris123 As in csv's?

hichris123

@Undo I got it, nevermind. dev.mysql.com/doc/refman/5.0/en/loading-tables.html

Undo

@hichris123 I dunno. I'm not really a master of datatypes, anyway.

@hichris123 latin1_swedish_ci

hichris123

Okay, thanks. Just wanted to make sure I wouldn't mess anything up by using that (it's the default for me too).

Undo

Yeah, it should be fine.

@hichris123 Want a copy of the Charcoal db to play around with?

hichris123

@Undo If you want to, I don't care. :P

Undo

5:43 PM

@hichris123 s3.amazonaws.com/erwaysoftware.mysqlbackups/…

hichris123

Thank you!

Undo

np

Got it?

(so that I can turn permissions back to private)

hichris123

Yep. Downloaded and everything.

Undo

Good!

Have fun!

hichris123

Me is/will/will continue. :P

Undo

5:47 PM

SES is kinda neat too - I use it for Charcoal's email-sending-on-register thingy, and for other stuff.

hichris123

SES?

Oh, console.aws.amazon.com/ses/home?region=us-east-1#

Undo

yup

You can configure it to send emails from your domain and everything

hichris123

Wow.

> If you are an Amazon EC2 user, you can get started with Amazon SES for free. You can send 2,000 messages for free each day when you call Amazon SES from an Amazon EC2 instance directly or through AWS Elastic Beanstalk.

Undo

Yup

It's dirt cheap too - to the tune of $0.01 per ten thousand emails.

hichris123

Woah. That is cheap.

Hmm, that would be an easy way for a spammer to spam places...

Undo

5:52 PM

Yeah, except there are... things in place to stop that :P

hichris123

Yay! * Affected rows: 165 Found rows: 0 Warnings: 0 Duration for 1 query: 0.109 sec. */

Undo

Yay!

hichris123

@Undo What emails do you send out with it?

Undo

@hichris123 Emails for one of my apps.

hichris123

Oh, next step will be getting up EC2.

@Undo Ah.

Undo

5:54 PM

EC2 is fun.

You have to play with pem files and stuff, but it's easy once you get that sorted out.

hichris123

pem files?

Undo

SSH keys

hichris123

Ah. I've done that with GitHub.

Undo

Yup

hichris123

So now I have this:

CREATE TABLE `transfers` (
	`from_stop_id` INT(11) NOT NULL,
	`to_stop_id` INT(11) NOT NULL,
	`transfer_type` INT(11) NOT NULL
)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB;

Undo

6:07 PM

Is GitHub slow for you, @hichris123?

hichris123

@Undo

> We have confirmed some reports of GitHub.com being unreachable. We're investigating further.

Undo

Awww

hichris123

status.github.com

Undo

Huh, that's kinda neat

> We have confirmed some reports of GitHub.com being unreachable. We're investigating further.

hichris123

It is neat.

Undo

6:09 PM

Whoa, I think it even has live updates!

hichris123

This too: status.github.com/messages

> This webpage is not available

Yeah, that's when I try to go to GitHub.

Undo

Uh oh

hichris123

Huh?

Undo

> This webpage is not available

Sounds bad :P

hichris123

It's Chrome. What do you expect? :P

Undo

6:11 PM

> Service to GitHub.com appears to be recovering. We'll keep status at yellow until we've confirmed all service is restored.

Yay!

hichris123

Yeah, it's up.

Undo

That was fast.

I bet a few people at GitHub have some adrenaline in their bloodstream :P

hichris123

Heh. Wonder what happened...

Undo

Probably the CHAOS team :P

hichris123

@Undo So can a VARCHAR have numbers in it?

Undo

6:18 PM

Not sure.

I think so.

hichris123

I assume so since it's basically just a String.

hichris123

6:32 PM

Yay! /* Affected rows: 90,970 Found rows: 0 Warnings: 0 Duration for 1 query: 39.655 sec. */

Undo

Charcoal db?

hichris123

Not yet, another one.

I'm too lazy to unzip the gz

Undo

Hehe

hichris123

It's the GTFS I was talking about earlier. That was the trips.csv file.

There's a 300-some MB file, but that's not it.

Undo

Wow

@hichris123 Have a second?

I'm getting a PHP Parse error: syntax error, unexpected '(', expecting ',' or ';' in /var/www/oauth/index.php on line 7 on this file - what am I doing wrong?

hichris123

6:38 PM

Are you trying to do a SE OAuth thingy?

Give me a second to look.

Undo

Yup, trying to

hichris123

Where do you define ClientId()?

Undo

In a config file

I think I figured it out, it didn't like those const's

changed them to normal vars and it works fine now.

hichris123

Huh, weird.

Now let's see if Excel will open my 350 MB csv!

Undo

hehe

If not, just cat it or something

hichris123

6:44 PM

It just said File not loaded completly

@Undo Huh?

Undo

Eh, nevermind.

Yay!

hichris123

Huh?

Does it work?

Undo

I think it might.

@hichris123 Try erwaysoftware.com/oauth

It won't make a token that can be used for writing, flagging, etc. yet.

But it's still a token!

I've got to go, hopefully last time today. Cya

hichris123

Bye. :(

Hmm, how long do you think it'll take to import that 350 MB csv?

:P

Heh, I wonder what I've been doing recently:

My MySQL client just crashed... :P

I'll be back later.

hichris123

8:13 PM

Hiya @Undo!

Undo

8:31 PM

Hiya @hichris123

hichris123

CSV splitters are amazing! @Undo

Undo

yes they are!

@hichris123 Are you into gaming?

hichris123

Well, depends what game...

Undo

Because there's a chance for a diamond

hichris123

Well, You need 300 reputation to nominate yourself.

Undo

8:43 PM

Yes. You have seven days to get 200 rep

hichris123

Oh, this sounds like some TV challenge. You have seven days to go around the world, fight people, and win

Undo

I'm going to get you a diamond somewhere...

We have the power to make huge helpful flag counts!

hichris123

Meh, I don't think I can really get that much rep on Arqade, and besides, would they elect me, who would just create an account today?

Undo

Probably not.

hichris123

That's why I'm playing around with my MySQL tables right now. :P

Undo

8:47 PM

That would be a fun exercise, though - every time an election pops up you work like crazy to get your helpful flag counts and your rep as high as possible at the end of seven days.

hichris123

Hmm, mod gets elected when they've been only there for 7 days, outrage umong the users.

Undo

No - users happy, outrage among CMs :P

hichris123

:P Probably true.

I'm hoping MySQL doesn't choke on my 41 MB csv's...

Undo

It shouldn't.

hichris123

@Undo What version of MySQL are you running for Charcoal?

Undo

8:55 PM

@hichris123 Not sure. Lemme check

hichris123

Heh:

There's a query that's supposed to look for deleted posts that have validated spam flags on them. Instead, it looks for deleted posts that have live spam flags on them. A live (active/unhandled) spam flag on a deleted post is a situation that shouldn't ever be allowed, and as a result generally doesn't exist. — Shog9 21 mins ago

Undo

mysql  Ver 14.14 Distrib 5.5.34, for debian-linux-gnu (x86_64) using readline 6.2

hichris123

5.6.13 for me. :P

Undo

:P

> I have one of the highest helpful flag counts on this site

with 3172 flags

We can beat that, no?

hichris123

Where, Arqade?

Undo

8:58 PM

Yup

hichris123

Is it even in Charcoal?

Undo

Not sure. But we can fix that fast.

hichris123

Oh, yay! * Affected rows: 700,000 Found rows: 0 Warnings: 0 Duration for 1 query: 00:04:57 */

File 1 of 9. :P

Undo

Wow :P

hichris123

No, not in Charcoal.

Undo

9:00 PM

We can fix that.

hichris123

Here: data.stackexchange.com/gaming/query/160967

plz-accepts

Undo

Wow

When the new mods are elected we should flood them.

hichris123

Edit ones: data.stackexchange.com/gaming/query/161035

@Undo We need to get 100/day first.

Undo

Ah, true

@hichris123 Arquade is there now.

hichris123

Yay! :P

Hiya @Schism!

@Undo Check your server...

Undo

9:16 PM

Uh oh, is it borked?

It works for me

hichris123

No, sometimes slow, sometimes not.

Undo

Odd

Undo

9:33 PM

@hichris123 Know anything about Earth Science?

hichris123

@Undo What, mod elections again? I don't know, lemme look at the scope.

Undo

@hichris123 No, brand new site.

area51.stackexchange.com/proposals/36296/…

hichris123

Done.

Yeah, meteorology is what got me. :P

Undo

Yay!

We get you diamond!

(or at least try :P)

hichris123

:P You never give up, do you?

Undo

9:37 PM

Nope!

hichris123

Hinty hinty: area51.stackexchange.com/proposals/58150/…

:P

Undo

@hichris123 But... I don't have an Arduino.

hichris123

Who cares, spam the site and make me look good! :P

Undo

hehe

hichris123

Me thinks you should run for mod on SO. :P

Undo

9:41 PM

Me thinks I would probably lose.

hichris123

Me thinks it's worth a try.

Undo

I suppose...

hichris123

Run with the kill all obsolete comments slogan.

Undo

> Hi, I'm a random guy on the interwebs. I would like to get a diamond so that I can burn comments faster and with less supervision. Plz vote for me!

hichris123

I like it. :P

1767 helpful flags.

Close to 1787, independence year... :P

Undo

9:47 PM

From what I hear, SO won't be needing more mods for quite a while.

hichris123

¿Por que? (Why?)

:P Me likes Spanish sometimes.

Undo

Because they improved the tool so much that they're almost running an excess now, I hear.

But it might be a good idea to ask in the Tavern if you want a real answer.

hichris123

Say that to the tons of other flags. :P

Undo

That was only because of animuson's laziness on one day. We shall soon cure him of that.

hichris123

@ThiefMaster?

Then please accept the answer ;) — ThiefMaster Apr 30 '11 at 8:17

Undo

9:49 PM

D:

Wow, that got deleted fast :P

ThiefMaster

> Y U NO FLAG THE COMMENTS

hichris123

Really?

> declined - Y U NO FLAG THE COMMENTS?

@ThiefMaster Cause Charcoal.

Undo

Last I hear mods don't like post flags for comment deletion requests.

ThiefMaster

^ this

hichris123

I blame you @Undo. :P

ThiefMaster

9:51 PM

1200 active flags on SO, no need for more :P

Undo

@ThiefMaster How's the workload on the SO mods? Do you need more, or are you getting close to the communication barrier of too many people?

hichris123

See why we need more mods?

ThiefMaster

for most stuff communication is not really necessary

Undo

@ThiefMaster So are we getting close to another election, or is that still far away?

hichris123

@ThiefMaster Any whispers about upcoming mod elections?

ThiefMaster

9:54 PM

none i know about

hichris123

@Undo My 9 CSV's are now in MySQL.

@hichris123 Yay!

@Undo ^

hehe

That's from RDS.

5,500,000 ish rows on that table @Undo...

Undo

10:00 PM

Wow

Can you query it and stuff?

hichris123

Yeah, it's kinda slow, though.

Undo

You would need SEDE-style hardware to query it fast.

hichris123

Yeah...

Select *
From stop_times
Where stop_id = 30078 AND trip_id =43065086586;
/* Affected rows: 0  Found rows: 1  Warnings: 0  Duration for 1 query: 10.249 sec. */

Undo

Hmm...

hichris123

Ooh. dev.mysql.com/doc/refman/5.6/en/mysql-indexes.html

Yay! @Undo Indexes are so good.

Now:

Select *
From stop_times
USE INDEX (trip_ids)
Where stop_id = 30078 AND trip_id =43065086586;
/* Affected rows: 0  Found rows: 1  Warnings: 0  Duration for 1 query: 0.203 sec. */

Undo

10:11 PM

Wow.

hichris123

Yeah.

Undo

I should do that to my local comment db copy.

How'd ya do it?

hichris123

10.25 seconds to 0.203 seconds.

ALTER TABLE tablename ADD INDEX indexname (columnname)

Undo

hmm

ok

hichris123

@Undo You mean SO comments?

Undo

10:13 PM

Yup

hichris123

You can do this:

ALTER TABLE tbl_name ADD FULLTEXT index_name (column_list)

Undo

Actually, let's try it with SO's 47 million row votes table

Currently running this:

select COUNT(*), PostId from Votes Group By PostId

Nevermind

hichris123

Why?

Undo

Was taking too long :P

hichris123

What was? Creating the index? :P

Undo

10:16 PM

I've got to go now, cya

hichris123

Bye again!

Undo

:P

Transcript for