Fake Programmers - 2010-09-17 (page 1 of 3)

« first day next day → last day (2401 days later) »

2:27 AM

@Casebash I changed the name, thanks for the suggestion;

@Sathya I also made you an owner of the room for when I'm not here

JNK

what sort of scripting are you doing @KronoS

It's just basic C... I'm taking a college course on it

JNK

ahhhhhhhhh

JNK

I know vbscript pretty well now

so simple "Calculate the root of a quad equation"

JNK

2:37 AM

learned in the last month or two

JNK

and will be doing powershell soon

JNK

gotcha

JNK

i hear "scripting" i think vbs/powershell/bash etc

JNK

python/perl

That's pretty cool. C is the only language that I'm "learning" and I've dabbled in the MATLAB which is a derivative of C

ya... I want to get into those, but need to learn the basic for now

JNK

2:39 AM

vbs is pretty awesome

JNK

and super easy

JNK

i knew zero 2 months ago

JNK

i made a script at work for transferring very large data sets

JNK

it makes an array of your source folder, breaks it down into XX GB size blocks (you pass a parameter) and creates batch files for the xfer of each block

JNK

then you can launch each batch from a different terminal session or w/e

2:41 AM

wow... that's intense

JNK

it was pretty easy after a bit

JNK

i borrowed alot of pieces

JNK

the hard part was making the batch files

JNK

cause it took like 3 nested loops

whoa...

JNK

2:42 AM

vbs integrates pretty completely with windows

JNK

and all you need is notepad :)

JNK

id recommend looking into it, you can do a lot of cool stuff

JNK

ok im off to bed

JNK

i have a 7 mo old who will be waking me up at 5

Ya that's my big beef right now, I'm using a Windows script writing program, but then have to upload it onto the school linux server to compile it and see if the syntax is correct.

JNK

2:43 AM

yeah no compiling is huge

JNK

also no debugger

JNK

but you can get notepad++ which has a lang template for it

I even tried using VM Ubuntu, but the compiler is different and I get errors within ubuntu that doesn't come up on the Solaris School Server

JNK

so it knows basic commands/syntax and can do highlighting

JNK

i wouldnt want to fool with compilers

2:44 AM

I actually use Crimson Editor which has the different language syntaxs

JNK

powershell/vbs are both compilerless

JNK

nice

JNK

ok have a GN man

MATLAB is the same... you just run it, it's pretty sweet

later... i should get some HW done

JNK

go google a bunch of answers

JNK

2:45 AM

;)

:P for sure

8 hours later…

10:25 AM

does this include Python?

fine by me, means I can ask Q's bout it as well :)

sure ;)

I feel my question would be fine for SO, but I don't know what my problem exactly is, so how would I ask it? :(

heh, I hate it when that happens

maybe asking a not-too-vague Q over there might get them to help you fix on the problem?

it's what you get when your a fake programmer

yeah, I might try it

went well last week, hehe

didn't it @ChrisF ;)

my question is the only non-list question on the Python list

yeah, spotted that from a link somewhere on here yesterday, I think

didn't it net you > 100 votes?

10:37 AM

The current room title is good. :-)

149

199

Q: Peak detection in a 2D array

I'm helping a veterinary clinic measuring pressure under a dogs paw. I use Python for my data analysis and now I'm stuck trying to divide the paws into (anatomical) subregions. I made a 2D array of each paw, that consists of the maximal values for each sensor that has been loaded by the paw over...

python image-processing

that was it

very nice

but I have one large script that just works for one measurement

so I tried refactoring it into functions

I think it worked, but I get random crappy errors when I try to import it

and I need it to run over several files in several folders

I managed to work my way around it

but now if I run it, it would overwrite my results all the time

so I need to make a class for each dog, that has class for all the measurements, that has a class for all the paws

but I have no idea where to start

so you need to ask SO about the class design? or more on how to refactor it?

well I guess both

though if I design a class, I could stuff in all the functions that apply to the class

which solves my refactoring

10:46 AM

I see funny member names coming up... ^^

this is strange :S I just made a function out of a piece of code. Now it starts nagging about the indentation

you've not got tabs/spaces mixed?

very strange the editor is messing things up too

I think I know what the problem is

I started writing the script in Notepad++ so I could have it open next to my regular script

and copy pasted

but it turned out to be a txt file :S

so I just changed it

and now, he's not liking anything coming out of that file

odd

it's not ended up in Unix format or something daft with the wrong CR/LF line ends?

I fear SOMETHINGS wrong

10:56 AM

lol

so I'm just going to type it over by hand

making sure it's correct

well, good luck

I've got some work to do I suppose, so I'll BBL

bye :)

2 hours later…

1:17 PM

how hard is it to create a sqlite database in python?

:7402 That depends... are you using jQuery?

;-)

nope, we're not on SO here, so jQuery doesn't solve all my problems

7

Q: Using SQLite in a Python program

I have created a Python module that creates and populates several SQLite tables. Now, I want to use it in a program but I don't really know how to call it properly. All the tutorials I've found are essentially "inline", i.e. they walk through using SQLite in a linear fashion rather than how to ac...

python exception sqlite

:7406 Bah, forgot about that!

user image

just to remind you

I'm flying

high...

Yes. I still love that one :-)

1:19 PM

and I can't see no jQuery from up here

:7404 jQuery isn't everything

:P

chant after me

mongoDB is webscale mongoDB is webscale mongoDB is webscale mongoDB is webscale

LOL

well I just want to ditch all my data in A database

:7415 I know jQuery is overrated -- I'm a Prototype.js guy myself XD

sadly, I don't even have my data arranged properly

but if I know what my database wants me to do, I might have something to aim for

1:23 PM

:7423 Hmmm. In my experience the database waits to know what you want it to do :-)

You'll need to plan out how you want your data broken into tables and relationships between tables

I know, but I need to start assigning tables to drop it in

exactly

no bloody idea :P

I have to run to a meeting but I can help when I return. But I'm sure my knowledge pales in comparison to @Sathya's :-)

hehe, well good thing he's around

have fun in the meeting

drink lot's of coffee and look ignorant

HA! great plan!!

bbs :-)

1:26 PM

can't watch it here sadly

what's it called?

nice spoof on the noSQL buzzwords being thrown about

@IvoFlipse what sort of data are you getting

I have measurements of 35 dogs, 24 measurements per dog, each measurement is a 3D array from which I'll extract n number of contacts

I should stuff some of these things into a class, because left, right, front, back, they're all just versions of a contact

but I have no idea where to start

2:28 PM

Ok @Ivo, I'm back... let's start with the easy stuff. If I'm being too over-simplified, let me know :-)

You'll probably want to start with a table for the dogs

(and I'm a MySQL guy so my syntax might not be perfect sqlite :-)

CREATE TABLE `dogs` (
`id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
`name` VARCHAR(128),

and any other columns you need to store about the animals, like owner_name, client_number, etc

You'll then probably want a measurements table...

CREATE TABLE `dog_measurements` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY_KEY,
`dog_id` INT UNSIGNED NOT_NULL,
`created_at` DATETIME NOT NULL,
-- some other columns...

I'll need some more info to help with the actual 3D data...

@Sathya ROFLMAO That's great!!!

> If you are stupid enough to totally ignore durability just to get benchmarks I suggest you pipe your data to /dev/null -- it will be very fast!

JNK

2:49 PM

oo are we talking databases?

:7521 Yeah, @Ivo needs some help designing one

in Fake Programmers on Super User Chat, Sep 17 at 13:22, by Ivo Flipse

but if I know what my database wants me to do, I might have something to aim for

JNK

is he using SQL?

But he's sleeping on the job ;-) @Ivo! Wake up! Stop playing frisbee with your dogs!!!! ;-)

:7526 I think python+sqlite

JNK

hmmmm i only use SQL Server but I would think it's similar

JNK

I like your normalization scheme

JNK

2:53 PM

assuming his 3d data is measured in 3 axes, yo ucould have an x, y, z field in your measurements table

:7530 Thanks. I agree about the 3d thing but I was unsure what he meant

JNK

dont know either

I wasn't sure if he meant "3D" as in, three relationships, thus three tables

or 3d as in X, Y, Z; one table

or 3d as on, 3 dogs for him to measure == no time to chat

JNK

its pressure data i think

JNK

from their feet or something

2:55 PM

Yeah I saw some (really cool) image

JNK

he posted it on SO last week or 2 weeks ago

so I think you're right @JKK

JNK

199

Q: Peak detection in a 2D array

I'm helping a veterinary clinic measuring pressure under a dogs paw. I use Python for my data analysis and now I'm stuck trying to divide the paws into (anatomical) subregions. I made a 2D array of each paw, that consists of the maximal values for each sensor that has been loaded by the paw over...

python image-processing

there it is

JNK

6

Q: How to write a multidimensional array to a text file?

In another question, other users offered some help if I could supply the array I was having trouble with. However, I even fail at a basic I/O task, such as writing an array to a file. Can anyone explain what kind of loop I would need to write a 4x11x14 numpy array to file? This array consist of...

python file-io numpy

3:00 PM

I gotta get back to work... @Ivo if you get back and want help, just @ me

I was indeed sleeping on the job @Josh, hard to work at home with a GF nagging for attention

I thought to use sqlite, because well it's light!

JNK

so what kind of data will you have Ivo?

3D as in n number of 2D arrays, so XYZ

:7546 LOL!

JNK

so could you store it in 3 different fields per measurement?

JNK

3:04 PM

with x, y, z

uhmmm well it's a numpy.array, so it's just one object

dogs, measurements

That's how I would do it

I didn't really know how to push it to the database

dogs.id, dogs.name ( :P )

JNK

3:06 PM

yeah you want at least those 2 tables

measurements.id, measurements.dogid, measurements.x, measurements.y, measurements.z

I feel like I'm missing your inside joke @Sathya :P

dogid is the fk to dogs.id

JNK

@sathya has it there

:7561 hhaha :D no inside jokes there.

3:07 PM

well from the code I saw so far, I'm not confident that you can assign values to a sql table by just using dogs.id :P

:7565 It's a relationship

the dogs table contains info on all the dogs. There's a "one to many" relationship between dogs and their measurements

so each dog has many measurements

each dog has an ID, measurement has an ID, and a dog ID

so to find the dog called "buster", you SELECT * FROM dogs WHERE name='BUSTER'

JNK

Then you can keep adding measurements for dogs

JNK

I would also add an identity column to measurements

JNK

just in case

that gives you, eg id: 4, name: buster, owner: JNK, age: 9

you can then SELECT * FROM measurements WHERE dog_id=4

3:10 PM

they have a name, which corresponds to a special condition yeah, so a measure.id would be useful

and that gives you all Buster's measurements

JNK

also a measurement.datetime

JNK

is something you probably want to capture

:7586 Yes

I called it created_at because I like Ruby conventions, but

taken_at works also

so I can check if I have newer versions?

3:12 PM

:7591 Or so you know what date the measurement was taken, like is it a day or a month or a year old?

I downloaded a 'viewer' so I can see what the sqlite database looks like

JNK

@IvoFlipse yes so you can sort by time

Think modification dates on files on your computer

but the relations between tables, where does it store that?

:7596 It doesn't, your code maintains that

JNK

3:12 PM

that way if you REMEASURE a dog, you can compare previous to new

ok

JNK

and pick which ones you want

JNK

or see if they are the same over time, etc

then I can just create the fields in the table like you just suggested

:7596 The convention is: each table has an id column

and each relationship has a *othertable*_id column

so each dog has an id

each measurement has an id and a dog_id

JNK

3:13 PM

Normally in a complex arrangement, your "Parent" table refers to child tables

awesome. google search for image upload and all 40 pages are blocked :-\

(This is the convention I follow anyway. Again, it's what Ruby on Rails does, but it makes perfect sense to me)

JNK

yeah @IvoFlipse thats normalization for you

:7613 yup.

Relational model

The relational model for database management is a database model based on first-order predicate logic, first formulated and proposed in 1969 by E.F. Codd. [[File:Relational Model 2.jpg|thumb|280px|Diagram of an example database according to the Relational model.]] Overview Its central idea was to describe a database as a collection of predicates over a finite set of predicate variables, describing constraints on the possible values and combinations of values. The content of the database at any given time is a finite (logical) model of the database, i.e. a set of relations, one per pre...

user image

The Wikipedia article isn't exactly a friendly intro though :-)

3:16 PM

lh3.googleusercontent.com/…

user image

put a "!" in front of it if it doesn't end in .jpg, .gif, etc.

3

thanks @balpha

:7621 Awesome!!

google image links are often just way too long @balpha

More tips straight from the Source :-)

3:18 PM

:7624 It's not length of the link; it's how you post it.

JNK

lol

( and yes Comic Sans was intentional :p )

:7621 metapedia.com/wiki/images/The-matrix-wwwdan-dareorg.jpg

I know, but if I search for images, it will give this crazy long link, because it's a search query @balpha. So chat says: too long!

:7628 9/10 dogs prefer Comic Sans...

3:19 PM

I think @IvoFlipse didn't get my joke :(

:7632 HAHAHA

JNK

@balpha he is scandanavian

I didn't at first either. I am slllooow today!

JNK

they have different idioms

Olivier Lalonde

hey

3:19 PM

:7637 Isn't that one pretty universal for all males? ;-)

Olivier Lalonde

just noticed this new feature

yeah I understand you were being a link @balpha

2

replace that with your own metaphor :P

JNK

lol @IvoFlipse

:)

:7641 Chat you mean?

3:20 PM

since when is Holland Scandinavia?

for starters, it's "Scandinavia"

JNK

@IvoFlipse sorry he's DUTCH

:7648 Since today, didn't you know? :-)

welcome @OlivierLalonde

JNK

is it north of france?

3:21 PM

I want to visit Holland someday

edit makes your remark much less fun @balpha

that's Belgium your making fun of @JNK

:7639 hi there

oh, post my link, ivo

JNK

I was just going to say everything north of france has about the same impact on my life

cheater

JNK

3:22 PM

im american, do you expect me to know the difference

nope, I can't even name 20 states

though I'm going to have a very good look at some in 2 weeks!

JNK

coming to the us?

yup, road trip through California and some neighboring states

JNK

fun

JNK

hello @MarcGravell

3:24 PM

hey @marc, we'll have to blacklist the word "link"

@IvoFlipse turned it into a bad word

Hey there. Just doing the rounds while my 4y/o does his homework. Won't be here long.

:7667 Sweet. Too bad you're on the icky coast. If you were on the real American coast I'd buy you some good Southern BBQ!

Can't we just blacklist @ivo instead ?

:7677 Homework for a four year old??

hehe, I'll remind that for my next visit

3:25 PM

What kinda homework does he have @Marc???

watch it, I'm visiting @Jeff on my way @MarcGravell

Sorry . Five now, but did hw when four too.

Reading, spelling, writing, math.

protobuf-net bugfixing.

That's next year @balpha :-)

He'll be answering your SO questions in three years...

Mini Gravell ♦ (12K)

Dammit Evan! You left-shift 3 to get the field id. And remember your endianness!

No play time until you can do variable length encoding...

3:29 PM

HAHAHA

Speaking of which

@Ivo, don't let @MarcGravell distract you from your homework!

;-) Did the Relational stuff make sense?

Because strangely enough I can't find a good, basic tutorial on the web

well that was my problem too

they all expect me to know like... everything?

what are the sqlite commands? (ok, ok, ill go over to their website)

:7719 I know MySQL commands, which should be similar enough to help you find their sqlite equivs

the most basic is SELECT

used for getting data. you SELECT something FROM somewhere WHERE some condition is true

:7719 SELECT, INSERT, UPDATE

well I would just want to know if I can dump my 3D array, so I know what happens if I do so

sulks at being beaten by @Josh

3:33 PM

so SELECT * FROM measurements WHERE dog_id=4 would give you all measurements for dog ID 4

:7725 Nah, you gave 3 commands, I'm still stuck at the first one.

:7725 Besides, don't sulk, go help @MarcGravell 's son -- I hear his next lesson is SQL JOINs ;-)

JNK

he wants to insert

:7728 haha :)

JNK

so you need to make your tables first

@IvoFlipse

Step 1: create tables

:7731 Ok, well... yes... that's a good point ;-)

3:35 PM

well I just made a bunch of tables @JNK

step 2: dump data

into tables

the files are csv ? I'd imagine you can import into them

well unless numpy arrays are csv :P

I guess it doesn't just take an object and inserts it?

JNK

@IvoFlipse are you still on your link?

guess it will be something like this

pastebin.com/sFWhrSM9

it's that values part that confuses me @Sathya

3:44 PM

Well this is starting to get python/sqlite specific, but if it's like MySQL, it would be like:

('First Dogs Name','First Dogs age','First Dogs gender','any','other',fields'),
('Second Dogs Name','Second Dogs age','Second Dogs gender','any','other',fields'),
-- etc

I cheated and used the SQLite Database Browser to just create the tables and some fields, so that part was easy

but I fear I'll just have to start reading a bunch of tutorials to grasp the basic syntax

JNK

the curse and blessing of SQL is it is pseudo-english

JNK

it makes it intuitive to do basic stuff and a HUGE HEADACHE to do more complicated stuff

:7757 Well said

:7751 What columns does table dogs have?

:7756 amen.

3:47 PM

Measurements, I just wanted to dump something in there

but it's NUMERIC, should that be something else?

:7763 so it's just an ID and a column called Measurements?

yeah

ivo - can you post a pic of SQLite viewer

wth happened

my database questions must have killed theirs :P

user image

JNK

careful @IvoFlipse

3:59 PM

to @JNK - any idea how I can filter by results of an analytic function ?

JNK

whatcha mean sathya

JNK

give me a forinstance

« first day next day → last day (2401 days later) »

Transcript for

Sep '1017

Fake Programmers

The room for elite Super Users who realize they know nothing a...

join this room
about this room

00:00

06:00

12:00

18:00

all times are UTC

site design / logo © 2024 Stack Exchange Inc; legal

mobile