The h Bar

« first day (3924 days earlier) ← previous day next day → last day (988 days later) »

ZeroTheHero

2:39 AM

@Rob hey…

I raised a couple of flags on a user and it seems the user admitted in a post that they closed…

but I will presume you can track down that flag nevertheless.

2 hours later…

John Rennie

4:29 AM

@satan29 It is an example of a tensor eating two vectors to produce a scalar.

satan 29

@JohnRennie I see

John Rennie

@satan29 Second rank tensors do that! :-)

8 hours later…

satan 29

12:31 PM

WHat happened to all the stars!?!?!?!

Nihar Karve

@satan29 are they invisible?

satan 29

yes

Nihar Karve

I've been getting this problem too sometimes, they usually load after some time

have a lot of your messages been timing out as well?

satan 29

YES OH GOD

for 2 days now. Its a menace

user178758

yup

ACuriousMind

12:36 PM

the stars are blinking out, one by one

must be the universe ending

satan 29

12:49 PM

any idea how to get the array in a more...readable format?

ACuriousMind

@satan29 redefine __str__ or __repr__ of the object you're printing there

satan 29

I dont follow...

Something like this can be done by pd.read_csv() and then printing it but we dont have csv file here...

ACuriousMind

the behaviour of a Python object when passed to print is defined by its __str__ or __repr__ methods

satan 29

ACuriousMind

what you see there is the default behaviour - the object is just printing its list of attributes as { <attribute_name> : print(attribute), ... }

so it has some attribute called data, and then that is an array that prints itself like that messy array

satan 29

12:55 PM

yeah

ACuriousMind

you can try passing it to pprint and see whether that improves things

but otherwise you have little choice but to implement the pretty-printing yourself by looping over data and printing it how you like

1 hour later…

Daniel Underwood

2:16 PM

pandas and pd.DataFrame(boston) may get you there, though you might need mappings for the data columns for it to actually look nice

3 hours later…

satan 29

5:23 PM

How can I normalise each column of a numpy array?

by normalise, i mean, $xi-->xi-\mu / \sigma$

where $\mu$ and $\sigma$ are mean and SD of that column

Daniel Underwood

Well `array.std()` and `array.mean()` are certainly available to do it
If you're using sklearn, it also has some preprocessing functions that you can use on their own or in a pipeline

satan 29

@DanielUnderwood but that will give a single value for the entire array, no?

I want to do it column by column

Daniel Underwood

Ah I didn't catch that part. I'm not sure off the top of my head, but I think it would be the axis parameter to those two functions

satan 29

Another issue is that whenever i try to print a certain column of the array, the values are in scientific format (eg 1.024 e 02) which is making things cumbersome

@DanielUnderwood My idea is to use what you suggested after extracting the particular columns in a loop

Daniel Underwood

5:41 PM

Yeah you could do it that way by slicing in a loop as well. A python loop will be a bit slower than having numpy do the whole thing at once, but that likely won't matter if the array is small

I think the scientific notation thing might be an option somewhere

satan 29

@DanielUnderwood hmm right, but right now I just want to get the whole thing done first, Ill look to optimize later

Nihar Karve

Compute the mean and stddev of the matrix using the respective numpy functions with axis=1

Then do x = x - mean/stddev

Broadcasting should take care of the rest

satan 29

okay hang on, I fixed the scientific notation problem, however for some reason I dont think the numpy array isnt showing the correct values

import sklearn
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_boston



boston=load_boston()
#print(boston)
X , y = load_boston(return_X_y = True)
y = y.reshape(506,1)

df_x=pd.DataFrame(boston.data,columns=boston.feature_names)
df_y=pd.DataFrame(boston.target)

print(df_x)

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)

np.set_printoptions(precision=3,suppress=True)

printing the dataframe df_x and the numpy array X-train gives different values...

Daniel Underwood

Does train_test_split shuffle by default?

satan 29

5:57 PM

ah damn

the dafult for shuffle is true hmm

@NiharKarve what exactly do you mean by broadcasting?

and with axis =1, stdev and mean would be arrays themselves, so how can you divide by an array? Or does "broadcasting" interpret this in the way we are looking for?

Nihar Karve

Yes, numpy should divide them element-wise

satan 29

Ah I see

Nihar Karve

This yields a length M vector, which you can't directly add to an MxN matrix

But broadcasting comes in again - it adds this same row vector to each row in the matrix, which I think is what you are looking for

satan 29

wait, I didn't catch that last part

Nihar Karve

Try this: jakevdp.github.io/PythonDataScienceHandbook/… (sorry I'm on phone right now so I can't really type fast or verify the code)

satan 29