Coding Projects and Duga Heaven :)

12:00 AM

RELOAD!

[Cardshifter/HTML-Client] 1 issue comment.

[Hosch250/VSDiagnostics] 2 commits. 2691 additions. 425 deletions.

[retailcoder/Rubberduck] 8 commits. 705 additions. 460 deletions.

[Vannevelj/VSDiagnostics] 3 commits. 7 opened issues. 1 closed issue. 8 issue comments. 360 additions. 63 deletions.

[Zomis/Machine-Learning] 9 commits. 1247 additions. 34 deletions.

Tweeting Duck

12:24 AM

twitter.com/rubberduckvba/status/686341397796917248

StackExchange

5:01 AM

In the latest round, 9-year-old Muhammad Ali beat 10-year-old JFK at air hockey, while Secretariat lost the hot-dog-eating crown to 12-year-old Ken Jennings. Meanwhile, in a huge upset, 11-year-old Martha Stewart knocked out the adult Ronda Rousey.

Simon Forsberg

7:10 AM

@skiwi resubmitted the webhook because nothing seemed to be happening.

@Phrancis Not a fan of the header font, otherwise I love it

@Phrancis Raises AssertionError actually. I should probably replace that with IllegalArgumentException instead.

Duga

[Zomis/Machine-Learning] Zomis commented on src/main/groovy/net/zomis/machlearn/images/ImageAnalysis.java of commit ce05abf8

> @skiwi2 this is not where my code spends the most time (it does that training the Neural Network), but it could be worth using another approach to get the RGB values indeed.

Simon Forsberg

@skiwi There's a bug in my neural network somewhere, but I'll try to work on that after work today.

Dan Pantry

8:02 AM

Phrancis

11:34 AM

@SimonForsberg The font being used at the moment for the header is Georgia. If you like I can use the same sans-serif font from the bottom (Eurostile) or use another serif font, or whatever you have in mind

Simon Forsberg

11:56 AM

@Phrancis I'd prefer some sans-serif font at least, the one at the bottom is fine :)

Phrancis

Simon Forsberg

12:17 PM

@Phrancis Nice!

Phrancis

12:43 PM

@SimonForsberg OK I just mailed a PNG version of the right size, and the larger size PDF version to your Hotmail address

skiwi

1:20 PM

hey

Phrancis

2:04 PM

hey

Simon Forsberg

5:05 PM

0

A: Community Promotion Ads - 2015

@Phrancis ^^

Not sure it has the right size though... it must be 220x250. My ad seems to be wider than the others.

Phrancis

I went with what Mug told me, they have changed the sizes recently to 300px wide by 250 tall

(not rolled out yet AFAIK)

Maybe super-ping Mug to clarify (kind of busy next door)

Dan Pantry

super-ping?

Phrancis

Mod-ping

in The 2nd Monitor, Jan 8 at 0:30, by Quill

22

Q: We're standardizing the sidebar width at 300px on all sites

Stack Exchange sites are ad-supported. We run relevant, unintrusive ads that don't get in your way--but they help us keep the lights on. Even sites that don't have paid external ads usually have a few internal ones, used to promote other sites on the network and whatever else each community feels...

in The 2nd Monitor, yesterday, by Mat's Mug

@Phrancis make it 300x250

Duga

5:57 PM

[Zomis/Brainduck] Zomis commented on issue #38: Brainfuck extensions

> primaryobjects.com/2015/01/05/… might also be interesting.

skiwi

@SimonForsberg What stuff is expected to work from Machine-Learning?

Simon Forsberg

@skiwi What do you mean?

skiwi

What can I use to see if my clone works?

And there's two modules? First time I work with modules then I think

Simon Forsberg

There are some tests I think, such as BasicPropagationTest

@skiwi technically only one in use... don't know why it says 2 at the moment

I have working implementations of the following, AFAIK: Linear Regression, Logistic Regression, Gradient Descent (for learning Linear and Logistic regression), Feed-forward Neural Network model, Back propagation for training a Neural Network

skiwi

@SimonForsberg I know some of these terms... definitely need to read more about it

Feed-forward and back propagation I'm familiar with, but never implemented it, I think?

Would need to check what the Hopfield Neural Network uses

Simon Forsberg

6:11 PM

@skiwi Feed-forward is I believe the most simple NN.

skiwi

Is there any thought behind using Java for some and Groovy for other files?

Trying to figure out why all files are marked with a weird symbol and why syntax highlighting isn't enabled

Simon Forsberg

@skiwi Not much thought. Although there was a case in the images code that using Groovy just didn't seem to work properly, so I switched to Java which worked fine

skiwi

> :test

net.zomis.machlearn.regression.LinearRegressionTest > simple1_9Line FAILED
groovy.lang.MissingMethodException at LinearRegressionTest.groovy:10

net.zomis.machlearn.regression.LinearRegressionTest > requiresMatchingParamSize FAILED
java.lang.Exception
Caused by: groovy.lang.MissingMethodException at LinearRegressionTest.groovy:10

9 tests completed, 2 failed
:test FAILED

This can't be right?

Simon Forsberg

I'll check

Ooops.

skiwi

Oh wait... I'm still on master branch

Is stuff expected to work there too?

Simon Forsberg

6:18 PM

It fails also on images branch

but I'm not sure why...

skiwi

Wait...

That red thing isn't an error?

Simon Forsberg

although this looks just like the error that caused me to switch to Java for something

skiwi

It's just a Groovy or Java icon?

I'm dumb.

Simon Forsberg

I think your src/main might not be marked as a sources root

src/main/groovy I mean

skiwi

This doesn't quite look right either

Looks much better ^^

Or maybe I've just messed up more stuff

Simon Forsberg

6:23 PM

@skiwi looks right

Phrancis

I have no idea how you manage to look at code with Courier font lol.

skiwi

The dependencies are listed on Neural, but I'm working with Machine-Learning

Duga

[Zomis/Machine-Learning] Zomis pushed commit 0be9c0fe to images: Default logRate to Integer.MAX_VALUE

Simon Forsberg

@skiwi As long as it works, ignore that. :)

skiwi

Problem is that it doesn't work

Simon Forsberg

6:25 PM

I fixed some testcases, for Backpropagation

I'm also getting error with LinearRegressionTest

I will switch LinearRegression to Java, hold on

Duga

[Zomis/Machine-Learning] Zomis pushed commit 114027c7 to images: Use Java for LinearRegression class

skiwi

I added NeuralOne as dependency of Machine-Learning, doesn't sound right either... But it works

Duga

[Zomis/Machine-Learning] Zomis pushed commit 3ca8facd to images: Extract LinearRegressionTest training set

skiwi

> LAYER
Node OUT-0
inputs [w0 -7.119017480526118, MIDDLE-0 --> OUT-0 w(0.6519400108793938), MIDDLE-1 --> OUT-0 w(11.954792930933996)]
outputs []
input 4.794736759249202 output 0.9917947074541579
Node OUT-1
inputs [w0 -5.239138212385757, MIDDLE-0 --> OUT-1 w(10.34905682038181), MIDDLE-1 --> OUT-1 w(6.52927319730159)]
outputs []
input 11.257394075287777 output 0.9999870886912933

Simon Forsberg

There we go, @skiwi

@skiwi Looks right

skiwi

6:29 PM

How serious is this library going to be?

LearningData(double[] inputs, double[] outputs) {
    this.inputs = inputs
    this.outputs = outputs
}

Should use defensive copying, but only if it matters

Else I'm happy to ignore those "issues"

Maybe my warnings are turned up too high ^^

Simon Forsberg

@skiwi For now: Not that serious.

This is not something I expect others to want to use. It is primarily for learning purposes

@skiwi true that.

@skiwi holy crap

skiwi

I barely have a clue of what you are doing in the code :) But it's the first time I see it

Exception in thread "main" java.lang.NullPointerException: challenge-flags-16x16.png not found
	at net.zomis.machlearn.images.Screenshoter.main(Screenshoter.java:20)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

Why are the resources not under resources/net/zomis/machlearn/images/ in this case?

Simon Forsberg

@skiwi They're in test/resources/...

perhaps your directory is not marked as test resources correctly?

skiwi

It's marked as resource root

Simon Forsberg

test resource root?

skiwi

6:43 PM

Yes, same issue

Simon Forsberg

works on my machine

it doesn't need to match the package name as I'm using the class loader to get resources, not the class

skiwi

Right

Simon Forsberg

quite strange that it works for me but not for you

EBrown

Alright, I need to get setup for this thing.

skiwi

How does the content tree look in your case?

Simon Forsberg

6:47 PM

@EBrown There are a few things you will need: IntelliJ IDEA with the Gradle plugin, Java 8 JDK, git :)

EBrown

Well I should have git installed.

Downloading and installing IntelliJ now.

skiwi

@SimonForsberg Is your project in IntelliJ called Machine-Learning or Neural?

Simon Forsberg

@skiwi yes

skiwi

@SimonForsberg X or Y? Yes.

Simon Forsberg

Never ask a programmer an "X or Y" question

EBrown

6:50 PM

> Should I do X, or Y?

> Most certainly.

Simon Forsberg

@skiwi ^^

skiwi

Mine doesn't have [NeuralOne]

EBrown

> Recommended only if you are familiar with vim.

Is anyone familiar with vim?

Simon Forsberg

@skiwi doesn't need it either I believe

skiwi

@EBrown It's similar to Emacs, right?

Simon Forsberg

6:53 PM

@skiwi Except that they're mortal enemies.

skiwi

I think that I should be on the NeuralOne module, but I don't know how I get there

In any case, I believe it should find the resources just fine

EBrown

@SimonForsberg I have no idea what I'm doing with this IntelliJ IDEA thing.

Ah, right. I don't have JDK installed.

Simon Forsberg

@EBrown That's an IDE, just like any other IDE.

@skiwi AFAIK, there is no such module.

that's the root project name it seems

settings.gradle:

EBrown

Does it matter if I get JDK 8u65 or 8u66?

Simon Forsberg

rootProject.name = 'NeuralOne'

include 'Neural'

@EBrown not AFAIK

skiwi

6:56 PM

Probably get the newest one

Still can't get it to work :|

Simon Forsberg

@skiwi then move the file out of there and load it differently

skiwi

Hmm

    public static void main(String[] args) throws AWTException, IOException {
        Rectangle screenRect = new Rectangle(Toolkit.getDefaultToolkit().getScreenSize());
        // BufferedImage capture = new Robot().createScreenCapture(screenRect);

        System.out.println("blablablablabla");

This isn't printing either

Simon Forsberg

@skiwi what are you running?

skiwi

Maybe IntelliJ broke

@SimonForsberg Screenshoter > Right-click > Run "Screenshoter.main()"

Simon Forsberg

delete the project and re-import?

I also cloned Machine-Learning today and could import it by opening the build.gradle easily

EBrown

6:59 PM

Ok, so I have JDK installed, IntelliJ installed, and the project cloned.

Simon Forsberg

How do you open the project in IntelliJ, @skiwi?

skiwi

@SimonForsberg I did checkout from GitHub, did you do it differently?

Simon Forsberg

@EBrown see if you can create a new simple Hello World project in IntelliJ, just to get you acquainted with the most basics of the IDE

@skiwi checking out from github does not open it in IntelliJ

what did you do after the checkout?

skiwi

Inside IntelliJ I checked out from Github

Simon Forsberg

never seen that option

I did git clone from command line, then went to IntelliJ and did File -> Open -> path/to/Machine-Learning/build.gradle

EBrown

7:03 PM

What is this GroupId and ArtifactId?

Simon Forsberg

@EBrown related to maven and library repositories. Each library has a groupId and artifactId

I always use net.zomis as groupId (as I own the zomis.net domain) and for artifactId I use lower-case hyphen-case project-name

skiwi

@SimonForsberg Works in one try, thanks

EBrown

Project created.

skiwi

It's outputting lots of data

Simon Forsberg

magic

@EBrown create a class, create a main method in that class (psvm + ctrl-enter)

and then do System.out.println("Hello World!");

EBrown

7:07 PM

Uh oh

It doesn't work...lol

Simon Forsberg

@EBrown Good! What's the problem?

As a programmer you should know better than to say "It doesn't work"

skiwi

What should the Screenshoter class do?

EBrown

Exception in thread "main" java.lang.ClassNotFoundException: com.evbpc.helloworld.Main
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:264)
	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:122)

Simon Forsberg

@EBrown show your code, all the file

EBrown

package com.evbpc.helloworld;

public class Main {
    public static void main(String[] args) {
        System.out.println("Test");
    }
}

Simon Forsberg

7:09 PM

@skiwi open the training image, create a Neural Network, train network, run some inputs on network.

@EBrown hmm... that looks right. How does your project tree look? (like the screenshots i showed to skiwi before)

skiwi

I suppose the boxes are 39x39?

EBrown

@SimonForsberg ^^

I did a new Console application, IIRC.

skiwi

I'm really unsure what you're doing @Simon, as far as I know NNs map input bits to output bits, but I cannot really find that structure

Simon Forsberg

@skiwi something like that. The input to the network is 39x39 greyscale pixels at least

skiwi

I'd expect an input pattern of 39*39 bits

Simon Forsberg

7:16 PM

@skiwi doubles, not bits.

ANN is more than just bits.

skiwi

However going from left to right, top to bottom doesn't quite learn the NN what you expect it to learn, it cannot really relate things in the vertical axis

Simon Forsberg

@EBrown that "out/production/untitled104" directory is weird.

EBrown

I'm gonna start over.

Simon Forsberg

@EBrown try a "Java Application" or something like that

skiwi

I've got to step away for a bit, what you're doing right now could (to me) use some clarification though, maybe even via Skype, I'm having a hard time understanding it

Simon Forsberg

7:18 PM

@skiwi Even if that is true (which I don't think it is, or I'm misunderstanding you), that doesn't explain the outputs I get when I run some examples after the training.

skiwi

@SimonForsberg I'm trying to understand what the outputs mean in the first place

Simon Forsberg

@skiwi The NeuralNetwork and NeuronLayer and Neuron classes are the core Neural Network model

EBrown

There we go.

Made a blank application, and then manually created the Main class.

Simon Forsberg

@skiwi Depends on whether or not the outputs work correctly. Right now, the image example clearly doesn't work. But it's meant to be the result classification about how if it detected the input as a flag, a clicked field, unclicked, or something else

@EBrown and it works?

EBrown

@SimonForsberg Yep.

Simon Forsberg

7:20 PM

Great

EBrown

Hello world.

Process finished with exit code 0

Simon Forsberg

then you could clone my Brainduck, and open the build.gradle file in IntelliJ

skiwi

{flag=0.14373051192873457, unclicked=0.15866862294774442, clicked=0.5633539422930751} that is the result of network.run(values), right?

Simon Forsberg

and then just click the default buttons to import it

EBrown

Well, I'm going to try to actually implement something in Java first, to refamiliarize myself with it.

(It's literally been years since I've done substantial Java.)

Simon Forsberg

7:21 PM

@skiwi yes. It takes the double[] output of the underlying NeuralNetwork and each output is mapped to an object (flag, unclicked, clicked)

@EBrown Sure :)

There's plenty of community challenges that are worth implementing in Java :)

skiwi

@SimonForsberg And what's the numbers we see there?

How likely it is to be of that type?

Simon Forsberg

@skiwi exactly. At least, that's what it is supposed to be

right now it seems more like gibberish, honestly.

skiwi

As I understand it so far you're learning your NN on unclicked/clicked/flag part, then later on you give it some other part and want to know the result?

JFrame frame = new JFrame();
BufferedImage tempImage = image.getSubimage(834, 284, 39, 39);
frame.getContentPane().add(new JLabel(new ImageIcon(tempImage)));
frame.pack();
frame.setVisible(true);

That's the good news

Figured it was worth to confirm ;-)

How sure are you that your NeuralNetwork is working correctly?

In NeuralNetwork#run there's some commented out code, doesn't exactly give me the confidance I want :P

EBrown

How do I view the line numbers in IntelliJ?

Simon Forsberg

@skiwi yes

@EBrown Ctrl + Shift + A, type "line numbers", go to the settings

@skiwi feel free to commit that code, I've been meaning to work on a GUI for this to show stuff

skiwi

7:33 PM

Have you also noticed that they are not all exactly the same? The one for "zero" is different

Oh... that's all zero doubles

@SimonForsberg That's literally all there was

Simon Forsberg

@skiwi quite sure, but I have my doubts as well. But it seems to work for the simple network in BasicPropagationTest

skiwi

So how exactly do I confirm that the NN is not just learning the wrong things?

Simon Forsberg

@skiwi yes, I've noticed, still can't figure out why.

@skiwi One guess I have is that it's not learning enough (I run very few iterations because it takes a long time)

but I still would have expected much more different output...

skiwi

BackPropagation [1521, 40, 40, 40, 3] iteration 50 : DoubleSummaryStatistics{count=1644, sum=-0,000000, min=-0,000000, average=-0,000000, max=0,000000}

Are those summary statistics correct?

I might have a clue what's going on

EBrown

So this is turning out horridly.

skiwi

7:36 PM

> clicked 2: {flag=0.9999999993111663, unclicked=0.9999999986905259, clicked=0.9999999999007874}
clicked 2: {flag=0.9999999993111663, unclicked=0.9999999986905259, clicked=0.9999999999007874}
clicked 4: {flag=0.9999999993111663, unclicked=0.9999999986905259, clicked=0.9999999999007874}
flag: {flag=0.9999999993111663, unclicked=0.9999999986905259, clicked=0.9999999999007874}
middle junk: {flag=0.9999999993111663, unclicked=0.9999999986905259, clicked=0.9999999999007874}
zero: {flag=0.9999999993111663, unclicked=0.9999999986905259, clicked=0.9999999999007874}

If you add more neurons/layers, your network keeps getting more confident

That's not how NN work ^^

Simon Forsberg

@skiwi yes, but they are for the deltas in the Backpropagation algorithm, not for the actual weights used in the network

skiwi

@SimonForsberg Ah okay

Also got the same results with a single layer of n=40

Simon Forsberg

@skiwi Did you try adding more layers and hidden neurons?

skiwi

n=640 is a bad idea appereantly

Simon Forsberg

@EBrown Java is not as horridly as your code :)

skiwi

7:39 PM

@SimonForsberg A little bit

EBrown

@SimonForsberg I know my code is the issue.

Simon Forsberg

@skiwi goes slow?

skiwi

I know the NN to generic MTG cards use two or three layers, on the GPU, with sizes of several hundreds

@SimonForsberg Very

> clicked 2: {flag=1.0, unclicked=1.0, clicked=1.0}
clicked 2: {flag=1.0, unclicked=1.0, clicked=1.0}
clicked 4: {flag=1.0, unclicked=1.0, clicked=1.0}
flag: {flag=1.0, unclicked=1.0, clicked=1.0}
middle junk: {flag=1.0, unclicked=1.0, clicked=1.0}
zero: {flag=1.0, unclicked=1.0, clicked=1.0}

Layers = [160] there

EBrown

Aha! Got it working.

Incoming CR question.

Simon Forsberg

@skiwi what happens if you introduce a second layer?

@EBrown Did you review it yourself first? :)

EBrown

7:41 PM

@SimonForsberg Yes. I just need to know if there are any other Java constructs that would be helpful.

skiwi

@SimonForsberg Then my PC dies :P It's too slow

Simon Forsberg

@EBrown I'm pretty sure the answer is Yes

skiwi

When I use n=20 then it just get smore confident

Simon Forsberg

@skiwi how about multiple layers with fewer neurons?

EBrown

This Hello World project is no longer a Hello World application.

3

Simon Forsberg

7:44 PM

@EBrown good sign

What have you made?

EBrown

Sieve of Eratosthenes.

skiwi

sizes = [1521, 40, 20, 10, 5, 3]

Why is the input layer of size 1521 and output of size 3?

EBrown

Though I think I'm going to implement a BitArray and post that, instead.

skiwi

assert 39 * 39 == 1521 probably...

EBrown

What's the Java idiomatic naming for a private field?

Simon Forsberg

7:48 PM

@skiwi input size = 39*39. Output size = flag, clicked, unclicked.

@EBrown camelCase

No underscore?

No!

Bah

lol

I'm starting to think that what you're doing is fundamentally wrong, I don't have any claims to back it up yet though...

Right, I'm taking a break now, continuing after that, not sure if you're still here then @SimonForsberg

Simon Forsberg

7:51 PM

@skiwi What would be wrong? How would it be wrong? From my understanding of image processing, this is how it is supposed to be done. (Although perhaps with more training data or training iterations)

@skiwi only one way to find out

I'll probably go to bed at about 22:30

EBrown

Gah, I keep getting lost in the fact this is Java.

It's no different than C#, damnit!

Simon Forsberg

@EBrown Would you have preffered Brainfuck?

EBrown

No lol

skiwi

I'm also going to check out my HopfieldNN first, as it is also classifying images, but it does work, now I only wonder why

EBrown

@SimonForsberg I do have a somewhat complex Java question for you.

Simon Forsberg

8:08 PM

@EBrown Shoot!

EBrown

Is there a way to create a custom array indexer in a class?

In C# I can override (or create) this[int index] which means I can create my own indexer for an array.

Simon Forsberg

@EBrown Nope. Groovy supports that, but not Java.

EBrown

Darn.

0

Q: BitArray (compacting 32 bool values into one integer)

I'm trying to refamiliarize myself with Java and working with it, so I've implemented my own version of a BitArray. Basically, it's an array of integers that maps 32 boolean values to each integer. This means an array of 32 boolean values only takes 4 bytes of space. I was attempting to implemen...

java performance array

Ripe Java question that could use answering. ;)

Simon Forsberg

upvoted, but I won't answer this time

Duga

8:29 PM

[Zomis/Machine-Learning] Zomis pushed commit 0505c76b to images: Pass an instance of Backpropagation to the ImageNetorkBuilder.learn() method

[Zomis/Machine-Learning] Zomis pushed commit 6dbc8b36 to images: Allow injection of Random to BackPropagation

EBrown

@SimonForsberg Personally I think it's pretty clean. Much cleaner than the Java I used to write.

Simon Forsberg

@skiwi Just noticed that you're using Swing

skiwi

@SimonForsberg Yes, that was a very quick hack

Not sure if that's possible with JavaFX

Simon Forsberg

@skiwi To do it at all, or to do a quick hack?

skiwi

A quick hack

@SimonForsberg Yes

Simon Forsberg

8:40 PM

@skiwi You're learning!

skiwi

Have you taken a look at github.com/skiwi2/HopfieldNeuralNetwork (ever)?

What (I think) I'm doing there is look at a pattern and return its closest match

Simon Forsberg

I've looked briefly on it, but not much

skiwi

As far as I know you cannot stop it, so it always crashes at the end

It uses a grid of 10x10 black/white cells, you can learn it a number of patterns and then it either recalls the best fitting pattern, or returns garbage (I believe) if it cannot find one

I need to read up more on backpropagation learning though

I see the HopfieldNN as something that "hardcodes" the information in the network

HopfieldNN is extremely simple though

@EBrown Should it not have reinventing-the-wheel? Because there's already BitSet in Java

EBrown

@skiwi Edit it in. I didn't know there was one.

Simon Forsberg

@skiwi I hate that tag.

skiwi

8:52 PM

You're a mod, you have power ;)

Simon Forsberg

@skiwi It's also my duty to respect the community.

As said, I have power.

EBrown

What's interesting is that, even after deleting the message, chat remembers the reply chain.

skiwi

What's the difference between NeuronLink and NeuronConnection?

StackExchange

CommitStrip: Meanwhile on Mars #10 – Life on Mars?

posted on January 11, 2016 by CommitStrip

David Bowie – Life on Mars ? Votre navigateur ne supporte pas l’élément audio element.

skiwi

@SimonForsberg What confuses me most as of now is that you are trying to map all neurons ultimately to an output layer of 3 neurons, but it might be that I don't quite understand yet how it works here

Simon Forsberg

8:59 PM

@skiwi one is interface, one is class?

skiwi

Wait...

double calculateInput() {
    double sum = 0
    for (NeuronLink link : inputs) {
        sum += link.calculateInput()
    }
    return sum
}

Shouldn't this be an average?

Maybe not

~~What's this deltas concept? Is that what's also called weights? Then I think it makes sense~~

I absolutely don't get why it returns the same values for different images, it's as if it hasn't learned it correctly

You know what would also be really useful? To log the error at every x iterations

That's a good way to see if the network itself is learning

I think you're also effectively only training the 3 output neurons, that doesn't seem like an awful lot

I'm now pretty sure your network is not learning anything (useful)

Simon Forsberg

9:22 PM

@skiwi considering how few iterations are run, it probably hasn't learned correctly. But I'd still expect different values

skiwi

The average neuron error stays at -0.73 forever

@SimonForsberg I'm running with 1000 now, that should work for this stuff

Simon Forsberg

@skiwi Do I have any error that I could log?

skiwi

@SimonForsberg neuron error

(The thing that makes backpropagation work)

Simon Forsberg

@skiwi And also training the nodes in between, but ultimately it is the output that I am interested in

@skiwi what variable name is that kept in?

Oh, there's a neuronError variable...

skiwi

@SimonForsberg double neuronError = expectedOutput[neuronIndexInLayer++] - neuron.output

At least I'm pretty sure we can blame the network

Do you have some resource where you got the algorithms from?

There's of course multiple on Google, but they're not all as good

Simon Forsberg

9:56 PM

@skiwi Your AI book that you sent to me a while ago

A Modern Approach something something

skiwi

10:07 PM

@SimonForsberg Oh... Well, that's a bit embarrassing

Hadn't thought of that at all

Duga

[Zomis/Machine-Learning] Zomis pushed commit 572d4b50 to images: Add support for loading and saving Neural Network

[Zomis/Machine-Learning] Zomis pushed commit 47bad9ab to images: Add .travis.yml

[Zomis/Machine-Learning] build for commit 47bad9ab on images: The Travis CI build passed

Simon Forsberg

At least now I should be able to train a network, save it, load it, continue to train it, save it again, etc...

SirPython

@Duga @SimonForsberg Did you add this through GitHub's services list, or did you do that through your IDE?

Simon Forsberg

@SirPython Copied it from an old project, added it to this project, commited it, done.

So.... neither.

I mean of course....

@SirPython No.

Phrancis

lol

Simon Forsberg

10:22 PM

@skiwi Good news: I understand a bit better why I get that output now. It is either because of a bad algorithm or because of not enough iterations in the training, not sure which yet.

SirPython

Ha! How did it get in that other project then?

Simon Forsberg

Printing the outputs of all layers helped

Layer HIDDEN 1: [0.524676790590632, 0.7151105285865519, 0.655854805199383, 0.6580715993931203, 0.5484418902735422, 0.6155674804608592, 0.6206487914449972, 0.5759816414014581, 0.721672196389395, 0.6573782568042311, 0.5042645783796387, 0.6648779614336485, 0.6988648261556257, 0.5361602752637759, 0.5714845158403853, 0.6278257637751041, 0.580189565609882, 0.6824358381323525, 0.5650591420295252, 0.714344407272292, 0.669114275384087, 0.515974574349947, 0.6644558166491858, 0.6197899921234079, 0.564735104743988, 0.5441213562257804, 0.683356355438126, 0.6691368160088264, 0.7253682079216585, 0.5133279…

@SirPython Copy + Paste

SirPython

Okay. Thanks.

Simon Forsberg

@skiwi Okay, it's not my algorithm that is the problem it seems. The neurons should not use an average of the input. Using the sum is correct. (Which makes perfect sense to me now that I think about it)

So I believe it is caused by too little training.

TTGTB

SirPython

'Night!

Phrancis

10:36 PM

Good night!

skiwi

11:02 PM

I don't believe it is because of the training as the error rate practically does not drop, maybe 0.00000000001 per 100 iterations

We'll see who is right :D

Duga

11:45 PM

200_success vs. rolfl: 6875 diff. Year: +39. Quarter: +39. Month: +39. Week: -40. Day: +10.

Mat's Mug vs. Simon Forsberg: 3427 diff. Year: +141. Quarter: +141. Month: +141. Week: -32. Day: -47.

Loki Astari vs. Simon Forsberg: 2543 diff. Year: -123. Quarter: -123. Month: -123. Week: -98. Day: -78.

200_success vs. janos: 16219 diff. Year: -551. Quarter: -551. Month: -551. Week: -190. Day: -145.

Transcript for

Coding Projects and Duga Heaven :)