You could always try going to the "other side", i.e. banks, where results are published still I guess (I suppose it's more mathematical, though, and the "framework" is generally and academically agreed upon)
This also tells us how to generalize the problem: If you had $A-\lambda I$ with $\lambda$ some nonzero real number, then this would also be invertible since $A-\lambda I$ can't possibly have any zero eigenvectors
Right. To put that more formally: Since $A$ is antihermitian, it only has imaginary or zero eigenvalues. Hence there exists no nonzero vector such that $Av=\lambda v$ whenever $\lambda$ is some complex number off the imaginary axis.
The kernel of a mapping is the part of the domain which gets mapped to zero
So for linear transformations it's vectors such that $Av=0$
So the kernel being trivial means that $Av=0$ is only true when $v=0$
It's on that list of invertible matrix stuff as well
this should be distinguished from the notion of an integral kernel. that's a different thing
(I'll never understand why kernel has two such different meanings. I guess they both arise from the word 'kernel' as root/seed, but it's a bit annoying)
@EmilioPisanty I think that's a better question than many that make the HNQ, and I think your answer is a wonderful example of straightforward common sense in physics and the sort of thing all physics students could learn from.
Hey John! Question: if every possible path is considered in the path integral formulation of quantum mechanics, and is equally valid, doesn't that violate the speed limit of nature, c? If so, how can unobserved particles violate this speed limit? Is it akin to saying that the particle is actually the entire universe?
@dm__ hi. You shouldn't take the path integral formulation too literally. It doesn't mean you have some point particle actually following some convoluted path faster than light.
When you get down to quantum field theory particles are very strange things. They are described by excitations in quantum fields and they look nothing like the intuitive notion of a little ball.
And aren't those fields manifest throughout the entire universe? If the particles are only concentrations of the fields, and those fields are everywhere, then can't we just as well say that the particle is everywhere?
The way we usually approach QFT we describe the particles as Fock states, which are effectively infinite plane waves so the particle has the same probability of existing everywhere in the infinite universe.
But the particles we observe in e.g. the LHC are clearly not well described by infinite plane waves. Instead we describe them by superpositions of plane waves to construct a wave packet.
So if a single particle has the same probability of existing everywhere in the infinite universe, is that what gives equal footing to the path integral formulation?
Of course we cannot prove that it takes any path other than one, but we cannot disprove it either, right?
What lots of students fail to appreciate is that physical theories like QFT are mathematical models that we construct in the hope they will correctly describe the results of our experiments. When a model gives the correct results it's tempting to think that model must be physically true i.e. reflect whatever it is that is really happening.
But there is no guarantee if this - none at all. So to take a mathematical model and start asking what it tells us about the fundamental nature of reality is a risky business.
If the LHC creates a Higgs boson that decays a fraction of a second later there is obviously no useful sense in which that Higgs boson existed everywhere in the universe, regardless of what the mathematical model suggests.
I can see that, for sure. Apart from the usefulness of it though, and with that formulation leading to all of the same experimental results, I think that it deserves more weight than, say, string theory -- sounds beautiful, but like a trap. For the Higgs boson example, apart from the useful sense of it, isn't that still a possibility? That once it decays, it has then been 'absorbed' back into the field, and then resumes all possible locations?
Just seems like a fascinating interpretation of simultaneity
No, the Higgs boson is an excitation in quantum field. It appears when energy is transferred to the Higgs field from some other quantum field, and it disappears when the Higgs field transfers the energy away again into some other field. Once the Higgs boson has decayed it's gone.
The Higgs field still exists of course, but remember that a quantum field is a mathematical object. It is a field whose value at every point in spacetime is a mathematical function called an operator. What it means physically is unclear.
So the Higgs field becomes excited into an actualized boson, and then that energy is transferred to other fields? It does not settle back into some lower state, only manifest in the spread of the field itself?
The field is just a function of position in spacetime, so it exists everywhere in the sense that for any point (t,x,y,z) you choose the field will have some value at that point.
@dm__ well the physical thing is the energy that's being transferred around and turned into different particles. The quantum field is telling us how this happens.
The quantum field obviously describes something physically real because, well, it works - it gives the correct predictions for experiments. But what exactly it represents is something the philosophers worry about. Physicists tend to get on with the job.
It seems that it would be very unlikely if something that yields the most detailed experimental results is solely a mathematical tool, that it does not have a physical basis in nature. Perhaps we blind ourselves by holding onto the intuitive notion of locality, which these fields, if interpreted physically, are inherently not.
You need to be a little cautious about the meaning of non-locality. Physicists have known that quantum mechanics is nonlocal from the very early days. That's exactly what the EPR thought experiment is about (and it's now been confirmed by experiment). But QM is still causal.
Does it violate causality to say that an unobserved particle, embedded into the path integral formulation, travels every possible path in the universe? It seems like these two things do not conflict. My professor said today "The paths considered in the path integral do not obey the speed limit of light. This is also true in a relativistic version." Which has led me down this entire train of thought
You're no doubt familiar with the idea that two entangled particles can be a large distance apart and yet can influence each other instantaneously (also experimentally proved). However this doesn't affect causality since you can't use this behaviour to generate any acausal behaviour.
The FTL travel in the path integral is sort of similar in the sense that it doesn't give any problems with causality.
So in the path that travel outside of what would be permitted by the speed of light, can we understand that with entanglement? Is the particle just relaying its state, in some sense, to entangled particles, and then we measure the result of that entanglement? We cannot distinguish between the particles, so then it would be just as well that 'our' particle did the travel
I'm not too sure about that. Feynman seemed to take it quite literally, as well as my current professor. Perhaps cautioning that we cannot know it has or has not, but the experimental result is the same as if it had. Why discard the possibility?
If anyone's familiar with Graph theory for Network analysis, how do we represent two elements which are connected in parallel? With a single line Or with two separate lines?
@dm__ I guess ultimately this comes down to personal opinion. All we know is that the maths works. We don't, and can't, know or certain how far this reflects what is actually happening. Personally I am agnostic.
Also is weight initialization something people often tinker with? I've always seen Gaussian initialization, but looks like Ng recommends He (square root of 2 / neurons in previous layer) for ReLU layers
I think Xavier-Glorot initialization is pretty standard now
and yeah, I think generally speaking Pytorch is "better" than TF from what I can tell
but again...no keras for pytorch
Pytorch code looks more like python code though because you're actually executing operations on a Variable rather than setting up a graph and then initiating a session to run stuff through that graph
i'm trying to extract relations between Persons (Father of, Mother of .. ) to construct my ontology.
Here is my code:
ny_bb = url_to_string('https://www.biography.com/people/brad-pitt-9441989')
article = nlp(ny_bb)
len(article.ents)
labels = [x.label_ for x in article.ents]
Counter(labels)
p...
> In this paper, we propose the sparsemax transformation. Sparsemax has the distinctive feature that it can return sparse posterior distributions, that is, it may assign exactly zero probability to some of its output variables
Like dropout but not random or something?
I should probably look more into softmax though. I thought it was a smooth version of ReLU rather than what I'd think of as a transformation
I can generally tell at a high level what's being done so I don't go through the details
I will go through the math if I ever have to implement a model myself from scratch -.-
it's tedious and boring
I'm not seeing how the "projection" is generally sparse in the sparsemax
"In words, sparsemax returns the Euclidean projection of the input vector z onto the probability simplex. This projection is likely to hit the boundary of the simplex, in which case sparsemax(z) becomes sparse."
I think I'm trying to visualize this geometrically and I'm failing since I can't visualize >3D lol
You lost me at "probability simplex"...though it seems to me that the boundary of such a thing may be where one class has probability 1 and the rest 0?
as far as I can tell, they are not applying any restrictions on the vectors that you project onto the simplex
specifically, they have not said that the vectors must all lie in the positive space either...they haven't said you MUST put the vector through a ReLU or some such.
so a projection is really generic and specifies any way you can map a higher dim space to a lower dim space I think
generally you want a "projection" to have some nice properties. The one we are familiar with in Linear Algebra for example would be problems like PCA where you're projecting points onto a linear subspace
in that case, one could generally think of things in terms of making right angles
but if the subspace is non-linear or with boundary or something then you either can't always make right angles or there are multiple ways you could make right angles
so I suppose in those cases it'd be nice to use "min distance"
min distance may not be unique either though
I think if you had some non-linear subsurface...it's probably just generically hard to find a "nice" projection absent some other restrictions
I dunno
I haven't had to think too much about projections lol
Referring to the Stanford course notes on Convolutional Neural Networks for Visual Recognition, a paragraph says:
"Unfortunately, ReLU units can be fragile during training and can
"die". For example, a large gradient flowing through a ReLU neuron
could cause the weights to update in such ...
This is what I found in that direction, but it looks like I'm going to have to do some reading on Boltzmann machines and belief networks to understand it
the sparse max really is more of a softmax analogue...the sparse attention I was thinking of was more along the lines of not even considering some inputs
say like something that may happen if you trained dropout
Actually the more that I think about that, the more non-trivial it seems. A couple of ways seem like they would either overfit or be the equivalent of training the weights like normal
I keep getting ^M character in my vimrc and it breaks my configuration.
i'm trying to extract relations between Persons (Father of, Mother of .. ) to construct my ontology.
Here is my code:
ny_bb = url_to_string('https://www.biography.com/people/brad-pitt-9441989')
article = nlp(ny_bb)
len(article.ents)
labels = [x.label_ for x in article.ents]
Counter(labels)
p...
Referring to the Stanford course notes on Convolutional Neural Networks for Visual Recognition, a paragraph says:
"Unfortunately, ReLU units can be fragile during training and can
"die". For example, a large gradient flowing through a ReLU neuron
could cause the weights to update in such ...
