Well argued! Yes, it is true that we don’t have causal relationships for a number of phenomena. But the absence of evidence is not the evidence of absence. So corner cases of failure may exist in all these phenomena which can cost lives. Does this mean that we halt the fast paced progress of AI research or any other scientific pursuit? No! But leveraging empirical evidence that can’t be fully explained in situations where lives are at stake, should require a very high bar of regulation. Responsible scientists and engineers agree that causality is important to understand and they do all they can to understand how systems work. However there are likely many among us who do not employ similar standards to the application of ill understood techniques. When it comes to regulation, we must pay heed to the worst in us.
Yann LeCun:
Actually, there is no clear definition of causality in classical and quantum physics, simply because the elementary equations of motions are time reversible (if you also reverse charge and parity). For every phenomenon that can occur, the same phenomenon can occur backwards in time (with corresponding anti-particles).
Take a Feynman diagram where an electron and a positron annihilate to produce a gamma ray photon. It can be interpreted as a gamma ray photon spontaneously creating an electron-positron pair. It's the same diagram where time goes right to left instead of left to right.
How can one possibly say that A causes B, when B could very have caused A in a time-reversed view point?
Even worse, most physical phenomena have loopy causal graphs. Motion "causes" friction. But friction limits the velocity of motion. Most differential equations have coupled terms with loopy interactions in which quantity x affects quantity y and vice versa. You rarely have y(t+dt)=f(x(t)) in physics. More often than not, you have coupled equations y(t+dt)=f(x(t)) and x(t+dt)=g(y(t)).
In these (frequent) cases, x cause y, but y also causes x.
There is something like that going on in fluid dynamics, which is why it's difficult to come up with "simple" causal explanations.
Only when collective phenomena are considered does the "arrow of time" appear to have a definite direction (with the 2nd law of thermodynamics).
>Actually, there is no clear definition of causality in classical and quantum physics, simply because the elementary equations of motions are time reversible (if you also reverse charge and parity). For every phenomenon that can occur, the same phenomenon can occur backwards in time (with corresponding anti-particles).
This is super misguided. Causation doesn't need physical grounding. The usual definitions are grounded in counterfactuals and interventions.
It's like defining the causal effect as the difference in potential lines of code I write today, depending on whether I do or don't eat breakfast this morning, but then arguing that under classical mechanics the universe is deterministic so I couldn't possibly take an intervention to not eat breakfast. Really, it's just another argument that counterfactuals are counter factual, and therefore don't have a clear definition. But yeah, that's in the name already.
Regardless, physical equations are pretty much all causal, because they are stable under intervention. If I intervene to do whatever, they still hold. That lets me cause a change in friction be intervening and swapping out a material.
Not quite, especially since time is such a weird concept when you dig far enough into the math. There are a bunch of competing definitions of a causal change, but arguably the most conventional one is the difference between an outcome if you do one thing versus another.
For example, looking at whether I write this comment now instead of getting back to work after lunch, you could imagine that I'm more inclined to slack off after a heavy meal. We have two hypotheticals. Scenario 1) I eat curry for lunch. Scenario 2) I eat salad for lunch. In scenario 1, I am a bit sleepy and log on to HN and write this. In scenario 2 (hypothetically) I just get right back to work and this comment never exists. The causal effect (hypothetically) is that having curry instead of salad for lunch made this comment exist.
Compare that to the effect of the comment I wrote before lunch. We can still talk about the causal effect of lunch on what I wrote before lunch. Assuming that physics works the way I think it does, the causal effect is that nothing changes, but it's still a thing you can formalize. You could imagine an episode of star trek involving time travel, where my actions today do affect things yesterday. Starfleet statisticians could still run time-reversed randomized trials, they'd just have to be really careful about their experimental designs.
I like your original comment further up, but disagree with the following bit:
> We can still talk about the causal effect of lunch on what I wrote before lunch.
To the extent that the situation occurs in the physical universe, that is meaningless. The future can never be a causal parent of the past. That says there is much more to the typical causality question than just the direction of time.
The difference between the “time reversibility” in physics and the typical ML example is that once you’ve decided on something as the effect, the other thing must be the cause. So the problem is quite simple if you’re not trying to figure out the nature/direction of time, and especially not with microscopic physics.
Anyone curious about the relation between time reversibility in physics and causal reasoning could look at Scholkopf+Janzing 2016 for an interesting line of thought.
Somebody should tell Yann Lecun that a little knowledge is dangerous.
>The future can never be a causal parent of the past.
That's a testable statement. The reason it's testable is because we can define a hypothetical causal effect of the present on the past, and then do experiments and show that it's always zero. I only mean that we can talk about it, not that we can show it to be anything but a non-effect, so I'm not sure I see where we're in disagreement.
> How can one possibly say that A causes B, when B could very have caused A in a time-reversed view point?
There is an old joke about a mathematician and an engineer invited to kiss someone they both fancied if they got there first. The catch was they could only move half the distance towards the inviter each period.
While the mathematician fretted about Zeno's paradox preventing from convergence, the engineer got close enough to the inviter, eventually, to take them up on the offer.
LeCun is the mathematician in this analogy.
Note: he responded to Mukherjee's comment here. Why?
Mukherjee made a comment.
LeCun then responded.
The causal reason that LeCun responded is because Mukherjee commented.
There may be a host of associated factors -- perhaps LeCun only monitors comments while bouncing on a pogo stick in a large conference room and simultaneously sipping a latte. It doesn't matter. Because if LeCun were bouncing on a pogo stick in a conference room and sipping his latte, he would not have made the response if Mukherjee had not first commented.
So, we have clear existence of causality. If needs be, we could build an entire framework with that observational evidence.
Placing this in a comment in case someone else is interested in some other aspect, but I am confused by this time-reversal point. Yes, the equation can go in either direction but time itself -- as we understand it -- goes in one direction, basically divided by our memory.
So why not just stipulate that the equation is one directional, and there is another (inverse, but otherwise identical) equation for the reverse process?
Gaurav Mukherjee:
Well argued! Yes, it is true that we don’t have causal relationships for a number of phenomena. But the absence of evidence is not the evidence of absence. So corner cases of failure may exist in all these phenomena which can cost lives. Does this mean that we halt the fast paced progress of AI research or any other scientific pursuit? No! But leveraging empirical evidence that can’t be fully explained in situations where lives are at stake, should require a very high bar of regulation. Responsible scientists and engineers agree that causality is important to understand and they do all they can to understand how systems work. However there are likely many among us who do not employ similar standards to the application of ill understood techniques. When it comes to regulation, we must pay heed to the worst in us.
Yann LeCun:
Actually, there is no clear definition of causality in classical and quantum physics, simply because the elementary equations of motions are time reversible (if you also reverse charge and parity). For every phenomenon that can occur, the same phenomenon can occur backwards in time (with corresponding anti-particles).
Take a Feynman diagram where an electron and a positron annihilate to produce a gamma ray photon. It can be interpreted as a gamma ray photon spontaneously creating an electron-positron pair. It's the same diagram where time goes right to left instead of left to right.
How can one possibly say that A causes B, when B could very have caused A in a time-reversed view point?
Even worse, most physical phenomena have loopy causal graphs. Motion "causes" friction. But friction limits the velocity of motion. Most differential equations have coupled terms with loopy interactions in which quantity x affects quantity y and vice versa. You rarely have y(t+dt)=f(x(t)) in physics. More often than not, you have coupled equations y(t+dt)=f(x(t)) and x(t+dt)=g(y(t)).
In these (frequent) cases, x cause y, but y also causes x. There is something like that going on in fluid dynamics, which is why it's difficult to come up with "simple" causal explanations.
Only when collective phenomena are considered does the "arrow of time" appear to have a definite direction (with the 2nd law of thermodynamics).
Edit: para spacing