If you're like me and have taken AP chem in high school, you probably had that thermodynamics unit, and if you're extra like me, you were basically told "entropy is a measure of disorderedness. Oh and also it actually has energy in it" with no further explanation. So naturally as I have no life I'm going to try to derive everything from first principles.
Disclaimer: I am not a good physicist. Lots of this post were fueled by Google searches and Fable 5. If I get any part of this wrong please let me know! I honestly want to understand this better too.
Defining
Let's do this with a toy example: you have $$$10$$$ particles bound tightly in a solid. They each have an integer energy level $$$\ge 0$$$.
We define a macrostate as every distinct measurable state of this system. For the toy example, this is the total energy: for example a macrostate of this system with energy 3 means $$$U_1 + ... + U_{10} = 3$$$.
We define a microstate as every actual possible arrangement of the particles. For example, if the macrostate is $$$1$$$, then there are 10 total microstates: particle 1 is 1 and rest are 0, ..., particle 10 is 1 and rest are 0. Let's assume that every microstate is equally likely.
This is where the first handwavy thing comes in — who decides what microstates and macrostates are? Theoretically, with good enough measuring tools, wouldn't every microstate be its on macrostate? You are correct. We will get to that.
Then here are the # of microstates that every macrostate can have:
Total energy States
0 = 1
1 = 10 ch 1 = 10
2 = 11 ch 2 = 55
3 = 12 ch 3 = 220
4 = 13 ch 4 = 715
5 = 2002
6 = 5005
7 = 11440
If every microstate is equally likely, then if we find the system in a random configuration, then we should expect the 2 state to be 5.5x as common as the 1 state, etc. So we are more likely to find it in a state with higher entropy than lower entropy.
The entropy of a macrostate is then $$$S = \ln(W)$$$ where $$$W$$$ is the number of microstates.
Temperature
Temperature is defined as
where $$$\frac{1}{T}$$$ is the inverse temperature, equal to the partial derivative of entropy w.r.t $$$U$$$, the total energy of the system.
Thermal equilibrium
Consider we bring two systems together. How would they interact?
After a period of interaction, we must have $$$U_1 + U_2 = U$$$, since energy is conserved. Let's calculate entropy here. The total number of microstates for the system to be in $$$U_1, U_2$$$ simply comes from multiplying system 1 and system 2's: $$$W_1 \cdot W_2$$$, and $$$S = \ln(W_1 \cdot W_2) = \ln(W_1) + \ln(W_2)$$$.
Now remember that probablistically, the system tends to move toward higher entropy. Consider a scenario where system 1 sends $$$\partial U$$$ amount of energy to system 2. System 1 loses $$$\ln W_1$$$ proportional to its inverse temperature, and system 2 gains $$$\ln W_2$$$ proportional to its inverse temperature. If system 1 is hotter, than system 1 loses less $$$\ln W_1$$$ and system 2 gains more $$$\ln W_2$$$, meaning that the new state where system 2 has more energy is more statistically favorable.
In other words, hotter systems "like" to send energy to colder systems, by definition. Eventually, the two systems will reach equal temperature, which maximizes $$$S = \ln(W_1) + \ln(W_2)$$$.
Here's the chart again to visualize it:
Total energy States S = ln(W) 1/Temp = (dS/dU) (slight abuse of derivative definition)
0 1 0.0 2.3 # colder systems gain a lot of states when heated, so cold systems like to receive energy
1 10 2.3 1.7
2 55 4.0 1.4
3 220 5.4 1.2
4 715 6.6 1.0
5 2002 7.6 0.9
6 5005 8.5 0.8 # hotter systems lose few states when cooled, so hot systems like to send their energy away
7 11440 9.3
One final note that you may be thinking to yourself is, the highest entropy state is not necessarily, just like how the mean of a normal distribution is the highest probability state, but you obviously never always get that. This is true, but in real life the number of microstates numbers something like like $$$10^{10^{20}}$$$ vs $$$10^{1.0001 \cdot 10^{20}}$$$ (the second one is $$$10^{16}$$$ times more likely), so it's a pretty big margin.
Also another funny thing: if you write entropy as $$$\log_2(W)$$$, entropy is measured in bits. Then $$$\frac{\partial S}{\partial U}$$$ would be bits per energy, and thus a valid unit of temperature could be Joules per byte. In fact, 1 Kelvin is $$$7.7 \cdot 10^{-23}$$$ J/byte or $$$0.48$$$ eV/kB.
Energy
So far, we've covered the theoretical basis of how entropy works and the theoretical framework behind temperature. But we still have the funny thing where entropy "makes energy unavailable for use" and the second law of thermodynamics. What's the connection here?
Firstly, entropy in real life
Real life is continuous. So instead of counting, we generalize to integrals in phase space. Specifically, if you have $$$n$$$ particles, the phase space includes $$$6n$$$ dimensions: for each particle, its 3-dimensional position and 3-dimensional momentum.
So then we say that if a system has total energy between $$$U$$$ and $$$U + \delta$$$, we'll integrate over this $$$6n$$$-dimensional space over all states that satisfy it. (You're just supposed to pick an arbitrary but small value of $$$\delta$$$. I honestly don't fully understand why but I think it's just a mathematical thing. Maybe it's more correct to describe the limiting behavior as $$$\delta \rightarrow 0$$$.)
Then to go to states, we use fun quantum mechanics: the uncertainty principle says a single quantum state is position * momentum = size $$$h$$$. So divide this by $$$h^{3n}$$$ and we get our number of states.
I'm still going to refer to microstates in a discrete sense because it's easier to reason with, but just know that this is how it actually works.
"Conservation of microstates"
This is Liouville's Theorem.
Intuitive statement: We will perform any physical procedure $$$P$$$. Suppose we perform $$$P$$$ separately on 1,000 distinct input states. Then, intuitively, $$$P$$$ should be bijective — we get 1,000 distinct output states. That's because physics must work backwards, so you can't have two different states initially that then somehow become identical states later.
Liouville's Theorem strengthens this in the continuous-phase-space case saying that you can't shrink the volume either. (Otherwise, the mapping $$$x \rightarrow \frac{x}{2}$$$ is bijective but cuts the number of microstates in half. Or you can think of discrete states again with quantum mechanics which just returns to the previous argument.
So, this is kind of like a "conservation of microstates". If we feed macrostate $$$x$$$ into, which contains $$$W$$$ microstates that we don't know about, the output could be one of $$$W$$$ microstates.
But...
Think of a gas confined to a left half of a container. With our state-of-the-art measuring tools, this gas has $$$W$$$ microstates. Now we open the valve and let the gas occupy the whole container. The gas is still in exactly $$$W$$$ microstates. But now we're in an interesting situation:
- If I knew the $$$W$$$ microstates of the gas beforehand (but I don't know which) and I had incredibly powerful computing resources, I could simulate each one find out that the gas is in one of $$$W$$$ microstates afterward.
- If all you could observe was the new gas, you would only see the new macrostate, which could consist of $$$W_2 \gt W$$$ microstates.
So this is how entropy can increase.
How does this affect available energy?
Temperature. Define free energy $$$A = U - TS$$$: total energy, with entropy multiplied by temperature subtracted. This is how much useful energy we can extract out of this system.
Now suppose we add in 1 J of energy to this system, so $$$U$$$ increases by 1 J. You would:
//TODO HERE
- Increase the total energy of the system $$$U$$$ by 1 J.
- This means that due to $$$\frac{1}{T} = \frac{\partial S}{\partial U}$$$, entropy increases by $$$\frac{1}{T}$$$.
Put this into useful energy; entropy doesn't change and $$$U$$$ increases by 1 J, so A increases by 1 J, as intended.
- Put the system in more microstates: 1 J would increase entropy by $$$\frac{1}{T}$$$ since , so in total the $$$U$$$ term increases by 1 J, but the $$$TS$$$ term also increases by 1 J, and you've obtained no additional useful energy because the new microstates are indistinguishable.
