If you're like me and have taken AP chem in high school, you probably had that thermodynamics unit, and if you're extra like me, you were basically told "entropy is a measure of disorderedness. Oh and also it actually has energy in it" with no further explanation. So naturally as I have no life I'm going to try to derive everything from first principles.

Disclaimer: I am not a good physicist. Lots of this post were fueled by Google searches and Fable 5. If I get any part of this wrong please let me know! I honestly want to understand this better too.

Defining

Let's do this with a toy example: you have $$$10$$$ particles bound tightly in a solid. They each have an integer energy level $$$\ge 0$$$.

We define a macrostate as every distinct measurable state of this system. For the toy example, this is the total energy: for example a macrostate of this system with energy 3 means $$$U_1 + ... + U_{10} = 3$$$.

We define a microstate as every actual possible arrangement of the particles. For example, if the macrostate is $$$1$$$, then there are 10 total microstates: particle 1 is 1 and rest are 0, ..., particle 10 is 1 and rest are 0. Let's assume that every microstate is equally likely.

This is where the first slightly handwavy thing comes in — who decides what microstates and macrostates are? Theoretically, with good enough measuring tools, wouldn't every microstate be its on macrostate? You are correct. Entropies (and temperatures) do change depending on who is measuring them. Strictly, a macrostate is an observable state of a group of a particles, and the microstates that compose of it are not distinguishable by any means with the tools we have. Also, the uncertainty principle from quantum mechanics does somewhat limit the resolution of our measurements.

Then here are the # of microstates that every macrostate can have:

Total energy   States
 0                       = 1
 1             = 10 ch 1 = 10
 2             = 11 ch 2 = 55
 3             = 12 ch 3 = 220
 4             = 13 ch 4 = 715
 5                       = 2002
 6                       = 5005
 7                       = 11440

If every microstate is equally likely, then if we find the system in a random configuration, then we should expect the 2 state to be 5.5x as common as the 1 state, etc. So we are more likely to find it in a state with higher entropy than lower entropy.

The entropy of a macrostate is then $$$S = \ln(W)$$$ where $$$W$$$ is the number of microstates.

Temperature

Temperature is defined as

where $$$\frac{1}{T}$$$ is the inverse temperature, equal to the partial derivative of entropy w.r.t $$$U$$$, the total energy of the system.

Thermal equilibrium

Consider we bring two systems together. How would they interact?

After a period of interaction, we must have $$$U_1 + U_2 = U$$$, since energy is conserved. Let's calculate entropy here. The total number of microstates for the system to be in $$$U_1, U_2$$$ simply comes from multiplying system 1 and system 2's: $$$W_1 \cdot W_2$$$, and $$$S = \ln(W_1 \cdot W_2) = \ln(W_1) + \ln(W_2)$$$.

Now remember that probablistically, the system tends to move toward higher entropy. Consider a scenario where system 1 sends $$$\partial U$$$ amount of energy to system 2. System 1 loses $$$\ln W_1$$$ proportional to its inverse temperature, and system 2 gains $$$\ln W_2$$$ proportional to its inverse temperature. If system 1 is hotter, than system 1 loses less $$$\ln W_1$$$ and system 2 gains more $$$\ln W_2$$$, meaning that the new state where system 2 has more energy is more statistically favorable.

In other words, hotter systems "like" to send energy to colder systems, by definition. Eventually, the two systems will reach equal temperature, which maximizes $$$S = \ln(W_1) + \ln(W_2)$$$.

Here's the chart again to visualize it:

Total energy   States   S = ln(W)  1/Temp = (dS/dU) (slight abuse of derivative definition)
 0             1        0.0        2.3  # colder systems gain a lot of states when heated, so cold systems like to receive energy
 1             10       2.3        1.7
 2             55       4.0        1.4
 3             220      5.4        1.2
 4             715      6.6        1.0
 5             2002     7.6        0.9
 6             5005     8.5        0.8  # hotter systems lose few states when cooled, so hot systems like to send their energy away
 7             11440    9.3

One final note that you may be thinking to yourself is, the highest entropy state is not necessarily, just like how the mean of a normal distribution is the highest probability state, but you obviously never always get that. This is true, but in real life the number of microstates numbers something like like $$$10^{10^{20}}$$$ vs $$$10^{1.0001 \cdot 10^{20}}$$$ (the second one is $$$10^{16}$$$ times more likely), so it's a pretty big margin.

Also another funny thing: if you write entropy as $$$\log_2(W)$$$, entropy is measured in bits. Then $$$\frac{\partial S}{\partial U}$$$ would be bits per energy, and thus a valid unit of temperature could be Joules per byte. In fact, 1 Kelvin is $$$7.7 \cdot 10^{-23}$$$ J/byte or $$$0.48$$$ eV/kB.

Energy

So far, we've covered the theoretical basis of how entropy works and the theoretical framework behind temperature. But we still have the funny thing where entropy "makes energy unavailable for use" and the second law of thermodynamics. What's the connection here?

Firstly, entropy in real life

Real life is continuous. So instead of counting, we generalize to integrals in phase space. Specifically, if you have $$$n$$$ particles, the phase space includes $$$6n$$$ dimensions: for each particle, its 3-dimensional position and 3-dimensional momentum.

So then we say that if a system has total energy between $$$U$$$ and $$$U + \delta$$$, we'll integrate over this $$$6n$$$-dimensional space over all states that satisfy it. (You're just supposed to pick an arbitrary but small value of $$$\delta$$$. I honestly don't fully understand why but I think it's just a mathematical thing. Maybe it's more correct to describe the limiting behavior as $$$\delta \rightarrow 0$$$.)

Then to go to states, we use fun quantum mechanics: the uncertainty principle says a single quantum state is position * momentum = size $$$h$$$. So divide this by $$$h^{3n}$$$ and we get our number of states.

I'm still going to refer to microstates in a discrete sense because it's easier to reason with, but just know that this is how it actually works.

"Conservation of microstates"

This is Liouville's Theorem.

Intuitive statement: We will perform any physical procedure $$$P$$$. Suppose we perform $$$P$$$ separately on 1,000 distinct input states. Then, intuitively, $$$P$$$ should be bijective — we get 1,000 distinct output states. That's because physics must work backwards, so you can't have two different states initially that then somehow become identical states later.

Liouville's Theorem strengthens this in the continuous-phase-space case saying that you can't shrink the volume either. (Otherwise, the mapping $$$x \rightarrow \frac{x}{2}$$$ is bijective but cuts the number of microstates in half. Or you can think of discrete states again with quantum mechanics which just returns to the previous argument.

So, this is kind of like a "conservation of microstates". If we feed macrostate $$$x$$$ into, which contains $$$W$$$ microstates that we don't know about, the output could be one of $$$W$$$ microstates.

But...

Think of a gas confined to a left half of a container. With our state-of-the-art measuring tools, this gas has $$$W$$$ microstates. Now we open the valve and let the gas occupy the whole container. The gas is still in exactly $$$W$$$ microstates. But now we're in an interesting situation:

• If I knew the $$$W$$$ microstates of the gas beforehand (but I don't know which) and I had incredibly powerful computing resources, I could simulate each one find out that the gas is in one of $$$W$$$ microstates afterward. However, in practice, you'd need an infinitely high precision measurement for each particle which is impossible to get in real life. (Lyapunov time / Lyapunov exponent.)
• If all you could observe was the new gas, you would only see the new macrostate, which could consist of $$$W_2 \gt W$$$ microstates.

So this is how entropy can increase.

Landauer limit

Warning: to the best of my knowledge, I think my argument is right, but it's definitely possible there are incorrect statements or accidental circular reasonings or bad assumptions in here.

The Landauer limit basically says that if you have a physical binary register (could be anything that stores information), permanently erasing it requires a small amount of energy.

The reasoning: Suppose your register could be in macrostate "0" or macrostate "1", each of which could be $$$W$$$ microstates (they have to be equal in number or else some weird things I don't understand happen), and you need to clear it to "0". Erasing must be a physical process. So by Liouville, the "blank" macrostate must have $$$2W$$$ microstates, because every one of $$$2W$$$ previous microstates must map to a different new microstate.

Suppose this erasure operation occurs at constant temperature. Then somehow the number of microstates doubled, aka $$$S$$$ increased by $$$\ln 2$$$. So energy had to be added to the system, specifically since $$$\frac{1}{T} = \frac{\partial S}{\partial U}$$$, adding $$$T \ln 2$$$ of energy is required.

Thus, clearing a memory register costs $$$T \ln 2$$$ of energy. In real life, that would be 0.018 electronvolts at room temperature.

The second law

...states that the entropy of the entire universe has to stay constant or increase over time.

If you define the microstates within a macrostate as "you will never be able to tell them apart no matter what physical process you apply", then I think it's fairly straightforward to understand why this law is true. Our universe is currently in $$$W$$$ possible microstates; if we perform a reversible operation, there are still $$$W$$$ forever-indistinguishable microstates of the universe. If we perform an operation like letting the gas expand, then we could end up with $$$W_2 \gt W$$$ permanently indistinguishable microstates. You'll never be able to go backwards.

Why is entropy and energy correlated anyway?

I sort of understand the theory now, but it still leaves behind a bit of a nagging question — why does entropy/disorderedness/microstates cause energy?

The answer, as always, is temperature. In order to increase the number of microstates something has, you need to put energy into it (proportional to the temperature). That's why energy matters — when particles have more energy, they have more microstates*.

In order to erase the bit in your register, you had to increase the number of microstates, which requires adding energy to it. If microstates could be increased for free (aka temperature zero), then these issues would not exist.

And then once you put that energy in, it's pretty much gone forever.

(*: In the quantum world you can have such a thing where adding energy actually decreases microstates — in these cases, you have negative temperature, which is actually "infinitely hot".)

Anyway this concludes the first physics post. I hope this was interesting if you were also curious about this topic. And I also hope everything I said is correct.

Also, if you are DuyMinh3005, click this link.