Dijkstra finds paths in non-decreasing order of cost. For regular shortest path problems, you do not continue searching paths that aren't the shortest for that vertex, however in second-shortest path you need to keep a counter for how many times a vertex has been dequeued from the priority queue / heap, and continue the search if it's the shortest or second-shortest path.

Codeforces graphs and shortest paths tags (Editorial if you can't solve them)

Cses graph problems

Book

get brain (and use it)

Try to understand and challenge each step of algorithm, try to come up with counter examples. Soon you will find out why it actually works and how to extend it to other problems. Post here if you get stuck.

Might as well use the USACO guide. Under the gold section in the Shortest Path with nonnegative weights, there are nice problems from various OIs and OJs with most of them having an internal solution