After I changed to a new font for the whole document, it also affects math font. So now $j$ becomes extremely long, and when \sqrt wants to adapt to its size, the result is not that good…

You can clearly see the difference for $i$ and $j$…

After some search, I found this. However, this won’t directly solve my problem. Then I combined two of his ideas:

\norm{\theta}_F = \sqrt{\smash[b]{\sum_{j} \theta_j^2} \vphantom{\sum_i}}

• \vphantom has the height and depth of its argument, but its width is zero (from tutorialspoint).
• \smash takes its contents and prints it as if its height and depth were zero (from stackexchange).

Hence this is a really weird solution but it works like a charm.