It doesn't matter if the server is running any mods or not, I don't even get a mismatch error.
When I disable all the mods, I get the appropriate errors (or can connect to unmodded servers)
I have seen similar errors reported before and some of them mention mod list packet fragmentation and port randomization, but I cannot do anything about router settings as my ISP does not provide me access to the device that would have that option.
Here are images of the error I get (Could not establish network communication) and the console log when it happens.
https://i.imgur.com/OHKqnSS.png https://i.imgur.com/39ucS1b.png
The server does not show any error, and using the same version and the same mod pack, five other friends have connected and played without issues.